go integration tests with ginkgo and gomega

Today, I’ve checked out the BDD¹ testing framework ginkgo with the matcher library gomega for integration tests with go. The motivation was to find an alternative for my go-to testing framework rspec. Although in ruby, I anyway used it for go projects like blogctl.

• rspec - Were I came from
• Integration test frameworks in go
• ginkgo
  • Setup
  • Show version
  • Real tests
• Thoughts

rspec - Where I came from

The rspec DSL together with ruby itself makes it easy to write down tests pretty quickly. At least for me its quicker than doing the same in go (until now?):

require 'rspec'
require 'open3'

describe 'CLI' do
  context 'Trivial test cases' do
    it 'shows a version' do
      out, err, _ = Open3.capture3("blogctl --version")
      expect(err).to eq ""
      expect(out).to match(/BuildVersion:/)
      expect(out).to match(/BuildDate:/)
    end
  end
end

Integration test frameworks in go

There are a bunch of testing frameworks on the awesome-go list that look interesting for BDD integration testing.

• Active development: last contribution was within the last year
• Specify tests within go: The test can be specified within go.
• Various nestable containers: Analog to rspecs decribe and context container that can be nested.
• (Before|After)(All|Each) scripts: rspec alike setup and tear down functions for all and test-wise
• ?: I skipped evaluating this feature as it wasn’t obvious from the readme and another feature was unsupported anyway.

ginkgo

Ginkgo offers a similar Given-When-Then approach after looking at the example in the readme. Gomega is the matcher library, used by ginkgo. They’re both from the same author.

I’ve used my renaming tool r to check it out.

Setup

// test suite
// r_suite_test.go
package r_test

import (
  . "github.com/onsi/ginkgo/v2"
  . "github.com/onsi/gomega"
  "testing"
)

func TestBooks(t *testing.T) {
  RegisterFailHandler(Fail)
  RunSpecs(t, "R Suite")
}

// spec
// r_test.go
package r_test

import (
  . "github.com/onsi/ginkgo/v2"
  . "github.com/onsi/gomega"
)

var _ = Describe("R", func() {
})

Show version

package main_test

import (
	"os"
	"os/exec"
	"path/filepath"

	"github.com/bsm/gomega/gbytes"
	"github.com/bsm/gomega/gexec"
	. "github.com/onsi/ginkgo/v2"
	. "github.com/onsi/gomega"
)

var _ = Describe("R", func() {
	When("binary is used correctly", func() {
		It("shows version", func() {
			session, err := gexec.Start(
				exec.Command("ci-build/r", "--version"),
				GinkgoWriter,
				GinkgoWriter,
			)
			Expect(err).ShouldNot(HaveOccurred())
			Eventually(session).Should(gexec.Exit())
			Expect(session.Out).Should(gbytes.Say(`(?m)BuildVersion:.*\nBuildDate:`))
			Expect(session.Err).Should(gbytes.Say(""))
		})
	})
})

• provides containers like Describe, When and It similar to rspecs describe, context and it
• few more lines, words and brackets, as expected in go
• gomega provides so-called Asynchronous Assertions like Eventually which checks the expection periodically for some time, which is very handy
• helper libs like gexec and gbytes simplify the matcher that would otherwise be a bit more fiddly in go
• gbytes directly awaits the saying to be a regex, like a charm!

Real tests

The following tests use before and after scripts that set up the temporary environment for some real2 tests:

package main_test

import (
	"os"
	"os/exec"
	"path/filepath"

	"github.com/bsm/gomega/gbytes"
	"github.com/bsm/gomega/gexec"
	. "github.com/onsi/ginkgo/v2"
	. "github.com/onsi/gomega"
)

var _ = Describe("R", func() {
	When("binary is used correctly", Ordered, func() {
		var tempFile *os.File
		var tempDir string

		BeforeAll(func() {
			var err error
			tempDir, err = os.MkdirTemp(os.TempDir(), "")
			Expect(err).Should(BeNil())
		})

		AfterAll(func() {
			os.RemoveAll(tempDir)
		})

		BeforeEach(func(ctx SpecContext) {
			var err error
			tempFile, err = os.CreateTemp(tempDir, "")
			Expect(err).ShouldNot(HaveOccurred())
		})

		It("renames file", func() {
			renamed := filepath.Base(tempFile.Name()) + "2"
			session, err := gexec.Start(
				exec.Command("ci-build/r", tempFile.Name(), renamed),
				GinkgoWriter,
				GinkgoWriter,
			)
			Expect(err).ShouldNot(HaveOccurred())
			Eventually(session).Should(gexec.Exit())

			_, err = os.Stat(tempFile.Name())
			Expect(err).To(MatchError(os.IsNotExist, "It is not existing"))

			_, err = os.Stat(filepath.Join(tempDir, renamed))
			Expect(err).Should(BeNil())
		})

		It("shows version", func() {
			session, err := gexec.Start(
				exec.Command("ci-build/r", "--version"),
				GinkgoWriter,
				GinkgoWriter,
			)
			Expect(err).ShouldNot(HaveOccurred())
			Eventually(session).Should(gexec.Exit())
			Expect(session.Out).Should(gbytes.Say(`(?m)BuildVersion:.*\nBuildDate:`))
			Expect(session.Err).Should(gbytes.Say(""))
		})
	})

	When("binary is used incorrectly", func() {
		It("fails without any argument", func() {
			command := exec.Command("ci-build/r")
			session, err := gexec.Start(command, GinkgoWriter, GinkgoWriter)
			Expect(err).ShouldNot(HaveOccurred())
			Eventually(session).Should(gexec.Exit())
			Expect(session.Out).Should(gbytes.Say("usage: r <path> <new_filename>"))
			Expect(session.Err).Should(gbytes.Say(""))
		})
	})
})

• Ordered container caveat: When defining before scripts, the superior container (which is When in this case) needs to have the Ordered Decorator set. Looking at their example, they assume that resources are reused outside of one It, as some variables are defined in the superior container. But I’m not sure why it can’t just run all before’s in hierarchical order like rspec can do it…
• Despite the previous point, there are before and after scripts for suite, all and each. It just lacks in rspec’s around.

Thoughts