Tech Deep Dives

Taming Complexity in Software Development with Behavior Driven Development (BDD)

OCTO Cloud Application Architecture Group

May 19, 2020

TL; DR Behavior Driven Development (BDD) is a testing methodology that brings a common language and process to large teams with diverse skillsets. It improves cohesion and facilitates participation in projects by breaking down (communication) silos and reducing misunderstandings. An example is given that shows how BDD is used on a gRPC application written in Go using the Gingko testing framework. The final app is packaged and orchestrated with the help of Docker.

Behavior Driven Development (BDD) is a testing methodology centered on writing specifications using the vocabulary of the associated business domain to describe expected application behavior. For example, “On a blog article submission page, given the user has logged in with author permissions, when an article is submitted, a draft post will be created.”

The blog example illustrates the style in which BDD conveys what is expected of an application. Compared to plain test code, it is easy for all team members to understand. Being human readable, projects implementing BDD have fewer barriers to contribution, especially when involving non-technical contributors.

These test specifications are written preferably before the application is developed. Adding BDD to the software development life cycle aims to drive standardization through the planning, implementation, testing, and validation phases.

The BDD Advantage

The vocabulary of BDD is conducive to the story format, including such words as “Describe”, “Context”, “Given, “When”, and “Expect”. Having a common language aligned to the business needs from a singular perspective—that of the end user—allows for consistency in the software development process. Additionally, developers can use BDD as a model for their code structure and defining application programming interfaces (APIs).

Where we have seen BDD show its full potential is in large teams, composed of diverse skillsets, often with specialization by individuals. As teams grow larger, it is easy for silos to form, leading to an increasingly complex challenge in communicating the direction and completion of a product. With a commonly understood vocabulary for communicating expectations, BDD provides a means of breaking through divisions creating an inclusive development process.

Within VMware Office of the CTO (OCTO), the Cloud Architecture team has been growing quickly and has found BDD to be helpful in software development. The primary programming language used by our group is Go, a language that has rapidly ascended in the last decade. Team members who are new to Go have greatly benefited from BDD due to its intuitive format for describing test specifications and acceptance criteria. For Go, Ginkgo is a popular framework, used by many open source projects, such as Kubernetes, and it allows for BDD tests to be utilized.

Go, BDD, and Ginkgo

However, at first glance some Go developers might be inclined to turn down testing frameworks such as Ginkgo. Having more human-readable tests does not come free. A layer needs to be in place that somewhat obscures the code beneath, making troubleshooting and understanding the behavior of the app during testing slightly more challenging. It is another sort of dialect that must be learned, one that may be difficult to master due to a level of subjectivity.

Frameworks also often go against the grain of Go which is known to be minimalist and pragmatic. Its no-frills philosophy has led developers to choose writing if err != nil repetitively instead of having built-in exception handling. Another perceived downside may be learning the subtleties of a new framework, including its dependencies such as a match library (Gomega) when comparison operators might suffice.

Depending on the project though, BDD may be the best way to attract new contributors and engage non-technical team members. While being free of frameworks appeals to a Go purist, having tools such as Ginkgo appeals to everyone else. BDD does what Go cannot, and that is to align different personas through a common language and work toward a unified goal that is based on user stories.

Containerization With Docker

Another decision made by the OCTO Cloud Architecture team was to make testing more robust and approachable by containerizing application code using Docker and orchestrating tests with Docker Compose.

Docker enables tests to be run on any environment without requiring software dependencies, such as Go, gRPC, Ginkgo, and Gomega which are used by the example later in this article. In addition, containerization ensures consistency between environments, users, and runs, while enabling anyone to run apps and tests without deeper knowledge of the underlying infrastructure. Docker containers are also conducive to being incorporated with a continuous integration and development process (CI/CD).

For this article, gRPC and the provided route guide example was selected as a technology that is commonly used in cloud-native applications. The Cloud Architecture team relies heavily on gRPC for inter-process communication. The Ginkgo BDD test files for the example Go application were containerized and orchestrated with Docker. The remainder of the article will cover how to run the example app, write Ginkgo BDD tests, and how to package and orchestrate it all with Docker.

Coding Objectives

Build and run gRPC route guide example app written in Go
Set up Ginkgo with Gomega matcher and write example test specs
Set up docker files and docker-compose
Write a negative test that expects an error when coordinates are out of range
Modify the app to return the expected error and pass the test

These are the prerequisites for the reader to follow along:

For the route_guide example, clone the repo where it resides:

$ git clone https://github.com/grpc/grpc-go.git

Install Ginkgo with the two commands as described in the official Ginkgo documentation:

$ go get github.com/onsi/ginkgo/ginkgo $ go get github.com/onsi/gomega/...

Set up a test suite bootstrap by running this command in the route_guide/client directory:

$ ginkgo bootstrap

A file called client_suite_test.go is created. This bootstrap file calls test files in a package. To generate a test file, run:

$ ginkgo generate routeGuideClient

A file called routeGuideClient_test.go is created.

Much of the files can be customized to fit the project, such as the package and function names, context descriptions, etc. Since the aim was to do end-to-end testing, and not on the client.go file in route_guide/client, the following organizational changes were made:

created a directory route_guide/bdd_tests
moved both generated files to the new directory

The objective is to end up with a directory tree that looks like the one below. The directory and files added by this blog article are in teal. The docker files will be covered later in this article.

└─route_guide │ ├── README.md ├── docker-compose-test.yml ├── client │ └── client.go ├── bdd_tests │ └── client_suite_test.go │ └── routeGuideClient_test.go │ └── Dockerfile.test ├── mock_routeguide │ ├── rg_mock.go │ └── rg_mock_test.go ├── routeguide │ ├── route_guide.pb.go │ └── route_guide.proto ├── server │ ├── server.go │ └── Dockerfile └── testdata └── route_guide_db.json

To bootstrap the test, initiate clients, and so forth, Ginkgo offers suite-level functions. BeforeSuite() will be used and it will run once, before all the spec tests. In this function we establish a gRPC connection to the gRPC server. AfterSuite() is typically used to tear down the test, e.g. closing the gRPC connection and cancel the context after all the tests complete. The connection, context and client variables are defined as global variables so that all tests can access them. This is what client_suite_test.go looks like with the additions:

// route_guide/bdd_tests/client_suite_test.go
package bdd_tests

import (
    "context"
    "testing"
    "time"

    . "github.com/onsi/ginkgo"
    . "github.com/onsi/gomega"
    "google.golang.org/grpc"

    pb "google.golang.org/grpc/examples/route_guide/routeguide"
)

func TestClient(t *testing.T) {
    RegisterFailHandler(Fail)
    RunSpecs(t, "BDD Test Suite")
}

var (
    clt pb.RouteGuideClient
    ctx context.Context
    cancel context.CancelFunc
    conn *grpc.ClientConn
    err error
)

var _ = BeforeSuite(func() {
    conn, err = grpc.Dial("localhost:10000", grpc.WithInsecure(), grpc.WithBlock())
    Expect(err).NotTo(HaveOccurred())

    ctx, cancel = context.WithTimeout(context.Background(), 10*time.Second)

    clt = pb.NewRouteGuideClient(conn)
})

var _ = AfterSuite(func() {
    cancel()
    conn.Close()
})

You can find the more test specifications on the Github repository at https://github.com/codegold79/grpc-go/blob/master/examples/route_guide/bdd_tests.

As described above, the wording of BDD tests tries to avoid specific technical/software development terms and typically follows the “Given-When-Then” style to improve readability. Dennis Hee in a ThoughtWorks blog post on BDD outlined the basic principles as:

‘Given’ is the precondition(s), state, parameters relevant to this particular scenario. Setting the scene.
‘When’ is a trigger, or a state change, the thing we’re testing
‘Then’ is the expected outcome(s) of the trigger given the context of the preconditions

Note that in Ginkgo Then is referred to as It().

For our example, this is what the user acceptance criteria would look like using BDD terminology:

Describe(“Route Guide Client”)
    Describe(“Get Feature(s)”)
        Context(“given a location”)
            When(“providing a location with a feature”)
                It(“should return a feature name”)
                It(“should not error”)
            When(“providing a location without a feature”)
                It(“should return an empty string”)
                It(“should not error”)
         Context(“given a rectangular area”)
            When(“stream from server has been established”)
                It(“should not error”)
                It(“should return (empty) features inside the area”)

Structuring and wording tests using BDD aligns well with user stories used in agile projects. Without a common vocabulary the development cycle can quickly become detached from its business domain and user expectations, leading to incorrect assumptions, communication overhead or worse, negatively impacting user experience.

Describe() blocks can be nested and used to group and structure the test cases which is especially useful in larger specs. Implementing the outlined structure above, with Ginkgo the first Context() block looks like the following:

// route_guide/bdd_tests/routeGuideClient_test.go
package bdd_tests

import (
    "io"
    "sync"

    . "github.com/onsi/ginkgo"
    . "github.com/onsi/gomega"

    pb "google.golang.org/grpc/examples/route_guide/routeguide"
)

var _ = Describe("Route Guide Client", func() {
    var _ = Describe("Get feature(s)", func() {
        Context("given a location", func() {
            var (
                feature *pb.Feature
                err     error
                point   *pb.Point
            )

            // Describe, Context, and When are functionally equivalent,
            // but provide semantic nuance.
            When("providing a location with a feature", func() {
                // BeforeEach runs before every spec, i.e. It(),
                // within this When() scope.
                BeforeEach(func() {
                    point = &pb.Point{Latitude: 409146138, Longitude: -746188906}
                    feature, err = clt.GetFeature(ctx, point)
                })

                It("should return a feature name", func() {
                    Expect(feature.Name).To(Equal("Berkshire Valley Management Area Trail, Jefferson, NJ, USA"))
                    Expect(feature.Location.Latitude).To(Equal(point.Latitude))
                    Expect(feature.Location.Longitude).To(Equal(point.Longitude))
                })

                It("should not error", func() {
                    Expect(err).NotTo(HaveOccurred())
                })
             })

             When("providing a location without a feature", func() {
                 BeforeEach(func() {
                     point = &pb.Point{Latitude: 0, Longitude: 0}
                     feature, err = clt.GetFeature(ctx, point)
                 })

                 It("should return an empty string", func() {
                     Expect(feature.Name).To(BeEmpty())
                     Expect(feature.Location.Latitude).To(Equal(point.Latitude))
                     Expect(feature.Location.Longitude).To(Equal(point.Longitude))
                 })

                 It("should not error", func() {
                     Expect(err).NotTo(HaveOccurred())
                 })
             })
         })
     })
 })

Writing tests after developing application features is against the spirit of BDD. Thus, the final section will cover writing a negative test (notice there isn’t one yet), then adjust the application to make the test pass. But first, set up the Docker files to ensure the server and tests can run and interact with each other.

# route_guide/server/Dockerfile
FROM golang:1.14 AS builder

# Install gRPC and repo containing route_guide example
WORKDIR /go/src/github.com/grpc/
RUN go get google.golang.org/grpc && \
    git clone https://github.com/grpc/grpc-go.git

# Copy over local changes to server code, build and run.
WORKDIR /go/src/github.com/grpc/grpc-go/examples/route_guide/server
COPY server/server.go .
RUN go build server.go

FROM photon:3.0
WORKDIR /go/src/github.com/grpc/grpc-go/examples/route_guide/server
COPY --from=builder /go/src/github.com/grpc/grpc-go/examples/route_guide/server .

ENTRYPOINT ["./server"]

The server/Dockerfile starts from an official Go image, which will have all that Go needs to build an application. Go get will install gRPC while git clone will clone the grpc repo which contains the route guide example app. Also copied over will be any modifications made on the local development machine, overwriting the original code in server.go.

Once the server binary is compiled with go build, a lightweight base image (Photon OS) is brought in using Docker’s multi-stage build capabilities. This image has only the bare minimum software installed to run the program. By copying over the binary through a multi-stage build process, the final image takes up fewer than 50 MB.

The docker image for the tests will end up taking up over 1 GB as nothing can be (easily) spared from the official Golang base image. The tests run using Go development tools, as well as Ginkgo, Gomega, and gRPC, which are installed with go get. The working directory inside route_guide is set, and all the test files are copied into it:

# route_guide/bdd_tests/Dockerfile.test
FROM golang:1.14

# Install gRPC per https://grpc.io/docs/tutorials/basic/go/
# Install ginkgo and gomega per https://onsi.github.io/ginkgo/
RUN go get google.golang.org/grpc && \
    go get github.com/onsi/ginkgo/ginkgo && \
    go get github.com/onsi/gomega/...

# Copy over tests into GOPATH
WORKDIR /go/src/github.com/grpc/grpc-go/examples/route_guide
COPY . .

WORKDIR /go/src/github.com/grpc/grpc-go/examples/route_guide/bdd_tests
ENTRYPOINT ["ginkgo", "-v"]

The route_guide/docker-compose-test.yml file is placed at the root of the route_guide example in order to set a build context at a level high enough such that route_guide/bdd_tests/Dockerfile.test is able to go outside its own context (the bdd_tests directory), and copy all the files on which the tests depend (e.g. route_guide.pb.go).

# route_guide/docker-compose-test.yml
version: "3.7"
services:
    route-server:
        build:
            context: .
            dockerfile: ./server/Dockerfile
        image: route-server:0.1
    route-client-bdd:
        build:
            context: .
            dockerfile: ./bdd_tests/Dockerfile.test
        image: route-client-bdd:0.1
        network_mode: service:route-server

The images are tagged and stored locally as route-server:0.1 and route-client-bdd:0.1. Since we do not push these images to a container registry and only run them locally (developer desktop, CICD pipeline), these image names can be arbitrary.

From within the route_guide directory, execute docker-compose which will build and run the Docker containers:

$ docker-compose -f docker-compose-test.yml up --abort-on-container-exit --build

Notice two flags were added in the docker-compose up command, abort-on-container-exit and build. When one container stops, abort-on-container-exit will shut down any other container still running. In this example, the server, which otherwise would not have stopped, will stop when the container with the tests complete (either successfully or when a test failure occurs).

The other flag, build, will always reconstruct images that are in the build section of the compose file to make sure they reflect code modifications. Docker will leverage caching so that the time to build an image can be reduced.

Once tests complete and both containers exit, clean up by running this command:

$ docker-compose -f docker-compose-test.yml down

Note that these commands would typically be defined in a Makefile with meaningful targets, e.g. make integration-test.

Now that the testing environment has been fully set up, let’s add a negative test (where an error is expected) that will fail. We will finally make the tests pass by modifying the gRPC server code.

Currently, the route guide app accepts all coordinates outside the range allowed by latitude and longitude for Earth. An empty string will be returned as the feature name if an invalid coordinate is submitted. We add a new test spec that expects an error to be returned for latitudes that are not between [-90E7, 90E7] and longitudes outside of [-180E7, 80E7].

Inside the function passed into Context(“given a location”), add this third When() block. It uses the Gomega matcher for a substring expected in an error message.

When("providing invalid coordinate(s)", func() {
    BeforeEach(func() {
        point = &pb.Point{Latitude: -910000000, Longitude: 810000000}
        feature, err = clt.GetFeature(ctx, point)
    })

    It("should return an invalid coordinate error", func() {
        Expect(err).To(HaveOccurred())
        Expect(err.Error()).To(ContainSubstring("coordinate(s) out of range"))
    })
})

Run the docker-compose up command. As intended, the test suite fails. The output would look something like this:

route-client-bdd_1 | Summarizing 1 Failure:
route-client-bdd_1 |
route-client-bdd_1 | [Fail] Route Guide Client Get feature(s) given a location when providing invalid coordinate(s) [It] should return an invalid coordinate error
route-client-bdd_1 | /go/src/github.com/grpc/grpc-go/examples/route_guide/bdd_tests/rg_test.go:67
route-client-bdd_1 |
route-client-bdd_1 | Ran 7 of 7 Specs in 0.013 seconds
route-client-bdd_1 | FAIL! -- 6 Passed | 1 Failed | 0 Pending | 0 Skipped
route-client-bdd_1 | --- FAIL: TestClient (0.01s)
route-client-bdd_1 | FAIL
route-client-bdd_1 |
route-client-bdd_1 | Ginkgo ran 1 suite in 40.0896564s
route-client-bdd_1 | Test Suite Failed
route_guide_route-client-bdd_1 exited with code 1
Aborting on container exit...
Stopping route_guide_route-server_1 ... done

The server-side code of the app will need to be updated to send the expected error when an invalid coordinate is provided. The error will require the codes and status imports listed at the top of route_guide/server/server.go. These imports enable sending custom error messages and statuses through gRPC.

"google.golang.org/grpc/codes" "google.golang.org/grpc/status"

Add a function to server.go that checks if the coordinates are valid:

func validatePoint(point *pb.Point) error {
    if point.Latitude < -900000000 || point.Latitude > 900000000 {
        return status.Errorf(codes.InvalidArgument, "coordinate(s) out of range")
    }
    if point.Longitude < -1800000000 || point.Longitude > 800000000 {
        return status.Errorf(codes.InvalidArgument,"coordinate(s) out of range")
    }

    return nil
}

Inside the GetFeature() method, before the for loop, call the validatePoint() function and return non-nil errors:

err := validatePoint(point)
if err != nil {
    return nil, err
}

Finally, rerun docker-compose and the tests should now pass.

route-client-bdd_1 | •
route-client-bdd_1 | Ran 7 of 7 Specs in 0.017 seconds
route-client-bdd_1 | SUCCESS! -- 7 Passed | 0 Failed | 0 Pending | 0 Skipped
route-client-bdd_1 | PASS
route-client-bdd_1 |
route-client-bdd_1 | Ginkgo ran 1 suite in 38.829469s
route-client-bdd_1 | Test Suite Passed
route_guide_route-client-bdd_1 exited with code 0
Aborting on container exit...
Stopping route_guide_route-server_1 ... done

It is now up to the reader to continue writing tests for the remaining features of the route guide example.

The examples above show how approaching testing with BDD in Go projects with the help of a BDD testing framework like Ginkgo can remove communication barriers between team members, i.e. the domain experts, service owners, scrum master, managers, quality engineers, operations teams and developers.

BDD enables this by using common and simple English words that put into context user actions and expectations of application behavior. The user stories used in BDD can be put to use from the very beginning, at the planning stages, and followed through development, testing, and validation. With the help of container technologies, such as Docker, code and dependencies can be packaged up, allowing for reproducible builds and ease of testing, especially for non-developers.

Learn more about the OCTO Cloud Architecture

SLOs – The Emerging Universal Language in the Enterprise by Michael Gasch and Emad Benjamin (https://octo.vmware.com/slos-emerging-universal-language-enterprise/)
VMware OCTO Application Platforms Position Paper by Emad Benjamin (https://octo.vmware.com/vmware-octo-application-platforms-position-paper/)
VMworld 2019, Achieving predictable response time for micro service based applications (CODE3458U) (https://www.youtube.com/watch?v=Vp1mzKQtxv4)
VMworld 2019, Multi-Cloud Application Platforms for Infrastructure Engineers (BCA1292BE) (https://videos.vmworld.com/global/2019/videoplayer/29657)
VMworld 2019, Hybrid Cloud Runtime Reference Architecture with the VMware OCTO (OCTO1079BE) (https://videos.vmworld.com/global/2019/videoplayer/28024)