All of these integration points require different layers of testing. Unit tests increase how fast you write code because you can mock all of your dependencies, set the expectation for your function and iterate until you get the desired transformation. But, we need more. We need to make sure that the integration with Redis, MongoDB or a microservice works as expected, not just that the mock works as we wrote it. Both are important but the difference is huge.
In this article, I will show you how to use testcontainer to write integration tests in Go with very low overhead. So, I am not telling you to stop writing unit tests, just to be clear!
Back in the day, when I was interested in becoming a Java developer, I tried to write an integration between Zipkin, a popular open source tracer, and InfluxDB. I ultimately failed because I am not a Java developer, but I did understand how they wrote integration tests, and I became fascinated.
Getting Started: testcontainers-java
Zipkin provides a UI and an API to store and manipulate traces, it supports Cassandra, in-memory, ElasticSearch, MySQL and many more platforms as storage. In order to validate that all the storage systems work, they use a library called testcontainers-java that is a wrapper around the docker-api designed to be “test-friendly.”Here is the Quick Startexample:public class RedisBackedCacheIntTestStep0 {
private RedisBackedCache underTest;
@Before
public void setUp() {
// Assume that we have Redis running locally?
underTest = new RedisBackedCache("localhost", 6379);
}
@Test
public void testSimplePutAndGet() {
underTest.put("test", "example");
String retrieved = underTest.get("test");
assertEquals("example", retrieved);
}
}
setUp you can create a container (redis in this case) and expose a port. From here, you can interact with a live redis instance. Everytime you start a new container, there is a “sidecar” called ryukthat keeps your Docker environment clean by removing containers, volumes and networks after a certain amount of time. You can also remove them from inside the test.The below example comes from Zipkin. They are testing the ElasticSearch integration and as the example shows, you can programmatically configure your dependencies from inside the test case.
public class ElasticsearchStorageRule extends ExternalResource {
static final Logger LOGGER = LoggerFactory.getLogger(ElasticsearchStorageRule.class);
static final int ELASTICSEARCH_PORT = 9200; final String image; final String index;
GenericContainer container;
Closer closer = Closer.create();
public ElasticsearchStorageRule(String image, String index) {
this.image = image;
this.index = index;
}
@Override
protected void before() {
try {
LOGGER.info("Starting docker image " + image);
container =
new GenericContainer(image)
.withExposedPorts(ELASTICSEARCH_PORT)
.waitingFor(new HttpWaitStrategy().forPath("/"));
container.start();
if (Boolean.valueOf(System.getenv("ES_DEBUG"))) {
container.followOutput(new Slf4jLogConsumer(LoggerFactory.getLogger(image)));
}
System.out.println("Starting docker image " + image);
} catch (RuntimeException e) {
LOGGER.warn("Couldn't start docker image " + image + ": " + e.getMessage(), e);
}
docker-compose to spin up your integration tests environment. By spinning it up from inside the test itself, you have a lot more control
over the orchestration and provisioning, and the test is more stable. You can even check when a container is ready before you start a test. Since I am not a Java developer, I ported the library (we are still working on all the features) in Golang and now it’s in the main testcontainers/testcontainers-go organization.
func TestNginxLatestReturn(t *testing.T) {
ctx := context.Background()
req := testcontainers.ContainerRequest{
Image: "nginx",
ExposedPorts: []string{"80/tcp"},
}
nginxC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: req,
Started: true,
})
if err != nil {
t.Error(err)
}
defer nginxC.Terminate(ctx)
ip, err := nginxC.Host(ctx)
if err != nil {
t.Error(err)
}
port, err := nginxC.MappedPort(ctx, "80")
if err != nil {
t.Error(err)
}
resp, err := http.Get(fmt.Sprintf("http://%s:%s", ip, port.Port()))
if resp.StatusCode != http.StatusOK {
t.Errorf("Expected status code %d. Got %d.", http.StatusOK, resp.StatusCode)
}
}
Creating the Test
This is what it looks like:ctx := context.Background()
req := testcontainers.ContainerRequest{
Image: "nginx",
ExposedPorts: []string{"80/tcp"},
}
nginxC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: req,
Started: true,
})
if err != nil {
t.Error(err)
}
defer nginxC.Terminate(ctx)defer nginxC.Terminate(ctx) command, you are cleaning up the container when the test is over. Remember ryuk? it is not a mandatory command, but testcontainers-go uses it to remove the containers
at some point. Modules
The Java library has a feature called modules where you get pre-canned containers such as databases (mysql, postgress, cassandra, etc.) or applications like nginx.The go version is working on something similar but it is still an open pr.If you’d like to build a microservice your application relies on from the upstream video, this is a great feature. Or if you would like to test how your application behaves from inside a container (probably more similar to where it will run in prod). This is how it works in Java:
@Rule
public GenericContainer dslContainer = new GenericContainer(
new ImageFromDockerfile()
.withFileFromString("folder/someFile.txt", "hello")
.withFileFromClasspath("test.txt", "mappable-resource/test-resource.txt")
.withFileFromClasspath("Dockerfile", "mappable-dockerfile/Dockerfile"))What I’m working on now
Something that I am currently working on is a newcanned container that uses kind to spin up Kubernetes clusters inside a container. If your applications use
the Kubernetes API, you can test it in integration: ctx := context.Background()
k := &KubeKindContainer{}
err := k.Start(ctx)
if err != nil {
t.Fatal(err.Error())
}
defer k.Terminate(ctx)
clientset, err := k.GetClientset()
if err != nil {
t.Fatal(err.Error())
}
ns, err := clientset.CoreV1().Namespaces().Get("default", metav1.GetOptions{})
if err != nil {
t.Fatal(err.Error())
}
if ns.GetName() != "default" {
t.Fatalf("Expected default namespace got %s", ns.GetName())Calling All Coders
The Java version for testcontainers is the first one developed, it has a lot of features not ported to the Go version or to other libraries as well such as JavaScript, Rust, .Net. My suggestion is to try the one written in your language and to contribute to it.
In Go we don’t have a way to programmatically build images. I am thinking to embed
buildkit or img in order to get a damonless builder
that doesn’t depend on Docker. The great part about working with the Go version is that all the container related libraries are already in Go, so you can do a very good work of integration with them. This is a great chance to become part of this community! If you are passionate about testing framework join us and send your pull requests, or come to hang out on Slack.
