In this article, you will find information on:

• how to write integration tests with testcontainers.org library.

Presentation

Integration testing

Integration testing is the phase in software testing in which individual software modules are combined and tested as a group. It occurs after unit testing.

Introduction to the

FIRST - Principles for Writing Good Tests

All tests (including integration tests) should follow principles defined as FIRST. Acronym FIRST stands for below test features below:

• [F]ast - A test should not take more than a second to finish the execution
• [I]solated/Independent - No order-of-run dependency. They should pass or fail the same way in suite or when run individually. Do not depend on any external resources.
• [R]epeatable - A test method should NOT depend on any data in the environment/instance in which it is running.
• [S]elf-Validating - No manual inspection required to check whether the test has passed or failed.
• [T]horough - Should cover every use case scenario and NOT just aim for 100% coverage

Pyramid of testing

As you can see in the pyramid of testing, the number of tests should be average - more than manual and system tests, more than component and unit tests.

Concept of integration tests with testcontainers.org library

testcontainers.org is a Java library that allows to run docker images and control them from Java code. (I will not cover topic what is Docker, if you need more information read more about it.)

The main concept of the proposal of the integration test is:

• Run your application
• Run external components as real docker containers. Here it is important to understand what I mean by external components. It can be:
  • database storage - for example run real PostgreSQL as docker image,
  • Redis - run real Redis as docker image,
  • RabbitMQ
  • AWS components like S3, Kinesis, DynamoDB and others you can emulate by localstack

Don’t run another microservice as docker image. If you communicate with another microservice via HTTP, mock requests by mockserver run as docker images.

Advantages

• You run tests against real components, for example H2 database doesn’t support Postgres/MySQL specific functionality.
• You can run your tests offline - no Internet connection needed. It is an advantage for people who are traveling or if you have slow Internet connection.
• You can mock AWS services by localstack. It will simplify administrative actions, cut costs and make your build offline.
• You can test cornercases like:
  • simulate timeout from external service,
  • simulate wrong http codes,
• All tests are written by developers in the same commit.

Disadvantages

• Continuous integration (e.g. Jenkins) machine needs to be bigger (build uses more RAM and CPU).
• You need to run containers at least once - it consumes time and resources.

How to run Integration Tests

To run PostgreSQL you can add it to your test:

protected static PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer("postgres:11.2")
        .withUsername("userName")
        .withPassword("password")
        .withDatabaseName("experimentDB");

Then run database migration scripts (eg. Flyway). Even empty test will validate if your migrations are executed properly. As a good practice, you should remember about cleaning the state - delete inserted rows.

Example of DB IntegrationTest

    @BeforeEach
    void setUp() throws Exception {
        name = UUID.randomUUID().toString();
    }
    @AfterEach
    void tearDown() throws Exception {
        accountRepository.findByName(name)
            .ifPresent(account -> accountRepository.delete(account));
    }
    @Test
    void shouldFindByName() throws Exception {
        Account account = Account.builder()
            .name(name).additionalInfo("additionalInfo").build();
        accountRepository.save(account);
    }
    Optional<Account> actual = accountRepository.findByName(name);
        assertThat(actual.get()).isEqualTo(account);
    }

Testing queues

To run RabbitMQ add it to the test:

    private static GenericContainer rabbitMqContainer =
        new GenericContainer("rabbitmq:3.7.7").withExposedPorts(5672);

Testing queues is more tricky, because it is asynchronous. You don’t know how much time you need to consume a message after putting it to the queue.

• If your production code receives the message, in the test put it in the queue. Then wait for consumption of the message and validate if changes made by consumption are what you expect. You can use Awaitility library to validate results of the test.
• If your production code sends a message, create a testConsumer in test scope and validate if you receive proper messages.

Always clean the queue after the test. Every sent message should be consumed. If you don’t do it, your build can became unpredictable.

Testing external HTTP calls REST / SOAP

To simulate interaction via HTTP use mockserver library (as docker image).

Testing external http calls - example:

mockServerContainer.getClient().reset(); //reset the mock server container
HttpRequest request = request("/api/entity/name.1").withMethod("GET");
    getMockServerContainer().getClient()
    .when(request)
    .respond(response()
    .withHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON.toString())
    .withStatusCode(200)
    .withDelay(TimeUnit.MILLISECONDS, timeoutInMs + DELTA_IN_MS)
    .withBody(readFromResources("additionalInfo1.json")));

The big advantage of this way of testing is that you can verify that the call happened:

    getMockServerContainer().getClient().verify(request);

AWS testing with localstack

You can mock interaction with AWS with localstack. Right now localstack supports:

• API Gateway
• Kinesis
• DynamoDB
• DynamoDB Streams
• Elasticsearch
• S3
• Firehose
• Lambda
• SNS
• SQS
• Redshift
• SES
• Route53
• CloudFormation
• CloudWatch
• SSM
• SecretsManager

How to start working with testcointainers library

One way of creating tests is extending abstract test class like the example below. It contains static references to docker containers. In static block, we start all images.

@ExtendWith(SpringExtension.class)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ContextConfiguration(initializers = AbstractIntegrationTest.Initializer.class)
public abstract class AbstractIntegrationTest {
    protected static PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer("postgres:10.4")
        .withUsername("userName")
        .withPassword("password")
        .withDatabaseName("experimentDB");
    protected static MockServerContainer mockServerContainer = new MockServerContainer("5.4.1");
    protected static GenericContainer rabbitMqContainer = new GenericContainer("rabbitmq:3.7.7")     
        .withExposedPorts(5672);
        
    static {
        postgreSQLContainer.start();
        rabbitMqContainer.start();
        mockServerContainer.start();
    }

Closing resources

You don’t need to release any resources, there is: Runtime.getRuntime().addShutdownHook() that closes all docker images. There is a ryuk docker image that does it.

Eureka

For service discovery I am using Eureka. At the beginning I was running Eureka as a docker image in my integration tests. To save resources on CI I decided to switch it off:

eureka:
  client:
    enabled: false

Then I mocked the addresses of external services, by setting system properties:

new ImmutableMap.Builder<String, String>()
   .put("commonsscriptsync.ribbon.listOfServers", format("localhost:%d", mockserverPort))
   .build()
   .forEach(System::setProperty);

Cloud Auth

My microservices are using Cloud Auth for authorization. To save resources on CI I decided to switch it off. I mocked the Cloud Auth by MockServer - hardcoded JWT token - valid for everything and forever.

requestAuth = request()
                .withPath("/oauth/token")
                .withMethod("POST");
getMockServerContainerClient()
                .when(requestAuth)
                .respond(response()
                        .withHeader("Content-Type", "application/json;charset=utf-8")
                        .withBody(readFromResources("integration/microservices/cloud_auth_response.json")));

Disadvantages of testcontainer library

TestContainers support docker_compose file, but only version 2.0; while the docker_compose current version is 3.6.

Alternative solutions

I liked TestContainers library. Personally I find http://testcompose.com library more interesting. The main advantage is that it simply runs docker_compose file and reduces boilerplate code.

Code example

You can find examples of usages in my github project: https://github.com/marekhudyma/testcontainers/