Integration testing with Docker and Maven
This post will use
to provide an integration test environment for Spring Boot applications, running at least JUnit 4.12, Spring Boot 1.4, the Failsafe plugin in the version managed by Spring Boot and the latest docker-maven-plugin.
Gerald Venzl asked for it on twitter especially in the context of integration tests with databases. In case you don’t know, Gerald and Bruno Borges are responsible for the official Oracle database docker images, which I am using at my company respectively for my upcoming talk at DOAG 2016.
Apart from Gerald asking I had several reasons to finally get this topic right. First, after upgrading my jugs site to Spring Boot 1.4 and with it, to Hibernate 5 I ran into issues with the ID generator which behaved differently than before (a million thanks to Vlad Mihalcea for his great input).
eurejug.eu is developed locally on an H2 database and runs on Pivotal Cloud Foundry in production where it uses a PostgreSQL database. So my problem was detected during the existing unit tests and broke the application in a way that wasn’t immediately obvious.
And last but not least: We at ENERKO INFORMATIK are creating mainly database centric application, some of them only 2 tier applications with a lot of SQL logic. That logic is fine for us and our customers, but automated integration testing always gives us a hard time, where I am convinced, that the following setup I developed for the Euregio JUG, will give us a lot of improvements, replacing the PostgreSQL docker image with Oracle ones. So everything you’ll read can be applied to other databases as well.
Configuring integration tests
There’s actually not much to do in a Spring Boot / Maven based application, just add the plugin like I did:
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-failsafe-plugin</artifactId> <executions> <execution> <goals> <goal>integration-test</goal> <goal>verify</goal> </goals> </execution> </executions> </plugin> |
I can safely omit the version number because Spring Boot has this plugin in its managed dependencies. The failsafe plugin automatically recognizes the following patterns in test classes as integration tests:
- “**/IT*.java” – includes all of its subdirectories and all Java filenames that start with “IT”.
- “**/*IT.java” – includes all of its subdirectories and all Java filenames that end with “IT”.
- “**/*ITCase.java” – includes all of its subdirectories and all Java filenames that end with “ITCase”.
The surefire plugin itself excludes them in the current version so that they aren’t run as Unit tests. I didn’t bother to move them into separate folders but that should be easy be doable by the Build Helper plugin.
Configure your containers with docker-maven-plugin
As I said before, the docker-maven-plugin has a superb documentation and is really easy to use. Here it is used to start a docker container based on the official postgresql image before the integration tests run. The integration tests are run by the failsafe plugin so its made sure that the container will be removed afterwards. See the commit for EuregJUG:
<plugin> <groupId>io.fabric8</groupId> <artifactId>docker-maven-plugin</artifactId> <version>0.20.1</version> <executions> <execution> <id>prepare-it-database</id> <phase>pre-integration-test</phase> <goals> <goal>start</goal> </goals> <configuration> <images> <image> <name>postgres:9.5.4</name> <alias>it-database</alias> <run> <ports> <port>it-database.port:5432</port> </ports> <wait> <log>(?s)database system is ready to accept connections.*database system is ready to accept connections</log> <time>20000</time> </wait> </run> </image> </images> </configuration> </execution> <execution> <id>remove-it-database</id> <phase>post-integration-test</phase> <goals> <goal>stop</goal> </goals> </execution> </executions> </plugin> |
Here it runs an existing image under the alias “it-database”. It waits either until the configured message appears in the Docker logs or the time expires (The plugin can also be used to create new images, for example based on the Oracle Docker images which is something I’ll use in my talk and publish afterwards). The container is started before the integration tests and stopped afterwards.
Also take note of the important port mapping: it-database.port and can be used throughout the pom file. If you would use the Oracle Database image, that would 1521 in all probability.
I use the new property as an environment variable for the integration tests by adding the following to the failsafe configuration above:
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-failsafe-plugin</artifactId> <executions> <execution> <goals> <goal>integration-test</goal> <goal>verify</goal> </goals> </execution> </executions> <configuration> <environmentVariables> <it-database.port>${it-database.port}</it-database.port> </environmentVariables> </configuration> </plugin> |
and make use of Spring Boots awesome configuration features, as you can see in this commit, where I add a file named application-it.properties containing the following line among others:
spring.datasource.url = jdbc:postgresql://localhost:${it-database.port}/postgres
recognize the property? This way, I don’t have to hardcode the port somewhere, which would lead into problems when several builds run in parallel (for example on a CI machine). Parallel build wouldn’t be possible if Docker would map the exposed port to a fixed port on the machine running the build.
Also that file configures the data to load through:
spring.datasource.data = classpath:data-it.sql
Per default it would load a file called “data-
Writing the actual tests for a Spring Boot application
So far, that was nothing, wasn’t it? I’m really impressed how much development change the last years. Yes, Maven is still XML based, but I didn’t have to do any “fancy” things or use special scripts or whatever to a sane testing environment up and running.
The actual test I needed looks like this:
import static org.hamcrest.Matchers.is; import org.junit.Assert; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.autoconfigure.orm.jpa.AutoConfigureTestDatabase; import static org.springframework.boot.test.autoconfigure.orm.jpa.AutoConfigureTestDatabase.Replace.NONE; import org.springframework.boot.test.autoconfigure.orm.jpa.DataJpaTest; import org.springframework.test.context.ActiveProfiles; import org.springframework.test.context.junit4.SpringRunner; @RunWith(SpringRunner.class) @DataJpaTest @AutoConfigureTestDatabase(replace = NONE) @ActiveProfiles("it") public class RegistrationRepositoryIT { @Autowired private EventRepository eventRepository; @Autowired private RegistrationRepository registrationRepository; @Test public void idGeneratorsShouldWorkWithPostgreSQLAsExpected() { // data-id.sql creates on registration with id "1" final EventEntity event = this.eventRepository.findOne(1).get(); final RegistrationEntity savedRegistration = this.registrationRepository.save(new RegistrationEntity(event, "foo@bar.baz", "idGeneratorsShouldWorkWithPostgreSQLAsExpected", null, true)); Assert.assertThat(savedRegistration.getId(), is(2)); } } |
Important here are three things:
- Run the test with
@RunWith(SpringRunner.class)and as a@DataJpaTest. I don’t want to fire up the whole application but only JPA and the corresponding repositories. - By default, @DataJpaTest replaces all data sources by an in memory database if such is on the class path, which is exactly what I don’t want here. So I prohibit this with
@AutoConfigureTestDatabase(replace = NONE) - Activate a profile named “it” with
@ActiveProfiles("it")so that Spring Boot takes the application-it.properties into account mentioned earlier.
Summary
I’m really, really happy with the solution, especially regarding the fact that I’m probably the right amount late to the Docker party. While it was relatively easy to install Docker on Linux, you would have to jump through hoops to get it running under OS X and especially on Windows. Lately, Docker natively supports libvirt under OS X and HyperV under Windows 10 pro so my colleagues don’t have a reason not to use it. The pre build images works fine and even the mentioned Oracle images above build without errors on several OS X and Windows machines I tested.
Even if you don’t want to have anything to do with Docker, the docker-maven-plugin is your friend. Nothing needs to be started manually, it just works. The EuregJUG site just build flawless on my Jenkins based CI.
If this post if of use to you, I’d be happy to hear in the comments.
Update: Here’s a snippet to build and run Oracle Database instances from the official Oracle Docker files: pom.xml.
Update 2: I just noticed that this commit on the above mentioned Oracle Database Images makes them somewhat less useful for integration testing: Now the Dockerfile only installs the database software but doesn’t create the database and the container takes ages to start up… I don’t have a solution for that right now and i’m staying with my older images, even if they are bit larger.
Update 3: In some cases a log messages appears twice in a container. The Maven-Docker-Plugin supports regex for log-waits since 0.20.1, which is reflected in pom snippet now. Also regarding Update 2: One just has to do it “right”, see this article.
