Integration testing with Docker-Compose, Gradle and Spring Boot

This post has been featured on This Week in Spring – November 28th, 2017.

Lately I have been preparing a small project called simple-meetup that I plan to use for different purposes. You’ll find the repository at GitHub: github.com/michael-simons/simple-meetup. It’s the first project where I used Gradle very intensive and I like my build file a lot.

The first intention of this repository is being an example for an article for OBJEKTSpektrum on testing. The project used JUnit 5 at the beginning, but I switched back to 4 for various reasons, which I explain in a follow up. Thanks upfront to Christian Stein, JUnit 5 committer and contributor, for your interest and feedback on this!

I already wrote about integration testing with Docker and Maven. In that post I used the docker-maven-plugin to fire up supporting services for my tests.

While researching similar options to use with Gradle, I stumbled upon this post by Thomas Kieffer from Codecentric. He describes the docker-compose-role in detail. This rule is a JUnit 4 rule designed to start containers defined in a docker-compose.yml before tests and tearing them down afterwards and also waiting for services to become available before running tests.

This is quite nice, as it doesn’t make your tests dependend on the build tool itself. There is activity to make this available as JUnit 5 extension as well, see #138.

While Thomas demonstrated a generic approach, I’d like to show you how to use this with Spring Boot and replace Spring Boot properties at the right point before the Spring context gets refreshed.

For my meetup project, I need a database and I want to run integration tests against the “real deal”. Here’s my Docker-Compose file:

And this is what the integration tests looks like:

@RunWith(SpringRunner.class)
@ActiveProfiles("it")
@DataJpaTest
@ContextConfiguration(initializers = PortMappingInitializer.class)
public class EventRepositoryIT {
 
    private static DockerComposeRule docker = DockerComposeRule.builder()
        .file("src/integrationTest/resources/docker-compose.yml")
        .waitingForService("it-database", HealthChecks.toHaveAllPortsOpen())
        .waitingForService("it-database", PostgresHealthChecks::canConnectTo)
        .build();
 
    @ClassRule
    public static TestRule exposePortMappings = RuleChain.outerRule(docker)
        .around(new PropagateDockerRule(docker));
 
    // Actual tests omitted
}

Two things to notice: First I didn’t use the DockerComposeRule as a class rule. Instead, I used a chain of JUnit rules. A rule chain works from outer to inner rules. The declaration public static TestRule exposePortMappings = RuleChain.outerRule(docker).around(new PropagateDockerRule(docker)) says: Apply the docker-compose-rule first and then an instance of PropagateDockerRule. Second: I apply ContextConfiguration with an instance of Springs ApplicationContextInitializer, dubbed PortMappingInitializer.

The initializers are called just before the Spring Boot context gets refreshed and autoconfiguration kicks in. What my initializer does is getting the instance of the docker compose rule out of a thread local and reads the name of all services and the mapped ports from it:

The important part is the usage of `TestPropertySourceUtils`. I use it to add inline properties to the environment. Those test properties have higher priority than any other properties except the devtools properties.

The purpose of the PropagateDockerRule is then plain and simple: It stuffs the current instance of the docker compose rule into the thread local:

That way, I can use inlined properties like this: spring.datasource.url = jdbc:postgresql://localhost:${it-database.port}/postgres. Spring Boot and it’s starters use it to point my database agains the instance I just started. The project repo includes additional health checks, but that’s all there is. The test database would also be subject to initialization, wether one uses Flyway, Liquibase or a simple schema or data script.

What’s nice about the setup is that I can fire up the test from my IDE, regardless of the build-tool. While one can usually run single Unit tests from the IDEs out there, its more complicated to run a single failsafe-based integration test with Maven.

| Comments (4) »

20-Nov-17


Past summer: October wrap-up

This year, time flies. Writing this post at about 7pm, it’s already dark. October had some brilliant moments regarding weather and I enjoyed the golden autumn very much. As announced last month, I send the “final” manuscript of Spring Boot Buch to dpunkt.verlag in the last week of September. 2 weeks later, I received three packets full of paper. 500 pages, with a lot of red corrections and annotations. I worked together with my fantastic wife Christina through the paper and fixed the LaTeX sources of the book. We still had to update some examples. There’s still so much change going on in Spring land. Whereas last month I had to update huge amounts of the Actuator chapter, it’s Spring Security this time. See the announcement of 5.0.0.M5. I just had it fixed and added a new starter (spring-boot-starter-security-reactive) and about 2 weeks later, it disappeared. So right now, I just didn’t bother fixing it again, because the finished correction (spelling and grammar that is) of the book was send to my publisher. The agreement we have is that the layout will be done now and the chapters marked as “will probably change” are done last. In the end, you can preorder the book right now: Preorder Spring Boot 2: Moderne Softwareentwicklung mit Spring 5, it will be published in time in January 2018.

My book will include a preface by Jürgen Höller himself. Jürgen and I met at Topconf last week, where he held a closing keynote and spoke about (literally) a life in Open Source. I like the non technical keynote and the inside a lot. Jürgen recently was awarded with the JAX Special Jury Award at JAX London for his work for Spring and I am more than happy to have him in my book:



I was in Darmstadt this month and spoke about Spring Boot myself:

Jörn Hameister wrote an excellent post about my content, thanks a lot! For that talk I prepared a new starter, that might come in handy for some with the release of Spring Boot 2.

The JUG Darmstadt meeting was great, meeting my friends Gerd, Ralf und Niko. Nikos Serverlessbuch is already finished and I’m currently reading the one he gave me as a gift. I’m really happy to be in his preface 🙂

The slides above are my first slides bearing innoQ logos and colors. I was so nervous (actually, I was a wrack) by the end of September. Departing the old company took his toll and being afraid if innoQ and consultancy would work for me and in what setting. I had a hard time thinking straight. Right now I’m with a customer in Aachen and actually not doing Spring. Despite that, I’m learning a lot thanks to a great internal slack and from my innoQ colleague Jerry. Wether it’s technology, domain, you name it. There seems to be an expert for everything. I’m really happy about the development. And what is also great is the fact that discussions, with colleagues and also customer, are really high level with mutual respect. At the moment, I’m happy and I feel finally good again. Looking forward to the coming weeks.

| Comments (1) »

28-Oct-17


Hibernate Search with Spring Data on Pivotal CF revisited

About a year ago I published an article called Running Hibernate Search with Elasticsearch on Pivotal CF. In several projects I use Hibernate Search. If you’re already have Hibernate-JPA-Entities in your project and you need a full text index for some of them, its really easy to add a Lucene based index by just using some simple annotations in addition to your standard JPA annotations. I also have a post on that topic, check Hibernate Search and Spring Boot: Simple yet powerful archiving.

You can now run Hibernate Search against a local disk based Lucene index as well as against the full blown Elasticsearch index. Together with Spring Data JPA repositories it’s a really valuable tool.

Back in September, Sanne announced the general availably of Hibernate Search 5.8, read it here. It now supports an even broader range of Elasticsearch and also introduces the nice feature of using a Simple Query String.

While upgrading the EuregJUG archive, I ran into two little problems. One got fixed nearly immediately after I reported it (see HSEARCH-2883) and the other one was more on my site.

The EuregJUG site runs on Pivotal Cloud Foundry using an Elasticsearch instance from the marketplace. The PaaS gives me the connection URL including user name and password in the form of user:password@host which is perfectly valid. Sadly that isn’t supported by Hibernate Search anymore.

In the process of splitting the environment variable I have I came across Springs EnvironmentPostProcessor interface. This interface can be used to change the environment after all possible property sources have been evaluated. To make this work, you have to declare it in META-INF/spring.factories. You’ll find the relevant parts here (the declaration to make Spring load this instance early) and the actual implementation (not fancy, just splitting a string). I really like the notion of the Environment post processor. I have every property ready to use and can force some of them to be in a format expected for a third party tool. Could be useful in a starter as well.

With that little change I have a nice search system in place, look for example for everything Spring and Mark. This is how simple the snippet from the extended Spring Data repository looks:

Thanks for the very nice work, Sanne, Yoann and Ales!

| Comments (0) »

06-Oct-17


September wrap-up

As the keen reader may have noticed, there hasn’t been an August summary about the work on Spring Boot Buch. There have been several reasons: At the end of July I gave in the manuscript for lecturing at dpunkt.verlag. After that, I took all holiday I had left at my current company and went on family vacation to Hamburg and Norway, which was really great.

End of August, I met with my new colleagues from innoQ in Edinburgh, my first “innoQ weit-weg-event”. That was quite something. 3 days of discussing all kinds of staff in an open space format. I probably was much quieter than my usual self because I was pretty impressed by this knowledge around.

Regarding my book, I had the chance to review the reviews together with Eberhard Wolff at the innoQ event and incorporate some of it into the manuscript. Thanks a lot for that 🙂 In fact, that was a huge part of what I’m doing right now: Reworking some stuff based on the reviews and reworking much more stuff that just got merged in to the master branch of Spring Boot 2.0 and is available in milestone 4, mainly autoconfiguration for Security and a lot of changes to the Actuator endpoints, including support for Micrometer. Madhura Bhave has published a blog about the changes regarding Spring Boot Security 2 and Stéphane Nicoll wrote about the new infrastructure for Actuators.

Spring Boot takes automagic away for the first time: It’s doing less autoconfiguration for Spring Security and even that backs off completely if you add your own customization. I think in this area that is a pretty good idea, but will lead to interesting upgrades.

Actuator endpoints changes a lot, too. Contentwise, facing monitoring tools and stuff, but also internal. That builds the foundation for technology agnostic implementation: Actuators can now be reactive as well, being based on JAX-RS or as one is used too, based on Spring Web MVC.

The metrics support through Micrometer is going to be awesome and will add value for many apps.

All those topics will be covered in my book. Right now I’m polishing the index, updating my examples to the latest milestones. After that, the manuscript gets out to spell checking and stuff. When the result is processed, all examples are hopefully based on a release candidate so that everything can go into layout. The book will be out not much later than January next year. I In any case, I’ll have it with me when I speak about Spring Boot internals at JavaLand next year.

| Comments (0) »

19-Sep-17


JDK 9 EA on macOS

The release date for Java 9 is getting closer and it’s about time to get your hands dirty with the early-access builds. Those are available here.

I’m on a Mac running the latest version of macOS. It’s easy to change $JAVA_HOME in a shell through the various settings of you preferred environment. Sadly, $JAVA_HOME (or any other environmental variable, even it’s from a login shell) isn’t picked up by macOS for quite some time now.

So when I run /usr/libexec/java_home -V it gives me

    9, x86_64:	"Java SE 9"	/Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home
    1.8.0_131, x86_64:	"Java SE 8"	/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home
    1.7.0_79, x86_64:	"Java SE 7"	/Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home

and it picks the latest, which is 9. Several things break with this setting at the moment, for example NetBeans 8.2 doesn’t work.

There’s however a simple solution to the problem. Goto /Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/ (i.e. in a terminal or even in finder) and move Info.plist to something else, like Info.plist.bak. The JDK is still fully functional and you can manually set $JAVA_HOME or configure your IDE to use it, but it won’t be picked up by the libexec-tool anymore.

| Comments (1) »

02-Aug-17