I have started working with a small team on an exciting project at Neo4j. The project is about database connectivity (what else) and we use Testcontainers in our integration tests, asserting the actual network connectivity and eventually the API.
The thing we are creating should of course also work when being compiled as part of an application into a native executable by GraalVM. For a bunch of older projects with a somewhat convoluted test-setup I used to create dedicated, small applications that produce some output. I compiled these apps into a native binary and used a scripted run to eventually assert the output.
For this new project I wanted to take the opportunity to use the Maven plugin for GraalVM Native Image building and it’s test capabilities directly.
The plugin works great and the maintainers are—like the whole GraalVM team—quite quick fixing any issue. We use it already at several occasions to produce native binaries as part of a distribution, but so far not for testing.
I personally find the documentation above linked not sufficient to create a proper setup for tests. Especially the section “Testing support” does not work for me: Neither the latest surefire nor failsafe plugins bring the required dependency org.junit.platform:junit-platform-launcher. This extension contains a test execution listener org.junit.platform.launcher.listeners.UniqueIdTrackingListener that tracks each executed tests and stores it in a file with unique ids. The GraalVM plugin will use that file to discover the tests it needs to run in native mode. If the file is not generated, it will yell at you with
[ERROR] Test configuration file wasn’t found. Make sure that test execution wasn’t skipped.
I can’t share the thing I am actually testing right now, so here’s something similar. The code below uses the Neo4j Java Driver and creates a data access object interacting with the Neo4j database:
package demo; import java.util.Map; import java.util.stream.Collectors; import org.neo4j.driver.Driver; public record Movie(String id, String title) { public static final class Repository { private final Driver driver; public Repository(Driver driver) { this.driver = driver; } public Movie createOrUpdate(String title) { return this.driver.executableQuery("MERGE (n:Movie {title: $title}) RETURN n") .withParameters(Map.of("title", title)) .execute(Collectors.mapping(r -> { var node = r.get("n").asNode(); return new Movie(node.elementId(), node.get("title").asString()); }, Collectors.toList())) .stream() .findFirst() .orElseThrow(); } } } |
and the integration tests looks like this. There are no surprises in it. It is disable without Docker support, uses a life cycle per class so that I can keep a reusable test container around for all tests and a sample test:
package demo; import static org.junit.jupiter.api.Assertions.assertNotNull; import org.junit.jupiter.api.AfterAll; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.TestInstance; import org.neo4j.driver.AuthTokens; import org.neo4j.driver.Driver; import org.neo4j.driver.GraphDatabase; import org.testcontainers.containers.Neo4jContainer; import org.testcontainers.junit.jupiter.Testcontainers; @Testcontainers(disabledWithoutDocker = true) @TestInstance(TestInstance.Lifecycle.PER_CLASS) class RepositoryIT { protected final Neo4jContainer<?> neo4j = new Neo4jContainer<>("neo4j:5.13.0") .waitingFor(Neo4jContainer.WAIT_FOR_BOLT) .withReuse(true); protected Driver driver; @BeforeAll void startNeo4j() { this.neo4j.start(); this.driver = GraphDatabase.driver(this.neo4j.getBoltUrl(), AuthTokens.basic("neo4j", this.neo4j.getAdminPassword())); } @AfterAll void closeDriver() { if (this.driver == null) { return; } this.driver.close(); } @Test void repositoryShouldWork() { var repository = new Movie.Repository(driver); var newMovie = repository.createOrUpdate("Event Horizon"); assertNotNull(newMovie.id()); } } |
Now, let’s walk through the pom.xml. For my dependencies, I usually look if they have a BOM file and import those into dependency management, so that I can rely on other projects organising their releases and transitive dependencies proper. Here I have the latest JUnit and Testcontainers plus Slf4j. From the latter I’ll use only the simple logger later, so that I can see Testcontainers logging:
<dependencyManagement> <dependencies> <dependency> <groupId>org.junit</groupId> <artifactId>junit-bom</artifactId> <version>5.10.0</version> <type>pom</type> <scope>import</scope> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-bom</artifactId> <version>2.0.9</version> <type>pom</type> <scope>import</scope> </dependency> <dependency> <groupId>org.testcontainers</groupId> <artifactId>testcontainers-bom</artifactId> <version>1.19.1</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> |
The relevant test dependencies then look like this. I kinda grouped them together, but please note how I include the JUnit launcher mentioned above explicitly. It is not part of the JUnit core dependencies and at least, was not brought in transitively by either Failsafe or Surefire for me:
<dependencies> <dependency> <groupId>org.junit.jupiter</groupId> <artifactId>junit-jupiter</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.junit.platform</groupId> <artifactId>junit-platform-launcher</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-simple</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.testcontainers</groupId> <artifactId>junit-jupiter</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.testcontainers</groupId> <artifactId>neo4j</artifactId> <scope>test</scope> </dependency> </dependencies> |
“But Michael, didn’t you think about the fact that you won’t usually get the latest Failsafe and Surefire plugins with default Maven?”—Of course I did. This is how I configure Failsafe. Take note here how I set a (Java) system property for the integration tests. While the UniqueIdTrackingListener is on the class path, it is disabled by default and must be enabled with the property below (yes, I did read sources for that. The rest is just the usual dance for setting up integration tests:
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-failsafe-plugin</artifactId> <version>3.2.1</version> <configuration> <systemPropertyVariables> <junit.platform.listeners.uid.tracking.enabled>true</junit.platform.listeners.uid.tracking.enabled> </systemPropertyVariables> </configuration> <executions> <execution> <goals> <goal>integration-test</goal> <goal>verify</goal> </goals> </execution> </executions> </plugin> |
Now onto the GraalVM native-maven-plugin. I usually wrap this into a dedicated profile to be activated with a system property like in the below listing. The documentation says that one must use <extensions>true</extensions> as part of the configuration in order to use the recommended JUnit Platform test listener mode, but that didn’t work for me. I guess it should in theory avoid having to set the above system property.
The next important part in the listing—at least when you want to use Testcontainers—is enabling the GraalVM Reachability Metadata Repository. This repository contains the required configuration shims for quite the number of libraries, including Testcontainers. If you don’t enable it, Testcontainers won’t work in native mode:
<profile> <id>native-image</id> <activation> <property> <name>native</name> </property> </activation> <build> <plugins> <plugin> <groupId>org.graalvm.buildtools</groupId> <artifactId>native-maven-plugin</artifactId> <version>0.9.28</version> <extensions>true</extensions> <configuration> <metadataRepository> <enabled>true</enabled> </metadataRepository> </configuration> <executions> <execution> <id>test-native</id> <goals> <goal>test</goal> </goals> <phase>verify</phase> </execution> </executions> </plugin> </plugins> </build> </profile> |
One cannot praise the people from VMWares Spring and Spring Boot teams enough for bringing their knowledge of so many libraries into that repository.
With that in a place, you can run your integration tests on the JVM and as a native image like this:
mvn -Dnative verify |
You first see the usual dance of integration tests, then the GraalVM compilation
[1/8] Initializing... (5,9s @ 0,25GB)
Java version: 17.0.9+11-LTS, vendor version: Oracle GraalVM 17.0.9+11.1
Graal compiler: optimization level: 2, target machine: armv8-a, PGO: off
C compiler: cc (apple, arm64, 15.0.0)
Garbage collector: Serial GC (max heap size: 80% of RAM)
1 user-specific feature(s)
- org.graalvm.junit.platform.JUnitPlatformFeature
[junit-platform-native] Running in 'test discovery' mode. Note that this is a fallback mode.
[2/8] Performing analysis... [*****] (21,0s @ 1,48GB)
13.384 (87,45%) of 15.304 types reachable
22.808 (62,52%) of 36.483 fields reachable
73.115 (60,68%) of 120.500 methods reachable
4.250 types, 1.370 fields, and 3.463 methods registered for reflection
99 types, 102 fields, and 102 methods registered for JNI access
5 native libraries: -framework CoreServices, -framework Foundation, dl, pthread, z
[3/8] Building universe... (2,6s @ 1,40GB)
[4/8] Parsing methods... [**] (2,6s @ 1,62GB)
[5/8] Inlining methods... [***] (1,4s @ 1,49GB)
[6/8] Compiling methods... [******] (37,1s @ 2,99GB)
[7/8] Layouting methods... [**] (4,0s @ 4,12GB)
[8/8] Creating image... [**] (4,4s @ 1,49GB)
37,40MB (58,34%) for code area: 40.731 compilation units
25,78MB (40,21%) for image heap: 338.915 objects and 80 resources
951,36kB ( 1,45%) for other data
64,11MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 origins of code area: Top 10 object types in image heap:
14,74MB java.base 7,80MB byte[] for code metadata
5,94MB testcontainers-1.19.1.jar 3,21MB byte[] for java.lang.String
3,85MB java.xml 2,46MB java.lang.String
3,79MB svm.jar (Native Image) 2,44MB byte[] for general heap data
1,12MB neo4j-java-driver-5.13.0.jar 2,39MB java.lang.Class
1,08MB netty-buffer-4.1.99.Final.jar 1,50MB byte[] for embedded resources
938,35kB docker-java-transport-zerodep-3.3.3.jar 879,17kB byte[] for reflection metadata
683,00kB netty-transport-4.1.99.Final.jar 627,38kB com.oracle.svm.core.hub.DynamicHubCompanion
655,80kB netty-common-4.1.99.Final.jar 438,19kB java.util.HashMap$Node
490,98kB jna-5.12.1.jar 384,69kB c.o.svm.core.hub.DynamicHub$ReflectionMetadata
3,95MB for 58 more packages 3,40MB for 2755 more object types
------------------------------------------------------------------------------------------------------------------------ |
And shortly after that:
[main] INFO org.testcontainers.DockerClientFactory - Checking the system... [main] INFO org.testcontainers.DockerClientFactory - ✔︎ Docker server version should be at least 1.6.0 [main] INFO tc.neo4j:5.13.0 - Creating container for image: neo4j:5.13.0 [main] INFO tc.neo4j:5.13.0 - Reusing container with ID: 56cc1b02f9b0ebfcc8670f5cdc54b5b3a85a4720e8d810548986369a557482ca and hash: aa81fad313c4f8e37e5b14246fe863c7dbc26db6 [main] INFO tc.neo4j:5.13.0 - Reusing existing container (56cc1b02f9b0ebfcc8670f5cdc54b5b3a85a4720e8d810548986369a557482ca) and not creating a new one [main] INFO tc.neo4j:5.13.0 - Container neo4j:5.13.0 started in PT0.280264S demo.RepositoryIT > repositoryShouldWork() SUCCESSFUL Test run finished after 608 ms [ 2 containers found ] [ 0 containers skipped ] [ 2 containers started ] [ 0 containers aborted ] [ 2 containers successful ] [ 0 containers failed ] [ 1 tests found ] [ 0 tests skipped ] [ 1 tests started ] [ 0 tests aborted ] [ 1 tests successful ] [ 0 tests failed ] |
What I like here is the fact that I don’t have anything special in my test classes, no weird hierarchies nor any additional annotation. In the project we are working on we have all our integration tests as a separate Maven module as we want to make sure we are testing the packaged jar proper (also for the fact that we have integration tests for both the Java class- and module path in separate Maven modules). This setup now gives us the additional advantage that the packaging of our library is subject to be tested under native image, too. This will let you discover issues with missing resources etc in native image, too.
Anyway, the whole project is shared as a gist, it’s only three files anyway.
The image of this post was generated with Dall-E by my friend Michael.
No comments yet
Post a Comment