info.michael-simons.eu

(*) As of now R, Ruby, Python and JavaScript 😉

GraalVM consists of a Java Compiler, a VM and a JDK based on HotSpot/OpenJDK, implemented in Java. These days it’s all about additional execution modes compared to the standard JVM, mainly ahead-of-time compilation and the creation of native images. Native images are an important piece to create Java based applications that are easier to scale in very elastic ways and large deployments.

However, the GraalVM has a couple of more distinctive features:

• Truffle Language Implementation framework and the GraalVM SDK, to implement additional programming language runtimes
• LLVM Runtime and JavaScript Runtime

The content of this post is based on the Truffle framework and with it, the polyglot aspect of GraalVM.

There have been a couple of great talks about Graal and especially Polyglot. Have a look at Polyglot Applications with GraalVM from @mesirii and JS, Rust, Python and the Holy Graal by @larsr_h.

The GraalVM is polyglot in many different ways: You can embedded a supported guest language in a host language through a so called polyglot context but you can also call access all supported languages from a supported, dynamic language by running the GraalVM version of the interpreter with the --polyglot.

Inside the GraalVM manual you’ll find the polyglot reference as well as the embedding reference. The embedding reference is mostly about running supported languages from Java programs. This won’t be our concern in this post. This is about the polyglot reference. The examples in the manual are designed for first selecting a start language and then a target language. The examples follow the pattern that a host languages executes some simple code in the guest language.

Me friend Michael however nerd snipped me with a different idea: How to access Neo4j from R? Or use the multi database features from Neo4j 4.0 from a language for which we (I work at Neo4j at the drivers team) haven’t yet come up with a driver?

GraalVM polyglot interoperability for the win. You find information about GraalVM’s polyglot interoperability per language inside the manual, for example here for Ruby.

In short: The interpreters for the supported languages all comes with an API to interact with all other supported languages.

That allows us the Neo4j Java Driver in it’s current 4.0.0 mainline to from R, Python, Ruby and for completeness from JavaScript (the 4.0 JavaScript driver for Neo4j is already there) as well.

Michael and I have setup a repository named “neo4j-graalvm-polyglot-examples” that demonstrates this approach.

Setting up Neo4j

These example are about access to Neo4j from various languages. I find the easiest way to have an instance up in running in no time is Docker.

docker run --publish=7474:7474 --publish=7687:7687 -e 'NEO4J_AUTH=neo4j/secret' neo4j:4.0.1

gives you a running instance. After that, you can open http://localhost:7474/browser/?cmd=play&arg=movies and install our Movie graph to have a dataset to work with.

Other options include our Desktop edition which you find among other downloads here: https://neo4j.com/download/

Setting up GraalVM

Michael Hunger uses SDKMan! for downloading and installing GraalVM, I went to the GraalVM Downloads page at GitHub and got the JDK 11 edition of GraalVM 20.0.0.

After downloading and installing, you should have a valid GRAALVM_HOME and a JAVA_HOME pointing to the former:

✗ echo $GRAALVM_HOME 
/Library/Java/JavaVirtualMachines/graalvm-ce-java11-20.0.0/Contents/Home
✗ echo $JAVA_HOME
/Library/Java/JavaVirtualMachines/graalvm-ce-java11-20.0.0/Contents/Home
✗ java -version
openjdk version "11.0.6" 2020-01-14
OpenJDK Runtime Environment GraalVM CE 20.0.0 (build 11.0.6+9-jvmci-20.0-b02)
OpenJDK 64-Bit Server VM GraalVM CE 20.0.0 (build 11.0.6+9-jvmci-20.0-b02, mixed mode, sharing)

GraalVM comes with gu, the GraalVM Component Updater. gu is used to install additional packages. We use it to install R, Ruby and Python packages as well as the native image tool for GraalVM:

$GRAALVM_HOME/bin/gu install R
$GRAALVM_HOME/bin/gu install Ruby     
$GRAALVM_HOME/bin/gu install Python
$GRAALVM_HOME/bin/gu install native-image

The list of installed components should now look likes this:

$GRAALVM_HOME/bin/gu list                
 
ComponentId              Version             Component name      Origin 
--------------------------------------------------------------------------------
graalvm                  20.0.0              GraalVM Core        
R                        20.0.0              FastR               github.com
llvm-toolchain           20.0.0              LLVM.org toolchain  github.com
native-image             20.0.0              Native Image        github.com
python                   20.0.0              Graal.Python        github.com
ruby                     20.0.0              TruffleRuby         github.com

JS respectively Node come by default with GraalVM.

Running the examples

The examples live in neo4j-graalvm-polyglot-examples. Clone this repository via standard Git means. The Neo4j driver lives under the Maven coordinates org.neo4j.driver:neo4j-java-driver. The driver has a single dependency to the reactive stream API. To make the download easier, the repository comes with a Gradle build that works both under Windows and Linux. Get and export the required dependencies via

./gradlew downloadDependencies
export CLASSPATH=lib/neo4j-java-driver-4.0.0.jar:lib/reactive-streams-1.0.2.jar

All examples can now be run like this:

# R
$GRAALVM_HOME/bin/Rscript --jvm --vm.cp=$CLASSPATH neo4j-graalvm-fastr-example.R
 
# Javascript
$GRAALVM_HOME/bin/node --jvm --vm.cp=$CLASSPATH neo4j-graalvm-javascript-example.js
 
# Python
$GRAALVM_HOME/bin/graalpython --jvm --vm.cp=$CLASSPATH neo4j-graalvm-python-example.py
 
# Ruby
$GRAALVM_HOME/bin/truffleruby --jvm --vm.cp=$CLASSPATH neo4j-graalvm-ruby-example.rb

They all execute the following query

MATCH (:Person {name:$name})-[:ACTED_IN]->(m)<-[:ACTED_IN]-(coActor) 
RETURN DISTINCT coActor

The query has one parameter called name and gives you all the actors that acted in the same movie like the request actor.

I take neo4j-graalvm-fastr-example.R as example, but the idea is the same for all the examples above.

First, you have to import the required classes via GraalVM’s Java API:

graphDatabase <- java.type('org.neo4j.driver.GraphDatabase')
authTokens <- java.type('org.neo4j.driver.AuthTokens')
config <- java.type('org.neo4j.driver.Config')

With our static factory method, a database connection is opened:

driver <- graphDatabase$driver('bolt://localhost:7687', authTokens$basic('neo4j', 'secret'), config$defaultConfig())

It looks similar in all the other languages. The Neo4j drivers are session oriented. That means, the driver instance is a long living object you keep around and the session is used for your tasks. The driver takes care of connection pooling.

Executing the above query looks like this in R:

query <- '
    MATCH (:Person {name:$name})-[:ACTED_IN]->(m)<-[:ACTED_IN]-(coActor) 
    RETURN DISTINCT coActor
'
 
session <- driver$session()
# The R list (which behaves like an associative array) is automatically converted to a Java Map 
coActorsRecords <- session$run(query, list(name="Tom Hanks"))$list()

This gives you a list of records that can be processed further.

Recap

The GraalVM is a fascinating piece of software. We are not combining trivial libraries in those examples, but loading a driver that manages a connection pool based on an embedded Netty.

While it is of course preferable to have everything “natively” to your language, a polyglot environment like this gives you the opportunity to use foreign functions without that much effort. It wouldn’t be that hard to wrap our Java driver with idiomatic code for someone that has actual R knowledge to make it feel like first class R citizen.

Another use case is to have access to all features of Neo4j 4.0, for example the multi database feature. This is not yet available in the Python driver or the community driven Ruby driver. Here’s an example on how to use them: Python and Ruby.

The official, native Neo4j drivers that are already fully Neo4j 4.0 are here:

• Neo4j Java Driver
• Neo4j JavaScript Driver
• Neo4j .NET Driver

The Python driver will be released the coming weeks with full 4.0 support.