info.michael-simons.eu

Weekly digest 2022-46

Last week, I received quite a lot of really positive feedback for my first attempt at writing a digest. Thank you for that Kevin and Frederik, super kind of you! Even Payara picked it up.

One thing that I have changed this week is my iPhone home screen from which I removed the Twitter app (which I had on the first page). Instead, I put Music back into it. Also, trying to be more conscious about my time spent, I added the Apple time tracker:

Let’s see how that goes.

I started my weekend after traveling last week with… Not cycling but with this awesome video by Mary Spender:

If you didn’t watch it, it’s actually not about todays music not sucking, but against nostalgia holding things back. I liked it a lot, good stuff and actually pretty positive.

I finished reading Verschwörungsmythen by Michael Blume:

Very good read. I never thought about Platons allegory of the cave as something not that positive, but when you have a closer look, it can indeed be connected to conspiracy and the “charismatic” leaders leading the poor people out of the cave. And on the other hand, the red pill is indeed the entrance to the rabbit hole, but not in a good way.

While watching it I thought: “I should post this.” Why, actually? I am not a content / link creation machine and I just can refuse. Let thinks sink in a bit and see if they are worth sharing a week later anyway.

Before we jump a bit into Java: Of course I did cycle on the weekend and it’s madness. I was riding roughly 200km on both days in short/short. In freaking November. The map on the right: Yes, the track reassembles the temperature of one of those rides. Still, those rides give me as much serenity as the guy on the mural seems to have:

A bit related to that, my friend Markus wrote an excellent article about changing his habits at LinkedIn: Dieting vs. Nutrition vs. Rethinking my lifestyle. I’m very impressed by his achievements. I vividly remember our Devoxx 2019 conversation about that topic and role models and such: Chapeau, Markus. Well done. Wrt to nutrition: I’m still on the hard, fast and wrong side of things (apart alcohol) but that works well enough for me atm.

This week was Nodes 2022 and I worked liked crazy to get the documentation of Neo4j-Migrations into our labs pages: Neo4j Migrations Docs / Introduction. Together with Guillaume and Adam we used the existing AsciiDoctor documentation and are now building it with Antora. See the related commit. Antora makes it necessary to rethink a bunch of things, but I am totally in love with the fact that “my” build now notifies our docs repository and that in turn pulls my docs and integrates it into a bunch of other modules. Sweet stuff.

As preparation of his Nodes2022 talk, Gerrit put up a video about Testcontainers and integration testing:

I also did a presentation yesterday day and here are my slides about Neo4j-Migrations or what I call it, the lean way of refactoring Neo4j database content:

I am unsure how good my presentation was in the end: I realised with last weeks Øredev and yesterdays Nodes2022 that I am much better offline than on video. At least, this is how I perceive it.

Maciej was on fire this month with excellent posts about getting Tailwind CSS into Spring Boot proper (at some point, I will revisit my biking application and remove all that SPA Angular stuff and go back to SSR with Thymeleaf for my own sanity) and this is great stuff. He has a bunch of more things that I am using regularly, too, i.e. Activating Maven Profile by Operating System.

From Maciej and my colleague Gerrit comes 🚀 Introducing YOLO – because life is too short for running tests btw. Just click if you’re up for a laugh.

I expected this a bit later this year, but here it is: Spring Framework 6:

As a major revision of the core framework, Spring Framework 6.0 comes with a Java 17+ baseline and a move to Jakarta EE 9+ (in the jakarta namespace), with a focus on the recently released Jakarta EE 10 APIs such as Servlet 6.0 and JPA 3.1. […] Don’t be stuck on Java EE 8, make the leap to the jakarta namespace, ideally straight to the Jakarta EE 10 level! The upcoming Spring Boot 3.0.0 release includes corresponding managed dependencies for you.

IMHO this is quite big for the ecosystem. I cooperated with Heise Online a couple of weeks ago and so we did have a comment ready (in German): Spring Framework 6 verarbeitet Native Images und baut auf Jakarta EE 9 oder 10.

I did already share this article on Twitter, but it’s super interesting still. It’s about databases and query planning. Funny enough, the week before I hang out with Andres Taylor, one of the minds behind it. Andres is one of the original creators of Neo4j Cypher and was also involved in the Cypher-DSL before I took this over.

While Twitter seems indeed to be burning, I will dive into a bit of nostalgia as today, November 18th, a rerelease of Queens 1989 album “The Miracle” will happen, including a bunch of unreleased Queen demos. Looking very much forward to that: Was it all worth it?

Filed in English posts | Comments (0) »

Weekly digest 2022-45

Tja, it had to happen at some point: Twitter is dying, I need a new old outlet. I have now this

but also this

I am not really happy about the situation and am reconsidering what to do social network wise. I never really missed my Facebook account that I deleted (at least I hope it’s deleted) more than 2 years ago. Might as well that I go a bit more silent and reluctant on those tools in total. I wish however to stay connected and do something reassembling a heartbeat. And this is my first attempt at it. Anyway, let’s go:

My week startet by participating in a 16,5km run:

I did this for the first time in 2019, then in 2021 again now this. My times went down from 1h:22min to 1h:17min and this year, 1h:15min for a light trail run with about 260 meters of altitude gain. I’m super stocked about that. Read more about the race here.

The working week I travelled to Malmö for the second time this year. I work for Neo4j (and we just released Neo4j 5, you might want to check out the release) and our engineering headquarters are in Malmö. However, I was invited to speak at Øredev there, which made me super happy and proud.

I spoke about GraalVM, Quarkus and how our driver works with it. You can see the slides here and there will be videos I heard:

Also on the conference was my friend Michael Hunger. We spent some really good quality time and also met Sebastian Daschner:

I also got some work done! Shortly before Twitter started going down in flames, I wrote one of my most successful tweets:

And in that light, I do now require Java 17 for neo4j-migrations. Java 17 provides better performance during runtime, but it is also so much nicer with text blocks, records and other smaller and bigger things. You might want have a look at this commit. Some of the big line savings come from the fact that I had a multi-release build that used some Java11/17 if available though.

I am now sitting at the Copenhagen airport after traveling the Øresund bridge by train:

More bridges, less walls I’d say. Next week, Neo4j will host Nodes2022 for free. Use this link to register and sign in if you’re interested technical talks about graph and ecosystem. Until then and please all, stay sane and healthy.

Filed in English posts | Comments (0) »

Use Oracle Cloud Infrastructure compute instances as custom GitHub runners

I have been publishing linux-aarch64 of Neo4j-Migrations for a while now. I am unsure if there are many people using it, but one scenario is other CI/CD based on Linux ARM infrastructure in which those binaries might come in handy. Those binaries are build with GraalVM using the official setup-graalvm action from Oracle Labs.

For the above project I use the ubiquitous GitHub actions. GitHub actions supports Windows, Linux and macOS operating systems, but only on x86_64 platforms (see https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources). So far I have not seen any indication that this will change in the near future.

For Linux on Arm Oracle Cloud Infrastructure (or in short OCI) offers a solution. The “Always-Free”-Cloud-Services gives you up to 4 instances of ARM Ampere A1 with 3,000 OCPU hours and 18,000 GB hours per month at the time of writing and you can sign up here. The sign up step are typical enterprise Oracle stuff, but alas, it’s free.

I’m not gonna screenshot all the steps, as I am pretty sure they have or will change over time, but this is the compute instance I set up for my purposes:

Make sure you don’t use the Oracle Linux image but something easier to deal with, like Ubuntu Linux or CentOS. I wasn’t able to setup the GitHub runner on the Oracle ones. It is currently hidden under “platform images” when you create new instance:

During creation you can either upload your own SSH public key or generate a new pair. If you do the latter, make sure to download it and store it safely, otherwise you can’t connect to the machine.

I basically followed the process and waited until the machine is ready.

Setting up the runner is dead simple and you can follow the official GitHub instructions: Adding self-hosted runners. I did this from the repository on, not from the org. I tagged the runner with “Linux” and “ARM64” like that:

GraalVM native image needs gcc and build essentials, which is the reason I needed to install them on the runner too. This is basically everything that was necessary in my case:

sudo apt-get install gcc libz-dev build-essential
mkdir actions-runner && cd actions-runner
curl -o actions-runner-linux-arm64-2.296.1.tar.gz -L https://github.com/actions/runner/releases/download/v2.296.1/actions-runner-linux-arm64-2.296.1.tar.gz
echo "ce11e5d9c1cc299bd8a197b7807f6146fbe7494abb19765fb04664afbdb3755e  actions-runner-linux-arm64-2.296.1.tar.gz" | shasum -a 256 -c
tar xzf ./actions-runner-linux-arm64-2.296.1.tar.gz
./config.sh --url https://github.com/your-org/your-repo --token your-token
./run.sh
sudo ./svc.sh install
sudo ./svc.sh start
sudo ./svc.sh status

Please take note that the above runner version might be outdated, you’ll find fresh instructions when setting this up from your repository or organization.

Now, how to use this? Here’s my release script: release.yml; this is the relevant part:

create_native_arm_binaries:
  name: 'Build with Graal on ${{ matrix.os }} ARM64'
  strategy:
    fail-fast: true
    matrix:
      os: [ linux ]
  runs-on:
    - self-hosted
    - ${{ matrix.os }}
    - ARM64

Key being self-hosted in the runs-on block and the remaining, corresponding tags.

A similar approach would be feasible for macOS M1 binaries, but the only option to use them in the cloud I found was AWS with some price tag. It may or may not be worth it for your use case.

Thanks to Fabio Niephaus who brought this possibility to my attention for the first time way back in April this year.

Feature image ARMCortexA57A53.jpg.

Filed in English posts, Java | Comments (1) »

Review: DevOps Tools for Java Developers

Note: Both JFrog and O’Reilly sent me a paper copy of DevOps Tools for Java Developers for review (or my reading pleasure, or hopefully both). The copies came with no strings attached and this article is my honest opinion.

The book is written by Ixchel Ruiz, Melissa McKay, Stephen Chin and Baruch Sadogursky. 3 of them I met personally and all of them come very much from the developer side of things and are known people in the Java world. All of them work at JFrog these days.

The latter is an interesting fact: JFrog is a public company that positions itself these days as the one stop solution for software supply chain management with the punchline “manage your binaries from developer to production”. A (long) while back I got to know JFrog primary for it’s (Maven) compatible software repository solution Artifactory which has evolved into a central solution to manage binaries, container-images and many more. Together with with build pipelines, distribution and a bunch of other things they target the complete requirements of developers and operations from build to production.

On that premise it only make sense that JFrog supports its people to publish about tooling for developers and it’s a great approach reading such a book written by developers.

The introductory chapter: DevOps for (or Possibly Against) Developers

The part really got me quickly. Of course Baruch refers to both the Phoenix Project Book by Gene Kin and others and The DevOps Handbook from the same author: Both are written coming from the operation side of things, not from the developers. So the question is implied: Is DevOps something coming from operation agains developers? Spoilers: It is not.

I really love how that chapter makes it clear that DevOps is very much not engineering per se, but is about collaborations and a shared endeavors. There are no DevOps engineers but site reliability, production, infrastructure, QA and some more disciplines of engineering and together the build an amalgam of development and operations that eventually fosters better quality, produces savings, deploys features faster and strengthens security.

Funny enough in these odd times we are living in: Baruch also addresses the fact that many companies – including modern software companies and not only companies of the past century – are cutting costs at all places. They can do so by layoffs, salary and benefits reductions or by doing better with the same resources, without grinding them down. This is also what DevOps is: Working together.

Contents

The table of contents reads like a proper build chain really. Taken verbatim from here you’ll find:

• (One introductory chapter I covered explicitly)
• The System of Truth
• An Introduction to Containers
• Dissecting the Monolith
• Continuous Integration
• Package Management
• Securing Your Binaries
• Deploying for Developers
• Mobile Workflows
• Continuous Deployment Patterns and Antipatterns

I like that order a lot! It starts with source control and I think it makes totally sense todo so: There are new people to the field everyday and we just cannot assume that everyone has an idea what that is and in addition, I would bet personal money on it that in way to many enterprise organization source control manage is still neglected, even in 2022. I skipped over most the Git specific things, but the history and intend of these topic are written well by Stephen.

Containers and its terminology is ubiquitous but seldom distinctly explained. That chapter by Melissa covers not only the what, but also the way and then the necessary best practices. It’s written in a refreshingly different way than the usual “you must / should do this” approach, either in written words or especially in the style of conference driven development. Excellent content.

What I like the most is how she explained that once secrets and stuff made it into an image, there’s no way to get them out again due to the layered nature and layered filesystem of images. Therefor I am gonna repeat this here: Be careful when crafting your first container image, don’t delay proper secrets management to a later stage.

That monolith chapter by Ixchel: Lucky me, in my day job, I am a library maintainer and can enjoy the luxury of watching the space from the outside. For better or worse. I was consulting in a previous life and have seen the heights of micro service architecture. These days, the pendulum seems to go into the different direction again. I read the anti patterns section closely and I violently agree. It’s good to start with them, especially if you hand this book to a junior person. I’m not so much in agreement with Spring Boot, Quarkus, Micronaut and Helidon being dedicated micro services frameworks though. They can be used in such a fashion, but you can also build (good or bad) monoliths with it. I recommend having an intensive look at the Moduliths effort by Herrn Drotbohm. There’s middle ground (of course, with different tradeoffs).

Can we assume that Continuous Integration (and later continuous deployment) is a given? I am not sure… Hearing how many manual steps it took in companies to react to log4shell back at the end of 2021 I have my doubts. Anyway, I really dig the reference to eXtreme Programming (XP) practices in the introduction of that chapter: “Code integration should be regular and rarely complicated”. If it is, you are already screwed. It goes on further stressing the importance of build and test failures and how they should be easy to investigate and fix. I would like to add that the day you opt to live with one red or one flaky test, is the second opportunity to shoot yourself. And with little surprise, I agree with the scripted builds, too. Nice UIs will only help you so much… And not everyone is equally fluid or able to navigate a UI and it’s pattern.

I did expect a bit more about central solutions in Package Management, but that chapter is a pretty solid introduction to build tools, especially how Maven handles package resolution. It is an important topic, though. Interesting note here is the rather short section about Docker: Here package management is complicated: Docker tags are fluid and prone to change… How to achieve reproducible builds with that? There seem to be no perfect answer.

Securing Your Binaries If you are familiar with any of the previously chapters and think the book isn’t for you, it might still be: Supply chain attacks and everything related to it are on the rise. This chapter by Stephen written together with my long time good acquaintance Sven is an essential read. It starts with the SolarWinds case and takes it from there, covering static and dynamic security analysis, the different roles of different people and many more things and even interactive application security testing and self-protection.

In addition, the chapter includes a proper introduction of the CVSS (Common Vulnerability Scoring System), which comes in quite handy. The section about the fact that vulnerabilities can be combined into different and new attack vectors cannot be underestimated.

The chapter ends with Quality Management and using the Quality Gate method as barriers as well as point of actions for risk assessment, countermeasures and risk tracking. I was a bit irritated for various reasons that “Clean Code” by Robert Martin is mentioned here, but its about making clear that “clean” code (whatever that is anyway) does not cover secure code. I do however disagree that only “clean implementations” opens up the full potential for security measures.

I’m very pleased to see Ana-Maria Mihalceanu in this book, too. She wrote Deploying for Developers. This chapter contains the culmination of several other chapters in the introduction of Kubernetes and how you can programmatically deploy from your build descriptor your application into a cluster, packaged as a OCI compatible container image. Monitoring, Tracing, Logging etc are all covered.

Maybe it’s my restricted experience in that special area, but I miss the mention of one-stop-solutions like Heroku or Pivotal Cloud Foundry of old where many of the things I would need to do for / in a cluster are done for me. Or maybe something like an assessment analogue to “make or buy”. Different tradeoffs, I know, but till to this day, not all applications are Netflix scale (neither in number of clients nor data produced. The latter being relevant in the context of the first chapter that speaks about the explosion of data created in the last years).

I guess a book covering the whole value chain of things these days isn’t complete without covering Mobile Workflows. I did only briefly skim this, though. A proper collaborative effort for being able to deploy new versions continuously is very important in this area: You want to grow and retain your user base with new features, keeping apps secure but also relevant to the algorithm of the app stores. It’s fun to read about device farms in this chapter… A category of problems I have never ran into luckily I guess.

The final chapter Continuous Deployment Patterns and Antipatterns is a logical conclusion of the book. It reemphasis the need for being able to continuously deploy and deliver with both the changed customer aspects and the need to act on security breaches and improving the timeline from identification to solution. The numbers stated about the Equifax security breach in 2017 are staggering (the costs go into the billions).

The case study about the iOS AppStore and how Apple change updating things (in many ways for the better) hits home with the version issue: It’s true that for many applications I don’t even know what version I have… It’s latest. At work there’s an ongoing discussion about that topic but with a database, a tool used as a foundational block for other software, it’s kinda hard to completely omit that information as long as you’ll never have breaking changes.

While all the case studies here are somewhat entertaining they aren’t meant to be, I guess. It can and will eventually happen to most of us.

Verdict

I enjoyed reading this book a lot! It gives – juniors and seniors a like – an exhaustive overview about end-to-end developing applications with the needs of 2022 and beyond in mind. And with developing I mean of course everything that is subsumed under the DevOps term: From actual coding, testing, packaging, deploying and observing with both security and availability and performance metrics in mind.

If you are in a hurry I would recommend at least reading the chapters “DevOps for (or Possibly Against) Developers”, “Securing Your Binaries” and the “Continuous Deployment Patterns and Antipatterns”.

Filed in English posts | Comments (4) »

How to become an Open Source committer?

A couple of days ago I was asked above questions: “How to become an Open Source committer?” I think the answer might be interesting to other as well, so I am sharing it here as well.

At some point I wrote a GitHub README.md, which you can read over there: github.com/michael-simons. I think it gives a pretty good idea what I am up to.

While that I probably would fail all of the FAANG assessment tests for getting a job and I am not the person to look for implementing low level optimizations and stuff like that, I think that I am a decently good generalists with deep knowledge in a couple of important things, such as databases and the bigger (server side) frameworks, but also on a language level. My aim is to keep an overview on how things work together and go rather deep into the topic that I am working on in a project.

Anyway, I created a GitHub account back in 2010, my first contribution was a pull request to a jQuery add-on I like but that was missing a piece I wanted. I needed that for my foto project Dailyfratze.de. At that time I was working for nearly 8 years already at a small German company in Aachen, mostly doing Oracle Database based stuff… At the frontend with Oracle Forms Client/Server: Not exactly the stuff that is popular on GitHub. However, I was often blogging about some work related stuff right here, for example this here bulk operations in Oracle, an article that still has hits, or integrating Hibernate of old with Oracle spatial. Many of these old posts have been accompanied by issues in the trackers of the respective projects.

And yes, I do absolutely think that good issue reports are valuable and Open Source contribution on their own rights, very much like contributions to documentation and heck, just fixing typos. So from my perspective, a proper first step can be reporting things. If you work in a company that uses Open Source projects and you ran into issues take the time to create a reproducer. Don’t just go to a project and yell “this doesn’t work”, but pay back by something that is runnable and shows the error. Most maintainers I know are happy to work with that. If you employer doesn’t permit this, I would rethink my time there: Your employer is saving money by using Open Source. While contributing features or fixes is actually not always legally possible, taking the time for proper issues is. And if your own project is that secretive, do a dummy.

My personal journey continues with a log of blogging about rails. I really used it a lot and I like it to this day. For whatever reason, it just worked back then and I found everything I wanted in there. Some posts might be more angry and ranty, but that usually boiled down to gems (those are projects you can pull in as libraries) required native counterparts on the system and well, let’s say that ecosystem was worse on macOS around 2010 than it is today.

Somewhen around 2011 I rewrote aforementioned foto project from Ruby on Rails to Java with Springframework which really got my deep into Java. We started to experiment with Spring in the company, replaced a couple of old applications for the customer with new ones and great success and in 2013, Spring Boot appeared and I have been using it ever since:

The application that I went public with is this biking.michael-simons.eu, still maintained and used (by me). Back in the early days I provided tons and tons of feedback to Boot and somewhen, also small features and bug fixes. In the end, it’s about trust.

At the same time, I had the great opportunity to visit ISAQB trainings and met Gernot Starke, a great personal inspiration. What an honor that he asked me to contribute to arc42byExample. Of course I took this chance.

Up until here the important part to understand is this: I am diligent at my work. Sometimes I overwork, but not too much. I have other interests than spent all day and night coding (More about this here). I was blessed that I started my professional live in a company that fostered and supported its employees: With a budget for trainings and a budget to take time for exploring and learning things. I cannot repeat this often enough: It depends so much on the first impression you get of work life which trajectory you take on and what expectations you allow yourself to develop. If you are a junior and your find yourself in a place that is all about the grind and hustle: Get out. Find something sustainable. I honestly don’t believe in “I hustle until my 30ths and than I stop” thing. Find a place that enables growth on a mutual level.

Anyway, I didn’t grind away with OOS contribution either. I started to experiment with giving talks. First at lokal meetups and (Java) user groups with good success. I somewhat like it, but it stresses the hell out of me. I am more of a writer. But it with some persistence it give me some kinda good name, which is of course valuable, both for contribution and growing a network.

Talks this days is a so-so topic. I personally find it really hard to justify traveling through the world and giving talks at every possible place (and not only eying the pandemic here). There’s no reason for me to fly to Brazil or so giving a talk about Spring Boot or Quarkus. There are excellent developers out in the world everywhere and I would rather coach a person from anywhere in the world to talk about Neo4j than flying ten thousand miles todo it myself just for a day off.

Back to open source: Make yourself a bit of a name. Report issues. Reach out over appropriate channels (tickets, Gitter, Slack; no unsolicited private messages). If you want to contribute code: Look for projects that have for contribution labels.

When you’re excited about a new idea and you might already have implemented it in someway or form and you think “hey, let’s just submit it!”, think again: Imagine the people at the receiving end. Is it a new feature? How will it be maintained in the future? Is it just something for a very small use case? Does it fit the rest? Who will own it? It is often safer to open up an issue and discuss if someone wants a new feature in their project or not. This will safe everybody’s time in the end.

Twitter is a good place for some discussions as well and from a question like this, a great learning across several people can come: Fix potential exponential backtracking in ReflectionUtils array parsing.

If its possible, go out to local meetups or conferences. Don’t just sit in there and spent your time passive, meet people. Talk with them. Listen. Build a network and contribute. In the end it’s a lot about trust and people, as I wrote already in 2017.

I personally was lucky: My close work with Spring and Spring Data people brought me into conversations with two people I would call both inspirational and friends these days: Oliver Drotbohm and later Michael Hunger. With a slight detour trying out consultancy at the good company INNOQ, I ended up at Neo4j. At Neo4j I maintain one of our Open Source projects, Spring Data Neo4j, together with Gerrit Meier. Several other modules have been spun off from there.

This month, I celebrated my 4th anniversary:

Of course, Neo4j doesn’t earn money by paying me and the teams that work on pure Open Source modules (such as connectors and drivers). We do need them however to facilitate the usage of our main products, such as Neo4j Enterprise and Neo4j AuraDB.

For me it was a once-in-a-lifetime opportunity. I get to learn from so many smart people in a world-class database company and on the same time do my work in the open. As said, don’t grind yourself mindlessly away, but also don’t let open doors pass.

Last but not least: Nobody is a worse developer if they don’t do Open Source. It is useful, educational and a lot of fun most of the time, but it’s not required at all to be good at a job.

Update: I had short conversation with Tim about the at when is a good time to enter a project: With rather young or mature projects:

If you follow that thread, you’ll see I spoke also about whether small equals insignificance or not (Hint: Small does not mean insignificance for me… Heck, you might even start something small that YOU need yourself and maybe attracting contributors on your own).

And while I was thinking about that topic, I remembered the initiative from iJUG last year, explicitly sponsoring new people to get into Open Source projects. Markus Karg wrote about this here (in German). While I am personally deeply into Spring and Quarkus these days, the Jakarta EE and Adoption projects are really valuable to the whole Java ecosystem.

And last but not least, sometimes things just don’t work out. Have a look at that small 5 Minute video:

Markus and Andres are both well known in the Java ecosystem, both avid Open source contributors and committers. And even though they followed the recommended approach, asking before contributing a new (small) feature, they weren’t able to get it in. This is super frustrating, but it happens and it happens to experience people as well. Don’t let it get to you if it happens to you.

Title photo by Peter Herrmann on Unsplash

Filed in English posts | Comments (1) »