Spring Security 5: New password storage format

Spring Security 5 has been out for some days now. People on Spring Boot 1 or plain Spring must manually upgrade their dependencies at the moment to notice the new kid on the block. But with Spring Boot 2 Spring Security 5 will be the default for new applications but also for applications that are migrated.

For me as the author of the upcoming Spring Boot Buch (Which you can preorder here) the migration turned out to be pretty hard (hard as in “I had to rewrite lots of examples and text”). The Spring Boot team changed a lot of the “magic” defaults in regard of security and also Actuator security. I’m not saying I don’t like the changes, quite the contrary actually, but it just took some time.

In this post I’m gonna explain the changes in the way password encoding works in Spring Security 5. Most certainly your application will be affected in some way if you upgrade to Spring Boot 2. I will explain migration scenarios if you already have secure hashes or none at all, but also an interesting way of converting insecure hashes to secure hashes and not only migrating them in the sense that they are usable with Spring Security 5 again.

Modernized Password encoding

Whether a password hash is secure or not is in constant flux. There was a time way back then when even a MD5-Hash was regarded secure. Long gone. SHA-1 and SHA-256: Insecure. Either there have been collisions or huge enough rainbow tables, it’s a constant “battle”. In Spring Security those things have been dealt with through a concept of PasswordEncoder. Historically it came from the org.springframework.security.authentication.encoding. The package from that interface has been deprecated for some time now and it’s finally gone from Spring Security 5. A good thing, as “encryption” does not depend on “authentication”. I really wish other frameworks and especially applications would do this kind of housekeeping more often.

So that’s the first thing taking in regard when upgrading to Boot 2: Those encoders are gone. If you’re user details system depends on something like the old ShaPasswordEncoder or Md5PasswordEncoder, you have to do an active migration. The algorithms are still available, but adapted to the current PasswordEncoder-interface residing in org.springframework.security.crypto.password. However, they are all deprecated to mark them as insecure.

A framework like Spring Security can only introduce breaking changes like this once in while, so the team behind Rob Winch made that one count.

What is a good, new password encoder for replacing the old default NoOpPasswordEncoder? You’d say BCryptPasswordEncoder? Nope, not really. That would tie everything again to a specific implementation. Spring Security 5 defaults now to DelegatingPasswordEncoder.

Delegating means being able to use multiple implementations to

  • encode new passwords
  • validate passwords in old and new formats
  • change encoding in the future

The encoder introduces a new password storage format for that purpose. It looks like {id}encodedPassword where {id} represents a logical name of a specific encoder and encodedPassword the actual hash.

The delegating encoder implements the following logic for matching:

  • If there is a known ID for an encoder, use that one to verify the password
  • If there is an encoder configured for having no id, use that, otherwise throw an exception

And for encoding: New passwords are encoded with the encoder identified with a parameter to the constructor of the delegating encoder.

That means you have to do an active migration in most situations.

Migration scenarios

You have been relying on the old default NoOpPasswordEncoder

That means that your passwords are stored in plain text. A bad situation in most cases. Creating an instance of NoOpPasswordEncoder and thus replacing the default delegating encoder allows you to postpone migrations of your passwords to a later date. The next better solution would be prefixing all existing passwords with {noop} and keeping the default encoder of Spring Security 5.

Best option in this case is actually hashing all existing passwords in a a batch run (i.e. in the most simple form of an ApplicationRunner instance) by using the default encoder instance like this:

String encoded = passwordEncoder.encode(plainTextPassword);

You’ll end up with bcrypt-hashed passwords.

Your passwords are already encoded (without a salt source)

Two options here: By all probability you are aware which hash you’re using and are able to update your hashed passwords to include the prefix.

If you you instantiate your password encoder like this

PasswordEncoderFactories.createDelegatingPasswordEncoder();

you get encoders for those prefixes as of Spring Security 5.0:

Map<String, PasswordEncoder> encoders = new HashMap<>();
String encodingId = "bcrypt";
encoders.put(encodingId, new BCryptPasswordEncoder());
encoders.put("ldap", new LdapShaPasswordEncoder());
encoders.put("MD4", new Md4PasswordEncoder());
encoders.put("MD5", new MessageDigestPasswordEncoder("MD5"));
encoders.put("noop", NoOpPasswordEncoder.getInstance());
encoders.put("pbkdf2", new Pbkdf2PasswordEncoder());
encoders.put("scrypt", new SCryptPasswordEncoder());
encoders.put("SHA-1", new MessageDigestPasswordEncoder("SHA-1"));
encoders.put("SHA-256", new MessageDigestPasswordEncoder("SHA-256"));
encoders.put("sha256", new StandardPasswordEncoder());

that means you can prefix your hashed passwords like {SHA-256}encodedPassword and you’re okayish to go. Okayish because you are now able to live on with insecure hashes. Nice thing here is that you can achieve by just updating all passwords at once, for example just using an SQL-query.

The Second option instead of migrating without touching the passwords themselves is this:

PasswordEncoder passwordEncoder = PasswordEncoderFactories.createDelegatingPasswordEncoder();
passwordEncoder.setDefaultPasswordEncoderForMatches(new MessageDigestPasswordEncoder("SHA-256"));

This way you won’t have to update your passwords as the delegating encoder has a fallback. New passwords would be encoded with bcrypt then.

That leads me to the third scenario

Rehashing and upgrading passwords

To rehash an existing password one has to know the plain text as all those hashes are one way functions. You just can invalide all accounts and force all users to change their password next time they login or you can do this automatically. If you want to do this automatically, you the the clear text password. The one and only moment in time to retrieve that is when a user authenticates against your system.

A word of warning: This can be insecure and normally, Spring Security prevents the credentials of a username and password authentication token dangling around.

Following is a migration scenario from a real bullsh*t encoder to a secure system. The author of the encoder was so bad that he even didn’t realize that he has been using a reversible hash all the time! (I kid you not, I actually have seen things like this in the not so distant past! (and no, it wasn’t me)).

The following example code is part of this repository michael-simons/passwordmigration. All beans are configured in the nested SecurityConfig class. Passwords have been encoded with an encoder called BSPasswordEncoder for a reason. The author realized that hashing passwords this way is very, very bad and wants to update them. He configures the delegating password encoder like this:

Please read through the comments, they are part of the story!

The current user and password repository comes from a database, but imagine for a second it’s something like this, so you can see the insecure password hash (you probably can rehash them without cleartext, can’t you?):

That user details service can actually be anything, it just illustrates stuff here. Next the system has to be hooked to Spring Securities event system, which fires successful and failed logins:

You than have to instantiate a WebSecurityConfigurerAdapter that connects the dots. Be careful: With Spring Boot 2.0 this turns Boots magic in regard to security off and all defaults come from Spring Security itself:

The last missing bit is a listener that is notified when a user logs in. It can look like this:

Summing this up: This is a scenario that shouldn’t be live for all the time in the world. I used it in my private project Daily Fratze around 2009 when I migrated all old SHA-1 hashes to BCrypt. It takes some time until all users have been logged in at least once. Maybe you don’t even get them all, but then you can still reset their passwords.

(Featured image on this post by portal gda.)

| Comments (0) »

13-Jan-18


Use Keycloak with your Spring Boot 2 application

This post has been featured in the 7th anniversary edition of This Week in Spring – January 2n, 2018.

Here’s a short post how to authenticate against Keycloak from within a Spring Boot 2 application. For Spring Boot 1.5.x there’s a community adapter from the Keycloak-team that takes the burden from you, but this adapter is not yet ready for Spring Boot 2 and Spring Security 5.

I had the following requirements for the setup I am gonna present:

  • Manage users outside one application (i.e. be ready for a bunch of services): Realized with Keycloak
  • Full integration with Spring Security, especially method security
  • Funktional with server side rendered Thymeleaf or other template systems supported by Spring Boot

Here is the fully functional demo project: keycloakdemo. I am not replicating some comments from the sources in the following paragraphs.

With Spring Boot 2 comes Spring Security 5 and the first class support for OAuth Login: New feature OAuth2-Login. That means one doesn’t need separate modules anymore. But to make this work, it’s not enough to have spring-boot-starter-security on the class path, you’ll need two more dependencies:

I’ll spare you the details on how to setup Keycloak and how to create a realm and what a realm is. Keycloak has an excellent documentation about that and the screenshots from the one existing tutorial on how to use the Spring Boot adapter have been copied around anyway, along with that post.

Next step, prepare your application and disable Security-Autoconfiguration completely. Right now, it doesn’t stop generating a default user and I don’t want that when I connect the application to an external user database.

Note: There’s an issue about the default user when using OAuth-login: #10531.

I have introduced a simple property-component to store all (for my use case) relevant Keycloak configuration elements. Should fit a standard use case in most times, too. All this stuff can be configured through new Spring Boot Properties for Security 5 as well (as described in there in the documentation), but those aren’t even fun for people who like YAML, not speaking of me, the last .properties-fan out there.

So these properties here

maps neatly to:

Read more about the client registration here. The instance of ClientRegistration generated from the properties here is put to use in my security configuration, the core of this tutorial:

It all boils down to the registered clients through the bean clientRegistrationRepository and then in oauth2Login in the security configurer. The following controller together with a simple Thymeleaf template is just for demoing purposes:

Assuming you have a Keycloak server running on port 8080, you can checkout the above linked project, build it with Java 9 and run it on port 8082. Open http://localhost:8082, hit login and you should be redirected to your Keycloak instance and back after a successful login.

Needless to say that this setup works well with other OAuth 2 providers.

| Comments (0) »

28-Dec-17


2017: It’s all about people

It’s the time of the year to write some recaps. I did this last year and the year before and I will continue that.

We live in highly technologized world, but what’s it all about if not people?



That picture was taken by Stephen Chin at the last day of JCrete 2017 and pretty much sums up the last year. If you click the link and look closely, you’ll find me right behind Heinz. I remember chatting with Heinz at the end of 2016 and the possibility of joining JCrete for one time and 7 months later, it actually happened. Incredible. Even if my first book arc42 by example has now been sold nearly 1500 times, that wouldn’t qualify me. Also, I am no Java Champion that would have qualified me and so I was very happy about the fact that JCrete opened up for more visitors. As written in the recap linked above: A once in a lifetime experience.

We organized eight talks at EuregJUG in 2017: Axel Fontaine, Johan Janssen, Michael Plöd, Stefan Tilkov, Mark Heckler, Maaike Brinkhof, Philipp Krenn and Mattias Weßendorf. I am incredible grateful for the success of this user group. I met new and old friends and have the feeling that I have connected some people.

I myself spoke at eight different meetups, user groups and conferences and I plan to continue that. It really enriched my life.

Yesterday I hopefully committed the last big fixes to Spring Boot Buch. My publisher and I agreed that it’s better to wait with publishing until Spring Boot 2 is final. It’s a bit annoying as I want to get this of my chest, but better for the overall quality. If you’re interested about the making of this, I have tagged all the posts about the book accordingly. As I already have published all examples to GitHub (and also did some contributions to other stuff throughout the year), my contributions chart looks very nice this year:



My plans for next year: Focus even more on communication and connecting to and with people. My impressions from current projects are: We have so much technology to solve problems but more often than not, problems that needs to be addressed are not so much of technical but people nature. My German post at INNOQ is mainly about that topic.

Following are some impressions from last year:

Getting wasted with Axel on Belgium beer in Aachen and opening Spring Community Meetup with Michael Plöd in Munich the next day:

 

“Playing” with Johanns Lego, recapping the event with Franz, enjoying Marks talk with my wife Christina and meeting Vlad in Cluj:

   

With Joe, Josh, Mark and Simon from the Spring team at Pivotal at Spring I/O in Barcelona:



At the same conference with Ingo, Sergi and Kevin:



Impressions from JCrete. Thanks to for your company throughout the week, to Felix for good conversations and Heinz for inviting me:

   

With Chris, Felix, Jürgen and Philipp:



And certainly not forgetting my new colleagues at INNOQ:



2017 was a good year with a lot of changes, but as my friends at Pivotal say “The only constant is change”, it was a good year. I’m looking forward to the next, having my first printed book published and meeting new and old friends and doing hopefully good work. Please take care of yourself, try to relax some times and step back a bit from the display and enjoy life and the people within.

| Comments (0) »

24-Dec-17


Git: Create a new, empty and unrelated branch

I wanted to start publishing to GitHub Pages from my learning and experimenting project simple-meetup. One of the learning scopes of the above project is “code as documentation” and the question to which extend such a project can be used to write articles and generate documentation. In this project I’m using Asciidoctor and the Gradle Plugin as suggested by my friends Ralf and Gernot in their column Hitchhiker’s Guide to Docs as Code and corresponding repo.

GitHub Pages can be setup separately for every repository and you can choose to select a branch for your HTML-files. As I didn’t plan this upfront, I obviously didn’t have an empty branch for that and I didn’t want to create one and have all the other commits in it. Enter --orphan for the checkout-command:

--orphan
Create a new orphan branch, named , started from and switch to it. The first commit made on this new branch will have no parents and it will be the root of a new history totally disconnected from all the other branches and commits.

The index and the working tree are adjusted as if you had previously run “git checkout “. This allows you to start a new history that records a set of paths similar to by easily running “git commit -a” to make the root commit.

See checkout --orphan.

Run git checkout --orphan gh-pages and you’ll end up with a new branch called gh-pages with all the files you have staged, but they can be safely removed with git rm -rf . and then the branch can be committed and pushed.

If you’re wondering what I am doing with that branch: Part of the build are ideas from the docToolchain. I am generating articles with this stance in Gradle:

tasks.withType(AsciidoctorTask) { docTask ->
    dependsOn check
 
    attributes \
          'sourceDir': "$projectDir/src/"
 
    sourceDir = new File("src/articles")
    outputDir = new File(project.buildDir, "articles")
    resources {
        from(sourceDir) {
            include '*.png'
        }
    }
}
 
task generateHTML (
        type: AsciidoctorTask,
        description: 'use html5 as asciidoc backend'
) {
    backends = ['html5']
}
 
defaultTasks "clean", "build", "generateHTML", "generatePDF"

This generation is an essential part of the build and it runs by default. In my Travis CI script I have the following after_success hook:

after_success:
- cd build/articles/html5
- git init
- git config user.name "${GH_USER_NAME}"
- git config user.email "{GH_USER_EMAIL}"
- git add . ; git commit -m "Deploy to GitHub Pages"
- git push --force --quiet "https://${GH_TOKEN}@${GH_REF}" master:gh-pages

It changes to the output directory, reinitializes the repository there and force pushes the new master to the remote gh-pages branch. The variables are my GitHub username and a token I created for the purpose of having Travis to be able to push.

| Comments (0) »

09-Dec-17


November recap and Java quickies

November has been a crazy month. Even though I joined Ralf in his Movember team and grew a mustache, I visited the JUG Münster at LVM Münster and gave my talk about SQL once again in a slightly updated version:

It felt very good being part of innoQ on this occasion.

Also very nice was the mentioning of my latest short tutorial in this week in spring.

Communications

What didn’t feel too good was the passing of my dear god father and especially his sad funeral. It really brought me down. As a result, I was in a bad mood for the last week and somewhat impatient and angry. Further down the road, communication got hard.

I already have been thinking a lot lately about communication problems and why they are so hard to solve in our business. As soon as things stress up (in a bad way), communication goes haywire. Often people build walls around themselves, myself included, and suddenly a bunch of good people who could form a good team, don’t. My friend and colleague Alex has written a great article (in German) on micro services, but honestly, it’s way more about various forms of communication as well. Check it out here.

While I tend to say I wanna do something with programming but without humans, the opposite is true. I love to listen, a good discussion, exchanging thoughts. Looking back at the past day, I spent several hours discussing options with the technical lead of our customer to evolve a platform and to onboard a new colleague into the project and the hours were well spent.

Spring Boot Buch

As hoped in last months recap, I got the preface from Jürgen himself and the book is somewhat jürgenized now.

The book itself is 95% done. It’s copy edited, spellchecked and ready to be set, but we are awaiting at least one release candidate of Spring Boot 2, which will see the light this month (December). I already did some changes on the Actuator parts and will do some big changes due to Spring Security 5. I have all the examples at GitHub. If you find something that doesn’t work, please let me know.

Java quickies and new, small projects

2 weeks ago I got nerd-sniped by Lukas, after complaining on twitter that I don’t have the amazing Spring Boot configuration mechanism for a task at hand. He proposed to just build it myself with plain JDK. I didn’t succeed fully, but did some nice, low-level stuff to map properties to classes. While working on that, I wrote this and some more in the pipeline. As I am a bit astonished how “new” Java 8 concepts still are for some, I will publish them the next months.

I have some new smaller projects:

The last project, AssertJ demos, come from experiments with AssertJ, both in the simple-meetup project as well as at the customers.

AssertJ got me while solving the following problem. Imagine this weird service, written in Java 9 but what was the other drinking while designing this return types?

public class WeirdService {
    public Object getSomething() {
        throw new UnsupportedOperationException("This is unsupported");
    }
 
    public Object getAString() {
        return "Hello";
    }
 
    public Object getTheRightThing() {
        return List.of(
            Map.of("firstName", "Michael", "lastName", "Simons"),
            Map.of("firstName", "Christina", "lastName", "Simons"),
            Map.of("firstName", "Anton", "lastName", "Simons"),
            Map.of("firstName", "Oskar", "lastName", "Simons")
        );
    }
}

AssertJ solves testing this freakish thing in every aspect:

class WeirdServiceTest {
 
    final WeirdService weirdService = new WeirdService();
 
    @Test
    @DisplayName("What to do? JUnit 5 or AssertJ?")
    public void demonstrateExceptionHandling() {
        // JUnit 5
        UnsupportedOperationException e = assertThrows(UnsupportedOperationException.class, () -> weirdService.getSomething());
        assertEquals("This is unsupported", e.getMessage());
 
        // AssertJ
        assertThatThrownBy(() -> weirdService.getSomething())
            .isInstanceOf(UnsupportedOperationException.class)
            .withFailMessage("This is unsupported");
    }
 
    @Test
    @DisplayName("Sometimes you have service that is just… weird… 🤡 Here it returns objects")
    public void niceTypeChecks() {
        assertThat(weirdService.getAString())
            .isInstanceOf(String.class)
            .asString()
            .isEqualTo("Hello");
 
        assertThat(weirdService.getTheRightThing())
            .isInstanceOf(List.class)
            .asList()
            .extracting("firstName")
            .containsExactly("Michael", "Christina", "Anton", "Oskar");
    }
}

First, look at the beautiful exception testing. I even find it nicer than the JUnit 5 way of doing stuff. And then, checking the returned type of the service, which comes with an elegant casting and then: extracting stuff from results is smart: It either works on properties or in this case, on map keys.

I collapsed several multi-page tests with that stuff into small chains of method calls. If you’re into testing, go have a look at AssertJ. Together with JUnit 5s possibilities to write the tests itself, its a great way to gain trust into your code.

| Comments (0) »

01-Dec-17