Mapping OAuth2 / OpenID Access / User Tokens to Sessions in SpringBoot Apps

Question

I am trying to learn techniques for implementing OAuth2 / OpenID Connect with servlet apps and react apps. I have Authorization Server functions working correctly in Keycloak and have commandline test authorization_code, token and refresh flows so that plumbing works. When building a MVC servlet, code to enforce required authorizations works, redirects a user's browser to Keycloak for authentication and code generation, the code is returned to my servlet which properly obtains an access token for the code. However, while redirecting the user to a "main page" in the authenticated realm, I am not correct mapping the OAuth2 layer tokens to session and SecurityContext objects used in Spring Security so the subsequent page request is treated as unauthenticated.

Here is a top level summary of the components being used:

SpringBoot 2.6.7 (latest as of 5/15/2022)
SpringBoot Thymeleaf Start
Keycloak 18.0.0 (latest as of 5/15/2022)
Java JDK 18.0.1

Key dependencies from the pom.xml (just the artifactId for brevity):

 <artifactId>spring-boot-starter-oauth2-client</artifactId>
 <artifactId>spring-boot-starter-security</artifactId>
 <artifactId>spring-boot-starter-oauth2-resource-server</artifactId>
 <artifactId>spring-security-oauth2-autoconfigure</artifactId>
 <artifactId>spring-boot-starter-thymeleaf</artifactId>
 <artifactId>spring-boot-starter-web</artifactId>
 <artifactId>thymeleaf-extras-springsecurity5</artifactId>
 <artifactId>spring-boot-starter-webflux</artifactId>
 <artifactId>spring-integration-http</artifactId>

Here are the implementation components that are working:

• Keycloak is installed, running on 192.168.99.10:8011 with a client configured for use by the app
• the Keycloak client is configured for OpenID Connect protocol
• curl tests of authorization_code, token and refresh queries to Keycloak all function
• a servlet app has been created with four key page areas:
  • /myapp/gui/public --- public pages, no authentication required
  • /myapp/gui/access -- used for functions to handle access login and logout and Oauth callbacks
  • /myapp/gui/clients --- pages for authenticated users with myfirmuser permissions
  • /myapp/gui/admin --- pages for authenticated users with myfirmadmin permissions
• redirection of unauthenticated users to /myapp/gui/access/oauthproviders page
• rendering of links on that oauthproviders page to /auth endpoint of defined Authorization Servers
• clicking on Keycloak displays its authentication page and sends back authorization code
• the /myapp/gui/access/oath2/callback/keycloak page is handled and calls the /token endpoint on Keycloak
• a token is returned and the callback page creates a JSESSION, addes the access_token and refresh_token in the HttpServletRequest then redirects to /myapp/gui/clients/mainpage

The actual (undesired) behavior is that after receiving the new access_token from the Authorization Server, the redirect sent to the browser forwarding the human user to /myapp/gui/clients/mainpage (the logged in "home page") is then processed by the security filters and no token is found so the user is redirected back to /myapp/gui/oauthproviders to start the login process again.

Clearly, I am not correctly populating the access token or JWT session token in the SecurityContext or HttpRequest or HttpResponse object for it to go out to the browser and come back. That logic is currently implemented in my AccessController class that handles the integration to the remote AuthorizationServer (Keycloak). I've tried creating classes to invoked for AuthenticationSuccessHandler and AuthenticationFailureHandler. Here are the key classes in the build.

src/main/java/com/myfirm/dependsgui/AccessController.java
src/main/java/com/myfirm/dependsgui/DependsAuthenticationFailureHandler.java
src/main/java/com/myfirm/dependsgui/DependsAuthenticationSuccessHandler.java
src/main/java/com/myfirm/dependsgui/DependsController.java
src/main/java/com/myfirm/dependsgui/DependsguiApplication.java
src/main/java/com/myfirm/dependsgui/KeycloakAuthoritiesExtractor.java
src/main/java/com/myfirm/dependsgui/KeycloakPrincipalExtractor.java
src/main/java/com/myfirm/dependsgui/SecurityConfiguration.java

The classes aimed at transforming OAuth2 layer user / authorization information are referenced in the configure() class and logs at startup DO show them firing to point to my custom classes. However, something between OAuth and Spring Security doesn't seem to be linked correct to fire those classes after successful authentication.

Key questions:

1. in configure(), should oauth2ResourceServer() only be used for web service builds (not MVC apps)?
2. in configure(), are formLogin() and oauth2Login() mutually exclusive and not to be used together?
3. should the mapping of userdetails from the access token into the SecurityContext Authentication object be implemented in a) filter layer classes? b) an AuthenticationSuccessHandler derived class? c) in PrincipalExtractor and AuthoritiesExtractor derived classes? c) my servlet controller class handling login / logout actions? d) somewhere else? I think it has to be performed in a filter layer or AuthenticationSuccessHandler. However, the run-time flow doesn't appear to be invoking my custom classes to give me a place to trace backward to the point where I'm inevitably not doing something correctly.

Code fragments are excerpted below. Any help would be greatly appreciated.

===========================

Here is the KeycloakPrincipalExtractor class:

package com.myfirm.dependsgui;

//imports omitted for brevity


public class KeycloakPrincipalExtractor implements PrincipalExtractor {

private final Logger thisLog = LoggerFactory.getLogger(this.getClass().getCanonicalName());

    @Override
    public Object extractPrincipal(Map<String, Object> map) {

    thisLog.info("extractPrincipal() -- extracting preferred_username from Oauth token - value=" +
         map.get("preferred_username").toString());
    return map.get("preferred_username");
    }

}

Here is the KeycloakAuthoritiesExtractor class:

package com.myfirm.dependsgui;

//imports omitted for brevity

public class KeycloakAuthoritiesExtractor implements AuthoritiesExtractor {

    private final Logger thisLog = LoggerFactory.getLogger(this.getClass().getCanonicalName());

    // for now, just mockup three capabilities and two sets of authorities
    // * everyone has MYFIRM_USER
    // * full will have MYFIRM_FULL
    // * get will have MYFIRM_GET
    List<GrantedAuthority> MYFIRM_USER  = AuthorityUtils.commaSeparatedStringToAuthorityList(
     "SCOPE_myfirmuser");
    List<GrantedAuthority> MYFIRM_ADMIN = AuthorityUtils.commaSeparatedStringToAuthorityList(
     "SCOPE_myfirmadmin");
    List<GrantedAuthority> MYFIRM_ANONYMOUS = AuthorityUtils.commaSeparatedStringToAuthorityList(
     "SCOPE_myfirmanonymous");



    @Override
    public List<GrantedAuthority> extractAuthorities (Map<String, Object> map) {

    thisLog.info("DEBUG -- extractAuthorities() - map --> " + map.toString());
    if (Objects.nonNull(map.get("realm-access"))) {
       if (!((LinkedHashMap) map.get("realm-access")).get("roles").equals("myfirmuser")) {
          return MYFIRM_USER;
          }
       if (!((LinkedHashMap) map.get("realm-access")).get("roles").equals("myfirmuser")) {
          return MYFIRM_ADMIN;
          }
        }
    return MYFIRM_ANONYMOUS;
    }

}

Here is the method in my AccessController.java handling the authorization response from the Authorization Server and calling the remote /token endpoint to get an access_token.

@GetMapping("/access/oauth2/callback/{oauthprovidername}")
public String oauthCallback(
      @PathVariable("oauthprovidername") String oauthprovidername,
      @RequestParam("session_state") String sessionstate,
      @RequestParam("code") String code,
      HttpServletRequest  servletRequest,
      HttpServletResponse servletResponse,
      Model model) {

thisLog.info("oauthCallback() - oauthprovidername=" + oauthprovidername +
    " code=" + code + " session_state=" + sessionstate);

ClientRegistration providerRegistration = clientRegistrationRepository.findByRegistrationId(oauthprovidername);

HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED);

LinkedMultiValueMap<String,String> params = new LinkedMultiValueMap<>();
params.add("grant_type","authorization_code");
params.add("scope","openid");
params.add("client_id",providerRegistration.getClientId());
params.add("client_secret",providerRegistration.getClientSecret());
params.add("code",code);
params.add("redirect_uri",providerRegistration.getRedirectUri());

RestTemplate restTemplate = new RestTemplate();

HttpEntity< LinkedMultiValueMap<String,String> > request = new HttpEntity<>(params,headers);

ResponseEntity<String> response = restTemplate.postForEntity(
    providerRegistration.getProviderDetails().getTokenUri(),
    request,
    String.class);

// the response has this structure:
//    {"access_token":"xxx","expires_in":300,"refresh_expires_in":1800,"refresh_token":"yyy", \\ others }
//

thisLog.info("oauthCallback() - completed restTemplate.postForEntity() -- response = " + response);

ObjectMapper mapper = new ObjectMapper();
String access_token = "";
String refresh_token= "";
try {
   JsonNode node = mapper.readTree(response.getBody());
   access_token = node.path("access_token").asText();
   refresh_token = node.path("refresh_token").asText();
   }
catch (Exception theE) {
   thisLog.error("oauthCallback() -- Exception=" +theE);
   }
// at this point, access_token can be used in any other web service call by adding
// it as a header "Authorization: Bearer $access_token"

// we need to send it back to the client so the client can re-submit it on subsequent
// requests to maintain session state at the client rather than in a cache in this servlet

thisLog.info("oauthCallback() - completed restTemplate.postForEntity() " );

Cookie accessJwtCookie = new Cookie("access_token",access_token);
Cookie refreshJwtCookie = new Cookie("refresh_token",refresh_token);
// in real implementations, these calls should be made to ensure communications is limited to HTTPS
// accessJwtCookie.setSecure(true);
// refreshJwtCookie.setSecure(true);
// these restrict the browser's ability to access the cookies to sending HTTP out, blocking script access
accessJwtCookie.setHttpOnly(true);
refreshJwtCookie.setHttpOnly(true);
// these allow browser to send back the cookie for any subsequent URLs on the site
accessJwtCookie.setPath("/");
refreshJwtCookie.setPath("/");
// these limit the retention of the cookie -- access are only good for 300 seconds, refresh for 1800
// so no point in the browser keeping them longer than that
accessJwtCookie.setMaxAge(300);
refreshJwtCookie.setMaxAge(1800);
servletResponse.addCookie(accessJwtCookie);
servletResponse.addCookie(refreshJwtCookie);

thisLog.info("oauthCallback() - attempting redirect to authenticated mainpage - servletResonse=" + servletResponse.toString());
// create a session and use that to create a JSESSION cookie in the response
HttpSession session = servletRequest.getSession(true);
session.setMaxInactiveInterval(5*60);  // set to 5 minute idle timeout

// for debuggging, use the refresh_token to test our refresh2Provider() logic
//ClientRegistration testClient = refresh2Provider(refresh_token);


model.addAttribute("diagnostics", response);
model.addAttribute("exception", "(none)");
model.addAttribute("stacktrace","(none)");
// NOTE! -- this redirect is "absolute relative" to the servlet context of /depends/gui
// "redirect:clients/mainpage.html" ---> /depends/gui/access/oauth2/callback/clients/mainpage (WRONG)
// "redirect:/clients/mainpage.html" --> /depends/gui/clients/mainpage
return "redirect:/clients/mainpage";

}

Here is the entire SecurityConfiguration class.

package com.myfirm.dependsgui;

//imports omitted for brevity


@EnableWebSecurity
public class SecurityConfiguration extends WebSecurityConfigurerAdapter {

private final Logger thisLog = LoggerFactory.getLogger(this.getClass().getCanonicalName());

String JWKSETURI = "http://localhost:8011/realms/myfirm/protocol/openid-connect/certs";


//------------------------------------------------------------------------------------
// webClient() - defines a bean that will map configured Authorizaton Server parameters
// from application.properties to a Http client that can call those endpoints to
// verify tokens, etc
//------------------------------------------------------------------------------------
@Bean
WebClient webClient(ClientRegistrationRepository clientRegistrationRepository,
      OAuth2AuthorizedClientRepository authorizedClientRepository) {

thisLog.debug("webClient() - instantiating new WebClient for this app to interact with each defined Authorization Server");
ServletOAuth2AuthorizedClientExchangeFilterFunction oauth2 =
          new ServletOAuth2AuthorizedClientExchangeFilterFunction(clientRegistrationRepository,
          authorizedClientRepository);
oauth2.setDefaultOAuth2AuthorizedClient(true);
return WebClient.builder().apply(oauth2.oauth2Configuration()).build();
}


//----------------------------------------------------------------------------------
// authorizationRequestRepository() - defines bean used by the auto-generated
// login handling to bounce an authorization request over to the
// Authorization Server selected by the interactive user
//----------------------------------------------------------------------------------
@Bean
public AuthorizationRequestRepository<OAuth2AuthorizationRequest>
  authorizationRequestRepository() {

thisLog.debug("authorizedRequestRepository() - instantiating new HttpSessionOAuth2AuthorizationRequestRepository()");
return new HttpSessionOAuth2AuthorizationRequestRepository();
}


//----------------------------------------------------------------------------------
// accessTokenResponseClient() - this mirrors the default function created by the
// OAuth2 libraries for accepting access tokens sent back from an Authorization
// Server.  This could be overriden / enhanced if additional info needs to be
// extracted from somewhere after successful authentication to stuff into the JWT
//---------------------------------------------------------------------------------
@Bean
public OAuth2AccessTokenResponseClient<OAuth2AuthorizationCodeGrantRequest>
  accessTokenResponseClient() {

thisLog.debug("accessTokenResponseClient() - instantiating default NimbusAuthorizationCodeTokenResponseClient() for post-processing of new access tokens");
return new NimbusAuthorizationCodeTokenResponseClient();
}


// -------------------------------------------------------------------------------
// Declare bean oauthPrincipalExtractor() that returns an instance of our
// customized OauthPrincipalExtractor class to extract the desired value of an
// Oauth reply from Keycloak we want used as principal in a Spring Authorization
// -------------------------------------------------------------------------------
@Bean
public PrincipalExtractor keycloakPrincipalExtractor() {

thisLog.debug("keycloakPrincipalExtractor() - instantiating bean of custom KeycloakPrincipalExtractor");
return new KeycloakPrincipalExtractor();
}


// -------------------------------------------------------------------------------
// Declare bean oauthAuthoritiesExtractor() that returns an instance of our
// customized KeycloakAuthoritiesExtractor() class to extract grants from Oauth
// tokens into Spring Security Authentication objects
// -------------------------------------------------------------------------------
@Bean
public AuthoritiesExtractor keycloakAuthoritiesExtractor() {

thisLog.debug("keycloakAuthoritiesExtractor() -- instantiating bean of custom KeycloakAuthoritiesExtractor");
return new KeycloakAuthoritiesExtractor();
}



// -------------------------------------------------------------------------------
// Declare bean dependsAuthenticationSuccessHandler() that returns an instance of our
// customized DependsAuthenticationSuccessHandler() class to perform post-processing
// after successful authentication
// -------------------------------------------------------------------------------
public AuthenticationSuccessHandler dependsAuthenticationSuccessHandler() {

thisLog.debug("dependsAuthenticationSuccessHandler() -- instantiating bean of custom DependsAuthenticationSuccessHandler");
return new DependsAuthenticationSuccessHandler();
}


// -------------------------------------------------------------------------------
// Declare bean dependsAuthenticationFailurHandler() that returns an instance of our
// customized DependsAuthenticationFailureHandler() class to perform post-processing
// after successful authentication
// -------------------------------------------------------------------------------
public AuthenticationFailureHandler dependsAuthenticationFailureHandler() {

thisLog.debug("dependsAuthenticationFailureHandler() -- instantiating bean of custom DependsAuthenticationFailureHandler");
return new DependsAuthenticationFailureHandler();
}



// ----------------------------------------------------------------------------
// keycloakJwtAuthenticationConverter() - defines a mapping that will be used
// by token processing to map claims at the token level to authorities in the
// Spring Security layer for the app
//-----------------------------------------------------------------------------
private JwtAuthenticationConverter keycloakJwtAuthenticationConverter() {

thisLog.debug("keycloakJwtAuthenticationConverter() -- instantiating critiera for grant converter within JwtAuthenticationConverter()");
JwtGrantedAuthoritiesConverter thisgrantauthconverter = new JwtGrantedAuthoritiesConverter();
// the roles we want to extract are under "realm-access": { roles": [ xx,yy,zz]  }
thisgrantauthconverter.setAuthoritiesClaimName("realm-access");
thisgrantauthconverter.setAuthorityPrefix("ROLE_");
JwtAuthenticationConverter thisauthconverter = new JwtAuthenticationConverter();
thisauthconverter.setJwtGrantedAuthoritiesConverter(thisgrantauthconverter);
return thisauthconverter;
}


//-----------------------------------------------------------------------------------
// configure(HttpSecurity) - key method for setting filters and OAuth2 parameters
//-----------------------------------------------------------------------------------
@Override
protected void configure(HttpSecurity http) throws Exception {

//------------------------------------------------------------------------------------------------
// NOTE: These patterns are APPENDED to the servlet context /depends/gui in application.properties
//------------------------------------------------------------------------------------------------

thisLog.info("configure(HttpSecurity) - defining access filters for application URI patterns");
// NOTE: using authorizeHttpRequests() instead of older authorizeRequests() -- many online examples
// have not reflected this new directional implmentation - older call is being deprecated
http.authorizeHttpRequests()
       .antMatchers(HttpMethod.GET, "/public/**").permitAll()
       .antMatchers(HttpMethod.GET, "/css/**").permitAll()
       .antMatchers(HttpMethod.GET, "/js/**").permitAll()
       .antMatchers(HttpMethod.GET, "/access/**").permitAll()
       .antMatchers(HttpMethod.GET, "/clients/**").hasAnyAuthority("SCOPE_myfirmuser","SCOPE_myfirmadmin")
       .antMatchers(HttpMethod.GET, "/admin/**").hasAuthority("SCOPE_myfirmadmin")
       .anyRequest().authenticated()
       .and() // return to the parent http object
    .oauth2ResourceServer()
       .jwt()
       .jwtAuthenticationConverter(keycloakJwtAuthenticationConverter())
       ;
http.formLogin()
       .loginPage("/access/oauthproviders")
       .successHandler(dependsAuthenticationSuccessHandler())
       .failureHandler(dependsAuthenticationFailureHandler())
       .and() // return to the parent http object
    .oauth2Login()
       .loginPage("/access/oauthproviders")
       .authorizationEndpoint()
       .authorizationRequestRepository(authorizationRequestRepository())
       ;
}



//-----------------------------------------------------------------------------
// jwtDecoder() - instantiates a JWT decoder using the JWKsetUri of an OAuth
//   provider to fetch strings to decode / unencrypt a token
// NOTE -- not clear how this approach works when a single app can use
//   multiple OAuth providers for google, facebook, github, keycloak, etc
//   For now, this is hardcoding the JwkSetUri for a local keycloak instance.
//-----------------------------------------------------------------------------

@Bean
public JwtDecoder jwtDecoder(OAuth2ResourceServerProperties properties) {

thisLog.info("jwtDecoder() - returning link to method for decoding / validating JWT via Nimbus library");
thisLog.info("jwtDecoder() - incoming properties = " + properties.getJwt().getJwkSetUri());
NimbusJwtDecoder thisDecoder = NimbusJwtDecoder.withJwkSetUri(JWKSETURI).build();

return thisDecoder;
}


} // end of entire class

Here are logs at startup showing the classes referenced in the configure() method ARE getting loaded:

2022-05-12 23:42:45,163 INFO com.myfirm.dependsgui.SecurityConfiguration$$EnhancerBySpringCGLIB$$df1fd8f6 - configure(HttpSecurity) - defining access filters for application URI patterns
2022-05-12 23:42:45,171 DEBUG com.myfirm.dependsgui.SecurityConfiguration$$EnhancerBySpringCGLIB$$df1fd8f6 - keycloakJwtAuthenticationConverter() -- instantiating critiera for grant converter within JwtAuthenticationConverter()
2022-05-12 23:42:45,178 DEBUG com.myfirm.dependsgui.SecurityConfiguration$$EnhancerBySpringCGLIB$$df1fd8f6 - dependsAuthenticationSuccessHandler() -- instantiating bean of custom DependsAuthenticationSuccessHandler
2022-05-12 23:42:45,179 DEBUG com.myfirm.dependsgui.SecurityConfiguration$$EnhancerBySpringCGLIB$$df1fd8f6 - dependsAuthenticationFailureHandler() -- instantiating bean of custom DependsAuthenticationFailureHandler
2022-05-12 23:42:45,209 DEBUG com.myfirm.dependsgui.SecurityConfiguration$$EnhancerBySpringCGLIB$$df1fd8f6 - authorizedRequestRepository() - instantiating new HttpSessionOAuth2AuthorizationRequestRepository()
2022-05-12 23:42:45,225 INFO com.myfirm.dependsgui.SecurityConfiguration$$EnhancerBySpringCGLIB$$df1fd8f6 - jwtDecoder() - returning link to method for decoding / validating JWT via Nimbus library
2022-05-12 23:42:45,225 INFO com.myfirm.dependsgui.SecurityConfiguration$$EnhancerBySpringCGLIB$$df1fd8f6 - jwtDecoder() - incoming properties = null
2022-05-12 23:42:45,381 DEBUG com.myfirm.dependsgui.SecurityConfiguration$$EnhancerBySpringCGLIB$$df1fd8f6 - webClient() - instantiating new WebClient for this app to interact with each defined Authorization Server
2022-05-12 23:42:45,586 DEBUG com.myfirm.dependsgui.SecurityConfiguration$$EnhancerBySpringCGLIB$$df1fd8f6 - accessTokenResponseClient() - instantiating default NimbusAuthorizationCodeTokenResponseClient() for post-processing of new access tokens
2022-05-12 23:42:45,591 DEBUG com.myfirm.dependsgui.SecurityConfiguration$$EnhancerBySpringCGLIB$$df1fd8f6 - keycloakPrincipalExtractor() - instantiating bean of custom KeycloakPrincipalExtractor
2022-05-12 23:42:45,591 DEBUG com.myfirm.dependsgui.SecurityConfiguration$$EnhancerBySpringCGLIB$$df1fd8f6 - keycloakAuthoritiesExtractor() -- instantiating bean of custom KeycloakAuthoritiesExtractor