RedirectToLogin handler not called when a role is not fulfilled (cookie authentication)

Question

I have a ASP.NET Core (.NET 6) API controller that makes sure that only authenticated users can access its endpoints using the Authorize attribute:

[ApiController]
[Authorize]
[Route("api/borrower")]
public class BorrowerController : ControllerBase
{
    ...
}

Since it serves a REST API, instead of returning 302 redirect responses, I prefer having 401 unautherized responses so I've configured my cookie authentication in Startup:

services.AddScoped<CookieAuthenticationEventsHandler>();
services.ConfigureApplicationCookie(o =>
{
    ...
    o.EventsType = typeof(CookieAuthenticationEventsHandler);
})

and:

public class CookieAuthenticationEventsHandler : CookieAuthenticationEvents
{
    public override Task RedirectToLogin(RedirectContext<CookieAuthenticationOptions> context)
    {
        context.Response.StatusCode = StatusCodes.Status401Unauthorized;
        return Task.FromResult(0);
    }
}

Works as expected. Instead of 302, I get a 401 on unauthenticated requests.

What does not work is when I introduce roles-based access control and I change the above [Authorize] to [Authorize(Roles = "Borrower")] for example.

A request to the controller with a principal that is authenticated but does not assume that role again results in a 302 instead of 401. Setting a breakpoint in the RedirectToLogin method shows that in this case the method is not being called (i.e. the event is not raised).

Also, I don't see any other event that would be suitable for this scenario.

Is this by design or is it a bug? Am I missing something here? And how could I still turn that 302 into a 401.