JAX-RS Jersey client gets 400 response when web method parameters are annotated

Question

Here is my web service method:

@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
@POST
@Path("login")
public Response login(@NotNull @Valid Credentials credentials) {
// do login
}

And here is the snippet of the client code:

WebTarget loginTarget = baseTarget
        .path("base")
        .path("login");

Credentials credentials = new Credentials(username, password);

Response resp = loginOperation
        .request()
        .post(
            Entity.entity(credentials, MediaType.APPLICATION_JSON_TYPE)
        );

When I do post, it doesn't reach the login method. Server returns 400 error with empty body.

When I remove @NotNull @Valid annotations from credentials parameter it works.

I noticed that Entity#entity method has an overloaded version which accepts Annotation[] as a 3rd parameter. And then I came across this section of Jersey documentation. So I went ahead a created annotation factories as suggested in the tutorial:

public static class ValidFactory extends AnnotationLiteral<Valid> implements Valid {
    public static ValidFactory get() {
        return new ValidFactory();
    }
}

and then changed client code to this:

.post(
    Entity.entity(credentials, MediaType.APPLICATION_JSON_TYPE,
        new Annotation[] {
            AuthenticationResource.NotNullFactory.get(), 
            AuthenticationResource.ValidFactory.get()
        }
    )
)

which unfortunately resulted in the same error. Google search didn't yield any results, and I don't have much time to dig in Jersey source code. So, maybe someone who knows the solution would share it, please?

UPDATE

Just to add to @peeskillet's response. I use custom ExceptionMapper:

@Provider
public class ValidationExceptionMapper implements ExceptionMapper<ConstraintViolationException> {

    @Override
    public Response toResponse(ConstraintViolationException exception) {
      // customize response
    }

}

So, in my case, instead of setting a ServerProperties.BV_SEND_ERROR_IN_RESPONSE property, I had to register the mapper in the server config:

GrizzlyHttpServerFactory.createHttpServer(URI.create(BASE_URI)
        new ResourceConfig() {
            {
                register(ValidationExceptionMapper.class)
            }
        }
);