Setting up multi-release JAR unit tests

Question

I have a project that uses a lot of reflection, also on "new" Java features such as records and sealed classes. I'm writing a class like this:

public class RecordHelper {
    public static boolean isRecord(Class<?> type) {
        return type.isRecord();
    }
}

Of course, this only works in Java 16 and higher, so I'm trying to set up a multi-release JAR file, with a default implementation like this:

public class RecordHelper {
    public static boolean isRecord(Class<?> type) {
        return false;
    }
}

I've been able to set that up with Maven using this post on Baeldung, and it works great. It's nice, because it doesn't rely on separate profiles for various versions, which keeps my pom file clean.

But now I need to write tests.

I want to be able to run the test suite on all platforms that I want to support, which means everything from JDK 8 and up. (I don't mind compiling on a different JDK before I run the tests.) Of course, on JDK 16 and up, I also want to test records-related things (and on 17 and up, sealed classes), so that means that I have to compile some records, which means I will inevitably have some class files that won't work on older JDKs.

In my mind, it would make sense to have something like a multi-release JAR file for tests as well, where the records tests get placed in the appropriate place in META-INF/versions, but of course tests aren't usually packaged in a JAR, so that doesn't work.

Is there a way to get this working in a single-module Maven project without too much repetition?

Of course, it would have to 'accumulate' test classes as the JDK version goes up, e.g. on JDK 8 I only have the 'regular' test classes, on JDK 16 I have the regular ones and the java16 ones, and on JDK 17 I have the regular ones, the java16 ones and the java17 ones. I haven't found a way yet to express this kind of thing in Maven in a concise way, but I'm not a Maven expert.

Or am I looking in the wrong direction, and is it preferable to make a multi-module Maven project, with the main code in one module, and the tests in another, and then generate a multi-module jar for the tests as well? If so, how would I run this jar file on different JDKs?

Answer 1

To test a MRJAR the classes must be packaged as a jar, so don't use surefire with target/classes, but instead use failsafe during the verify phase. And you must run it at least twice, once per targeted Java version. I would write a unittest, that works for all Java versions, but might skip certain tests.

import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assumptions.assumeTrue;

import org.junit.jupiter.api.Test;

class RecordHelperTest
{

    @Test
    void isNotARecord()
    {
        assertFalse( RecordHelper.isRecord(Object.class));
    }

    @Test
    void isARecord() throws Exception
    {
        assumeTrue( Integer.parseInt( System.getProperty( "java.specification.version" ) ) >= 16 );
        
        Class c = Class.forName( "jdk.net.UnixDomainPrincipal" );
        assertTrue( RecordHelper.isRecord(c));
    }

}

How to run it twice, that's up to you, e.g. configure the failsafe plugin twice with different a Toolchain, or rely on a CI server, that builds the project with both JDKs.

https://maven.apache.org/plugins/maven-compiler-plugin/multirelease.html describes several options with their pros and cons as there is no one-fits-all solution.

Answer 2

TL;DR

A Maven multi-module solution like the one from @jqno and @khmarbaise should be preferred over this solution. The Maven Invoker Plugin should be used only when a multi-module setup is not an option.

Detailed Answer

The Maven Invoker Plugin could be used for this purpose.

Following the standard directory layout suggested by the plugin and assuming versions 8, 11, and 16 as JDK targets, JDK specific tests should be structured in separate directories:

• src/it/jdk-8 for tests targeting the JDK 8.
• src/it/jdk-11 for tests targeting the JDK 11.
• src/it/jdk-16 for tests targeting the JDK 16.

Each group should have its own POM where the Java version is specified in the maven-compiler-plugin configuration, for example by using the maven.compiler.release property.

Each test group can run on multiple Java versions without duplicating the test classes. For example, to run JDK 11 tests on both Java 11 and 16 executions:

• A test JAR should be created in the JDK 11 related POM using the maven-jar-plugin
• The test JAR should be installed in the local Maven repository used by the invoker (can be done setting invoker.goals = install for the test specific group)
• The test JAR should be declared as a dependency and added to the Maven Surefire Plugin configuration of the JDK 16 related POM, using the dependenciesToScan parameter.

Pros

• Single-module projects do not need to switch to a multi-module setup only for test executions.
• All the complexity is locked into src/it.

Cons

• Files under src/it are not IDE friendly, e.g., the src/it/jdk-*/pom.xml files are not detected as Maven projects and the src/it/jdk-*/src/test subdirectories are not detected as test sources root.

A full example is available at https://github.com/scordio/invoker-plugin-example.