Multiple CollectionDefinition per test class in XUnit

Question

We have many XUnit tests in our project that we use to test our API (each with many InlineData). The problem is that we can access a subset of the API via two other connection methods. Today we use a CollectionDefinition to keep the connection to the main API (it is time consuming to create the connection) and we intend to have three separate CollectionDefinition, one for each connection path.

I imagine I will create a new attribute I can add to each test class to tell which connection methods it should use. And then it uses the CollectionDefinitions that are connected to the various connections.

I am considering using [assembly: Xunit.TestFramework ("name", "assembly")] and implementing my own XunitTestFramework. But I can not figure out how to achieve what I want. Do you have any suggestions on how I should proceed?

Answer 1

You can accomplish this as follows:

Make an abstract class that defines the methods common to all your API's, and derive specific classes from it which create the specific connection. Make a CollectionDefinition and a fixture for each type of connection:

using Xunit;

namespace SO71300821_multipleCollectionDefinition
{
    public abstract class APIConnectionMethod 
    {
        public bool isOK { get; } = true;
    };

    public class APIConnectionMethod0 : APIConnectionMethod
    {
        public APIConnectionMethod0()
        {
            // ... initialize API connection ...
        }
    }

    [CollectionDefinition("Collection0")]
    public class Collection0 : ICollectionFixture<APIConnectionMethod0>
    {
        // This class has no code, and is never created. Its purpose is simply
        // to be the place to apply [CollectionDefinition] and all the
        // ICollectionFixture<> interfaces.
    }

    public class APIConnectionMethod1 : APIConnectionMethod
    {
        public APIConnectionMethod1()
        {
            // ... initialize API connection ...
        }
    }

    [CollectionDefinition("Collection1")]
    public class Collection1 : ICollectionFixture<APIConnectionMethod1>
    {
        // This class has no code, and is never created. Its purpose is simply
        // to be the place to apply [CollectionDefinition] and all the
        // ICollectionFixture<> interfaces.
    }

    public class APIConnectionMethod2 : APIConnectionMethod
    {
        public APIConnectionMethod2()
        {
            // ... initialize API connection ...
        }
    }

    [CollectionDefinition("Collection2")]
    public class Collection2 : ICollectionFixture<APIConnectionMethod2>
    {
        // This class has no code, and is never created. Its purpose is simply
        // to be the place to apply [CollectionDefinition] and all the
        // ICollectionFixture<> interfaces.
    }
}

Make an abstract base test class that implements all the tests that need to work for each of the various API's:

using Xunit;

namespace SO71300821_multipleCollectionDefinition
{
    public abstract class UnitTestBase
    {
        APIConnectionMethod connection;

        public UnitTestBase(APIConnectionMethod connection_)
        {
            connection = connection_;
        }

        [Fact]
        public void connectionIsOK()
        {
            Assert.NotNull(connection);
            Assert.True(connection.isOK);
        }

    }
}

Derive a public class from the base test class for each type of connection:

using Xunit;

namespace SO71300821_multipleCollectionDefinition
{
    [Collection("Collection0")]
    public class UnitTestSpecific0 : UnitTestBase
    {
        public UnitTestSpecific0(APIConnectionMethod0 connection_) : base(connection_)
        {
        }
    }

    [Collection("Collection1")]
    public class UnitTestSpecific1 : UnitTestBase
    {
        public UnitTestSpecific1(APIConnectionMethod1 connection_) : base(connection_)
        {
        }
    }

    [Collection("Collection2")]
    public class UnitTestSpecific2 : UnitTestBase
    {
        public UnitTestSpecific2(APIConnectionMethod2 connection_) : base(connection_)
        {
        }
    }
}

If there are tests that only work for certain specific types of connections, you can implement them in these classes.

The test discoverer will find the tests in UnitTestBase in each of the derived classes and run them in those classes, so you'll have a separate test in the Test Explorer for each of the derived classes.

The information from https://xunit.net/docs/shared-context helped me prepare this answer and some of the code is copied from there.