Register Autofac Service Provider Factory from services instead of CreateHostBuilder(ASP.Net 5)

Question

I am trying to use autofac to register my open bound generic Mediator handlers, which works in my main project but does not in my tests(using Mytest.Aspnet.Mvc library)

EDIT The only difference being that my project calls UseServiceProviderFactory, while the test project cannot(no API for it sadly). Upon combing through the github repo, I found these lines here

var provider = serviceCollection.BuildServiceProvider();
var factory = provider.GetService<IServiceProviderFactory<IServiceCollection>>();

Which leads me to believe that, if I can change this to AutofacServiceProviderFactory would solve my problem. What I've tried

services.AddTransient(x => (IServiceProviderFactory<IServiceCollection>)new AutofacServiceProviderFactory());

But it throws Invalid Cast Exception System.InvalidCastException : Unable to cast object of type 'Autofac.Extensions.DependencyInjection.AutofacServiceProviderFactory' to type 'Microsoft.Extensions.DependencyInjection.IServiceProviderFactory1[Microsoft.Extensions.DependencyInjection.IServiceCollection]`

ORIGINAL QUESTION

I've used breakpoints to check even if my ConfigureTestContainer method is invoked, it is but still Mediator throws Handler not found for... please register your handler with container.

The fun fact is that everything works on my ASP project.

Autofac Version: 6.3.0 MyTested.AspNetCore.Mvc Version: 5.0.0 .net 5

My constraints are to class which inherits my main layout Model.

public class Main
    {
        public class Query<T> : IRequest<Result<T>> where T : MainLayout, new()
        {
             //some props
        }
        public class Handler<T> : IRequestHandler<Query<T>, Result<T>> where T : MainLayout, new()
        {
             public async Task<Result<T>> Handle(Query<T> request, CancellationToken cancellationToken)
            {
            }
         }
    }

and this is my Test Startup class to add autofac

    public class TestStartup : Startup
    {
        static TestStartup()
            => MyApplication
                .IsRunningOn(server => server
                    .WithServices(services =>
                    {                        
                        services.AddAutofac();
                    }));

        public void ConfigureTestContainer(ContainerBuilder builder)
        {
            builderbuilder.RegisterGeneric(typeof(Sunspot.Application.Main.Main.Handler<>)).AsImplementedInterfaces();
        }
     }