How to identify the right assembly to reference when abstracting logic to class libraries and choose between referencing an assembly or NuGet package

Question

I am attempting to abstract some of our common WebApi bootstrapping logic into projects that all WebApis can reference. The general idea being that all the builder.Services.Blah() and app.UseOtherBlah() calls are standardised across our projects.

My assumption was that I would just create a class library and reference any assemblies I needed.

But I'm running into confusions due to the class library obviously not being a Web API project that's been all nicely set up for the web bootstrapping process.

The code below exists in a class library project as the only class. It is my attempt to abstract out the logging configuration. It doesn't do anything useful at the moment as I am just proofing the concept of abstracting this out into a class library.

using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;

namespace XXXX.Hosting
{
    public static class ExtendILoggingBuilder
    {
        public static ILoggingBuilder ConfigureLoggingProviders(this ILoggingBuilder logging, ConfigurationManager configuration)
        {
            // TODO:  load providers from config

            logging.ClearProviders();
            logging.AddConsole();

            return logging;
        }
    }
}

I need references (of some kind) to Microsoft.Extensions.Configuration and Microsoft.Extensions.Logging.

VS offers me the choice for Microsoft.Extensions.Logging between an assembly and a NuGet package.

But for Microsoft.Extensions.Configuration, I only get offered a NuGet package.

When I install the package, the dlls inside are not the dlls listed in the documentation page for the ConfigurationManager class here.

And I can't find the dll listed anywhere on my disk. Nor is it offered via NuGet.

My question is:

How do I work out a) what dll I need and b) where to get it from?

I don't really want a mixture of NuGet package references and assembly references. And I don't really want to reference anything I don't need to reference.