Getting EF to automatically create 2 databases when running in Azure App Service

Question

I recently just deployed a web app in Azure App Service. This web app uses 2 separate database (2 connection strings) to be able to function fully. Upon deploying and browsing the web app, the first database got automatically created but the 2nd one was not. I was expecting for both to be automatically created but I can't make it happen for the 2nd database.

I checked the connection strings and it is correct. What do I need to so the 2nd database gets created as well?

EDIT:

Code was requested but I am not really sure what relevant code I need to post. This is the only thing I can think of that might be involved in the database creation.

in ConfigureServices:

services.AddDbContext<FirstDbContext>(options =>
            options.UseSqlServer(Configuration.GetConnectionString("FirstConnection")));

        services.AddDbContext<SecondDbContext>(options =>
            options.UseSqlServer(Configuration.GetConnectionString("SecondResourceConnection")));

Not sure if this is the correct code I need to post relating to this.

EDIT 2:

One thing to note is that this worked in my local, both DBs were created.