How to host multiple Create-React-App production builds with NGINX, each in a separate directory (without Docker)

Question

I'm trying to find a way how to host multiple (currently 2) React applications, each created with CRA. Both apps should run behind a single instance of NGINX and be accessible in a separate directory:

app1 -> http://localhost/app1

app2 -> http://localhost/app2

The 2 React applications where created like this:

npx create-react-app app1
npx create-react-app app2

To host both React applications using NGINX, the production build of each CRA app (npm run build) is copied to NGINX static directory /var/www/app[1|2]

# App1
npm run build
cp build/* /var/www/app1

# App2
npm run build
cp build/* /var/www/app2

This is how NGINX can be setup to host a single CRA app:

# /etc/nginx/conf.d/default.conf
server {
    listen 80;
    listen [::]:80;

    server_name localhost;

    location / {
        root /var/www/app1;
        index index.html;
    }
}

Now I'm trying to extend this example so both CRA apps are hosted by one NGINX.

Which modifications are necessary, both to NGINX site.conf and each React application itself?

I pushed a code showing my (incomplete) example above to Github for reference: https://github.com/wolkenarchitekt/multiple-cra-apps-behind-nginx. The code is using docker-compose for simplicity, but in the end the whole stack should run without Docker, so running separate Docker services for each React app is not an option.