How to set the http.server module to use a custom 404.html page?

Question

I want to change the default 404 page of the python3 module http.server to a custom 404.html page, so it will redirect the client to that page.

I have confirmed the 404.html works, but putting a 404.html in the web root doesn't help.

Any solution?