React component library can not load its own static asset

Question

I have a react app (using create react app) and then a component library (using webpack directly). One component in the component library needs to load a png file in one of the components that it exports. In the webpack config for the component library I have a section such as:

      {
        test: /\.(png|jpg|jpeg|gif)$/i,
        type: 'asset/resource',
      },

This successfully results in a file like: 29b6b66cf1a691be2b3f.png in the component libraries output directory. The issue, is that when the application uses that component and the component attempts to load the image, the img element is <img ... src="29b6b66cf1a691be2b3f.png" /> and that fails to load, since that image actually lives in the component library folder (within the react application's node_modules/component-library/ at that point).

I have scoured the internet to the best of my ability, and can not seem to figure out what the best solution would be here. Any help is appreciated. I will quickly offer clarification if needed.

UPDATE: I have discovered CopyWebpackPlugin but it is quite unfortunate that this would require me to eject the "parent" application from create react app. Something I would very much prefer to avoid.

UPDATE2: Current plan is to try following something like what is explained here. The jist of it is to utilize something like rewire to avoid needing to eject and still be able to edit the webpack config via something like:

// in ./build.js
const rewire = require('rewire');
const defaults = rewire('react-scripts/scripts/build.js');
const config = defaults.__get__('config');

// edit webpack config values here

... I will answer my own question here if I find that this approach works.