Remove event listeners from all nodes except

Question

I am trying to build a utility function that follows the following criteria:

• (selectors: string[]) => void
• Removes all event listeners from the page, except for on elements matching the selectors provided in the array
• Doesn't use the getEventListeners function in Chrome's API

Some important factors:

• I don't know or have access to the functions that were passed in when adding and event listener to each of these elements, so removeEventListener is out of the question
• Ideally, I don't want to inject a script prior to the page's scripts running to override the addEventListener method. But if that's necessary, then so be it.

Here's an example of something I've already tried:

// only using "getEventListeners" because I'm testing this in devtools
getEventListeners(document.querySelector('#server')) // has event listeners

const removeAllEventListenersExcept = (selectors = []) => {
    // loop through all dom nodes
    for (const node of [...document.querySelectorAll('*')]) {
        // if it matches one of our selectors, skip it
        // this gets logged, so why is the listener still being removed?
        if (selectors.some((s) => node.matches(s))) return console.log('hi');
        
        // else, make a clone of the node (which removes its event listeners)
        const clone = node.cloneNode(true)
        // replace the one in the dom with the new clone
        node.parentNode.replaceChild(clone, node)
    }
}

removeAllEventListenersExcept(['#server'])

getEventListeners(document.querySelector('#server')) // no event listeners on it

Here is my implementation using getEventListeners that works nicely (focusing on click listeners); however, once again, I can't use this:

const getAllWithClickListeners = () => {
    const elems = Array.from(document.querySelectorAll('*'))
    elems.push(window)
    elems.push(document)
    const listeners = []
    elems.forEach((elem) => {
        const result = getEventListeners(elem)
        if (Object.values(result).length && result.click) {
            listeners.push({
                node: elem,
                ...result
            })
        }
    })
    return listeners;
}

const removeAllClickListenersExcept = (selectors = []) => {
    const allWithClickListeners = getAllWithClickListeners()
    for (const { node, click } of allWithClickListeners) {
        if(selectors.some((s) => node.matches(s))) return;
        for (const listener of click) { // click === array of the node's click listeners
            node.removeEventListener('click', listener.listener, listener.useCapture)
            node.removeAttribute('onclick')
        }
    }
}
removeAllClickListenersExcept(['#server'])

All suggestions welcome. I've been scouring the internet for ideas, so please don't snipe my question just 'cause the title is similar to other questions. Even my project manager is stumped.