How to find regions in a graph from an image in Mathematica?

Question

Our aim is to find the shortest path to an exit. We have an image of a floor plan where we have painted the exit yellow. Then we turn all the walkable space (including said exits) into a mesh and then into a graph using MeshConnectivityGraph. When doing this we lose all information about where the exit points are, but we need to recover that information to find the shortest path.

We have stored the pixel value positions of the exits, but they have nothing to do with the vertices "naming".

Our code works the following way:

1. We apply some Image Processing and store the exit pixel position values.
2. We apply TriangulateMesh to the processed image.
3. We create a graph using the MeshConnectivityGraph function. The vertices labels seem to be random and have nothing to do with the image.

How can we find the vertices that are exits, marked in yellow in the original image and saved as pixel value positions?

We have thought of naming the vertices based on their pixel value position in the original image, but we do not know how to do this as the graph comes from the mesh and not the image. Any other ideas is welcomed.