How to fix misalignment between image and plot in python?

Question

My goal is to show a binary image and then plot the boundary contours as lines overlaying the image. If I do this and export the result as a PDF, I see a progressively worsening misalignment between the image and contours as one moves across the image from bottom left. So it seems like there is a multiplicative error in the position of either the background image or the contours.

I think the issue is caused by the PDF renderer. If I output the result in PNG with a very high DPI, I can remove the problem, but I would prefer PDF for other reasons. Does anyone know if there is a setting I can change to make the PDF render correctly?

Here is an example and the resulting image. You can see that the bottom left corner has good alignment between image and contour and the top right is the worst.

import numpy as np
import matplotlib.pyplot as plt
import cv2

# Make a test image
img = np.zeros((100,100), dtype=np.uint8)
img[20:99,1:80] = 1
img = np.matlib.repmat(img, 9, 6)

# Extract contours
cntrs, hier = cv2.findContours(img, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)

# Make overlay
fig = plt.figure(figsize=(6,9), dpi=300)
ax = fig.add_subplot()
ax.imshow(img, interpolation='none', cmap=plt.cm.gray)
for cntr in cntrs:
    x = np.append(cntr[:, 0, 0], cntr[0, 0, 0])
    y = np.append(cntr[:, 0, 1], cntr[0, 0, 1])
    ax.plot(x, y, c='r', linewidth=0.5, alpha=0.7)
ax.axis('off')

# Save overlay
plt.savefig('test.pdf', dpi=fig.dpi)