Matplotlib: Making axes fit shape limits

Question

I'm trying to draw a rectangle in matplotlib using the following code:

import matplotlib.pyplot as plt
import matplotlib.patches as patches

fig, ax = plt.subplots()
width = 20
height = 10

rect = patches.Rectangle((0,0),width, height, linewidth=4,edgecolor='r',facecolor='none')
ax.add_patch(rect)

plt.show()

Which results in:

The axes do not fit the rectangle limits in this case. I could solve it with:

import matplotlib.pyplot as plt
import matplotlib.patches as patches

fig, ax = plt.subplots()
width = 20
height = 10

ax.set_xlim(0,width)
ax.set_ylim(0,height)

rect = patches.Rectangle((0,0),width, height, linewidth=4,edgecolor='r',facecolor='none')
ax.add_patch(rect)

plt.show()

This gives me the following picture which solves the problem in this case:

However, as I am trying to plot many rectangles and other shapes in the same figure, I need a way that matplotlib smartly determines the proper axes limits itself, like the way it does when plotting normal diagrams.