How to design to avoid duplicate column in two models that have the same ForeignKey?

Question

I have the following models where a User has many Photo objects and many Album objects. However, after a User has added a given photo they may choose to put it in a single Album (a Photo does not have to have an album linked to it). They may also choose to take it out of the album and or put it in a different album.

class User(models.Model):
    id = ...

class Photo(models.Model):
    id = ...
    user = models.ForeignKey(User, ...)
    album = models.ForeignKey(Album, null=True...)

class Album(models.Model):
    id = ...
    user = models.ForeignKey(User, ...)

How do I get around the redundant user column (Album.user and Photo.user) when a Photo is associated with an Album to avoid db anomalies?

I can't get rid of the user column on Photo since I wouldn't be able to get it if it is not linked to an album. The same goes for Album. I have some experience with normalizing tables but I don't know how to avoid this without using db constraints.