Finding the angle of rotation relative to a normal of a 2D image with TensorFlow

Question

I am new to TF, so what is the general process of finding the angle of a 2D image? What I want to do is find the angle of rotation a particular object real-time.

Here's an example of what I am trying to achieve:

and every angle in between (to at least .1 angle).

I am assuming that I might need to detect the base image (angle=0) then compare any other rotated images to it?