AR foundation: when phone camera is covered, instantiated object is glued to the camera

Question

I downloaded the AR Foundation samples from Unity (link: https://github.com/Unity-Technologies/arfoundation-samples/tree/4.1) , and made a build of one of the scenes called ImageTrackingWithMultiplePrefabs and put it on my android phone.

The build works fine when scanning markers, but as soon as the camera is covered, the object is glued to the camera, and I have to either look at the marker again, or restart the app to fix it. This is going to be an issue because we're making an app and it could happen a lot, for example when the user puts their phone down.

I made a video to better explain the issue: https://youtu.be/BgCHbZeSWd4

More info:

Unity version: 2020.3.20f1

AR Foundation version: 4.1.9

phone: google pixel 5 (but it also happened on my Samsung galaxy S10)

I put a log in the PrefabPairImageManager.cs script and noticed that when we look away, the tracking changes to limited, and stays the same, even when the camera is covered. So there's no way for me to find out when the camera is covered in code based on the tracking info.

What am I doing wrong?

Thanks a lot