Did you mask out motion?
Yes
Masks are necessary when:
There is a prominent moving subject (person, car, leaves)
There is a reflective surface (metal, windows, etc)
There is a sudden lens flare
Tricking CameraTracker into doing an object solve