I believe what at least partially is causing this is the mouse and TrackIR "freelooks" become desyncronized and each have their own "center". A temporary way to work past this is to simply unbind the mouse from the four freelook directions. This leaves them always in sync and stops the centering discrepancies.