Definitely don't need the magnets, you could even replace the magnet in the trigger with 1/8" round metal and still adjust the frame magnets.
I'll check the mounting screw and I think I may have a new safety I can try. The current safety is older so it is possible that could be causing it.
Check to see if the T-Rex mounting screw is loose.
Also, if you have a different safety try it.
These are extremely close tolerances