Finally solved this!
I must have had the door card off at least 5 times, putting various bits of foam in places, and the no avail.
It turns out, the door lock had come slighty loose, allowing it to rattle around in the door. Fixed by tightening the torque screw inside the door skin.