So I got the car in the air and re-checked the torque on all the bolts on the front cross member. They were all tight, except the two that bolt the rear of the cross member to the body - those could take some more torque (up to spec).
Symptom gone
Took it to alignment because I replaced the Tie Rods. Those I did not do during the suspension rebuild, so hoped these would eliminate it, but it did not.