As mentioned, use a clay bar. Removing paint overspray
safely is one of the primary reasons clay was adopted by the auto body repair industry. I would be leery using a lubricant such as WD-40 as you might have some paint adhesion issues in the future.
Realize that the clay will also remove any wax or acrylic protectants and these will need to be reapplied.
Lubricate the clay and surface well with soapy water, and fold the bar often. This should work fine with little risk of ruining the paint.
