well in theory you're supposed to do the sandwich bag test to see if your paint requires clay at all. Place hand in bag and run fingers across the paint to see if there is any drag,bumpy lumps vs. smooth gliding.
Claying only "shaves" the top surface. If you have swirls that haven't been corrected you may have pollution and particles trenched in the swirl channels. Using Menzerna with an orbital buffer will flatten the paint surface so the particles have no place to hide. If you don't want to go the buffer route you can try paint decontamination which flushes out the swirls channels.
FK1 1119/883 is one popular decontamination kit for about $50.
p.s.
if you are claying and nothing is coming off that's a good thing. Kinda like spraying for roaches before you get them.