Today I 'inspected' the steering rack. I was mainly searching for the part number to ensure I order the right boots. Didn't succeed in that, got the 996 and a bit, but not the last numbers that actually matter.
Anyway, I noticed the driver side being dirty;

Having read that the main failure of the racks can be found by a boot full of power steering fluid, this did scare me a bit. I popped the boot off but found nothing but a greasy rack
I was now able to rock the tie-rods in different directions on both side. Passenger side, no play found, no noises, all good. On the drivers side I could get the clunk by rocking the tie-rod up and done (fore-aft was silent). It didn't sound like it was coming from the tie-rod itself. With the boot shifted aside I could see the rack (to toothed part) move along with the clunk.
So I guess I am looking at getting a replacement rack....