Yes you have heard someone who owned both say less HP was more fun. Heck my 85HP 914 was more fun that either but then I was much younger then... I could keep my foot down longer and harder without attracting the government's attention (In the '99 201HP and '01S 250HP I never got a ticket in Arctic Silver cars, not saying there weren't times I didn't deserve one.).
Now are there suspension, brakes, engine power improvements in the S? Yes. See
here.)
The big difference comes in the '03 and I'd gladly have the 228HP base '03 over my '01S 250HP S. But you can change the entire roof assembly from an '03-'04 onto a earlier '97-;02 chassis. Not so the glove compartment.
The downside of the S is the wear in the driveshaft CV and CV boots due to the different angle the 6 speed imposes. A matter of personal preferences is the ratios of the 5 speed from the base are often preferred to those of the 6 speed of the S.