Yep, almost certainly a vacuum leak.
The booster itself is a pretty dumb part, a vacuum vessel, rubber diaphram and a check valve. It's usually the internal diaphram which fails, but for the Boxster, you can't repair/replace it; you must replace the whole part - 996.355.923.00 (for 986) and it isn't cheap.
But there is also a gasket on the housing which the vac line plugs into and these too can leak so it's worth checking them b4 forking out for the whole unit.
From the way you describe it and the sound, it does sound like the a leak in the line or gasket, so check these first.
