The secondary air pump will turn on under two situations: (1) At cold start for 90 sec to reduce emissions, and (2) to test the pressurization integrity of the SAI and fuel system.
The pressurization test usually happens the first time that the car is at idle as the engine warms up. However, this test can be delayed substantially if the car never has an opportunity to idle long enough for the test to complete. For example, the car is started and driven directly down the street and stops at a light, but the light turns green and you drive away before the test can complete.
There is no absolute need to replace the SAI pump unless you can't stand the whine or if the pump fails and you need it working to pass a smog test (and can live with a CEL). The engine might run a little rough during cold start but nothing else bad will happen (except you'll be polluting more at startup).
You can find an affordable used SAI pump from Woody or eBay (here is one for $50:
996/986 SAI Pump). They are easy to replace yourself.