I'm just going to guess... and based upon what PW suggested...
Are there instances wherein you are listening to the stereo at a normal, conversational level, volume?
Are there instances where you might be blasting some metal (ROCK)... and that's when the fuse blows?
Maybe at lower volume there's less amps running. But, when you're blasting those amps go up. I don't know. I failed radio electronics late 70's. Trade institute. Couldn't wrap my head around it.
Have you considered installing, temporarily or otherwise, a different stereo and seeing if the problem repeats? That might be the ticket.
My $00.02