I don't either. The brake rotor surface is cleaned by the pads each time you stop. If you are talking about the hub that the rotar sits on, that should be handled by whatever wheel cleaner you use.
Also, a pressure washer does a nice job on the innards of the brake system.