I did not remove the bumper. You'll have to lift the car a bit higher, but it can definitely be done without.
I'll second obtaining the triple square tool (I had never heard of such a thing either), and new exhaust hardware. Exhaust hardware leads a very tough life. Even if you get it all off successfully this time it will be stuck bigger than crap next time you're in there. Just throw it out and get new.