Well I didn't think and evap issue would cause the hard starting problems but after a lot more research I found that I need to replace the "Regeneration Valve from Throttle Body for Fuel Vapor System" PN 996-110-129-06-M100.
I didn't relate it to hot starts because I could immediately start the car right back up after having trouble initially. It seems that it only affects starting from around 30 minutes to a couple hours after shutdown. Cold starts are usually fine.
I'm hoping this should clear the O2 sensor codes as well because other owners had these exact codes with the valve issues.
Hot Start Problem, Code P0446 - Page 2 - Pelican Parts Technical BBS Success! Finally fixed the hard start while hot and fueling issues.
http://986forum.com/forums/general-discussions/25572-99-engine-dying-after-normal-startup.html