Could your impression of the "sometimes works" remote operation be after 5 days or more not driving the car? If so, you may be encountering the "battery saving" feature of the security system (turns off the remote "standby" after a few days to save power).
I just did a little experiment to help me think this through - removed the battery from my key (Panasonic CR 2032 / 3V, BTW) and could open (of course, manually) and start the car. So it appears there is no "magic" with the key talking to the ignition system, etc. relative to starting the car. So the key's battery health shouldn't matter.
You don't mention if you have a tiptronic or manual tranny - but the clutch switch is another suspect if you have a manual. If not fully engaged or working properly, the car won't crank. Similarly, for the tip, there is a sensor or interlock for Park/Neutral relative to starting which would operate similarly. If I were debugging my own car, I'd probably check these out next - they should be easier to get to and check out than the ignition key switch.
-Don
BTW, you may or may not know that you only get one shot at cranking the engine without having to turn the key all the way back to the left... unlike most american and japanese cars. We had some "fun" with this when a friend stalled my car in traffic - it "wouldn't start" repeatedly... until turned *all* the way off. And then the steering locks, so you don't want to be rolling, like we were at the time...
