There is some misleading info here.
The deteriorated foam absolutely
will affect both heat and AC, as James pointed out. The flap in front of the heater core, which is always hot, either makes air blow over the core for heat, or blocks it off. If that flap is full of holes, your AC will partially blow over that hot heater core.
Also, as Tom said, it's nice to keep a clean interior and not be pummeled with foam bits that turn to black tar when you touch them.
