Well just because the ignition switch isn't sticking doesn't rule it out. They can go tits up and never stick.
But, I too am suspecting the starter. This is pretty classic when the pinion gear and the solenoid plunger get gunked up with dirt and dust. The solenoid isn't available separately and the whole unit is fairly expensive. Consider having it rebuilt locally for half the price. On many cars, pulling it and cleaning it with solvent and compressed air, then re-greasing it can give it a new life, but since it's not the easiest swap, I'd opt for a rebuild straight away and avoid possible added frustration should it only continue to work a short time.
Now, there is something else also creeping in my thoughts and that has to do with the new alternator, well two things actually.
The first is the belt. It could squeak, especially on startup and since the alternator was swapped, it may be tensioned differently than before (read probably is). This could lead the idler or tensioning pulley (either or both) to have developed a bad bearing.
Next is the new alternator (it's bearing actually) which cannot be ruled out either. Just because it's new, or rebuilt, means little. Mechanical devices usually fail either in their early life (due to mfg., quality control issues) or in their old age (simply worn out).
Since any of these possibilities involves some $$ and some semi-serious wrenching, I'd pull the engine firewall cover and try to isolate it before just throwing a random fix at it. It's the smartest way to go.