It's because of a couple things: The battery is very dead and the alternator cannot put out enough power at idle RPMs to run the whole system and charge a battery. This leads to a lack of power which may kill one or more systems most likely the spark "ignition" system since this takes the most power and is necessary to run the car.
In Fact you can remove the battery all together making sure the positive cable is insulated and run the car that way too. At Least once you get it started.
Andre Borie has a good point. removing the battery can potentially cause damage to some of the more delicate electronics in the car. This is due to the battery acting like a big capacitor smoothing out some of the larger spikes in the electrical system. These spikes can originate from many different sources: the alternator, power surges from the coil packs, or even a wire crossing something like a spark plug wire causing some induction.
kmarsh below has a good point here is a video on how a self starting alternator works and if the car has been sitting too long the alternator can go flat too. Next time you think your alternator died try jumping it and forcing it on.