The engine operates on high compression to ignite the fuel at very low RPMs which gives it it's distinctive sound. The lower RPM makes the ignition of each cylinder easier to hear and the compression ignition system gives it it's rattle like sound. This is as opposed to gasoline engines that operate by a spark/compression ignition system. The key to understanding this lies in understanding the diesel engine. See: the Wiki page for this.
In short, the distinctiveness of your typical diesel sound comes from a combination of it's properties of low RPMs and the high compression ignition system. The actual sounds are generated at combustion and the opening and closing of valves. You really cannot her the injection of the fluids into the chambers. Also, as another poster fist pointer out (DucatiKiller), there is sometimes a piston slap that make a audible sound but not always in newer engines.
The high pitch hiss noise you hear from some diesels as they accelerate comes from the turbo charger turbine spinning up and sucking in air at a high rate. Don't mistake the hiss I refer to with the air-brake system, as this is altogether something else.
See:How a Turbo Charger works.