The water pump is a mechanical pump that is one of the belt driven devices in your engine.
This pump is what circulates the coolant (when the parts are referred to for water, they mean coolant. Water hoses, water outlet, water inlet, water pump, etc are all carrying coolant).
The water outlet is simply a pipe, on one end of it is the engine block, on the other end is a hose that is connected to the radiator.
There is both a water inlet and a water outlet, these are the two points in which the coolant flows in and out of your engine block.
Since yours was a water outlet "housing" it may be the location of your thermostat. On my engine, the thermostat housing is connected to the water inlet, not outlet. I'm sure manufacturers use different locations and naming theories all the time though. The housing contains the thermostat which is a temperature sensitive device that determines whether your coolant cycles through the radiator or not.
When you've first turned on your vehicle, you want it to get up to operating temperature, so you don't yet want to be cooling the coolant through the radiator, so the thermostat keeps it blocked off until reaching a certain temperature. Once it reaches that temperature it opens up allowing the coolant to flow through the radiator to get cooled.
The water outlet could leak due to corrosion, some failure point in the metal or to a faulty gasket.
Depending on your vehicle and the shop you took it to (I'm guessing dealership), $1200 doesn't surprise me to replace it. The water pump is typically done at the same time as the timing belt because you have to take off almost all of the same parts to get in there.
I'll be changing my water pump on my car this weekend just because I'm already going to have my radiator out and hoses off. The water pump if I had purchased OEM would have been $100 instead of the $30 I'm paying for an aftermarket one so there is some difference there but most of that $1200 is probably labor.
