First, everyone who said that the braking effect comes from the compression stroke is wrong...the air in the cylinder is compressed which takes energy, yet after top dead center acts as a spring and helps force the piston back down, returning the exact same force as was put into it in the first place. Probably more, actually, since the compression heats the air charge up and causes it to expand, like a very minor version of a power stroke.
No, the braking effect comes from the INTAKE stroke, from the engine drawing air through the closed off throttle-body. It's like trying to suck air through a tiny straw. Braking is from vacuum, not compression. That and a bit of engine friction, which is less significant with modern engines than it used to be.
As for using engine braking, using it mildly for keeping speed on downhills isn't really bad for your car. I'd avoid driving like a rally racer and aggressively downshifting to slow down for every turn or stoplight, but when done right, it doesn't effect anything any differently than driving normally. Yes, it causes more engine revs, but they are no worse than the revs you get accelerating onto an on ramp. If you're planning to keep the car for 250,000 miles, yeah, you might want to keep the revs to a minimum, but otherwise, assuming nothing is wrong with the motor itself, the car will be done-for long before you wear the engine out by adding revs.
The same applies for the clutch...yes, every time you use it, it wears a little, but no differently than using it during normal driving. If you're worried enough about the clutch that you try to keep your car in one gear as much as possible, sure, avoid engine braking. But if you're like most people, you use the clutch 5 times every time you get up to speed, two or three times every time you leave a stoplight. A few more engagments isn't a big deal. And you don't always have to use the clutch; often you just have to let off the gas and leave it engaged and it will work to slow you down.
It will add some wear and tear to the drive train, but like the above, you could consider it the same as driving an extra half-mile every day, only without the fuel cost. If that wouldn't scare you, neither should engine braking.
The only thing that wears out differently is, as the other poster said, the rear faces of the gear teeth, and that really doesn't matter. You use them every time you back up, the rest of the time they do nothing. IF they should wear significantly, the only effect will be that the transmission will be louder when your reverse. And you'd have to put a lot of wear and tear on them to get them that worn out.