It's not time that wears the pads off, it's friction. Effect of friction increases with temperature. Braking itself causes brake components to heat up, and cooling of brakes is quite limited. Heavy braking causes lots of heat to build up for which there isn't enough time to dissipate, so the pads will be softer from heat and wear more. In more severe cases, the heated discs/drums might bend out of shape under the tension, or melted pad material may stick to them; the uneven surface will then cause more wear on subsequent braking attempts.
Heavy braking also increases chances the next vehicle will collide into yours. Even though your expenses might be covered, is all the trouble and wasted time really worth it?
Light braking has to be started earlier, so braking lights will light up earlier, giving fair advance warning to the driver behind, and lifting the foot from throttle earlier also helps in saving fuel - your engine will be revolving on cars kinetic energy (manifested through engine braking, which also takes some load off brakes) instead of burning fuel or batteries.
Quite often, when braking lightly well before a red light equipped with approaching vehicle sensors, the light will change just in time so that I don't even have to brake all the way to stop and then start off again, but instead can slide on just by slightly adding throttle again.
Economical driving should be striving for smoothness.