Lugging an engine is like hammering the engine parts with every explosion in a cylinder. It racks on the rod bearing/journals, makes the pistons slap the side of the cylinders hard, and if is done enough, could probably break piston rings. It also creates a hammer effect all the way through the drivetrain. There are springs in the friction disk (of the clutch) which takes up some of this hammering, but they will only take so much before they wear out as well. This causes premature wear all the way around. It's not like it will wear out tomorrow, but it will cause wear over time. If you want you car to last a while, you don't want to be doing this.
I think the major problem with what you are doing is, you aren't doing it right. It sounds like you are just trying to take off from a stop (or slow roll) without reving the engine at all. You should bring the engine up in rpm to around 1500 to 2000 rpm when starting to slip the clutch. You should get used to slipping the clutch in either situation (stopped or slow roll) no matter where you are at. To get it down, you should press on the pedals like you would a bicycle. Basically, as you press on one, you should allow the other to come up, so: gas pedal down slightly while allowing the clutch pedal to come out; and visa-versa - clutch pedal down while allowing the gas pedal to come out. Over time and experience, this becomes second nature. You will figure out (mainly by feel) where the engine needs to be at in speed in order to get the car moving. This will keep you from lugging the engine at all.