It can actually be the solenoid or the brushes which are at fault and take to getting a beating ... those masochistic machines! They just like it for some reason, lol.
Most solenoids are built with a large copper washer in the end of them. When the solenoid is energized, the plunger is sucked into the solenoid via electromagnetic forces and pushes the copper washer into two large contacts. This allows current to flow to the starter motor itself, which spins the armature. If the washer or the two contacts get "bad spots" in them, contact will not be made and electricity doesn't flow. Rapping on the starter can move the washer slightly where good contact can be made and everything works correctly until the washer comes back around to the bad spot (the washer can spin freely inside it's solenoid home).
Likewise, the brushes inside the starter can suffer a similar fate. Since the brushes ride on the commutator, they wear out over time. Here are a couple of images of what they look like:

Usually in the its the commutator which will have a "bad spot" on it. This will cause the starter motor not to turn, because electricity cannot flow correctly. Banging on the starter can sometimes give enough of a jolt to the brushes where they will once again have the correct contact with the commutator and will allow electricity to flow.
As an aside, I once had a '91 Chevy Suburban which the starter died on it. In this application, banging on the starter didn't work. I took the starter apart to find that brushes, which were held onto swing arms with screws, had worn down to the screws (a couple of the screws were riding on the commutator). I replaced the brushes for about $4 and the starter worked like new again.