There are a lot of rotating parts in the engine, e.g. shafts, etc. Below idle they enter their critical revolutions, then they start to wobble (Layman's term) and you feel it inside, in some case in the trucks and tractors you can even see it from the outside, especially when they have troubles igniting the engine (then the whole vehicle is almost jumping).
The above can be reinforced by the non-periodic engine ignition, when cylinders are ignited at very low rates and they almost fire one by one.
To answer your second question after the edit:
Idle speed of engine is determined by the characteristics of the engine's largest rotating parts camshaft, crankshaft, etc(such that they avoid critical RPM's) and by cylinder number and arrangement, i.e. straight engine, V - engine, W - engine, Wankel - engine (their ignition sequences are different and there is a lower bound on the rate of firing of the cylinders).