So I ended up driving a few blocks at low speeds without any power steering fluid in the reservoir. It was only 4 turns. Would this dumb mistake have killed the power steering pump or the belt?
If ps fluid was still in the pipes that may have been enough to save the pump, if you had drained the system of fluid completely then you may have damaged the pump or reduced its life.
If the pump is noisy now or becomes noisy then you will need a replacement.
For the belt - if the punp seized - partially or fully and that has damaged the belt the it needs replacing but if it is still in good condition it should be fine. If you have to replace the pump then, personally, I would fit a new belt as well - the belt is cheap...