The culprit in this specific case was the steering pump pulley - it developed a slight play along the axis and the auxiliary belt started rubbing the sides of the pulley ridges.
Since no one has a more comprehensive answer for similar issues I can try answering it myself now that I am more knowledgeable in this area.
Causes of the noise
The auxiliary belt can produce an annoying squeaky noise for two main reasons:
- Slipping on the pulleys;
- Rubbing a misaligned pulley.
Which one it is can be determined from the noise it makes. If it is a constant "squeal" that occurs or increases in volume when the RPM is increased (especially when the engine is cold), the belt is likely to be slipping on the pulleys, making a god-awful noise. If the noise is a more subtle periodic "chirp" - the culprit is probably the belt ridges/sides excessively rubbing the groove sides of the pulley.
To be sure there is one easy test that can be done. Take a spray bottle of water and lightly spray some on the belt and on the pulleys while the noise is occurring. If it suddenly goes silent - it indeed is a misaligned pulley. You just lubricated the part that was rubbing and the noise went away, unfortunately it will come back soon. If the noise suddenly got louder - your belt is slipping and it temporarily started slipping even more.
You can also inspect the belt itself if you're still not sure, but it usually requires the belt to be removed. If you see fraying on the edge of a belt this usually indicates rubbing a misaligned pulley, if it was happening for long enough. Glazed grooves or edges of the belt indicate that the belt was overheating from excessive slipping.
Causes of slipping
The most basic cause for slipping would be a simply worn belt. It usually shouldn't cause the belt to slip, but it might it if wasn't replaced after previous repairs. The only way here is to replace it.
Another simple cause is fluid contamination. Coolant, oil, power steering fluid and brake fluid can seep into the ridges, attracting dirt and causing slippage. Cleaning the belt and pulleys off with a rubber friendly degreaser should help.
If the belt is in pristine condition and dry, yet still slips - the reason is most likely insufficient tension provided by the belt tensioner. Usually on newer cars the tensioner is spring loaded and retains tension automatically. If the tensioner does not do its job - it needs to be replaced. I believe manual tensioners exist as well, which means they might need to be adjusted as the belt ages and stretches.
A temporary fix for a slipping belt could be belt dressing, as suggested in the comments. It's a sticky substance that basically increases the friction between the belt and the pulleys, reducing slippage. However it also attracts dirt, which will need cleaning off the pulleys when the belt will finally be replaced.
Causes of rubbing
If the belt is rubbing a misaligned pulley, then one of the pulleys has a different offset (parallel misalignment) or a different angle (angular misalignment) than the rest. Parallel misalignment occurs when the pulley is not pressed onto the shaft far enough or pressed too far. This can be checked by placing a straight ruler across adjacent pulleys - they should be perfectly parallel. There might also be lateral looseness in the pulley because of a failing bearing, which would not be shown by this test. If the misalignment is angular, it is likely caused by a worn bearing as well. In either case a bearing problem can be found by taking off the belt and grabbing each of the pulleys, giving them a few tugs and shakes - there should be practically no wobble.

(borrowed from flowcontrolnetwork.com)
If there are any inaccuracies in this text - please feel free to edit, my knowledge might be flawed here.