Edit: I wrote this when the title mentioned "squealing" noises. Since the noise description has changed, I'll have to get to a computer with audio to see if the video helps. I left the original post intact, because it may still be worthwhile to eliminate the serpentine belt and pulleys as a noise source.
Original post:
First thing that comes to mind is the serpentine belt is slipping. Cold weather makes the rubber harder and more susceptible to slippage. After things start warming up, they become more pliable and expand a bit, which stops the slippage. You can examine the belt and see if it's cracked or frayed.
If the belt it looks okay, the belt tensioner may be loose.
Auto parts store usually carry some spray specifically for noisy belts. This isn't a long term solution, but a diagnostic tool. If you spray the belt and the noise changes, then you've limited the area of concern to the belt and tensioner.
However, if the belt looks bad, I'd replace that before spending money on spray that I'd never use again.
Another possible cause is the bearings in one of the pulleys around the serpentine belt. Auto techs use a stethoscope thing with a metal tip to narrow down those types of noises. Given that the noise goes away, I doubt it's a pulley bearing.
It's something to be a bit concerned about. The belt may come off and leave you stranded without an alternator/power steering/water pump.
