Not to worry. Your car uses refrigerant in a closed loop system to keep things cool when its hot outside.
The refrigerant system works on pressure. There are sensors that keep things sane. At ultra cold temperatures, the pressure is low, and the compressor is disengaged from pumping. At ultra high pressures, same thing, the compressor is disengaged from pumping. (Think 110 deg F, hauling a trailer up a steep mountain. The control computers turn off the A/C to protect the engine from overheating.) Normal operating pressures within the system range from 30 psi to 350 psi and higher.
One thing that's pretty cool (pun intended). The cold evaporator core helps remove water vapor from the air. In fact, when your vehicle is placed in the DEFROST mode, the air conditioning (A/C) is automatically engaged. This removes water vapor and keeps the inside of your windshield clear. Typically this works behind the scenes from around 40 deg F and above. Below that, the A/C compressor should NOT be engaged.
Note that all air in the heating ventilation system is routed through the evaporator core. Thats the part that gets cold when you are driving in the summer.
Now lets talk the red/blue temperature control. Remember in your car there is hot water flowing to the heater core ALL the time. That red/blue control moves a little door (blend door). When it's set to full red, then all the air inside the heating ventilation system is routed thru the heater core. At full blue, no air is routed through the heater core.
Your A/C refrigerant system is totally protected in cold temperatures.
You may well have some other issue related to the red/blue temperature control. I'd expect full blue (full cold) to be just a little bit warmer than outside air temperature (there is a little bit of heat given off from the engine that affects the air you are trapping just in front of the windshield)
Note: I'm assuming you have the recirculation setting (inside/outside) set to Fresh outside air. The recirc/inside/Cabin air setting is bad to be using in cold weather, particularly if its raining or snowing outside. If that setting is set to inside/ Cabin Air (recirculation) you are not getting air from outside, instead you are just recirculating air within the passenger cabin. If there is snow on your boots, that will melt, and pretty quickly fog up the inside windows, affecting visibility. This is unsafe.
Easy way to quantify the issue? Use a thermometer. Heres the best one for the money. I used to do a whole lot of work measuring temperature, and I broke about ten different thermometers. That one in aluminum housing, is just awesome. Always works, even if you drop it on cement.