Whatever you do, don't try to unclog the nozzles with anything solid. This is a sure way to break them - you may end up washing the windshield of the car behind you. Even dislodging ice from around them could be tricky because it could change their alignment.
If the clog is caused by ice, my suggestion is to find a heated parking place (like in a shopping mall or supermarket) and leave your car in there for a while (driving in the cold doesn't heat up all of your car, just the passenger compartment and the engine). After they start working again, use up all the liquid you already have and fill up with winter washer liquid. Then give it a few more squirts to push the new one into the pipes.
If the clog is just at the outside of the nozzles, either caused by dirt, or ice, you can pour some mildly hot water on them and see if that helps, and again, if it's cold, use up all the remaining liquid and replace it with winter one.