Set your multimeter to the "20 VDC" or similar setting. NOT "AC", be sure it's on "DC".
With the car off, measure between the two battery posts. NOT the clamps, but the posts themselves that are part of the battery. Note the reading - it should be above about 12.5V if the battery is fully charged.
Now start the car and measure again. With the engine idling, you should now read something between about 13.4V and 13.8V. If your helper gives it a little throttle to speed up the engine, it should never rise above about 13.8 unless the weather outside is very cold and everything in the engine compartment is cold (a good voltage regulator will raise the charging voltage in cold weather because car batteries don't charge as readily in the cold).
If you read lower than about 13.4V or higher than about 13.8V with the car running, you probably have a voltage regulator problem. If you read lower than about 13.4V with the engine running FAST, you may have a diode trio (inside the alternator) problem. If you read below about 12.5V with the engine running, you have a serious charging issue with any of a number of possible causes - brushes or alternator windings or diode trio or voltage regulator or poor connections.
If you read higher than about 13.8V in warm weather, then you definitely have a voltage regulator problem and it needs to be tended to soon, before you boil your battery dry.
If the problem turns out to be intermittent, make up a simple adapter that fits your cigar/cigarette lighter socket and has two wires - one positive and one negative - extending out of it. The positive wire should be shorter than the negative one, so short that it CANNOT accidentally touch any part of the metal inside the passenger compartment. You can leave the wire ends stripped back about 1/2" and wrap the stripped wire around your multimeter probe tips, even securing the connections with any sort of tape... then you can read voltages while you drive the car. If the battery light comes on again, immediately (as immediately as is safe, of course) glance at the meter to see what the voltage is. If you want to make it a little fancier, you could put an alligator clip on each of the wires to make connection & disconnection simpler & faster.