Try different scales on the multimeter: does the lowest millivolt scale overflow? Is the value the same on the 1000 V scale? It's genuinely possible you obtain different voltage values at different scales.
Put some resistor, let's say 1000 ohm - 10 000 ohm, between the 12v outlet leads. Then measure. The voltage should be much lower now, very close to 0V.
A multimeter is a very high valued resistor, usually in the high kilo-ohms to low megaohms range. The resistance may depend on the measurement scale.
Or if you dare, you can measure the maximum current you can obtain by using the current position in the multimeter (warning: don't do this unless you know what you're doing! NEVER do this if the key is in the "acc"/"on" position!). I bet the maximum current you can obtain from those 4.2 volts is a very low value, so low that you won't damage anything by using the multimeter.
Yet, there is a genuine possibility it's really outputting 4.2 volts and can provide some significant current, so to avoid blowing a fuse, do try with 1000 - 10 000 ohm resistor first and proceed to the second test only after you have verified the voltage is essentially zero with the resistor.