I have a 2005 accord and have recently had the alternator and the battery replaced from the honda dealer. I then install the voltage meter on the cigarette lighter port. I notice the voltage output is at around 14V (good) while the engine is running. However, the voltage meter will occasionally show 12.6V. Sometimes, it will stay at 12.6V for a while, and sometimes it will show ~14V. Unfortunately, it's so random and I can't find a way to deterministically repeat it so I can show to the repair show. Is that the problem with the alternator or what? Any suggestion how to get that fixed?
I suggest that you rule out an intermittent alternator. The symptoms could be quite similar, and would most likely be caused by failing brushes, which intermittently make contact with the slip rings. Another common failure mode is that a overrunning clutch on the alternator intermittently grabs and then releases, which results in a period of intermittent output.
I do not know if your engine has an alternator with an overrunning clutch.
If a wire is loose on the alternator connection it could cause similar issues. I understand that the alternator and battery were replaced, so perhaps your best best is to verify all the wiring.
A clamp-on current meter on the alternator output will tell you the alternator current output. The same clamp-on on the battery will give you an idea of whether the battery is charging or discharging, as you seem to be concerned about.
Knowing current flow is actually better information than knowing battery voltage, but since the lighter circuit probably doesn't have large loads on it, it will provide reasonably accurate voltage measurements.
So in summary, the likely issues are: intermittent alternator (clutch or brushes) and wiring or connector issues. The best diagnostic tool will be a clamp-on DC current meter.