In the image below notice that there are no rocker arms or push rods. The cam acts directly on the shim and bucket which acts on the valve stem. You adjust the valves by changing out different shims to get the appropriate valve clearence. In this image, the shim is over the bucket. This is what your Yamaha has. You have an Over the Bucket Shim to adjust valve clearance.***
Here is an image of a shim and bucket valve train.

Solution
Even though all of your symptoms seem like carburetors. It is not.
The issue you are experiencing is out of adjustment valves. The symptoms are these depending on the severity.
What you are looking for is counter-intuitive. You are looking for no valve clearance as opposed to too much.
A shim and bucket setup will lead to zero clearance situations where the valve can actually hang open and there is a tiny gap between the valve seat and valve face in the head. When this condition arises the vehicle is hard starting, poor idling and general runs like a bag of hammers. One valve hanging open can lead to a very poor running/idling condition that improves with higher RPM. Running a compression test can give you breadcrumbs and hints but a leak down test will truly determine if this condition is occurring. It's a very common issue and is considered general maintenance to adjust your valves. This bike is notorious for needing frequent valve adjustments.
This happens as the valves get super hot and get beat in and then the engine cools, the valve shortens. So you get into a situation where the condition can get worse when the engine heats up and the valve doesn't make contact with face. It will actually hang open just a bit.
Along with the mechanical beating of the valve face against the valve seat, you get a gap.
All you need to do is adjust the valves. You remove the cams after measuring everything out with a feeler gauge and documenting it and then subtract the value you need to get valve clearance and goto the Honda shop and order up some shims. Be sure you put the shims in a cupcake backing pan or ice tray, something where you can keep track of where they go, exhaust cylinder 3 etc...
This was my sons first track bike. I was adjusting the valves endlessly because he was a kid beating the bike up and keeping it at very high RPM's every single time he rode it. A race weekend was three days of hard riding and I would have to adjust the valves at least once for every three weekends. You have a bullet proof bike that will take a ton of punishment. Adjust those valves and you will love it like the day you got it. I promise. It's a great bike.