I have a 2001 Ford Explorer, 4.0L V6 and It likes to hard start after running. If I start it up in the morning it has no problems if it sits for 3-4 hours when its cold outside it starts up no problem. However when I start it after letting it sit for less that 3ish hours it struggles to start, it tends to Stall out after trying to start. It starts no problem with starter fluid. It also has two codes for Running Lean, I'm thinking it might be the fuel pump, as I've checked the MAF, and has been replaced within the past year, the Throttle Position Sensor is also new, it has new Plugs & wires, Fuel Filter is kinda new with ~10-15k Mi on it. And the PCV has been replaced aswell.
Since you stated it will start on starting fluid, I'd highly suggest you're on the right track with it being fuel related. Doubly so considering the lean codes which have popped. More than likely I think you're barking up the right tree with thinking it's the fuel pump.
You could try throwing a fuel pressure gauge onto the fuel rail and see what it shows. According to this reference, the fuel pressure should be 65+/-8 psi. Considering it's hard to start when it's warm, it could be the pump loses its gusto when it gets warm. This would account for the hard starting and is very typical of a worn out pump.
From what I'm reading, this is a returnless system where the fuel pressure is regulated in the tank at the pump. If it's reading any less than the 65+/-8 psi, you're going to need the pump replaced. Ensure you check the pressure when the engine is hot (to ensure the pump is warmed up as well), then also check it when revving the engine, as this will put more stress on the pump and give you a better idea what could be going on.