In Short its based on low burn rate of diesel plus the longer stroke of the diesel engine.

First you must understand the difference between these engines, the diesel works on purely compression of fuel , heating and generating bang to produce power, the gasoline on the other hand is natively twitchy and needs a spark to explode and produce power on its own.

That said,

Think of the petrol engine as a cheetah,it has lightweight bones and has a streamlined body thus is very fast but is not powerful enough to kill a lion or a rhino. 

The Diesel engine is like an elephant, its slow and has a ton of power , but needs heavier legs to support the huge body mass thus it cant run as fast as the cheetah but has a lot of power.By power I mean torque.

The petrol engine natively combusts so it does not need heavy duty parts to withstand the explosion , yes you are compressing the fuel but its not nearly as much as the diesel, that is why 100cc motorcycles rev like crazy they have lightweight construction(cranks, pistons).

The diesel does not combust as much as petrol , it needs to be compressed to a much higher extent to combust thus the cylinder head , the piston, the crank everything needs to be heavy duty to withstand the compression explosion thus slowing down the speed of the piston.

Finally to answer your question:

Diesel burns slowly compared to petrol and for most diesel engines the limit is around 4800 to 5000 RPM.

Adding clarity to the above point, in a petrol engine almost 95% of the fuel is burnt in every stroke but in the diesel engine not all of the diesel is burnt in each stroke due to the slow combustion rate some diesel remains in the cylinder before the next stroke so no matter how hard you push , that small quantity of diesel will not burn before the piston is ready for the next stroke thus limiting the speed or redline.