Diesel does damages rubber. Diesel fuel may cause stripping of certain types of rubber hose, gradually perforating the fuel hose. Thus, it is strongly advised to make sure if the fuel hose to be installed is rated for diesel.
To clean a diesel spill, you first need to pour lots of warm water on the spill to instantly reduce the risk of rubber damage. Then using a kitty litter, you need to wipe the diesel. Do use small brushes to reach the all the narrow places where the diesel might have spilled. Then again repeat the process of pouring warm water and follow it up with scrubbing with kitty litter and brush. Finally, use a engine wash specced soap with warm water on the rubber parts and scrub generously. In the end, the rubber part should not smell of diesel.
You should clean the spilled diesel as soon as possible if it is spilled on a rubber part. However, to be even more critical, you need to know that the warmer the fuel is, the sooner it needs to be cleaned, as it is more likely to damage rubber when it is warm. Thus, as a thumb rule, whenever you will see diesel spill immediately pour warm water over it, as it will reduce the risk of damage drastically.
A bit off topic:
Whenever there is a fuel spill, please ensure you follow the legal guidelines of cleaning and sanitizing the place. Fuel spills need to be treated according to government guidelines.