The rubber seals that are staying on the inner fork tube after riding are called dust seals. They are one of two seals that are used to seal the fork slider onto the inner fork tube.
The first seal you see, the dust seal, does not prevent the oil from leaking out. When you look into the the area where the dust seal would go you will see your oil seal.
I recommend you change your dust seals if they are not staying seated into the slider.
IF you are leaking ANY oil whatsoever from your forks you will need to change your oil seals.
The seal cost is very low but of course there is a labor component. I believe for those types of forks you are looking at about 4 hours of labor if you have a shop perform the task.