Recently, I have come across many bikes which have upside down front forks.
From browsing around the internet, I was not able to get a concrete answer for why it is preferred over conventional telescopic forks.
There are several advantages to upside down forks
Reduced Unsprung Weight
Reduced Sprung Weight
Reduced Stanchion Friction Levels
Increased Stanchion to Slider Overlap
Image of the lower portion of a pair of inverted forks
The largest delta between the legacy standard fork and an inverted fork is in their strength. Bending occurs just below the bottom of the yoke. When brakes are applied you are slowing the front wheel. This reduced forward motion
All the momentum of the bike acts through the forks, trying to push the front wheel along. The reduced inertia is felt at the yolk and the upper slider mounted within it. The larger diameter slider allows for increased rigidity thereby transferring more energy directly to the frame rather than bending the slider.
The slider is made of lightweight material and it's increased diameter from standard forks does not increase weight significantly. The stanchion which inserts into the slider is significantly shorter and has thinner walls. This component is typically made of ferrous material and is significantly heavier than the slider. Due the construction of the stanchion in an inverted fork, reduced length, this component in an inverted configuration offers great gains in weight and effects the unsprung weight of the front end. Reduced unsprung weight results in performance gains regarding the wheels contact with the pavement making for a greater 'feel' for the rider.
The increased stanchion diameter has an effect on the bushings in the slider which are spaced further apart vertically and have a significant gain in surface area contact with the stanchion. This allows for greater hydrodynamic lubrication and reduced friction.
Caliper mounting points are also effected and now allow for radial or monoblock style brakes. The mounting point at the bottom of the fork and the reduced stanchion length create less leverage, the larger bushings and increased surface are in conjunction allow for harder braking into corners. The legacy standard design had a tendancy of locking the slider and the stanchion together under heavy braking when the stanchion broke through hydrodynamic lubrication and made metal to metal contact with the bushing. This front end lockup under hard braking in a corner could have catastrophic consequences as the normally absorbed energy through uneven roadway contact was transferred directly to the contact patch of the tire.
USD's have a bit more complexity than standard forks, especially the nitrogen charged versions. Additional components and bladders for nitrogen retention require professional maintenance with special tools.
Seal failure in USD's require a higher level of urgency for maintenance as the oil within the fork is retained above the seal in this design. Under those circumstances fork oil will be discharged upon the brake disc, brake caliper, wheel and tire. Due to the retention mechanism for the seal it is rare that a catastrophic seal failure will occur where oil is sprayed upon these components but it has been known to happen under rare circumstances, typically due to the human error in re-assembly of the system.
Additionally, due to added complexity of components, cost of USD's over standard forks are significant. This also applies to the plethora of tools required to rebuild USD's.
Wet and dry seal replacement
Maintenance tasks are basically the same as standard forks as the list above denotes. Special tools are required for driving seals, oil level height measurement, fork cap installation and nitrogen charging (if they are nitrogen charged).
Inverted forks decrease internal friction of components and "add lightness". Increased rigidity creates stability and brake mount points increase safety while braking in turns.
* image from RaceTek
Greater Torsional Stability.
Why should flipping the orientation cause any difference?
Without getting into a detailed analysis, you can imagine that most of the stresses on the fork can be assumed to be concentrated just below the bottom yoke.
Eg, while braking (front), the wheel slows down- pushing the bike back through the forks causing a bending moment to the fork.
The upside down fork will be stiffer as the slider has a larger diameter (see bending stiffness, and this) compared to the stanchions. So , there you go.
For the Rider, this results in a more predictable front end under braking, turning and any other maneuver you might put it through.
There's also this matter about reduced unsprung mass, which ducati killer has explained in his answer here . Though, in practice, there's very little difference/advantage.
So though the general argument presented above in my answer still holds true about larger diameter cylindrical sections providing more resistance to bending, for the same material use, there's another thing to consider. That is , the number of points where one can introduce lateral braces. Addition of lateral braces serves to only provide additional lateral stiffness and prevent the tendency to 'twist'. On a mountain bike, I guess this design serves to optimize the structural requirements the best. Again, this is a generalization and for XC (cross country) mtb, upside down forks are becoming popular.
For sports bikes, the maximum stress on the fork has to arguably be when it's braking (or braking and turning). This force out measures all the other forces by so much that a fork designed to optimize on that structural requirement would meet lateral stiffness and twist requirements quite conveniently (things are not that simple, but you get the picture) .
Main advantage is the mentioned above lower unsprung weight that increases the responsiveness of the suspension through reducing the inertia of the moving part of the suspension.
Second important advantage is the fact that the fork stanchions aren't in the path of dirt from the front wheel as is the case in a classic fork - this is the most important advantage of an upside down fork in my opinion as it increases the fork's life, reduces its maintenance requirements and makes it very easy to completely shield the fork stanchions. That's why it's a standard for cross motorcycles and starts getting into enduro bicycles.
As to other advantages mentioned in other responses - I wouldn't count on those. I'll explain:
On an upside-down fork you have no connection between sliders (lowers), while on a classical setup you do have an arch connecting those and making that part of the fork very stiff. This makes the fork very resistant to twisting stresses. Additionally the distance between the wheel axle and the sliders arch is the smallest possible distance between those components, and that also adds stiffness.
In an upside-down fork you have a huge distance between the fork crown and the wheel axle, and as mentioned before the sliders have no other connection between them except the wheel axle. So all the forces going through such a fork have a big leverage on the stanchions and those have to be very beefy to be comparable to a classical design. Additionally a beefy through-axle is an absolute must for an upside-down fork as without it the wheel would twist in relation to stanchions (this is because the sliders are only solidly connected via the wheel axle).
Weight - if you create equivalent stiffness forks of both designs with similar cost and materials because of point 1, it's highly doubtful that the upside-down fork would be stiffer. This assumption comes from the fact that upside-down forks are always positioned as higher-end forks and they use better materials and machining, and usually those are the factors that cause those forks to be lighter, not the fact that they're upside-down design.
Brakes mounting - force exerted on the sliders and stanchions depends only on the size of the disk brake and the friction force that the caliper can exert on the disc brake - the leverage of this force in relation to the wheel axle is always the same for the same brake setup and it doesn't matter how the caliper mount looks like. Also it's important to note that not a single type of brakes exist that can be used only on upside-down fork and not on classical design.