I know that proper compression is necessary for all engines to run well, but I have never understood why. I researched it and I learned that compression makes the explosion more powerful but it still didn't explain why. I was wondering what the science was behind compressing the fuel and air mixture before igniting it and why it is so important.
There's an article from Hot Rod Magazine which explains it:
Cylinder pressures and output will increase as the CR is raised, but what is less obvious is that the increase in compression also increases the engines thermal efficiency. Thermal efficiency is a measure of how effectively the engine converts heat into mechanical power. To appreciate this it is better to consider the engines expansion ratio (ER). This is the opposite side of the coin to the CR and describes what is occurring as the piston moves down on the power stroke rather than what happens as it moves up on the compression stroke.
In other words, the greater the compression ratio, the better the thermal efficiency and the more power which can be extracted from the bang.
This can be extrapolated directly from the Otto cycle:
Basically, the Otto cycle is:
... an idealized thermodynamic cycle that describes the functioning of a typical spark ignition piston engine. It is the thermodynamic cycle most commonly found in automobile engines.
It goes on to say:
From analyzing equation ... it is evident that the Otto cycle efficiency depends directly upon the compression ratio r. Since the γ (gamma) for air is 1.4, an increase in r will produce an increase in η (eta).
On the HRM page, there's also this chart which gives you a rough estimate of the power gains you can expect. With all things being the same except for the increase of compression ratio, the following is the percentage increase in power you can expect:
You read the from the top line (original CR) to the left side (new CR). For instance, if you went from 8:1 to 9:1, you could expect a 3.5% increase in power output from the engine. This is strictly due to extracting more power from the mixture.
To answer why there is compression at all, an explosion is just an explosion unless you can make it do some work. The rapid expansion of gases needs to be harnessed. By compressing the fuel-air mixture you provide a baseline pre-explosion environment. The expanding force of the explosion is confined, except for the moveable piston.
Without any initial compression, there would be only a small amount of fuel to mix with the air. That would result in a mild explosion with very little work output. The Otto cycle (intake, compression, power, exhaust) allows the steps to occur and the output force to be extracted.