Fairly straightforward question, would I be able to charge my car battery with a DC power supply that outputs 12V 2.5A? I know real car battery chargers put out 12V 10A. Would it just take 5 times longer?
If your power supply is well-regulated (with linear regulators or switched mode power supply), so that it outputs 12V at all load levels, the voltage is way too low. Common float charging voltages that can be left indefinitely on are 13.5V - 13.8V. Fast charge occurs at voltages of about 14.5V, but then you need the ability to stop the fast charge and switch to float charge once the low current indicates the charging is done.
A lead-acid battery with resting voltage below 12.6V will not have a full charge. At 12V, it is nearly empty! And lead-acid doesn't like being empty for especially for extended periods of time. An empty lead-acid battery will permanently sulfate, meaning it will no longer work properly.
If, on the other hand, your power supply is not regulated (just a transformer, rectifier and capacitor), the fact that it says 12V will mean it will be probably way over 15V at low current demands. This means the charger will convert water into hydrogen and oxygen via electrolysis slowly. This leads to water loss in the battery. Many new batteries do not have the possibility of adding distilled water, so this is a good way to destroy your battery. Many of these batteries have the possibility to recombine hydrogen and oxygen back to water (maintenance free battery), but the rate at which this can be done is very very slow, and your charger will very surely exceed this rate, if it's unregulated.
So, no matter whether your power supply is regulated or unregulated, charging a battery with it is a bad idea, but the reason for it being a bad idea are different in different cases.
To see if your power supply is regulated, measure it with a multimeter. Regulated ones measure the exact nominal voltage, unregulated ones with no load measure much higher voltages than the nominal voltage. You can also judge by the weight whether there's a heavy iron core transformer inside; if not, it is probably a switched-mode power supply which means it's regulated.