Computer arithmetic has limited precision

< Previous | Contents | Manuals Home | Boris FX | Next >

Computer arithmetic has limited precision

While you trust that your accounting software always adds everything up to the penny perfectly accurately and repeatably, that's really because you don't have fantastic numbers of pennies, or need to worry about tiny fractions of a penny.

In reality, computers only maintain about 17 digits of accuracy, so 1 + 1.0E-18 is still1 exactly, not 1.000000000000000001: information has been lost. For a single addition of a tiny number, this doesn't sound like a problem.

However, if you are adding many numbers of widely differing values, some large, some small, this becomes a problem. If you add them up first to last like you'd enter

them into a calculator, then the final answer depends very much on the order of the numbers: you'll get a different result for different orderings.

SynthEyes uses various techniques to limit the error, but it exists. The most accurate methods would take much longer.

©2024 Boris FX, Inc. — UNOFFICIAL — Converted from original PDF.