Main Content

Precision

Limitations on precision, effects of rounding and padding

Computer words consist of a finite number of bits. This means that the binary encoding of variables is only an approximation of an arbitrarily precise real-world value. Therefore, the limitations of the binary representation automatically introduce limitations on the precision of the value.

Topics

Featured Examples