Special Numbers

1Learning Outcomes¶

• Understand how the IEEE 754 standard represents zero, infinity, and NaNs

• Understand what overflow or overflow mean with floating point numbers

• Understand how denormalized numbers implement “gradual” underflow

• Convert denormalized numbers into their decimal counterpart

Normalized numbers are only a fraction (heh) of floating point representations. For single-precision (32-bit), IEEE defines the following numbers based on the exponent field (here, the “biased exponent”):

In this section, we will motivate why these “special numbers” exist by considering the pitfalls of overflow and underflow. Then, we’ll define each of the special numbers.

2Overflow and Underflow¶

Coming soon!

3Special Numbers¶

Unlike integer representations, floating point representations similar to the IEEE 754 standards can more “gracefully” handle overflow, underflow, and errors with special numbers. We discuss the four non-normalized categories shown in Table 1.

3.1Zero¶

Just like in the sign-magnitude zero, IEEE 754 floating point has two zeros (Table 2). Recall that the standard was built for scientific computing! Having two zeros is mathematically useful. Two examples: limits towards zero and computing $\pm \infty$, the latter of which we discuss next).

If we consider its mathematical representation, zero is our first encounter with a floating point representation that is not normalized. After all, there 0.0 in scientific notation has no leading 1!

Floating point hardware often implements zero by reserving the biased exponent value zero 00000000 to signal no normalization, i.e., not to implicitly add 1. If the significand is additionally all zeros, then the hardware knows it is zero. If the significand is non zero, we represent other non-normalized numbers, which we discuss below as denormalized numbers.

3.2Infinity¶

The IEEE 754 standard defines positive infinity ($+\infty$) and negative infinity ($-\infty$), as shown in Table 2. To represent infinity, we reserve the biased exponent value 11111111 and set the significand to zero.

Because infinity is such an important concept in mathematics, the standard differentiates infinity from other arithmetic errors (which we discuss next). Importantly, dividing by $\pm 0$ yields $\pm \infty$. Computations like $x / 0 > y$ should be representable,^[1] even if not as actual “numbers.”

3.3Not a Number (NaN)¶

What if we try to compute invalid arithmetic, like $\sqrt{-4}$ or $0/0$? For scientific computing, it may be more valuable to “bubble” these errors up to the user–instead of explicitly crashing the program or computing incorrect values due to wrap-around (e.g., in integer overflow).

NaNs (Not a Number) are values of the following form (Table 4):

Because these values are triggered upon overflow (note the high exponent), they contaminate: $\text{op}(\text{NaN}, x) = \text{NaN}$.

Certain proprietary hardware for floating point go further and use the significand to encode or identify where errors occur. This practice of error codes is not defined in the standard.

3.4Denormalized Numbers¶

Coming soon!