Floating Point: IEEE 754
Read a floating-point word as sign, exponent and fraction. Name its class, see why its gaps double, and round a result to the nearest even value.
You Will Be Able To
- Split a half-precision word into its fields, name its class and compute its value.
- Find the gap between neighbouring values in any binade.
- Round to the nearest even value with the guard and sticky bits, and explain why order changes a sum.
- Lesson
- G1.5
- Time
- 30 min
- Runs on
- Your browser
- Check
- 5 questions
A sensor hub counts events in a half-precision register. It counts 2046, 2047 and 2048, and then every new event is lost. The adder works, but the format has no number between 2048 and 2050.
- Cycle 2
2047 + 1 is 2048. The sum is exact.
- Cycle 3
2048 + 1 is 2049, half way between 2048 and 2050. The tie goes to the even pattern, which is 2048.
- Cycle 4
The same thing happens on every later step. The count is stuck.
Measured on the simulator with the rounding model of this lesson.
Three Fields in One Word
Fixed point puts the binary point in one place. Floating point stores where the point is. A word has three fields: a sign, an exponent and a fraction.
- Neutral
- Data
- Split
The word 3e00 is 0 01111 1000000000. So s is 0, e is 15 and f is 1000000000.
- Classify
e is between 1 and 30, so the word is a normal number.
- The hidden 1
A normal number has a 1 before the point that is not stored. The significand is 1.1 in binary, which is 1.5.
- The bias
The exponent is e minus 15, the bias. Here it is 0, so the value is 1.5 x 2^0 = 1.5.
Read the diagram as text
- 16-bit word: for example 3e00
- Sign s: bit 15
- Exponent field e: bits 14 to 10
- Fraction f: bits 9 to 0
- Class: from e and f
- Value: (-1)^s x 1.f x 2^(e - 15)
- 16-bit word → Sign s: 1 bit
- 16-bit word → Exponent field e: 5 bits
- 16-bit word → Fraction f: 10 bits
- Exponent field e → Class: 0, 1 to 30, or 31
- Fraction f → Class: zero or not
- Class → Value: which formula
- Sign s → Value: plus or minus
The bias lets the field store exponents from -14 to 15 without a sign bit of its own.
The hidden 1 is free precision. Every normal significand starts with 1, so the format does not store it.
Each button is one bit: S is the sign, then the exponent field with its weights, then the fraction with its weights.
- Hexadecimal
- 0x3E00
Sign 0Exponent 01111Fraction 1000000000
- Format
- Half (binary16)
- Class
- normal
- Exponent
- 0
- Value
- 1.5
(−1)0 × 1.10000000002 × 20
The exponent field is 15, so the exponent is 15 − 15 = 0 and the significand has a hidden 1.
- 3e00 is 1.5. The exponent field 01111 is 15, so the exponent is 0. The top fraction bit is worth 1/2, so the significand is 1.5.
- The largest finite value. Exponent field 30 and every fraction bit set: 1.1111111111 x 2^15, which is 65504.
- The smallest subnormal. Exponent field 0 and only the last fraction bit set. There is no hidden 1, so the value is 2^-24.
Click any bit. The panel shows the three fields, the class and the exact value.
| Format | Bits | Exponent bits | Bias | Fraction bits | Largest | Smallest normal | Gap above 1.0 |
|---|---|---|---|---|---|---|---|
| half | 16 | 5 | 15 | 10 | 65504 | 6.10352e-05 | 0.000976562 |
| single | 32 | 8 | 127 | 23 | 3.40282e+38 | 1.17549e-38 | 1.19209e-07 |
| double | 64 | 11 | 1023 | 52 | 1.79769e+308 | 2.22507e-308 | 2.22045e-16 |
The three common formats of IEEE 754. The last three columns are measured on the simulator from the field widths.
Five Classes of Value
Two exponent fields are reserved. All zeros means zero or a subnormal number. All ones means infinity or NaN, "not a number".
| Exponent field | Fraction | Class | Value | Example from the lab |
|---|---|---|---|---|
| 0 | 0 | zero | +0 or -0 | 8000 is -0 |
| 0 | not 0 | subnormal | (-1)^s x 0.f x 2^-14 | 0001 is 5.96e-08, the smallest |
| 1 to 30 | any | normal | (-1)^s x 1.f x 2^(e - 15) | 7bff is 65504, the largest |
| 31 | 0 | infinity | +inf or -inf | 7c00 is +infinity |
| 31 | not 0 | NaN | no value | 7e00 is NaN |
The class depends only on the exponent field and on whether the fraction is zero.
Subnormals fill the gap between zero and the smallest normal number, 2^-14. They have no hidden 1 and keep the exponent at -14. Their steps are all 2^-24.
Infinity is the result of an overflow or of 1/0. NaN is the result of an operation with no answer, such as 0/0. A NaN is not equal to anything, itself included.
Gaps That Double
Between two powers of two lies a binade. Every binade holds 1,024 half-precision values, evenly spaced. The next binade is twice as wide, so its gaps are twice as large.
- Cycle 3
Up to 2047 the patterns are 1 apart in value. The fraction counts in steps of 1.
- Cycle 4
Pattern 6800 is 2048. The exponent went up by one and the fraction went back to 0.
- Cycle 5
The next pattern is 2050. The fraction still counts by 1, but each count is now worth 2.
Measured on the simulator. There is no half-precision number 2049.
| Binade starts at | Gap to the next value |
|---|---|
| 0.25 | 0.000244141 |
| 0.5 | 0.000488281 |
| 1 | 0.000976562 |
| 2 | 0.00195312 |
| 4 | 0.00390625 |
| 8 | 0.0078125 |
| 16 | 0.015625 |
| 32 | 0.03125 |
| 64 | 0.0625 |
| 128 | 0.125 |
| 256 | 0.25 |
| 512 | 0.5 |
| 1024 | 1 |
| 2048 | 2 |
| 4096 | 4 |
Measured on the simulator. The gap at 1 is 2^-10, one step of the 10-bit fraction. Each binade doubles it.
The gap at a value is one ulp, a "unit in the last place". For a normal number the gap is always 2^-11 to 2^-10 of the value. So floating point keeps the same number of digits over a wide range.
Round to Nearest Even
An exact sum often falls between two neighbours. The default IEEE rule picks the nearer one. At a tie it picks the one whose last fraction bit is 0: the even one.
Hardware decides with three bits. L is the last bit kept. G, the guard, is the first bit dropped. S, the sticky bit, is the OR of every bit below G. Round up when G is 1 and S or L is 1.
| Exact value | What it is | Truncate | Round half up | Nearest even | Why |
|---|---|---|---|---|---|
| 1.000244141 | 1 + 2^-12 | 3c00 | 3c00 | 3c00 | below half a step: G = 0 |
| 1.000488281 | 1 + 2^-11 | 3c00 | 3c01 | 3c00 | a tie, kept LSB even: G = 1, S = 0, L = 0 |
| 1.001464844 | 1 + 3 x 2^-11 | 3c01 | 3c02 | 3c02 | a tie, kept LSB odd: G = 1, S = 0, L = 1 |
| 1.000489235 | 1 + 2^-11 + 2^-20 | 3c00 | 3c01 | 3c01 | just above a tie: G = 1, S = 1 |
| 2049 | 2048 + 1 | 6800 | 6801 | 6800 | a tie, kept LSB even |
| 2051 | 2050 + 1 | 6801 | 6802 | 6802 | a tie, kept LSB odd |
Measured on the simulator; Node's IEEE half type gives the same nearest-even patterns. Yellow marks a result that differs from nearest even.
The fourth row shows why the sticky bit exists. G alone says "half a step or more". Only S tells a tie from a value just above it.
The Order of Additions Matters
Every operation rounds its own result. Two orders of the same additions round at different places, so their sums can differ.
(a + b) + c, in double
real a = 0.1, b = 0.2, c = 0.3;
$display("%0.17f", (a + b) + c);
Measured: 0.60000000000000009.
a + (b + c), in double
real a = 0.1, b = 0.2, c = 0.3;
$display("%0.17f", a + (b + c));
Measured: 0.59999999999999998.
Half precision shows the same thing with whole numbers. (2048 + 1) + 1 is 2048, because each add rounds back. 2048 + (1 + 1) is 2050.
Two consequences follow for hardware. A floating-point unit must define the order of its operations. A testbench must compare results with a tolerance, or match the exact order of the design.
Asked in interviews
Why is the exponent biased and not two's complement?
With a bias, the exponent field grows with the value. Positive floats then sort in the same order as their bit patterns read as integers.
What is a subnormal number?
A word with an exponent field of 0 and a fraction that is not 0. It has no hidden 1, so it fills the gap between zero and the smallest normal number.
How does hardware round to nearest even?
It keeps the last kept bit L, the guard bit G and the sticky bit S. It adds one when G is 1 and S or L is 1.
Try It
Each line shows the word, its three fields and its value. 3c00 has e = 15, so its exponent is 0 and its value is 1. c000 differs in the sign and in one exponent bit: -2.
7bff is the largest finite word: the largest normal exponent and every fraction bit set. 0400 is the smallest normal number, 2^-14. 0001 is the smallest subnormal, 2^-24.
Add a line decode(16'h0200);. It is a subnormal with half of its fraction set, so it prints half of 2^-14. Then change int'(e) to e and decode 16'h3800, which is 0.5.
3c00 s=0 e=15 f=000 +1 c000 s=1 e=16 f=000 -2 7bff s=0 e=30 f=3ff +65504 0400 s=0 e= 1 f=000 +6.103515625e-05 0001 s=0 e= 0 f=001 +5.960464478e-08 8000 s=1 e= 0 f=000 -0 7c00 s=0 e=31 f=000 +infinity 7e00 s=0 e=31 f=200 NaNCheck Yourself
Common Mistakes
Wrong Compare two floating-point results with ==.
Each operation rounds, so two routes to the same number can end one step apart.
Fix Compare the difference with a tolerance that suits the size of the values.
if (x == 0.3)if ((x - 0.3) < 1e-9 &&
(0.3 - x) < 1e-9)Wrong Add the hidden 1 to every word.
A word with an exponent field of 0 is zero or subnormal, and it has no hidden 1.
Fix Use 1.f only when the exponent field is 1 to 30. Use 0.f with an exponent of -14 when it is 0.
Wrong Subtract the bias from the unsigned exponent field as it is.
For a field below 15 the difference wraps to a huge unsigned number. Measured: the word for 0.5 decodes to infinity.
Fix Convert the field to a signed integer first.
2.0 ** (e - 15)2.0 ** (int'(e) - 15)Wrong Expect more exponent bits to give more precision.
The exponent sets the range. The fraction bits alone set the step between neighbours.
Fix Count fraction bits for precision and exponent bits for range.
Remember
- A float word is a sign, a biased exponent and a fraction; a normal number has a hidden 1.
- Exponent field 0 means zero or subnormal; all ones means infinity or NaN.
- Each binade holds the same count of values, so the gap doubles with every power of two.
- Round to nearest even uses L, G and S: round up when G is 1 and S or L is 1.
- Every operation rounds, so the order of additions can change the sum.