Floating Point: IEEE 754

Read a floating-point word as sign, exponent and fraction. Name its class, see why its gaps double, and round a result to the nearest even value.

You Will Be Able To

  • Split a half-precision word into its fields, name its class and compute its value.
  • Find the gap between neighbouring values in any binade.
  • Round to the nearest even value with the guard and sticky bits, and explain why order changes a sum.
Lesson
G1.5
Time
30 min
Runs on
Your browser
Check
5 questions

A sensor hub counts events in a half-precision register. It counts 2046, 2047 and 2048, and then every new event is lost. The adder works, but the format has no number between 2048 and 2050.

A Counter That Stops at 2048Each step adds 1 and rounds the sum to half precision.
01234step01234count2046204720482048 + 1 rounds back
Scroll horizontally ↔
  1. Cycle 2

    2047 + 1 is 2048. The sum is exact.

  2. Cycle 3

    2048 + 1 is 2049, half way between 2048 and 2050. The tie goes to the even pattern, which is 2048.

  3. Cycle 4

    The same thing happens on every later step. The count is stuck.

Measured on the simulator with the rounding model of this lesson.

Three Fields in One Word

Fixed point puts the binary point in one place. Floating point stores where the point is. A word has three fields: a sign, an exponent and a fraction.

From a Half-Precision Word to Its ValueArrows show data. A half-precision word has 1 sign bit, 5 exponent bits and 10 fraction bits.
  • Neutral
  • Data
From a half-precision word to its valueArrows show data. A half-precision word has 1 sign bit, 5 exponent bits and 10 fraction bits. neutral 16-bit word for example 3e00 neutral Sign s bit 15 neutral Exponent field e bits 14 to 10 neutral Fraction f bits 9 to 0 neutral Class from e and f neutral Value (-1)^s x 1.f x 2^(e -15) 1 bit5 bits10 bits0, 1 to 30, or 31zero or notwhich formulaplus or minus
  1. neutral 16-bit wordfor example 3e00
    • 1 bit → Sign s
    • 5 bits → Exponent field e
    • 10 bits → Fraction f
  2. neutral Sign sbit 15
    • plus or minus → Value
  3. neutral Exponent field ebits 14 to 10
    • 0, 1 to 30, or 31 → Class
  4. neutral Fraction fbits 9 to 0
    • zero or not → Class
  5. neutral Classfrom e and f
    • which formula → Value
  6. neutral Value(-1)^s x 1.f x 2^(e - 15)
  1. Split

    The word 3e00 is 0 01111 1000000000. So s is 0, e is 15 and f is 1000000000.

  2. Classify

    e is between 1 and 30, so the word is a normal number.

  3. The hidden 1

    A normal number has a 1 before the point that is not stored. The significand is 1.1 in binary, which is 1.5.

  4. The bias

    The exponent is e minus 15, the bias. Here it is 0, so the value is 1.5 x 2^0 = 1.5.

Read the diagram as text
  • 16-bit word: for example 3e00
  • Sign s: bit 15
  • Exponent field e: bits 14 to 10
  • Fraction f: bits 9 to 0
  • Class: from e and f
  • Value: (-1)^s x 1.f x 2^(e - 15)
  • 16-bit word → Sign s: 1 bit
  • 16-bit word → Exponent field e: 5 bits
  • 16-bit word → Fraction f: 10 bits
  • Exponent field e → Class: 0, 1 to 30, or 31
  • Fraction f → Class: zero or not
  • Class → Value: which formula
  • Sign s → Value: plus or minus

The bias lets the field store exponents from -14 to 15 without a sign bit of its own.

The hidden 1 is free precision. Every normal significand starts with 1, so the format does not store it.

Click the Bits of a Half-Precision Word
S
24
23
22
21
20
2-1
2-2
2-3
2-4
2-5
2-6
2-7
2-8
2-9
2-10

Each button is one bit: S is the sign, then the exponent field with its weights, then the fraction with its weights.

Hexadecimal
0x3E00

Sign 0Exponent 01111Fraction 1000000000

Format
Half (binary16)
Class
normal
Exponent
0
Value
1.5

(−1)0 × 1.10000000002 × 20

The exponent field is 15, so the exponent is 15 − 15 = 0 and the significand has a hidden 1.

  1. 3e00 is 1.5. The exponent field 01111 is 15, so the exponent is 0. The top fraction bit is worth 1/2, so the significand is 1.5.
  2. The largest finite value. Exponent field 30 and every fraction bit set: 1.1111111111 x 2^15, which is 65504.
  3. The smallest subnormal. Exponent field 0 and only the last fraction bit set. There is no hidden 1, so the value is 2^-24.

Click any bit. The panel shows the three fields, the class and the exact value.

FormatBitsExponent bitsBiasFraction bitsLargestSmallest normalGap above 1.0
half1651510655046.10352e-050.000976562
single328127233.40282e+381.17549e-381.19209e-07
double64111023521.79769e+3082.22507e-3082.22045e-16

The three common formats of IEEE 754. The last three columns are measured on the simulator from the field widths.

Five Classes of Value

Two exponent fields are reserved. All zeros means zero or a subnormal number. All ones means infinity or NaN, "not a number".

Exponent fieldFractionClassValueExample from the lab
00zero+0 or -08000 is -0
0not 0subnormal(-1)^s x 0.f x 2^-140001 is 5.96e-08, the smallest
1 to 30anynormal(-1)^s x 1.f x 2^(e - 15)7bff is 65504, the largest
310infinity+inf or -inf7c00 is +infinity
31not 0NaNno value7e00 is NaN

The class depends only on the exponent field and on whether the fraction is zero.

Subnormals fill the gap between zero and the smallest normal number, 2^-14. They have no hidden 1 and keep the exponent at -14. Their steps are all 2^-24.

Infinity is the result of an overflow or of 1/0. NaN is the result of an operation with no answer, such as 0/0. A NaN is not equal to anything, itself included.

Gaps That Double

Between two powers of two lies a binade. Every binade holds 1,024 half-precision values, evenly spaced. The next binade is twice as wide, so its gaps are twice as large.

Eight Neighbouring PatternsEach column adds 1 to the 16-bit pattern. The value row shows what the pattern means.
01234567pattern67fc67fd67fe67ff6800680168026803value20442045204620472048205020522054new binade
Scroll horizontally ↔
  1. Cycle 3

    Up to 2047 the patterns are 1 apart in value. The fraction counts in steps of 1.

  2. Cycle 4

    Pattern 6800 is 2048. The exponent went up by one and the fraction went back to 0.

  3. Cycle 5

    The next pattern is 2050. The fraction still counts by 1, but each count is now worth 2.

Measured on the simulator. There is no half-precision number 2049.

Binade starts atGap to the next value
0.250.000244141
0.50.000488281
10.000976562
20.00195312
40.00390625
80.0078125
160.015625
320.03125
640.0625
1280.125
2560.25
5120.5
10241
20482
40964

Measured on the simulator. The gap at 1 is 2^-10, one step of the 10-bit fraction. Each binade doubles it.

The gap at a value is one ulp, a "unit in the last place". For a normal number the gap is always 2^-11 to 2^-10 of the value. So floating point keeps the same number of digits over a wide range.

Round to Nearest Even

An exact sum often falls between two neighbours. The default IEEE rule picks the nearer one. At a tie it picks the one whose last fraction bit is 0: the even one.

Hardware decides with three bits. L is the last bit kept. G, the guard, is the first bit dropped. S, the sticky bit, is the OR of every bit below G. Round up when G is 1 and S or L is 1.

Exact valueWhat it isTruncateRound half upNearest evenWhy
1.0002441411 + 2^-123c003c003c00below half a step: G = 0
1.0004882811 + 2^-113c003c013c00a tie, kept LSB even: G = 1, S = 0, L = 0
1.0014648441 + 3 x 2^-113c013c023c02a tie, kept LSB odd: G = 1, S = 0, L = 1
1.0004892351 + 2^-11 + 2^-203c003c013c01just above a tie: G = 1, S = 1
20492048 + 1680068016800a tie, kept LSB even
20512050 + 1680168026802a tie, kept LSB odd

Measured on the simulator; Node's IEEE half type gives the same nearest-even patterns. Yellow marks a result that differs from nearest even.

The fourth row shows why the sticky bit exists. G alone says "half a step or more". Only S tells a tie from a value just above it.

Round 2049 to Half Precision
S
24
23
22
21
20
2-1
2-2
2-3
2-4
2-5
2-6
2-7
2-8
2-9
2-10

Each button is one bit: S is the sign, then the exponent field with its weights, then the fraction with its weights.

Hexadecimal
0x6800

Sign 0Exponent 11010Fraction 0000000000

Format
Half (binary16)
Class
normal
Exponent
11
Value
2048

(−1)0 × 1.00000000002 × 211

The exponent field is 26, so the exponent is 26 − 15 = 11 and the significand has a hidden 1.

Round 2049 to half precision. 2049 lies between 2048 (toward zero) and 2050 (away from zero).

Rounding directionBitsValue
Nearest, ties to even0 11010 00000000002048
Nearest, ties away from zero0 11010 00000000012050
Toward zero0 11010 00000000002048
Toward +∞0 11010 00000000012050
Toward −∞0 11010 00000000002048

The table rounds 2049 in all five IEEE directions. Nearest even is ringed; the form below it takes any decimal number.

The Order of Additions Matters

Every operation rounds its own result. Two orders of the same additions round at different places, so their sums can differ.

(a + b) + c, in double

real a = 0.1, b = 0.2, c = 0.3;
$display("%0.17f", (a + b) + c);

Measured: 0.60000000000000009.

a + (b + c), in double

real a = 0.1, b = 0.2, c = 0.3;
$display("%0.17f", a + (b + c));

Measured: 0.59999999999999998.

Half precision shows the same thing with whole numbers. (2048 + 1) + 1 is 2048, because each add rounds back. 2048 + (1 + 1) is 2050.

Two consequences follow for hardware. A floating-point unit must define the order of its operations. A testbench must compare results with a tolerance, or match the exact order of the design.

Asked in interviews
  1. Why is the exponent biased and not two's complement?

    With a bias, the exponent field grows with the value. Positive floats then sort in the same order as their bit patterns read as integers.

  2. What is a subnormal number?

    A word with an exponent field of 0 and a fraction that is not 0. It has no hidden 1, so it fills the gap between zero and the smallest normal number.

  3. How does hardware round to nearest even?

    It keeps the last kept bit L, the guard bit G and the sticky bit S. It adds one when G is 1 and S or L is 1.

Try It

Each line shows the word, its three fields and its value. 3c00 has e = 15, so its exponent is 0 and its value is 1. c000 differs in the sign and in one exponent bit: -2.

7bff is the largest finite word: the largest normal exponent and every fraction bit set. 0400 is the smallest normal number, 2^-14. 0001 is the smallest subnormal, 2^-24.

Add a line decode(16'h0200);. It is a subnormal with half of its fraction set, so it prints half of 2^-14. Then change int'(e) to e and decode 16'h3800, which is 0.5.

RUNIcarus Verilog · in your browser
Edit the code, then run it
Expected: 3c00 s=0 e=15 f=000 +1 c000 s=1 e=16 f=000 -2 7bff s=0 e=30 f=3ff +65504 0400 s=0 e= 1 f=000 +6.103515625e-05 0001 s=0 e= 0 f=001 +5.960464478e-08 8000 s=1 e= 0 f=000 -0 7c00 s=0 e=31 f=000 +infinity 7e00 s=0 e=31 f=200 NaN

Check Yourself

Predict the outputThe program adds two doubles and compares the sum with 0.3. What exact line does it print?
module tb;
  real x;
  initial begin
    x = 0.1 + 0.2;
    $display("%0d", x == 0.3);
  end
endmodule
Spot the bugEach option sets v, the value of a subnormal half-precision word with the fraction field f. The bench tries all 1,023 subnormal words. Which option fails?
ConceptWhat is the gap between 1024 and the next larger half-precision number?
ConceptA half-precision word has the exponent field 11111 and a fraction that is not zero. What is it?
ConceptA rounder ignores the sticky bit and treats every G = 1 as a tie. Which exact value does it round wrongly to half precision?

Common Mistakes

Wrong Compare two floating-point results with ==.

Each operation rounds, so two routes to the same number can end one step apart.

Fix Compare the difference with a tolerance that suits the size of the values.

Before
if (x == 0.3)
After
if ((x - 0.3) < 1e-9 &&
    (0.3 - x) < 1e-9)

Wrong Add the hidden 1 to every word.

A word with an exponent field of 0 is zero or subnormal, and it has no hidden 1.

Fix Use 1.f only when the exponent field is 1 to 30. Use 0.f with an exponent of -14 when it is 0.

Wrong Subtract the bias from the unsigned exponent field as it is.

For a field below 15 the difference wraps to a huge unsigned number. Measured: the word for 0.5 decodes to infinity.

Fix Convert the field to a signed integer first.

Before
2.0 ** (e - 15)
After
2.0 ** (int'(e) - 15)

Wrong Expect more exponent bits to give more precision.

The exponent sets the range. The fraction bits alone set the step between neighbours.

Fix Count fraction bits for precision and exponent bits for range.

Remember

  • A float word is a sign, a biased exponent and a fraction; a normal number has a hidden 1.
  • Exponent field 0 means zero or subnormal; all ones means infinity or NaN.
  • Each binade holds the same count of values, so the gap doubles with every power of two.
  • Round to nearest even uses L, G and S: round up when G is 1 and S or L is 1.
  • Every operation rounds, so the order of additions can change the sum.

Practice

Report a problem with this page