Architecture Trade-Offs: Parallelism, Pipelining and Race-to-Idle

Spend extra hardware to lower the supply voltage, and decide when to race through work and when to run slowly.

You Will Be Able To

  • Find the lowest supply at which one unit, two units or a pipeline meets a throughput.
  • Compare their energy per operation, including the multiplexer and register they add.
  • Choose between racing to idle and running slowly from measured energy and leakage.
Lesson
W2.7
Time
30 min
Runs on
ngspice
Check
6 questions

A filter must finish one operation every nanosecond, and one unit does it at 1.5 V. A colleague proposes two copies at half the rate. Does the bigger design use less energy?

Three Ways to Build the Same ThroughputArrows show operations moving from the input to the result. The same critical path, a chain of inverters, sits in every version.
  • I/O
  • Datapath
  • Data
Three ways to build the same throughputArrows show operations moving from the input to the result. The same critical path, a chain of inverters, sits in every version. io Operations One every period T datapath One unit Must finish in T datapath Unit 0 Even operations, 2Teach datapath Unit 1 Odd operations, 2Teach datapath Multiplexer Picks the result thatis due datapath Half, register,half Each half gets T io Results One every period T
  1. io OperationsOne every period T
    • One unit
    • Unit 0
    • Unit 1
    • Half, register, half
  2. datapath One unitMust finish in T
    • Results
  3. io ResultsOne every period T
  4. datapath Unit 0Even operations, 2T each
    • Multiplexer
  5. datapath Unit 1Odd operations, 2T each
    • Multiplexer
  6. datapath MultiplexerPicks the result that is due
    • Results
  7. datapath Half, register, halfEach half gets T
    • Results
  1. One unit

    The whole path must settle in one period. That sets a high voltage.

  2. Two units

    Each unit takes every second operation, so it has two periods. It can run at a lower voltage. The multiplexer is new hardware.

  3. A pipeline

    A register cuts the path in half. Each half has one period, so the voltage can also fall. The register is new hardware, and it is clocked in every cycle.

Read the diagram as text
  • Operations: One every period T
  • One unit: Must finish in T
  • Unit 0: Even operations, 2T each
  • Unit 1: Odd operations, 2T each
  • Multiplexer: Picks the result that is due
  • Half, register, half: Each half gets T
  • Results: One every period T
  • Operations → One unit
  • One unit → Results
  • Operations → Unit 0
  • Operations → Unit 1
  • Unit 0 → Multiplexer
  • Unit 1 → Multiplexer
  • Multiplexer → Results
  • Operations → Half, register, half
  • Half, register, half → Results

Extra hardware buys time per operation. Time per operation buys a lower voltage, and energy follows C·VDD².

Measured below: two units need only 1.21 V and spend 21 % less energy per operation.

Two Units at Half the Rate

Energy per operation follows C·VDD² (W1.1), and speed falls with the voltage (W1.4). A design that gives each operation more time can use a lower voltage.

Two copies of a unit, each taking every second operation, give each operation two periods. A 2:1 multiplexer picks the result that is due.

TransientTwo Units at 1.21 V, One Operation per NanosecondUnit 0 takes the even operations, unit 1 the odd ones. Each output settles within two periods. The select line s switches every period and the multiplexer output yb follows the unit whose result is due.
  • y0 (unit 0)
  • y1 (unit 1)
  • yb (result)
  • s (select)
-0.5 V00.5 V1.0 V1.5 Vy0 (unit 0), y1 (unit 1)-0.5 V00.5 V1.0 V1.5 V2.0 Vyb (result), s (select)02 ns4 ns6 ns8 ns10 ns12 ns14 nsTime-0.5 V00.5 V1.0 V1.5 Vy0 (unit 0), y1 (unit 1)-0.5 V00.5 V1.0 V1.5 V2.0 Vyb (result), s (select)05 ns10 ns15 nsTime
Read the plot as a table
Timey0 (unit 0)y1 (unit 1)yb (result)s (select)
0 s134.9 nV135 nV134.9 nV1.21 V
2 ns113.6 nV134.9 nV134.9 nV1.21 V
4 ns1.21 V1.179 V1.209 V1.212 V
6 ns8.216 µV1.497 mV161 µV1.208 V
8 ns1.21 V1.179 V1.209 V1.212 V
10 ns8.216 µV1.497 mV161 µV1.208 V
12 ns1.21 V1.179 V1.209 V1.212 V
14 ns8.216 µV1.497 mV161 µV1.208 V

Measured on ngspice 42, tt, 27 °C. All 12 results arrive before their deadlines. One operation costs 144 fJ.

ArchitectureLowest supply for 1 nsEnergy per operationAgainst one unit
One unit1.5 V183 fJ—
Two units and a multiplexer1.21 V144 fJ21 % less
Pipeline: two halves and a register1.44 V211 fJ15 % more

Typical chip, 27 °C. The lowest supply is found in steps of 10 mV.

The voltage ratio squared is 0.65, yet the energy ratio is 0.79. The multiplexer and its select line take back part of the gain.

What the Extra Hardware Costs

At the same 1.8 V the voltage gain disappears, and only the cost of the added hardware remains.

Architecture at 1.8 V, 1 nsEnergy per operationAgainst one unit
One unit267 fJ—
Two units and a multiplexer308 fJ15 % more
Pipeline: two halves and a register336 fJ26 % more

Same bench, same throughput, every design at 1.8 V. Probe data: tools/probes/lowpower/arch-probe.mjs --extra.

The multiplexer and its select driver add 15 %. The register adds 26 %, because its clock nodes switch in every cycle.

Two units also double the area and the leakage. The extra hardware pays only if the voltage it allows saves more than the hardware costs.

Pipelining: Cut the Path in Two

A pipeline gets more time without a second unit. A register in the middle lets each half take a full period. Each result appears one period later.

TransientThe Pipeline at 1.44 V, One Operation per NanosecondThe first half h must settle before the register clock clk rises. The register output r then starts the second half, whose output yc is read one period later.
  • h (first half)
  • clk
  • r (register)
  • yc (result)
-0.5 V00.5 V1.0 V1.5 V2.0 Vh (first half), clk-0.5 V00.5 V1.0 V1.5 V2.0 Vr (register), yc (result)02 ns4 ns6 ns8 ns10 ns12 ns14 nsTime-0.5 V00.5 V1.0 V1.5 V2.0 Vh (first half), clk-0.5 V00.5 V1.0 V1.5 V2.0 Vr (register), yc (result)05 ns10 ns15 nsTime
Read the plot as a table
Timeh (first half)clkr (register)yc (result)
0 s183.4 nV1.44 V175.8 pV183.4 nV
2 ns1.44 V-749.8 nV-273.3 nV183.4 nV
4 ns1.44 V2.204 µV459.3 nV22.69 µV
6 ns1.44 V2.204 µV459.3 nV22.69 µV
8 ns1.44 V2.204 µV459.3 nV22.69 µV
10 ns1.44 V2.204 µV459.3 nV22.69 µV
12 ns1.44 V2.204 µV459.3 nV22.69 µV
14 ns1.44 V2.204 µV459.3 nV22.69 µV

Measured on ngspice 42, tt, 27 °C. Every result is on time, at 211 fJ per operation.

The pipeline needs 1.44 V, not much below 1.5 V. The register's own delay takes a large part of each period.

So with this short path the pipeline loses: 211 fJ against 183 fJ. Make the path longer and the register matters less.

32-stage unit, one operation every 2 nsLowest supplyEnergy per operation
One unit1.49 V365 fJ
Two units and a multiplexer1.19 V254 fJ
Pipeline: two halves and a register1.3 V309 fJ

Probe data, same bench with a twice-as-long chain and twice the period. Typical chip, 27 °C.

With 32 stages the pipeline saves 15 %. Two units still save more, 30 %: a multiplexer costs less than a clocked register.

Energy Against Throughput

Measured sweepEnergy per Operation Against ThroughputEach point is one architecture at its lowest supply (to 10 mV) that keeps every result on time. Typical chip, 27 °C.
  • one unit
  • two units
  • pipeline
80 fJ100 fJ120 fJ140 fJ160 fJ180 fJ200 fJ220 fJ240 fJ260 fJ280 fJEnergy per operation0.4 GHz0.5 GHz0.6 GHz0.7 GHz0.8 GHz0.9 GHz1.0 GHz1.1 GHz1.2 GHzThroughputthe 1 ns example50 fJ100 fJ150 fJ200 fJ250 fJ300 fJEnergy per operation0.4 GHz0.6 GHz0.8 GHz1.0 GHz1.2 GHzThroughputthe 1 ns example
Read the plot as a table
Throughputone unittwo unitspipeline
400 MHz95.49 fJ93.66 fJ122.1 fJ
500 MHz108.9 fJ102 fJ134.1 fJ
600 MHz121.9 fJ109 fJ148.3 fJ
700 MHz135.6 fJ117.6 fJ162.3 fJ
800 MHz149.4 fJ126.8 fJ176.3 fJ
900 MHz166.3 fJ135.4 fJ193.9 fJ
1 GHz183.2 fJ144 fJ211.5 fJ
1.1 GHz203.6 fJ153.5 fJ229.5 fJ
1.2 GHz224.1 fJ162.9 fJ247.5 fJ

Measured with ngspice 42 through the site's spice route: 6 throughputs × three architectures, each at its own lowest supply.

At a low throughput every design runs at a low voltage. As the throughput rises, one unit must climb towards 1.8 V first.

Two units stay cheapest at every throughput measured here. Their advantage is largest where one unit is pushed hardest: at 1.25 GHz they spend 28 % less.

The pipeline never wins with this short path.

The gap closes at low throughput. As the supply falls toward the threshold voltage, the delay grows steeply. So halving the rate saves less and less voltage.

Race to Idle, or Run Slowly

When a deadline is longer than the work, there is a choice. Race to idle runs fast, finishes early and then idles. Running slowly stretches the work over the whole time.

Measured sweepPower Against Time for One Frame of 200 OperationsA fast chip at 100 °C, one 16-stage unit. Racing runs at 1.8 V, 0.8 ns per operation, then idles with its clock stopped. The slow runs take 5 ns per operation.
  • race, then idle
  • slow at 1.8 V
  • slow at 0.83 V
1 µW10 µW100 µW1 mWSupply power00.2 µs0.4 µs0.6 µs0.8 µs1.0 µsTime in the frame1 µW10 µW100 µW1 mWSupply power00.5 µs1.0 µsTime in the frame
  1. Time in the frame 80 nsrace, then idle = 345.8 µW · slow at 1.8 V = 56.74 µW · slow at 0.83 V = 11.07 µW

    Racing burns 346 µW while it works. The slow run at 1.8 V burns 56.7 µW, and at 0.83 V only 11.1 µW.

  2. Time in the frame 580 nsrace, then idle = 1.682 µW · slow at 1.8 V = 56.74 µW · slow at 0.83 V = 11.07 µW

    After 160 ns the racer idles. Its leakage is 1.68 µW, far below every working power. The slow runs work until the frame ends.

Read the plot as a table
Time in the framerace, then idleslow at 1.8 Vslow at 0.83 V
0 s345.8 µW56.74 µW11.07 µW
80 ns345.8 µW56.74 µW11.07 µW
200 ns1.682 µW56.74 µW11.07 µW
400 ns1.682 µW56.74 µW11.07 µW
580 ns1.682 µW56.74 µW11.07 µW
600 ns1.682 µW56.74 µW11.07 µW
800 ns1.682 µW56.74 µW11.07 µW
1 µs1.682 µW56.74 µW11.07 µW

Energy per frame is the area under each line. Racing takes 56.7 pJ, slow at 1.8 V 56.7 pJ, slow at 0.83 V 11.1 pJ. Drawn from measured values on a logarithmic axis.

Strategy for 200 operations in 1 µsTypical chip, 27 °CFast chip, 100 °C
Race at 1.8 V, then idle with the clock stopped (still leaking)53.5 pJ56.7 pJ
Run slowly at 1.8 V (lower clock only)53.5 pJ56.7 pJ
Race at 1.8 V, then switch the unit off (no idle leakage)53.5 pJ55.3 pJ
Run slowly at the lowest supply for 5 ns (0.96 V and 0.83 V)14.3 pJ11.1 pJ

Energy per frame is the measured energy per operation times 200. The first row adds the measured leakage of the idle unit for 840 ns. The third row assumes a perfect power switch (lesson W3.1).

Compare the first two rows. At one voltage every operation costs the same switching energy. Both units also leak for the whole microsecond: one while it waits, the other between operations. So they come out equal, here within 0.01 % on the hot chip.

Racing at one voltage gains only if the idle state is cheaper than leaking. Switching the unit off after the race saves 1.41 pJ per frame on the hot chip. On a typical chip it saves only 5.12 fJ.

The fourth row wins by far. A lower voltage makes every operation cheaper: 5.12 times less energy than racing on the hot chip.

DC sweepSupply Current of the Idle Unit, Fast Chip at 100 °COne 16-stage unit with its input held low, the supply swept from 0.6 V to 1.8 V.
10 nA100 nA1 µASupply current0.6 V0.8 V1.0 V1.2 V1.4 V1.6 V1.8 VSupply voltage10 nA100 nA1 µASupply current1.0 V1.5 VSupply voltage
Read the plot as a table
Supply voltageidle current
600 mV79.97 nA
800 mV125.6 nA
1 V192.1 nA
1.2 V288.5 nA
1.4 V428.4 nA
1.6 V632.9 nA
1.8 V934.3 nA

Measured on ngspice 42: 934 nA at 1.8 V and 134 nA at 0.83 V. On a typical chip at 27 °C the same unit draws 3.38 nA at 1.8 V.

The same sweep on a typical chip at 27 °C
DC sweepSupply Current of the Idle Unit, Typical Chip at 27 °CThe same deck with .lib sky130 tt and .temp 27.
100 pA1 nA10 nASupply current0.6 V0.8 V1.0 V1.2 V1.4 V1.6 V1.8 VSupply voltage100 pA1 nA10 nASupply current1.0 V1.5 VSupply voltage
Read the plot as a table
Supply voltageidle current
600 mV117.8 pA
800 mV220.2 pA
1 V402 pA
1.2 V715.3 pA
1.4 V1.237 nA
1.6 V2.077 nA
1.8 V3.385 nA

Measured on ngspice 42: 3.38 nA at 1.8 V and 357 pA at 0.96 V.

Racing beats a low voltage only when other power sleeps with the block. A PLL or a bus kept awake for it are examples.

Take a racer that switches the unit off when it is done. It wins once that other power exceeds 46.7 µW on a typical chip. On the hot, fast chip the break-even is 52.7 µW.

Asked in interviews
  1. Why can two slow units use less energy than one fast unit?

    Each unit gets two periods per operation, so the supply can fall. Energy follows C·VDD², which more than pays for the multiplexer: 21 % less here.

  2. When does pipelining not save power?

    When the path is short compared with the register. The register adds delay to each stage and burns clock energy in every cycle.

  3. What is race to idle, and when does it win?

    Run fast, finish early, then idle or power down. At one voltage the switching energy is the same either way. So racing gains only if the idle state leaks less, for example behind a power switch. Against running slowly at a lower voltage it also needs platform power that sleeps with the block.

Try It

Run the netlist. It builds all three architectures, but only the one that V1 powers runs. wrong counts late or wrong results, and eops is the supply energy of 8 operations.

Divide eops by 8: one unit at 1.5 V spends 183 fJ per operation. Now lower V1 to 1.49 V and run again. Some results arrive late, and wrong is no longer 0.

Move the source to two units: write V1 dual 0 1.21. Then try V1 pipe 0 1.44. Compare eops with the table in the first section.

Change tt to ss and find the lowest voltage again. A slow chip needs more.

SPICEngspice 42 · simulator service
Edit the netlist, then run it
Expected measurements: va = 1.500000e+00 eops = -1.46520e-12 from= 3.00000e-09 to= 1.10000e-08 wrong = 0.000000e+00 at= 1.500000e-08

Check Yourself

ConceptOne unit meets a throughput at 1.5 V. Two units behind a multiplexer meet it at 1.21 V. What did the run measure for the energy per operation of the two-unit design?
ConceptA block runs from a regulator that cannot go below 1.2 V. One unit already meets the throughput at 1.2 V. What do two parallel units bring?
ConceptWhy did the pipeline of the 16-stage path need almost as high a supply as one unit?
ConceptA chip must do a fixed task before a deadline. Its supply voltage is fixed, and when idle it only stops its clock. Which uses less energy: racing at full clock speed, or a slower clock until the deadline?
ConceptWhen does racing to idle beat running slowly at the lowest voltage?
Predict the outputA racer switches its unit off when done, so it saves the platform power for the time it sleeps. This deck computes the platform power at which racing breaks even. Type the line (format: p_break = 1.00000e+00).
* break-even power for racing to a switched-off idle state
.param ops=200 e_fast=267f e_slow=71f t_saved=840n
V1 a 0 1
R1 a 0 1k
.tran 1n 2n
.meas tran p_break param='ops*(e_fast-e_slow)/t_saved'
.end

Common Mistakes

Wrong Assume two units at half speed always save energy.

The saving comes only from the lower voltage. If the supply cannot fall, for example because a regulator or a memory sets a floor, the second unit only adds energy.

Fix Check that the voltage can really drop, then compare energy at the voltages each design can use.

Wrong Size the voltage of the parallel design from the delay of one bare unit.

The multiplexer adds delay and energy. One bare unit meets two periods at 1.17 V, yet the two-unit design needed 1.21 V.

Fix Simulate the whole datapath, multiplexer included, at the slow corner.

Wrong Pipeline a short path to save power.

A register costs clock energy in every cycle and eats part of each period. With 16 stages the pipeline spent 211 fJ per operation against 183 fJ for one unit.

Fix Pipeline for voltage only when each stage is long compared with the register's own delay and energy.

Remember

  • Extra hardware can buy time per operation, and time lets the supply voltage fall.
  • Two units at half the rate spent 21 % less energy per operation here, multiplexer included.
  • A pipeline pays only on a long path; on a short one the register costs more than the voltage saves.
  • All of it depends on the voltage really dropping. A supply floor cancels the gain.
  • Race to idle gains nothing at one voltage unless the idle state stops leaking. Against a lower voltage it needs power that sleeps with the block.

Practice

Report a problem with this page