Memory Power: Banking, Chip Selects and Sleep Modes

Split a memory into banks, enable only the bank an access needs, and put idle banks to sleep without stalling the program.

You Will Be Able To

  • Decode the bank of an address and drive one chip select per access.
  • Count how long each bank sits awake and unused on real traffic.
  • Choose a sleep timeout from the gaps in the traffic, and explain what a wake costs.
Lesson
W2.5
Time
25 min
Runs on
Your browser
Check
5 questions

A small processor has a 1024-word data memory. It works on one region for a while, then moves on. In the lab its memory sits awake and unused in 95.6 % of bank-cycles.

One Memory as Four Banks of 256 WordsArrows show the address and the data. The top two address bits pick the bank; only that bank gets its chip select.
  • I/O
  • Control / FSM
  • Storage
  • Datapath
  • Data
  • Control
One memory as four banks of 256 wordsArrows show the address and the data. The top two address bits pick the bank; only that bank gets its chip select. io Address 10 bits control Bank decoder addr[9:8] storage Bank 0 words 000-0FF storage Bank 1 words 100-1FF storage Bank 2 words 200-2FF storage Bank 3 words 300-3FF datapath Read mux selects the bank'sdata addr[9:8]cs0cs1cs2cs3rdatardata
  1. io Address10 bits
    • addr[9:8] → Bank decoder
  2. control Bank decoderaddr[9:8]
    • cs0 → Bank 0
    • cs1 → Bank 1
    • cs2 → Bank 2
    • cs3 → Bank 3
  3. storage Bank 1words 100-1FF
    • rdata → Read mux
  4. datapath Read muxselects the bank's data
  5. storage Bank 2words 200-2FF
    • rdata → Read mux
  6. storage Bank 0words 000-0FF
  7. storage Bank 3words 300-3FF
  1. Read 0x1A3

    The top bits are 01, so only cs1 is high. Bank 1 swings its bit lines; banks 0, 2 and 3 stay still.

  2. Read 0x1A4

    The same bank again. A program that stays in one region keeps the other three banks idle for long stretches.

  3. Read 0x2F0

    The program moves on: cs2 now. Bank 1 becomes idle and could go to sleep.

  4. No chip selects

    Without a decoder, every access enables all four banks. That costs as much as one 1024-word array, and banking saves nothing.

Read the diagram as text
  • Address: 10 bits
  • Bank decoder: addr[9:8]
  • Bank 0: words 000-0FF
  • Bank 1: words 100-1FF
  • Bank 2: words 200-2FF
  • Bank 3: words 300-3FF
  • Read mux: selects the bank's data
  • Address → Bank decoder: addr[9:8]
  • Bank decoder → Bank 0: cs0
  • Bank decoder → Bank 1: cs1
  • Bank decoder → Bank 2: cs2
  • Bank decoder → Bank 3: cs3
  • Bank 1 → Read mux: rdata
  • Bank 2 → Read mux: rdata

A bank's bit lines run past every row of that bank. A smaller bank has shorter bit lines, so each access charges less capacitance.

One Chip Select per Access

Each read or write charges and discharges the bit lines of the enabled bank. So the energy of an access grows with the size of what is enabled, not with the size of the word read.

The lab drives 600 requests from a program-like stream into the same four banks, twice. Once every bank is enabled on every access; once only the addressed bank is.

OrganizationBank enables for 600 accessesWords on the enabled bit lines per accessAwake but unused bank-cycles
All four banks enabled on every access2,4001,02413,168
Chip select: only the addressed bank60025613,168

Measured by the lab on Icarus. Both versions read back every word correctly and finish in 3,442 cycles.

With chip selects the memory enables a quarter of the array per access. The read data is the same, and no cycle is lost.

The last column has not moved: 13,168 bank-cycles awake and unused, 95.6 % of all bank-cycles. An idle bank still leaks, and a chip select does nothing about that.

Sleep Modes and the Wake Cost

A bank that no one uses can go to a low-leakage mode. Arm's Power Control System Architecture defines three modes for a power domain, and a memory bank can have the same three.

ModeContentsCan be accessedLeakageCost to get back
ONKeptYesFullNone
RET (retention)Kept, if the RAM cells support retentionNoLowerA wake time; lower than OFF
OFFLostNoLowestPower-up, then refill or restore the contents

From Arm PCSA (DEN0050 2.1) §4.2.1. Memory macros name their modes in their own datasheets; check the names, wake times and leakage there.

The lab's banks use a retention-like mode. After SLEEP cycles without an access a bank sleeps and keeps its data. An access to it then waits two cycles while it wakes.

Measured sweepSleep Timeout Against Idle Time and StallsEach point is one run of the lab with the same 600 requests and a different SLEEP value.
  • awake but unused bank-cycles
  • stall cycles
0100020003000400050006000Count1 cycles10 cycles100 cyclesSleep timeouttimeout 40100020003000400050006000Count1 cycles10 cycles100 cyclesSleep timeouttimeout 4
  1. Sleep timeout 1 cyclesawake but unused bank-cycles = 471 · stall cycles = 1401

    Timeout 1: banks sleep in every short gap. The program stalls 1,401 cycles and takes 41 % longer.

  2. Sleep timeout 4 cyclesawake but unused bank-cycles = 1131 · stall cycles = 282

    Timeout 4: the gaps of 0 to 3 cycles no longer put a bank to sleep. Stalls fall to 282.

  3. Sleep timeout 64 cyclesawake but unused bank-cycles = 4916 · stall cycles = 84

    Timeout 64: banks rarely sleep. Stalls are 84, but awake idle time is back up to 4,916.

Read the plot as a table
Sleep timeoutawake but unused bank-cyclesstall cycles
1 cycles4711401
4 cycles1131282
10 cycles1742264.3
64 cycles491684

Measured by the lab. Without sleep: 13,168 awake idle bank-cycles and no stalls.

The best timeout sits just above the short gaps of the traffic. Here a timeout of 4 cuts the awake idle time 11.6 times. The program runs 8 % longer.

Whether a wake pays also depends on the energy of the wake itself. A later lesson turns this into a break-even time. This lesson counts only the time.

Asked in interviews
  1. Why does banking save memory power?

    An access charges the bit lines of the enabled array. A smaller bank has shorter bit lines with less capacitance. With a chip select only one bank switches per access, and idle banks can sleep.

  2. What is the difference between retention and power-off for a memory?

    Retention keeps the contents at reduced leakage, and the memory cannot be accessed until it wakes. Power-off loses the contents and saves more. The data must then be restored or refilled.

  3. How do you pick a memory sleep timeout?

    From the idle gaps of real traffic. A timeout shorter than the common gaps causes many wakes and stalls. A long one leaves banks awake and leaking. Pick it just above the frequent short gaps.

Try It

Run it. Eight copies of the memory get the same 600 requests. The first two differ only in their chip selects; the rest sleep after a different idle time.

Change WAKE = 2 to WAKE = 8 in bankmem. Predict which column grows for every sleeping copy, and which copy suffers most.

Change the long pause from 8'd40 to 8'd4. The traffic now has no long gaps. Check whether any timeout still saves idle time without stalls.

RUNIcarus Verilog · in your browser
Edit the code, then run it
Expected: all banks enabled: bank enables=2400 awake idle=13168 stalls=0 cycles=3442 errors=0 chip select only: bank enables=600 awake idle=13168 stalls=0 cycles=3442 errors=0 sleep after 1: bank enables=600 awake idle=471 stalls=1401 cycles=4843 errors=0 sleep after 2: bank enables=600 awake idle=814 stalls=1020 cycles=4462 errors=0 sleep after 4: bank enables=600 awake idle=1131 stalls=282 cycles=3724 errors=0 sleep after 8: bank enables=600 awake idle=1514 stalls=273 cycles=3715 errors=0 sleep after 16: bank enables=600 awake idle=2223 stalls=246 cycles=3688 errors=0 sleep after 64: bank enables=600 awake idle=4916 stalls=84 cycles=3526 errors=0

Check Yourself

ConceptWhy does an access to a 256-word bank cost less energy than an access to a 1024-word array?
ConceptThe four banks of the lab are enabled on every access. What did the bank enables count show for 600 accesses?
ConceptWith a sleep timeout of 1 cycle, what happened to the lab's program?
ConceptA cache must still hold its data after a short idle period. Which mode fits?
Predict the outputSame program as the section check, with a timeout of 2 instead of 4. What exact line does it print?
module tb;
  int gaps [5];
  int timeout = 2, wake = 2, wakes = 0;
  initial begin
    gaps[0] = 1; gaps[1] = 4; gaps[2] = 2; gaps[3] = 9; gaps[4] = 3;   // idle cycles before each access
    for (int k = 0; k < 5; k++)
      if (gaps[k] >= timeout) wakes++;     // the bank fell asleep during this gap
    $display("wakes=%0d waited=%0d", wakes, wakes * wake);
  end
endmodule

Common Mistakes

Wrong Split a memory into as many small banks as possible.

Each bank needs its own decoders, sense amplifiers and wiring, so the overhead grows while the saving per access shrinks.

Fix Pick a bank size from the access pattern, and stop when one more split saves little.

Wrong Read the same word from the memory in every cycle while it has not changed.

Each read enables a bank and swings its bit lines, even when the result is the value already held.

Fix Keep the last value in a register and read again only when the address or the data changes.

Wrong Count only the chip select, and drive the address and write data into every bank.

The shared address and data wires still swing the input pins of every bank on every access.

Fix Check whether the macro ignores its inputs while deselected, and hold the shared buses still between accesses.

Remember

  • An access charges the bit lines of every enabled array, so the energy follows the enabled size.
  • A bank decoder with one chip select per access enables a fraction of the memory.
  • Chip selects do nothing for idle banks. Real traffic leaves most bank-cycles awake and unused.
  • Retention keeps the contents at lower leakage; power-off loses them.
  • A sleep timeout trades idle time against stalls. Set it just above the frequent short gaps.

Practice

Report a problem with this page