Serial vs Parallel
Measure how lane-to-lane skew shrinks the one sample window a parallel bus shares, and say why fast off-chip links use few wires.
- Lesson
- P5.1
- Time
- 20 min
- Runs on
- Your browser
- Check
- 4 questions
A wide parallel bus between two chips passed on the prototype and failed on the production board. The new routing had stretched a few traces, and the errors always came from those lanes. Each lane was fine alone; together they no longer agreed on when a word was complete.
- Cycle 0
One clock edge launches a new word on all eight lanes at once.
- Cycle 4
Lane 4 changes first: its flight time is 1.92 ns.
- Cycle 9
Lane 5 changes last, at 4.17 ns plus a little jitter. Until then the word on the wires is half old, half new.
- Cycle 10
From 4.85 ns to 6.60 ns after launch every lane is steady and inside its flop's setup and hold. One shared clock edge must land here.
Measured: the example's bus, 5 ns per bit, one column per 0.5 ns. The shared window is 1.75 ns of the 5 ns bit.
One Wire per Bit, or One Wire for All
- Parallel
Every lane is launched by the same edge, and one clock edge samples them all. The lanes must arrive together.
- Serial
One lane carries the bits in turn. No lane can arrive early or late against another, but the receiver must find each bit in time (lesson P5.2).
Read the diagram as text
- Sender: one word per clock
- Lane 0: its own flight time
- Lanes 1 to 7: each a little different
- Clock lane: one edge for all
- Receiver flops: sample all at once
- Serializer: word to bits
- One lane: bits in a row
- Deserializer: bits to word
- Sender → Lane 0
- Sender → Lanes 1 to 7
- Sender → Clock lane
- Lane 0 → Receiver flops
- Lanes 1 to 7 → Receiver flops
- Clock lane → Receiver flops: sample
- Serializer → One lane
- One lane → Deserializer
Lesson P6.1 shows the five serial links an SoC uses. This lesson measures what the parallel shape costs.
A parallel bus moves a word per clock, so it looks faster. Its weakness is the shared sample point: the receiver has one clock edge, and every lane must be steady around it. Any difference in arrival time between lanes, called skew, comes out of that window.
Every Lane Has Its Own Eye
The bench launches 32 random words on eight lanes and samples them at every offset after the launch edge, 50 ps apart. A lane's eye is the range of offsets that read every word right. The flop needs 0.4 ns of setup and 0.3 ns of hold, and each edge lands up to 0.25 ns late.
| Lane | Flight | Eye opens | Eye closes | Eye width |
|---|---|---|---|---|
| 0 | 3.15 ns | 3.80 ns | 7.85 ns | 4.05 ns |
| 1 | 3.13 ns | 3.80 ns | 7.85 ns | 4.05 ns |
| 2 | 3.67 ns | 4.35 ns | 8.35 ns | 4.00 ns |
| 3 | 3.33 ns | 4.00 ns | 8.00 ns | 4.00 ns |
| 4 | 1.92 ns | 2.60 ns | 6.60 ns | 4.00 ns |
| 5 | 4.17 ns | 4.85 ns | 8.85 ns | 4.00 ns |
| 6 | 3.58 ns | 4.25 ns | 8.30 ns | 4.05 ns |
| 7 | 2.36 ns | 3.00 ns | 7.05 ns | 4.05 ns |
| All 8 | 4.85 ns | 6.60 ns | 1.75 ns |
Measured. Every lane alone has an eye of 4.00 or 4.05 ns. The shared window opens with the latest lane (5) and closes with the earliest (4).
Each eye is the bit time minus jitter, setup and hold. The lanes differ only in where their eye sits. Lane 4 arrives 2.25 ns before lane 5, so the window all eight share is 1.75 ns, not 4.
Width Eats the Window
Each extra lane is another draw of flight time. It can only push the latest arrival later or the earliest earlier, so the shared window can only shrink. The bench repeats the sweep on eight seeded boards of 64 lanes and reads the window for the first 1, 2, 4 and up to 64 lanes.
| Lanes sampled together | Shared window, 8 boards | Of a bit of |
|---|---|---|
| 1 | 4.00 to 4.05 ns | 5.00 ns |
| 2 | 2.30 to 4.05 ns | 5.00 ns |
| 4 | 2.15 to 3.50 ns | 5.00 ns |
| 8 | 1.55 to 2.45 ns | 5.00 ns |
| 16 | 1.40 to 2.25 ns | 5.00 ns |
| 32 | 1.40 to 1.90 ns | 5.00 ns |
| 64 | 1.20 to 1.65 ns | 5.00 ns |
Measured: the narrowest and widest window over eight seeded boards. The window never grows with width.
The fix is a slower clock. The bench searches each width for the shortest bit time that still leaves a sample point, on board 1.
| Lanes | Pins with the clock | Shortest bit | Total | Per pin |
|---|---|---|---|---|
| 1 | 2 | 0.95 ns | 1.05 Gbit/s | 0.53 Gbit/s |
| 8 | 9 | 3.20 ns | 2.50 Gbit/s | 0.28 Gbit/s |
| 16 | 17 | 3.25 ns | 4.92 Gbit/s | 0.29 Gbit/s |
| 32 | 33 | 3.25 ns | 9.84 Gbit/s | 0.30 Gbit/s |
| 64 | 65 | 3.70 ns | 17.29 Gbit/s | 0.27 Gbit/s |
Measured shortest bit and total; per pin is the total over the pins. 64 lanes carry 16.5 times one lane's data on 65 pins.
This is the case for serial. One fast lane has no partner to be skewed against, so it runs near its own limit, 0.95 ns per bit here. Pins, package balls and board layers are expensive, and a serial link spends them where they work hardest.
What Serial Pays Instead
Serial links trade the skew problem for a timing problem: with one lane, the receiver must know when each bit is there. It can learn that from a clock wire, a strobe sent with the data, its own fast clock, or the data edges themselves. Lesson P5.2 measures all four on one byte.
| Where it shows up | What to look for | Lesson |
|---|---|---|
| QSPI flash | Four data lanes and one clock: a small parallel bus, kept short and slow enough | P7.6 |
| QSPI turnaround | The four lanes change direction together | P7.7 |
| SPI timing | Clock and data skew in a half-cycle budget | P7.3 |
| UART | One lane each way, no clock wire | P6.3 |
| I2C and I3C | Two wires shared by many devices | P8.1, P9.1 |
| Setup and hold | Why a flop needs its input steady around the edge | P1.2 |
The window you measured here returns whenever two signals must arrive together.
Try It
Each lane line is one lane alone: its seeded flight time and the offsets, after the launch edge, at which a sample reads all 31 checked words right. The last line is the window every lane shares, the overlap of the eight eyes.
Try bus.lanes = 16;, or a slower bus with bus.period = 8000;. Then bus.jitter = 0; to see how much of each eye the jitter took. bus.place(2); draws another board.
lane 0: flight 3.15 ns, eye 3.80 to 7.85 ns, 4.05 ns wide lane 1: flight 3.13 ns, eye 3.80 to 7.85 ns, 4.05 ns wide lane 2: flight 3.67 ns, eye 4.35 to 8.35 ns, 4.00 ns wide lane 3: flight 3.33 ns, eye 4.00 to 8.00 ns, 4.00 ns wide lane 4: flight 1.92 ns, eye 2.60 to 6.60 ns, 4.00 ns wide lane 5: flight 4.17 ns, eye 4.85 to 8.85 ns, 4.00 ns wide lane 6: flight 3.58 ns, eye 4.25 to 8.30 ns, 4.05 ns wide lane 7: flight 2.36 ns, eye 3.00 to 7.05 ns, 4.05 ns wide all 8 lanes: window 4.85 to 6.60 ns, 1.75 ns wideRead included file: skew_bus.sv
`timescale 1ps/1ps
module skew_bus;
int lanes = 8; // lanes in use (at most 64)
int period = 5000; // ps per bit on every lane
int base = 1000, spread = 1000; // flight = base + four draws of 0..spread-1 ps
int jitter = 250; // each edge lands 0..jitter ps after its flight time
int t_su = 400, t_h = 300; // the receiving flop
int words = 32, step = 50; // words per offset; offset step in ps
int flight[64], last_change[64], errors[64];
reg [63:0] lane = 0, sent[0:63], pass[0:1023];
int n_off = 0, sample_time = -1000000; bit sampling = 0;
reg [31:0] seed[0:2];
function automatic int draw(input int which, input int range); // 0..range-1
seed[which] = seed[which] * 32'd1664525 + 32'd1013904223;
return (seed[which] >> 8) % range;
endfunction
// Give every lane its flight time.
task automatic place(input int s);
seed[0] = s;
for (int i = 0; i < 64; i++) flight[i] = base + draw(0, spread) + draw(0, spread) + draw(0, spread) + draw(0, spread);
endtask
// A lane that changes less than t_h after a sample spoils that sample (hold).
for (genvar g = 0; g < 64; g++) begin : track
always @(lane[g]) begin
if (sampling && $time - sample_time < t_h) errors[g]++;
last_change[g] = $time;
end
end
// Send `words` random words and sample each one `offset` ps after its launch edge. The same data and the
// same jitter for every offset, so a sweep of offsets maps one fixed set of edges.
task automatic one_offset(input int offset);
reg [63:0] w;
for (int i = 0; i < 64; i++) errors[i] = 0;
seed[1] = 32'd20261108; seed[2] = 32'd5;
fork
for (int k = 0; k < words; k++) begin
w = 0; for (int q = 0; q < 4; q++) w = (w << 16) | draw(1, 65536);
sent[k] = w;
for (int i = 0; i < lanes; i++) lane[i] <= #(flight[i] + draw(2, jitter + 1)) w[i];
#(period);
end
begin
#(offset);
for (int k = 0; k < words; k++) begin
sampling = (k > 0); sample_time = $time;
if (k > 0) for (int i = 0; i < lanes; i++)
if ($time - last_change[i] < t_su || lane[i] !== sent[k][i]) errors[i]++;
#(period);
end
end
join
sampling = 0; #(20000);
endtask
// Every offset, `step` ps apart, from the earliest lane's flight time to the latest lane's flight time
// plus one bit: pass[n] has bit i set when lane i read every word right at offset off0 + n * step.
int off0 = 0;
task automatic sweep();
int lo = 1000000, hi = 0;
for (int i = 0; i < lanes; i++) begin if (flight[i] < lo) lo = flight[i]; if (flight[i] > hi) hi = flight[i]; end
off0 = lo / step * step; n_off = 0;
for (int off = off0; off <= hi + period + jitter; off += step) begin
one_offset(off);
pass[n_off] = 0;
for (int i = 0; i < lanes; i++) pass[n_off][i] = (errors[i] == 0);
n_off++;
end
endtask
// The first and last passing offset of the lanes in mask (-1 when none passes).
task automatic window(input reg [63:0] mask, output int lo, output int hi);
lo = -1; hi = -1;
for (int n = 0; n < n_off; n++) if ((pass[n] & mask) == mask) begin if (lo < 0) lo = off0 + n * step; hi = off0 + n * step; end
endtask
function automatic string ns(input int ps); return $sformatf("%0d.%02d", ps / 1000, (ps % 1000) / 10); endfunction
endmodule
Check Yourself
Remember
- A parallel bus samples every lane on one clock edge, so all lanes must be steady around that edge.
- Each lane alone has an eye of about the bit time minus jitter, setup and hold; the bus only gets the overlap.
- More lanes mean more spread in flight time, so the shared window never grows with width.
- A single fast lane has no lane-to-lane skew, which is why fast off-chip links are serial; the cost is that the receiver must find each bit.