Why Serial at Speed
Explain why fast links use a few serial lanes with an embedded clock, and find the rate where a parallel bus breaks.
You Will Be Able To
- Predict the rate at which a parallel bus with a given skew fails.
- Explain why a forwarded clock does not remove skew between wires.
- Explain what an embedded clock asks of the data.
- Choose the right word: lane, link, unit interval, baud or bit rate.
- Lesson
- P10.1
- Time
- 25 min
- Runs on
- Your browser
- Check
- 5 questions
An eight-bit bus works at 2 Gb/s on each wire. The team raises the clock to 2.5 Gb/s, and 22 of 40 words come out wrong. Every wire still has clean edges. The clock edge no longer lands where all eight wires are stable.
| Rate on each wire | Bit time | Shared window | Wrong words | Wrong words, one wire 400 ps longer |
|---|---|---|---|---|
| 100 Mb/s | 10000 ps | 9710 ps | 0 of 40 | 0 of 40 |
| 200 Mb/s | 5000 ps | 4710 ps | 0 of 40 | 0 of 40 |
| 500 Mb/s | 2000 ps | 1710 ps | 0 of 40 | 0 of 40 |
| 1 Gb/s | 1000 ps | 710 ps | 0 of 40 | 0 of 40 |
| 2 Gb/s | 500 ps | 210 ps | 0 of 40 | 20 of 40 |
| 2.5 Gb/s | 400 ps | 110 ps | 22 of 40 | 31 of 40 |
| 5 Gb/s | 200 ps | -90 ps | 31 of 40 | 37 of 40 |
| 10 Gb/s | 100 ps | -190 ps | 33 of 40 | 35 of 40 |
Bench run: eight data wires and a clock wire, 40 random words per rate. Trace delays from 450 to 700 ps, setup and hold 20 ps each. All of these are bench values.
Skew Eats the Bit Time
- Storage
- Channel / handshake
- Clock / reset
- Datapath
- Data
- Clock
- Parallel: launch
All eight bits leave on one clock edge. Each wire has a slightly different length, so the bits arrive at eight different times.
- Parallel: capture
One clock edge must catch all eight wires. It has to fall after the slowest wire settles and before the fastest wire changes again.
- Serial: one wire pair
The serializer sends the same bits one after another on one lane. No second data wire exists to be early or late.
- Serial: clock from the data
The receiver finds the bit timing from the edges of the data itself. The lane delay moves data and timing together.
Read the diagram as text
- Transmit register: 8 bits at once
- 8 data wires: each with its own delay
- Clock wire: one more delay
- 8 flip-flops: one clock edge for all
- Serializer: 8 bits, one after another
- One lane: one differential pair
- Clock recovery: clock from the data edges
- Transmit register → 8 data wires
- 8 data wires → 8 flip-flops: arrive at 8 different times
- Transmit register → Clock wire
- Clock wire → 8 flip-flops: samples all 8
- Serializer → One lane
- One lane → Clock recovery: data and timing together
The time difference between wires is called skew. The serial lane has no skew against itself.
A flip-flop needs its input to be stable for a short time before the clock edge (setup) and after it (hold).
On a bus, one clock edge samples every wire. So there must be a moment in each bit when all the wires are stable together. Call it the shared window.
- bit time
- shared window
Read the plot as a table
| Rate on each wire, Mb/s | bit time | shared window |
|---|---|---|
| 100 | 10 ns | 9.71 ns |
| 1000 | 1 ns | 710 ps |
| 10000 | 100 ps | -190 ps |
Bench run. The gap between the curves is constant: 250 ps of skew plus 40 ps of setup and hold. The window reaches zero while the bit time is still 290 ps.
The skew does not shrink when the clock gets faster. The traces are as long as they were. So skew takes a growing share of every bit.
The Clock Wire Has Skew Too
Sending the clock along with the data is called source-synchronous clocking. It removes the delay that all wires share. It cannot remove the differences between them.
| Wire | Trace delay | Setup margin at 2.5 Gb/s | Hold margin at 2.5 Gb/s |
|---|---|---|---|
| 0 | 500 ps | 180 ps | 180 ps |
| 1 | 520 ps | 160 ps | 200 ps |
| 2 | 480 ps | 200 ps | 160 ps |
| 3 | 610 ps | 70 ps | 290 ps |
| 4 | 450 ps | 230 ps | 130 ps |
| 5 | 530 ps | 150 ps | 210 ps |
| 6 | 700 ps | -20 ps | 380 ps |
| 7 | 490 ps | 190 ps | 170 ps |
Bench run. The clock wire has a 500 ps delay and its edge is sent in the middle of the bit. Wire 6 is the slowest, so it settles last. The first wrong word had the wire pattern 01000000, with wire 7 on the left.
The clock wire is one more trace with its own delay. Here it sits early in the window, so the slowest wire runs out of setup time first. In 100 Mb/s steps the bus first fails at 2.3 Gb/s. Wire 6 then has a setup margin of -3 ps.
At that rate the shared window is still open. The clock edge is simply not in the middle of it. The twin in the first table makes one trace 400 ps longer, and the bus already fails at 2 Gb/s.
One Lane with the Clock Inside the Data
A serial lane sends the bits one after another on one differential pair. There is no clock wire. The receiver recovers the clock from the edges in the data. This is an embedded clock.
The bench sent 80 bits at 10 Gb/s down one lane with trace delays of 450, 700, 900 ps. No bit was wrong in any run. The delay moves the data and its timing together, so it cancels.
The embedded clock has a price. The receiver needs edges, so the data must change often enough. Raw data makes no such promise. Line codes and scramblers exist to keep it (TI SLYY057B, p.3).
To go faster still, a link uses several lanes. Each lane recovers its own clock. The lanes are lined up later, on whole symbols, where a skew of many bit times is easy to absorb.
The Counter-Example: Wide Buses That Train Each Wire
Wide parallel buses did not disappear. DDR5, GDDR7 and HBM memories still send data on many wires beside a strobe, at several Gb/s per wire. HBM4 has 2,048 data wires at up to 10 Gb/s each (Rambus ISCA 2025 DRAM tutorial, slide 14).
They survive skew by measuring it. At start-up the controller trains each wire (slides 43 and 44). It sweeps that wire's delay, finds where the data is caught right, and parks the delay in the middle.
| Wire | Trace delay | Passing extra delay at 5 Gb/s | Trim chosen | Delay after training |
|---|---|---|---|---|
| 0 | 500 ps | 220 to 380 ps | 300 ps | 800 ps |
| 1 | 520 ps | 200 to 360 ps | 280 ps | 800 ps |
| 2 | 480 ps | 240 to 400 ps | 320 ps | 800 ps |
| 3 | 610 ps | 110 to 270 ps | 190 ps | 800 ps |
| 4 | 450 ps | 270 to 400 ps | 335 ps | 785 ps |
| 5 | 530 ps | 190 to 350 ps | 270 ps | 800 ps |
| 6 | 700 ps | 20 to 180 ps | 100 ps | 800 ps |
| 7 | 490 ps | 230 to 390 ps | 310 ps | 800 ps |
Bench run on the same eight wires, with the clock wire moved to 800 ps. Each extra delay was swept from 0 to 400 ps in 10 ps steps. The sweep cut off wire 4's window.
| Rate per wire | Untrained: words wrong of 40 | Trained: words wrong of 40 |
|---|---|---|
| 2,500 Mb/s | 22 | 0 |
| 5,000 Mb/s | 31 | 0 |
| 10,000 Mb/s | 33 | 0 |
| 12,500 Mb/s | not run | 0 |
| 16,000 Mb/s | not run | 21 |
| 20,000 Mb/s | not run | 21 |
Bench runs. Untrained, the bus fails from 2.5 Gb/s. Trained, it runs clean to 12.5 Gb/s.
After training the skew is gone except the 15 ps that the 10 ps steps leave. The limit is now the setup and hold times, and in real silicon jitter and noise.
The training has a cost. It must run at every power-up and track temperature, and each wire needs its own delay line. DDR5 adds dedicated training modes and patterns for this (Kingston DDR5 overview).
So the rule is not that parallel cannot go fast. It is that every wire must be measured and trimmed, while a serial lane carries its own timing for free.
The Words of a Serial Link
| Term | Meaning |
|---|---|
| Differential pair | Two wires that carry one signal and its inverse. The receiver looks at the difference. |
| Lane | One differential pair in each direction. |
| Link | One or more lanes between two ports, used together. |
| Unit interval (UI) | The time of one symbol on the wire: 100 ps at 10 Gb/s with two levels. |
| Baud | Symbols per second on the wire. It equals the line rate only for two-level signalling. |
| Line rate | Bits per second on the wire, line-code overhead included. USB Gen 1's 5 Gb/s is a line rate. |
| Data rate | Data bits per second after the line code: the payload rate. |
| Embedded clock | No clock wire. The receiver recovers the clock from the data edges. |
USB 3.2 calls its lane "dual-simplex": four wires, one pair for each direction (Table 3-1).
The two wires of one pair must still match each other closely. TI asks for 5 ps inside a pair in one of its layout guides (SPRABC1 §3.2). That is one short pair to match, not a whole bus.
Try It
The bench sends 40 random words at two rates and prints the margins of each wire at the second rate.
A negative setup margin means the wire changes less than 20 ps before the clock edge. The receiver then captures an unknown value.
Change clk_delay_ps to 575, the middle of the data delays. Wire 6 gets its setup margin back, and both rates pass.
2000 Mb/s: 0 of 40 words wrong 2500 Mb/s: 22 of 40 words wrong wire 0: delay 500 ps, setup margin 180 ps, hold margin 180 ps wire 1: delay 520 ps, setup margin 160 ps, hold margin 200 ps wire 2: delay 480 ps, setup margin 200 ps, hold margin 160 ps wire 3: delay 610 ps, setup margin 70 ps, hold margin 290 ps wire 4: delay 450 ps, setup margin 230 ps, hold margin 130 ps wire 5: delay 530 ps, setup margin 150 ps, hold margin 210 ps wire 6: delay 700 ps, setup margin -20 ps, hold margin 380 ps wire 7: delay 490 ps, setup margin 190 ps, hold margin 170 ps first wrong word 6, wrong wires 01000000 (wire 7 on the left)Read included file: hs_pkg.sv
`timescale 1ns/1ns
// hs_pkg.sv: shared pure functions, and the bench random generator (read-only).
package hs_pkg;
// xorshift32 (Marsaglia 2003): a linear generator over GF(2), period 2^32 - 1. A zero state is
// replaced by 1, because zero maps to itself.
function automatic [31:0] xs32(input [31:0] s);
reg [31:0] x;
x = (s == 32'd0) ? 32'd1 : s;
x = x ^ (x << 13); x = x ^ (x >> 17); x = x ^ (x << 5);
return x;
endfunction
// A Fibonacci PRBS generator x^n + x^m + 1 (PRBS7: 7,6; PRBS9: 9,5; PRBS15: 15,14; PRBS23: 23,18;
// PRBS31: 31,28). The output bit is s[n-1]; the next state shifts the feedback into bit 0.
function automatic [31:0] prbs_next(input [31:0] s, input int n, input int m);
reg fb;
fb = s[n-1] ^ s[m-1];
return ((s << 1) | {31'd0, fb}) & ((n == 32) ? 32'hffffffff : ((32'd1 << n) - 32'd1));
endfunction
function automatic int ones(input [63:0] v);
int n; n = 0;
for (int i = 0; i < 64; i++) n += v[i];
return n;
endfunction
// GF(16) with x^4 + x + 1, and GF(1024) with x^10 + x^3 + 1: shift-and-add multiply, no tables.
function automatic [3:0] gf16_mul(input [3:0] a, input [3:0] b);
reg [3:0] p, x; p = 4'd0; x = a;
for (int i = 0; i < 4; i++) begin
if (b[i]) p = p ^ x;
x = x[3] ? ({x[2:0], 1'b0} ^ 4'b0011) : {x[2:0], 1'b0};
end
return p;
endfunction
function automatic [3:0] gf16_pow(input int e); // alpha^e, alpha = x
reg [3:0] p; int k; p = 4'd1; k = e % 15; if (k < 0) k += 15;
for (int i = 0; i < k; i++) p = gf16_mul(p, 4'd2);
return p;
endfunction
function automatic int gf16_log(input [3:0] a); // -1 for 0
reg [3:0] p; p = 4'd1;
if (a == 4'd0) return -1;
for (int i = 0; i < 15; i++) begin if (p == a) return i; p = gf16_mul(p, 4'd2); end
return -1;
endfunction
function automatic [9:0] gf1024_mul(input [9:0] a, input [9:0] b);
reg [9:0] p, x; p = 10'd0; x = a;
for (int i = 0; i < 10; i++) begin
if (b[i]) p = p ^ x;
x = x[9] ? ({x[8:0], 1'b0} ^ 10'b0000001001) : {x[8:0], 1'b0};
end
return p;
endfunction
function automatic [9:0] gf1024_pow(input int e);
reg [9:0] p; int k; p = 10'd1; k = e % 1023; if (k < 0) k += 1023;
for (int i = 0; i < k; i++) p = gf1024_mul(p, 10'd2);
return p;
endfunction
endpackage
// hs_rng: the bench's random numbers. Same seed, same numbers, on every engine.
module hs_rng;
int unsigned s = 32'h2545f491;
function automatic void seed(input int unsigned v); s = (v == 0) ? 32'd1 : v; endfunction
function automatic int unsigned next(); s = hs_pkg::xs32(s); return s; endfunction
function automatic int below(input int n); s = hs_pkg::xs32(s); return (n <= 0) ? 0 : int'(s % n); endfunction
function automatic bit bit1(); s = hs_pkg::xs32(s); return s[16]; endfunction
function automatic bit chance_ppm(input int ppm); s = hs_pkg::xs32(s); return (s % 1000000) < ppm; endfunction
function automatic [7:0] byte8(); s = hs_pkg::xs32(s); return s[23:16]; endfunction
endmodule
Check Yourself
Common Mistakes
Wrong Say that serial is slower because it sends one bit at a time.
A lane has no skew against another wire, so the shared window does not cap its rate.
Fix Compare total throughput: lanes times line rate times the share of line bits that are data.
Wrong Count a differential pair as two data streams.
The two wires carry one signal and its inverse, and the receiver uses only their difference.
Fix One pair is one stream in one direction. A lane has two pairs, one for each direction.
Wrong Use baud and bit rate as the same number.
Baud counts symbols on the wire, and a symbol may carry more or less than one data bit.
Fix State both: a 5 Gb/s line with 8b/10b coding carries 4 Gb/s of data.
Wrong Expect a forwarded clock to remove every timing problem.
The clock wire has its own delay, and it does nothing about the spread between data wires.
Fix Budget skew, setup and hold against the bit time, and place the clock edge in the middle of the shared window.
Remember
- On a parallel bus one clock edge samples every wire, so all wires must be stable together.
- The shared window is the bit time minus skew, setup and hold. Skew stays the same as the bit time shrinks.
- A forwarded clock is one more wire with its own delay. It cannot remove skew between data wires.
- A serial lane carries its clock in the data edges, so its own delay cancels. The data must then have enough transitions.
- Baud counts symbols on the wire, and bit rate counts data bits. Line coding makes them differ.