P11.38.1 Debug the Link That Never Completes Training

MediumDebugIcarus Verilog · serverReview the topic

Two backplane ports trade training frames, and the link never comes up. The log says only that. Find the first bad frame and the cause.

Hardware context: An Ethernet MAC and physical coding sublayer: frame builder and checker, reconciliation sublayer and block encoder. Build, check and pace Ethernet frames, and code them for the physical layer as IEEE 802.3 specifies.

The design below is the PMD control function of one 10GBASE-KR port. It handles one training frame in each call.

It finds frame lock from the markers and reads the two fields of the control channel from their DME cells.

It moves its three transmitter taps as the partner asks and reports a status for each tap.

It also runs the training state diagram, from SEND_TRAINING to SEND_DATA.

In the bench a partner asks for seven changes. First comes a preset, then c(+1) down three times.

Then it asks for c(−1) down twice and for c(0) up once.

The partner follows the handshake. It asks until the tap reports a status. Then it holds until the status is not_updated again.

The log says: ERROR: after 400 frames the link is not up.

Step 1: find the first cycle where a signal differs from a correct design, and name the signal. Cycle N is frame N.

Step 2: choose the cause from five bugs. Each is real, and each one ends in this same log line.

Step 3 is optional: fix the design and run the tests.

This design is a model: the line is one level for each half of a DME cell, and the timers count frames.

The lesson's readable library is the sender of a frame. The bench calls it as tx.

Signals in the Timeline

SignalWidthRole
test_number8input of each check
Full specification
The API (keep it)
function automatic void lt_reset(input int max_wait, input int wait_frames);
function automatic [15:0] lt_frame(input [31:0] marker, input [63:0] ctl, input bit rx_trained);   // one training frame; returns status
ItemRule
markerthe 32 UI of the frame marker as received. Bit 31 was sent first. A good marker is FFFF0000h (72.6.10.2.1)
ctlthe control channel: 32 DME cells, two levels a cell. Bit 63 was sent first. Bits 63:32 are the coefficient update field, bits 31:0 the status report field. Cell 15 of a field is sent first
DMEthe level changes at the start of every cell. A change in the middle of a cell is a 1 (72.6.10.2.2). The level before the first cell is marker[0]
Violationa cell that starts with no change of level. Then neither field of this frame is used (72.6.10.2.2)
Frame lockFigure 72-4, one marker a call. Two good markers in a row give frame_lock. In lock, five bad markers in a row take it away. Without lock, one bad marker starts the count again
Reading a framea course choice: the two fields are used only when frame_lock is true after this frame's marker and no cell has a violation
Coefficient update fieldcell 13 is preset and cell 12 is initialize. Cells 5:4 ask c(+1), cells 3:2 c(0) and cells 1:0 c(−1). The codes: 00 hold, 01 increment, 10 decrement, 11 reserved. The other cells are ignored (Table 72-4)
A tapFigure 72-6. At not_updated, a preset, an initialize, an increment or a decrement is carried out once. The tap then reports updated, maximum or minimum
Maximum and minimuma tap that reaches a limit, or is already there, reports that limit (72.6.10.2.5)
Back to not_updatedonly by a hold: the tap's request is 00, and preset and initialize are both 0. Any other field leaves the status as it is
Prioritypreset, then initialize, then the tap's own request (72.6.10.3.4)
Taps of this benchin steps of 1. c(−1): −6 to 0. c(0): 20 to 40. c(+1): −15 to 0. INITIALIZE and reset: −2, 30, −8. Preset: 0, 40, 0. A real PHY has its own limits (72.7.1.10)
Status report fieldcell 15 is receiver ready. Cells 5:4 report c(+1), cells 3:2 c(0) and cells 1:0 c(−1). The codes: 00 not_updated, 01 updated, 10 minimum, 11 maximum. The other cells are sent as 0 and ignored (Table 72-5)
remote_rx_readythe standard asks for at least three frames in a row with receiver ready (72.6.10.3.1). The rest is a course choice. It is true while the last three or more frames that were read all had the bit. A frame that is not read starts the count again, and so does a frame without the bit
TrainingFigure 72-5. SEND_TRAINING to TRAIN_LOCAL on frame_lock. To TRAIN_REMOTE on rx_trained: local_rx_ready is now 1. To LINK_READY on remote_rx_ready. To SEND_DATA when the wait timer ends: signal_detect is 1
Going backTRAIN_LOCAL without frame_lock, and TRAIN_REMOTE without frame_lock or without rx_trained, go back to SEND_TRAINING. That clears local_rx_ready and remote_rx_ready, and the count of ready frames starts again. The taps and their status stay
Two exits at oncea course choice: in TRAIN_REMOTE, going back is tested before remote_rx_ready. The standard gives no order (21.5.3)
LINK_READYhas one way out: the wait timer. Nothing else moves it
No timetransitions take no time (1.2.1, 21.5.1): in one call the diagram moves on until no condition holds
Timersa course choice: they count calls. max_wait is started by lt_reset only. The standard gives 500 ms, about 1,176,000 frames, and 100 to 300 frames for the wait timer (72.6.10.3.2)
max_waitthe call number max_wait after lt_reset ends in TRAINING_FAILURE if it began in SEND_TRAINING, TRAIN_LOCAL or TRAIN_REMOTE. A course choice: this check comes before any other transition of that call
Wait timerentered in call E, LINK_READY ends in SEND_DATA in call E + wait_frames. So the port is in LINK_READY at the end of wait_frames calls, the entering call included
After trainingin SEND_DATA training is false, so frame lock is held off (Figure 72-4). From the call after SEND_DATA is entered, frame_lock is 0 and no frame is read
Order in one callframe lock, then the control channel, then the training diagram, then status
The testsmax_wait is at most 600 and wait_frames is 100 to 300

Random tests compare every output with a private model after every frame.

Diagnose the Failing Run

The test below runs the buggy design. Read its log, then find where it first goes wrong in the waveform.

Log

Waveform

Click a signal at a cycle to select it. Or focus the waveform and use the arrow keys: Left and Right move one cycle, Up and Down one signal.

Step 1 · The first bad cycle and signal
Step 2 · The cause

Step 3 is optional: fix the code in the editor, then run the tests.

Examples

Example 1

A Partner That Asks for Six Changes
Input
`fx_story(3, 2, 0, 12, 100)`
Output
c_plus_1 = -3, c_0 = 40, c_minus_1 = -2, status = 0x8000, training_failure = 0, signal_detect = 1, remote_rx_ready = 1, frame_lock = 1, state = SEND_DATA (5), check_of_every_frame = 0x7511374b

The preset gives 0, 40, 0. Three steps down make c(+1) −3, and two make c(−1) −2. The partner is ready from frame 15 and this port from frame 12. LINK_READY comes at frame 17 and SEND_DATA 100 frames later.

Example 2

Bad Markers in LINK_READY
Input
`fx_quiet(600, 100, 20, 5, 6, 8, 60)`
Output
c_plus_1 = -8, c_0 = 30, c_minus_1 = -2, status = 0x8000, training_failure = 0, signal_detect = 0, remote_rx_ready = 1, frame_lock = 1, state = LINK_READY (4), check_of_every_frame = 0xc8c428a0

The third ready frame is frame 10: LINK_READY. Five bad markers take the frame lock away at frame 24. LINK_READY has one way out, the wait timer, so the state stays. The lock comes back at frame 26.

Example 3

A Partner That Is Never Ready
Input
`fx_quiet(40, 100, 0, 0, 6, 999, 60)`
Output
c_plus_1 = -8, c_0 = 30, c_minus_1 = -2, status = 0x8000, training_failure = 1, signal_detect = 0, remote_rx_ready = 0, frame_lock = 1, state = TRAINING_FAILURE (6), check_of_every_frame = 0xd0a99a0

This port waits in TRAIN_REMOTE with receiver ready set. Frame 40 ends max_wait: TRAINING_FAILURE, and nothing leaves it but a reset.

Constraints and Contract

  • Keep lt_reset and lt_frame with their arguments.
  • Keep the names state, frame_lock, remote_rx_ready, signal_detect, training_failure, status, cm1, c0 and cp1: the fixture reads them.
  • lt_frame returns status.
  • lt_reset must clear all state.
  • Off limits: the names ref_k, ref_tx, rng, tb, root and the modules of the bench.
  • The fixture's own names are off limits too. They begin with fx_.

Hints

Hint 1

Follow upd. Which request does the partner repeat until the end of the run?

Hint 2

Look at the tap that this request names. Where does it stand, and where are its limits?

Hint 3

A tap that cannot move must still answer. Read 72.6.10.2.5 again: which status does it report?

What the grader checks: The failing run is simulated live with the faulty design. The first bad cycle and each option's symptom come from runs of each version on the same bench. The fix is checked on 67 scripted runs, with expected values from a JavaScript model. Each of those folds the outputs after every frame into a check value. Seven more tests compare every output with a private model after every frame, on random frames seeded for each submission. The private model is written in another way than the solution, and it agrees with the JavaScript model on random scripts. This design is a model. The line is one level for each half of a DME cell. So only a missing transition at the start of a cell is a violation. The timers count frames with terminal counts from the bench. The hunt for the marker position (SLIP) is one call for each candidate. The receiver's own tuning is the input rx_trained, and the requests this port sends are the bench's. Six things are course choices. The fields are read only in frame lock. remote_rx_ready follows the run of ready frames. A frame that is not read restarts that run. The timer check comes first. In TRAIN_REMOTE, going back is tested before remote_rx_ready. The tap limits are the bench's own. The training pattern, the electrical limits of the taps and the management variables are not graded.

74 checks on Icarus Verilog · server. Run the examples first, then check the full regression. Full checks use the server judge. Downloads contain your source only; samples require the simulation service. A passing result establishes the tested cases, not a proof for all possible inputs.

See all 1 steps and the final capstone

Report a problem with this page

Your code SystemVerilogSaved locally
Ln 1, Col 1Tab indents · Ctrl/⌘ + Enter runs samples
Not run yet

A Partner That Asks for Six Changes

Input
`fx_story(3, 2, 0, 12, 100)`
Expected output
c_plus_1 = -3, c_0 = 40, c_minus_1 = -2, status = 0x8000, training_failure = 0, signal_detect = 1, remote_rx_ready = 1, frame_lock = 1, state = SEND_DATA (5), check_of_every_frame = 0x7511374b

The preset gives 0, 40, 0. Three steps down make c(+1) −3, and two make c(−1) −2. The partner is ready from frame 15 and this port from frame 12. LINK_READY comes at frame 17 and SEND_DATA 100 frames later.

Run samples executes all 3 public checks. Check all tests runs 74 checks on the server.

Icarus Verilog · server