H1.5 FIFO Subsystem Regression

MediumFixVerilator · serverReview the topic

Fix a regression summary that reports green while required coverage is missing.

Hardware context: Clocked FIFO plus ready/valid output adapter. Qualify the complete FIFO/adapter subsystem with an independent stream oracle.

A regression is done when three things hold. The correct design passes every test. Every planted bug fails at least one test. Every required scenario really happened. This summary checks all three, but it never reports a coverage hole.

The class subsystem_regression collects results from four tests. Each test runs on six FIFO-and-adapter designs: variant 0, the baseline, and variants 1 to 5, each with one planted bug. record() is called once per test and variant. finish() returns the summary.

A test detects a variant when the trusted checker report shows a failing edge. Coverage comes only from the baseline. The verdict has three separate causes: missing coverage, a bug no test caught, and a failing baseline.

Find the bug in finish() and fix it.

Signals in the Timeline

SignalWidthRole
profile2input of each check
baseline_mode1input of each check
controls096input of each check
values0256input of each check
controls196input of each check
values1256input of each check
controls296input of each check
values2256input of each check
controls396input of each check
values3256input of each check
Full specification
The class (keep these names and types)
class subsystem_regression;
  function new();
  function void record(input int test_index, variant, input bit [79:0] result, input bit [7:0] measured_hits);
  function bit [79:0] finish();
endclass
InputMeaning
test_index0 to 3.
variant0 is the baseline, the design believed correct. 1 to 5 each have one planted bug, listed below.
resultThe 80-bit report of the trusted checker from Reset During Adapter Backpressure. Bits 79:48 are its failing-edge mask; any 1 bit means this test failed on this variant.
measured_hitsThe coverage bins this test hit on the baseline.
VariantPlanted bug
1The held value survives reset.
2The read in flight survives reset.
3The held value is dropped while stalled.
4Bit 0 of the data is flipped.
5A read is issued over a stalled value.
Coverage binHit when
0A write is offered while the FIFO is full.
1A valid stream offer is stalled by ready = 0.
2Reset arrives while the output holds a value.
3Reset arrives while a FIFO response is valid, so a read is in flight.
4Reset arrives while the FIFO holds entries.
5The test ends fully drained: ledger, FIFO and output all empty.
6A stalled output is later taken with unchanged data. A reset in between does not count.

Only variant 0 adds to coverage. All seven bins are required, so required_coverage = 0x7f and missing_coverage = required & ~observed.

For variant n, tests_v<n> is the set of tests that detected it. first_test_v<n> is the first of them, counting from 1, or 0 for none. survivors has bit n - 1 set when no test detected variant n. baseline_fail has bit t set when test t failed on the baseline.

verdict bit 0 means coverage is missing, bit 1 means a bug survived, and bit 2 means the baseline failed. Set each one on its own.

Examples

Example 1

A Complete Campaign
Input
Depth 3, correct baseline; tests: long stall with a full FIFO, reset while holding, reset with a read in flight, two round trips
Output
tests_v1 = {1}, first_test_v1 = 2, tests_v2 = {2}, first_test_v2 = 3, tests_v3 = {0}, first_test_v3 = 1, tests_v4 = {0, 1, 2, 3}, first_test_v4 = 1, tests_v5 = {0}, first_test_v5 = 1, observed_coverage = {0, 1, 2, 3, 4, 5, 6}, required_coverage = 0x7f, missing_coverage = {}, baseline_fail = {}, survivors = {}, verdict = {}

The four tests hit every bin and catch all five bugs. The baseline stays clean, so missing, survivors and verdict are all empty. The per-variant fields show which test caught each bug first.

Example 2

Four Easy Tests Prove Little
Input
Depth 3, correct baseline; all four tests are the same two round trips
Output
tests_v1 = {}, first_test_v1 = 0, tests_v2 = {}, first_test_v2 = 0, tests_v3 = {}, first_test_v3 = 0, tests_v4 = {0, 1, 2, 3}, first_test_v4 = 1, tests_v5 = {}, first_test_v5 = 0, observed_coverage = {5}, required_coverage = 0x7f, missing_coverage = {0, 1, 2, 3, 4, 6}, baseline_fail = {}, survivors = {0, 1, 2, 4}, verdict = {0, 1}

Only bin 5, a clean drain, is hit. The flipped data bit is caught, but the other four bugs survive. The verdict has bits 0 and 1 set. Repeating an easy test does not add evidence.

Example 3

A Bad Baseline Fails the Campaign
Input
The complete tests, but the baseline is a design with a flipped data bit
Output
tests_v1 = {1}, first_test_v1 = 2, tests_v2 = {2}, first_test_v2 = 3, tests_v3 = {0}, first_test_v3 = 1, tests_v4 = {0, 1, 2, 3}, first_test_v4 = 1, tests_v5 = {0}, first_test_v5 = 1, observed_coverage = {0, 1, 2, 3, 4, 5, 6}, required_coverage = 0x7f, missing_coverage = {}, baseline_fail = {0, 1, 2, 3}, survivors = {}, verdict = {2}

Coverage and bug detection still look complete. Every baseline test that moves data fails, so verdict bit 2 is set on its own.

Constraints and Contract

  • Coverage needs an observed event. Detection needs a real checker failure.
  • Full coverage cannot excuse a failing baseline or a surviving bug.
  • Keep the class name, the method names and the argument types.

Hints

Hint 1

Keep baseline evidence, coverage and failures, apart from bug detection in the variants.

Hint 2

Write first_test for a variant only while it is still 0, so the earliest test stays.

Hint 3

Look at how finish() computes missing_coverage. Can it ever be anything but 0?

What the grader checks: Each case replays four 32-edge tests on a baseline and five faulty designs at depth 1, 3 or 5. The summary must report complete and incomplete campaigns truthfully. This covers these designs and tests, not coverage closure in general.

39 checks on Verilator · server. Run the examples first, then check the full regression. Full checks use the server judge. Downloads contain public samples only. A passing result establishes the tested cases, not a proof for all possible inputs.

See all 9 steps and the final capstone

Report a problem with this page

Your code SystemVerilogSaved locally
Ln 1, Col 1Tab indents · Ctrl/⌘ + Enter runs samples
Not run yet

A Complete Campaign

Input
Depth 3, correct baseline; tests: long stall with a full FIFO, reset while holding, reset with a read in flight, two round trips
Expected output
tests_v1 = {1}, first_test_v1 = 2, tests_v2 = {2}, first_test_v2 = 3, tests_v3 = {0}, first_test_v3 = 1, tests_v4 = {0, 1, 2, 3}, first_test_v4 = 1, tests_v5 = {0}, first_test_v5 = 1, observed_coverage = {0, 1, 2, 3, 4, 5, 6}, required_coverage = 0x7f, missing_coverage = {}, baseline_fail = {}, survivors = {}, verdict = {}

The four tests hit every bin and catch all five bugs. The baseline stays clean, so missing, survivors and verdict are all empty. The per-variant fields show which test caught each bug first.

Run samples executes all 3 public checks. Check all tests runs 39 checks on the server.

Verilator · server