Fix a regression summary that reports green while required coverage is missing.
Hardware context: Clocked FIFO plus ready/valid output adapter. Qualify the complete FIFO/adapter subsystem with an independent stream oracle.
A regression is done when three things hold. The correct design passes every test. Every planted bug fails at least one test. Every required scenario really happened. This summary checks all three, but it never reports a coverage hole.
The class subsystem_regression collects results from four tests. Each test runs on six FIFO-and-adapter designs: variant 0, the baseline, and variants 1 to 5, each with one planted bug. record() is called once per test and variant. finish() returns the summary.
A test detects a variant when the trusted checker report shows a failing edge. Coverage comes only from the baseline. The verdict has three separate causes: missing coverage, a bug no test caught, and a failing baseline.
Find the bug in finish() and fix it.
Signals in the Timeline
| Signal | Width | Role |
|---|---|---|
profile | 2 | input of each check |
baseline_mode | 1 | input of each check |
controls0 | 96 | input of each check |
values0 | 256 | input of each check |
controls1 | 96 | input of each check |
values1 | 256 | input of each check |
controls2 | 96 | input of each check |
values2 | 256 | input of each check |
controls3 | 96 | input of each check |
values3 | 256 | input of each check |
Full specification
class subsystem_regression;
function new();
function void record(input int test_index, variant, input bit [79:0] result, input bit [7:0] measured_hits);
function bit [79:0] finish();
endclass
| Input | Meaning |
|---|---|
| test_index | 0 to 3. |
| variant | 0 is the baseline, the design believed correct. 1 to 5 each have one planted bug, listed below. |
| result | The 80-bit report of the trusted checker from Reset During Adapter Backpressure. Bits 79:48 are its failing-edge mask; any 1 bit means this test failed on this variant. |
| measured_hits | The coverage bins this test hit on the baseline. |
| Variant | Planted bug |
|---|---|
| 1 | The held value survives reset. |
| 2 | The read in flight survives reset. |
| 3 | The held value is dropped while stalled. |
| 4 | Bit 0 of the data is flipped. |
| 5 | A read is issued over a stalled value. |
| Coverage bin | Hit when |
|---|---|
| 0 | A write is offered while the FIFO is full. |
| 1 | A valid stream offer is stalled by ready = 0. |
| 2 | Reset arrives while the output holds a value. |
| 3 | Reset arrives while a FIFO response is valid, so a read is in flight. |
| 4 | Reset arrives while the FIFO holds entries. |
| 5 | The test ends fully drained: ledger, FIFO and output all empty. |
| 6 | A stalled output is later taken with unchanged data. A reset in between does not count. |
Only variant 0 adds to coverage. All seven bins are required, so required_coverage = 0x7f and missing_coverage = required & ~observed.
For variant n, tests_v<n> is the set of tests that detected it. first_test_v<n> is the first of them, counting from 1, or 0 for none. survivors has bit n - 1 set when no test detected variant n. baseline_fail has bit t set when test t failed on the baseline.
verdict bit 0 means coverage is missing, bit 1 means a bug survived, and bit 2 means the baseline failed. Set each one on its own.
Examples
Example 1
A Complete Campaign- Input
Depth 3, correct baseline; tests: long stall with a full FIFO, reset while holding, reset with a read in flight, two round trips
- Output
tests_v1 = {1}, first_test_v1 = 2, tests_v2 = {2}, first_test_v2 = 3, tests_v3 = {0}, first_test_v3 = 1, tests_v4 = {0, 1, 2, 3}, first_test_v4 = 1, tests_v5 = {0}, first_test_v5 = 1, observed_coverage = {0, 1, 2, 3, 4, 5, 6}, required_coverage = 0x7f, missing_coverage = {}, baseline_fail = {}, survivors = {}, verdict = {}
The four tests hit every bin and catch all five bugs. The baseline stays clean, so missing, survivors and verdict are all empty. The per-variant fields show which test caught each bug first.
Example 2
Four Easy Tests Prove Little- Input
Depth 3, correct baseline; all four tests are the same two round trips
- Output
tests_v1 = {}, first_test_v1 = 0, tests_v2 = {}, first_test_v2 = 0, tests_v3 = {}, first_test_v3 = 0, tests_v4 = {0, 1, 2, 3}, first_test_v4 = 1, tests_v5 = {}, first_test_v5 = 0, observed_coverage = {5}, required_coverage = 0x7f, missing_coverage = {0, 1, 2, 3, 4, 6}, baseline_fail = {}, survivors = {0, 1, 2, 4}, verdict = {0, 1}
Only bin 5, a clean drain, is hit. The flipped data bit is caught, but the other four bugs survive. The verdict has bits 0 and 1 set. Repeating an easy test does not add evidence.
Example 3
A Bad Baseline Fails the Campaign- Input
The complete tests, but the baseline is a design with a flipped data bit
- Output
tests_v1 = {1}, first_test_v1 = 2, tests_v2 = {2}, first_test_v2 = 3, tests_v3 = {0}, first_test_v3 = 1, tests_v4 = {0, 1, 2, 3}, first_test_v4 = 1, tests_v5 = {0}, first_test_v5 = 1, observed_coverage = {0, 1, 2, 3, 4, 5, 6}, required_coverage = 0x7f, missing_coverage = {}, baseline_fail = {0, 1, 2, 3}, survivors = {}, verdict = {2}
Coverage and bug detection still look complete. Every baseline test that moves data fails, so verdict bit 2 is set on its own.
Constraints and Contract
- Coverage needs an observed event. Detection needs a real checker failure.
- Full coverage cannot excuse a failing baseline or a surviving bug.
- Keep the class name, the method names and the argument types.
Hints
Hint 1
Keep baseline evidence, coverage and failures, apart from bug detection in the variants.
Hint 2
Write first_test for a variant only while it is still 0, so the earliest test stays.
Hint 3
Look at how finish() computes missing_coverage. Can it ever be anything but 0?
What the grader checks: Each case replays four 32-edge tests on a baseline and five faulty designs at depth 1, 3 or 5. The summary must report complete and incomplete campaigns truthfully. This covers these designs and tests, not coverage closure in general.
39 checks on Verilator · server. Run the examples first, then check the full regression. Full checks use the server judge. Downloads contain public samples only. A passing result establishes the tested cases, not a proof for all possible inputs.
See all 9 steps and the final capstone