Choosing the checking strategy
Pick how a scoreboard will know the right answer, from read-back self-checking to a full model, and know what each choice can and cannot catch.
- Lesson
- U9.1
- Time
- 30 min
- Runs on
- Verilator with UVM
- Check
- 5 questions
A memory controller passes a thousand regression runs with a scoreboard that writes a byte, reads it back and compares. Then a firmware engineer sets the controller's inversion mode, and every read returns the complement of what was written. The scoreboard reports a hundred mismatches against a design that does exactly what the specification says. Nobody had decided what the checker should know, so it never knew the mode existed.
| Request taken | Memory returns | Read-back checker | Model checker |
|---|---|---|---|
| WRITE bank 0 addr 1 = 11 | - | remembers 11 | stores 11 |
| READ tag 1 bank 0 addr 1 | 11 | match | match |
| READ tag 2 bank 1 addr 5 | d5 (power-up) | skipped: never written | skipped: unspecified |
| CTRL inv = 1 | - | ignored | inv = 1 |
| READ tag 3 bank 0 addr 1 | ee | false mismatch: expected 11 | match: 11 inverted |
| READ tag 4 bank 1 addr 2 | 22 | match | match |
The same traffic through two checkers. One of them is wrong about a correct design.
The questions a plan has to answer
Before anyone writes a scoreboard, the verification plan answers a short list of questions for each block. How many agents does the block need? Which kind of scoreboard: in-order queue, or out-of-order matching by a key as in U8.2? Does the checker need a reference model, a second implementation of the function that computes the expected output? And if the model works in transactions while the design works in cycles, where do the two meet? That meeting is the sync point. It has two halves: the prediction point, when the model is asked, and the comparison point, when its answer meets the design's result.
The last two questions are the subject of this module. U8 took the prediction for granted: model0[t.addr] was one line inside the scoreboard. That line is a choice. The figure walks it for one output at a time; the table adds what each strategy costs and where it is simply wrong.
- Plain data paths
A FIFO or bridge that must deliver bytes unchanged needs no model; the checker assumes exactly that.
More context
One question settles it. The checker's only assumption, what goes in comes out, is the specification itself.
Follow the calls, values and waits
- data FIFO output byte: Is the data transformed? → Write, read back, compare
- Stateless transforms
An incrementer (U7.3) is one expression,
x + 1.More context
The data changes, but each result depends only on its own input, so the predictor needs no memory.
Follow the calls, values and waits
- data incrementer y: Is the data transformed? → Does history matter?
- data incrementer y: Does history matter? → Transfer function
- State and modes
A model keeps the state. Write it as a class, or reuse a C model if one exists.
More context
This lesson's memory: the stored byte and the inv mode set by an earlier control write both decide the answer. The value does not depend on how many cycles the read takes.
Follow the calls, values and waits
- data read data, inv mode: Is the data transformed? → Does history matter?
- data read data, inv mode: Does history matter? → Is timing the question?
- data read data, inv mode: Is timing the question? → Untimed model
- Timing
When the question is when, not what, the check needs time: a cycle-accurate model or a meter.
More context
The same read, asked a different question. Its value still goes to the untimed model; its latency goes to a timing check.
Follow the calls, values and waits
- data when the read returns: Is the data transformed? → Does history matter?
- data when the read returns: Does history matter? → Is timing the question?
- data when the read returns: Is timing the question? → Timing check
Read the diagram as text
- Is the data transformed?: Modes, arithmetic, encodings
- Write, read back, compare: Assumes data comes back unchanged
- Does history matter?: Modes, state, earlier inputs
- Transfer function: One expression in the predictor
- Is timing the question?: Latency, cycle order
- Untimed model: SV class (U9.2) or C over DPI (U9.3)
- Timing check: Cycle-accurate model or latency meter (U8.4)
- Is the data transformed? → Write, read back, compare: no
- Is the data transformed? → Does history matter?: yes
- Does history matter? → Transfer function: no
- Does history matter? → Is timing the question?: yes
- Is timing the question? → Untimed model: no
- Is timing the question? → Timing check: yes
Most outputs need two answers: an untimed model for values plus a timing check. For a whole processor subsystem, an ISS or virtual platform (U9.5).
| Strategy | What the checker knows | Cost | Where it is wrong |
|---|---|---|---|
| No model: write, read back, compare | Only what the test sent | Lowest | Its assumption, data back unchanged, fails for any transform (modes, encodings); locations never written |
| Transfer function | One expression per output, as in U7.3's x + 1 | Low | State: anything where the answer depends on history |
| Untimed class model | Storage and modes, answered at once (U9.2) | Medium | Timing: a result that is right but late or early looks fine |
| Cycle-accurate model | The design's timing too | High, and it changes with every pipeline change | Shares the design's assumptions; two copies of the same misreading agree |
| C or C++ over DPI | A model that already exists, or one that must run fast (U9.3) | Medium plus a build step | Type and width mismatches at the hand-off |
| Instruction-set simulator or ESL platform | The whole programmer's view of a subsystem (U9.5) | A team's work, usually bought | Only as timed as it was built: an untimed one compares at instruction or transaction level |
| Assertions | Protocol rules: one response per read, a response only when a read is waiting | Low per rule | These protocol rules check no data values |
Seven ways to know the right answer. Most benches combine two or three.
Every strategy encodes an assumption. A read-back check assumes the path returns data unchanged: cheap to state and, as the hook shows, wrong for a design with modes. A model encodes a fuller reading of the specification, so it checks more, and a bug in the model is a bug in the test.
Two strategies on one memory
- Writes15 ns (illustrative)
Both checkers record
11at bank 0 address 1.More context
One
writeon the monitor's request port reaches both subscribers in turn, in the same time step. The bank 1 write of 22 follows at 25 ns.Follow the calls, values and waits
- data WRITE bank 0 addr 1 = 11: mem_monitor → raw_checker
- data WRITE bank 0 addr 1 = 11: mem_monitor → model_checker
- The control write55 ns (illustrative)
Only the model records the control write that sets
inv. The raw checker receives it too, and ignores it.More context
Same transaction, two readings of the specification. To the raw checker a CTRL request is not a storage write, so it changes nothing.
Follow the calls, values and waits
- data CTRL inv = 1: mem_monitor → raw_checker
- data CTRL inv = 1: mem_monitor → model_checker
- The read with inv set65 ns (illustrative)
Tag 3 reads address 1 while
invis 1. Each checker queues its own prediction for tag 3.More context
The predictions are made when the read is taken, from what each checker knows at that edge. The model applies inv now; a CTRL write after this edge must not change it.
Follow the calls, values and waits
- data READ tag 3, addr 1: mem_monitor → raw_checker
- data READ tag 3, addr 1: mem_monitor → model_checker
- data raw: expect 11: raw_checker → Tag 3: expects 11
- data model: expect ee: model_checker → Tag 3: expects ee
- Waiting: Tag 3: expects 11 — waiting for tag 3
- Waiting: Tag 3: expects ee — waiting for tag 3
- The response85 ns (illustrative)
The memory returns
eefor tag 3. The model matches; the raw checker reports a false alarm against a correct design.More context
Log line
85 ns model tag=3 data=ee matches. The run ends with raw at one mismatch and model at zero:UVM_ERROR : 1.Follow the calls, values and waits
- data response tag 3 = ee: mem_monitor → raw_checker
- data response tag 3 = ee: mem_monitor → model_checker
- data mismatch: 11 vs ee: raw_checker → Tag 3: expects 11
- data match: ee: model_checker → Tag 3: expects ee
- Released: Tag 3: expects 11
- Released: Tag 3: expects ee
Read the diagram as text
- mem_monitor: Requests taken, responses given
- raw_checker: Last byte written per {bank, addr}
- model_checker: Storage plus the inv mode
- Tag 3: expects 11: Memory returned ee: false alarm
- Tag 3: expects ee: 11 inverted: match
- mem_monitor → raw_checker: req, rsp
- mem_monitor → model_checker: req, rsp
- raw_checker → Tag 3: expects 11: predict
- model_checker → Tag 3: expects ee: predict
Same pins, two readings of the specification. A disagreement says the readings differ; the specification says which one is right.
The raw_checker remembers only what the test wrote, keyed by {bank, addr}; a location never written gets no prediction, and control writes mean nothing to it. The model_checker also tracks the control write and predicts the stored byte XOR {8{inv}}. Both queue predictions per tag, as in U8.2.
Observed at the earliest two edges after the read for bank 0 and four for bank 1; bank 0 wins a tie, so tag 2 waits behind tag 3 and is seen at 95 ns. Tag 3 reads addr 1 with inv set, so ee is right.
Response ee at 85 ns is where the two part. The memory inverted the stored 11 because inv was 1 when the read was taken. The model predicted ee and matched. The raw checker still expected 11 and reported a mismatch, a false alarm against a correct design. The run ends with raw at one mismatch and model at zero, on the same pins.
Tag 2 shows the opposite limit. Nobody wrote bank 1 address 5, and the specification leaves the power-up contents open. The model cannot predict what it does not know either, so both checkers skip the response. A model is not magic: it carries exactly what the specification says, and no more.
When may the run stop? The test waits until neither checker has a read waiting, but at most 200 ns; if the bound wins, it reports DRAIN. It then watches 50 ns more, the slowest read-to-response time in this run (tag 2), so an extra response is still reported as having no read waiting. Finally check_phase lists every read still pending. Rerun with tag_mem #(.LOSE_TAG(4)): tag 4 is never answered, and both checkers name it.
When the cheap strategy is the right one
Read-back checking looks like the loser here, and for a design with modes it is. But it has a real place. For a plain storage path, a FIFO, or a bus bridge that must deliver bytes unchanged, "what went in comes out" is the specification. The check costs a few lines, and its one assumption is easy to check against the document.
It also keeps working when the model is missing. Early in a project the model may not exist yet, or it may belong to another team and arrive late. A read-back scoreboard finds dropped and corrupted data from day one. When the model arrives, it joins the bench as a second checker rather than replacing the first.
One checker that knows everything
// model_checker only
mon.req_ap.connect(model.req_imp);
mon.rsp_ap.connect(model.rsp_imp);
Every verdict depends on one reading of the specification. A model bug and a design bug look the same.
Two checkers that know different things
mon.req_ap.connect(raw.req_imp);
mon.rsp_ap.connect(raw.rsp_imp);
mon.req_ap.connect(model.req_imp);
mon.rsp_ap.connect(model.rsp_imp);
A disagreement means their assumptions differ, and the specification decides which is wrong. Agreement can still share a wrong assumption.
The pattern generalises. Assertions guard the protocol: one response per read, no response without a read. A read-back or transfer-function checker guards the data path, and the model guards the modes and the history. Each is small enough to review, and each covers behaviour the others do not.
When the model is the thing that is wrong
A model is written by a person who read the same specification as the designer. If the sentence "inv inverts the data" is ambiguous, both may read it the same wrong way, and the bench passes. Or they read it differently, and the bench fails against a design that is right. The quiz below has a model that applies inv on the write path instead of the read path. It disagrees with the memory as soon as the mode changes between a write and a read.
So a mismatch is a question, never a verdict. The report must carry enough to answer it: the request that was predicted, when it was taken, what the model said and what the pins said. The example's error line names all four. U9.3 returns to this when the model is in C and the two sides disagree on a corner neither author thought about.
Try it
Both checkers see the same requests and responses. They agree on tags 1, 2 and 4. On tag 3 the raw checker reports data=ee, expected 11: the one UVM_ERROR in the summary is a false alarm, because inv was 1 when that read was taken. At each time the model's line comes first: the monitor's port calls its listeners in name order, as U7.2 showed.
Remove the two CTRL lines from the script and rerun: both checkers agree on everything, which is exactly why the bug in the hook survived a thousand runs that never set the mode.
55 ns model tag=1 data=11 matches 55 ns raw tag=1 data=11 matches 85 ns model tag=3 data=ee matches 95 ns model tag=2 data=d5 skipped: no expectation 95 ns raw tag=2 data=d5 skipped: no expectation 125 ns model tag=4 data=22 matches 125 ns raw tag=4 data=22 matches raw checked 3, skipped 1, mismatches 1 model checked 3, skipped 1, mismatches 0 UVM_ERROR : 1Check yourself
Remember
- The plan decides, per output, how the checker knows the right answer: read-back, transfer function, untimed model, cycle-accurate model, C over DPI, a platform, or assertions.
- Read-back self-checking is cheap and its one assumption, data back unchanged, is easy to review; it is blind to any transform of the data and to locations the test never wrote.
- A model encodes a reading of the specification: it catches modes and history, and it is wrong exactly where that reading is wrong.
- Checkers that know different things cover different behaviour; when they disagree, the specification decides which assumption is wrong, and agreement can still share a wrong one.
- A run may stop only when every read is answered or a reported bound expires, plus a quiet window for late extras;
check_phaselists what is still pending.