Scoreboard accounting and drain checks

Build a scoreboard that predicts each result from the observed input, checks results in order, and refuses to pass while any prediction is still waiting.

Lesson: U7.3
Time: 25 min
Runs on: Verilator with UVM
Check: 5 questions

A regression has passed for weeks with zero errors. Then a change corrupts the last result of each burst, the one that leaves the pipeline after the input stream stops, and the regression still passes. The test dropped its objection as soon as the sequence finished, two edges before the last result came out, so the scoreboard never saw it.

Time	Event	Scoreboard
15 ns	Design takes 16	Predicts 17
35 ns	Result 17	Checks 17: match
75 ns	Design takes 48; sequence ends; objection dropped	Predicts 49
75 ns	Run phase ends	One prediction still waiting
95 ns	The corrupted result would have come out here	Never runs

The test ended between the last input and its result. "Zero errors" meant "zero checks of the last value".

Predict from input; compare the output

Checking ends after the work, then observation endsFollow the final accepted input, 48. Prediction and comparison are separate roles inside the example’s one scoreboard.

One obligation remains75 ns (illustrative)
The final accepted input creates prediction 49. Two earlier pairs have been checked, but this prediction is still pending.
More context
Accepted inputs = 3; checked pairs = 2; pending = 1.
Follow the calls, values and waits
1. data x = 48: Accepted input → Predict
2. data expected 49: Predict → Expected queue
3. Waiting: Expected queue — 49 is still owed
Close the source80 ns (illustrative)
The only sequence has ended and the driver returns the bus to idle. The test still owns its objection because one accepted request is unanswered.
More context
An empty queue earlier in the run would only have meant that the checker had caught up.
Follow the calls, values and waits
1. data pending = 1: Expected queue → Test owns the end
2. Waiting: Test owns the end — waiting for required checks
Finish the comparison95 ns (illustrative)
The observed 49 meets the oldest prediction, 49. write_out removes the prediction, compares the values and increments checked before it returns.
More context
Accepted = checked = planned = 3. This comparator has no background worker left busy.
Follow the calls, values and waits
1. data observed 49: Observed output → Compare
2. data expected 49: Expected queue → Compare
3. data checked = 3: Compare → Test owns the end
4. Waiting: Test owns the end — watching two more edges
5. Released: Expected queue
Close observation120 ns (illustrative)
The quiet window: monitors keep sampling at 105 and 115 ns, so an extra result would still reach write_out. The test drops its objection at the following falling edge.
More context
The quiet window is two edges, 20 ns: the longest time from an accepted input to its observed result.
Follow the calls, values and waits
1. data pending = 0: Expected queue → Test owns the end
2. data checked = 3: Compare → Test owns the end
3. Released: Test owns the end

Read the diagram as text

Accepted input: Input monitor publishes x = 48
Predict: write_in computes 48 + 1
Expected queue: 49 waits for its observed result
Observed output: Output monitor publishes y = 49
Compare: write_out checks and counts a pair
Test owns the end: Required work, then a quiet window

Accepted input → Predict: accepted x
Predict → Expected queue: enqueue 49
Expected queue → Compare: oldest prediction
Observed output → Compare: observed y
Compare → Test owns the end: checked count
Expected queue → Test owns the end: pending count

Arrows carry transaction data or completion facts, not connect() calls. A counted comparison can still be a mismatch; reports determine pass/fail.

A reference model computes what should happen from trusted inputs. Here write_in models the specified operation: an eight-bit increment, so 255 wraps to 0. It does not read the design's y to discover the expected answer.

The comparator pairs each observed output with a prediction and checks their values. These are separate responsibilities, even though this small scoreboard implements both. U8 separates the components; U9 compares model choices.

This bench assumes one result per accepted input, results in acceptance order, and no reset. The predictor runs immediately; the comparator finishes inside write_out. No prediction or comparison task remains busy after those functions return.

Under those assumptions, a FIFO queue of predictions is enough: push each expected value, then pop the oldest when an output arrives. The closing section lists what breaks when an assumption fails.

Give each stream its own receiving method

The suffix selects the receiving function. It does not change the transaction or create a worker process.

// Before the class:
`uvm_analysis_imp_decl(_in)
`uvm_analysis_imp_decl(_out)
// Component members (constructed with new):
uvm_analysis_imp_in  #(incr_item, incr_scoreboard)   in_imp;
uvm_analysis_imp_out #(incr_result, incr_scoreboard) out_imp;
// In connect_phase:
mon.in_ap.connect(sb.in_imp);   // calls write_in(t)
mon.out_ap.connect(sb.out_imp); // calls write_out(r)

U3.5 introduced suffixed imps. Both receiving functions run in the publishing monitor's process, so when an input and an output share an edge, this monitor publishes the input first. That order comes from this monitor's code; UVM gives no ordering between separate monitors.

Account for work that never became a comparison

A wrong value is only one failure. An output with no prediction is unexpected; a prediction with no output is pending. Keep those cases separate from comparisons that ran and found a mismatch.

Accounting invariants for this no-reset, one-result-per-input checker. checked includes both matching and mismatching pairs.

inputs_seen  == checked + expected.size()
outputs_seen == checked + unexpected

The example updates these counts in the receiving methods. check_phase rejects pending predictions, broken accounting and zero observed inputs. Reporting counts makes a disconnected checker visible; zero mismatches alone cannot establish useful work.

Read the failure before changing the model

A mismatch suggests the output value, model or pairing is wrong. An unexpected output can indicate a duplicate result or a missing input observation. Pending predictions can indicate a lost result, a missed output observation or an early test end. The counters identify the data path to investigate; they do not prove which component is at fault.

For a larger model, keep the original input, acceptance time and tag with each prediction. A message such as “y=50, expected 49, for x=48 accepted at 75 ns” identifies the relevant transfer. Implement the specified behavior in the model, not the design's implementation mistakes.

One owner closes the test

The test owns the objection because it knows when its only stimulus source has finished. After the final acceptance at 75 ns, it waits to the falling edge at 80 ns. The driver returns to idle, and the monitor has already counted that acceptance.

The main example completes three comparisons at 95 ns, observes two more rising edges, and ends at 120 ns.

// planned was captured from seq.values.size() before seq.start.
wait (sb.inputs_seen == planned &&
      sb.checked == planned &&
      sb.expected.size() == 0);
repeat (2) @(posedge drv.vif.clk);
@(negedge drv.vif.clk);
phase.drop_objection(this);

The planned count prevents a silently empty checker from satisfying completion. Here queue removal, comparison and checked++ happen inside one function, so completed counts also cover consumer work. With a FIFO-fed worker, additionally require its input queues empty and its busy work finished.

After the final comparison, the monitors keep sampling at 105 and 115 ns: a quiet window of two edges, 20 ns, the longest time from an accepted input to its observed result. An extra result in that window reaches write_out with nothing waiting and is reported as unexpected. A result invented after the window ends goes unseen, so derive the window from the design's latency.

A drain time is the common shortcut. phase.get_objection().set_drain_time(this, 10ns) keeps the phase alive for 10 ns after the objection count reaches zero. Dropped when the sequence ends at 75 ns, it ends the run at 85 ns and still misses the result at 95 ns. Drain starts when the objection count reaches zero; dropping with the sequence starts it at stimulus completion, so it guesses the latency. The main example waits for the last comparison before its quiet window.

The 2000 ns global timeout bounds a missing-input or missing-result wait with PH_TIMEOUT, which is a failed test. U10.3 adds a local watchdog with outstanding-work diagnostics.

If the scoreboard owns the objection instead

Avoid zero-count gaps
Raise before the first prediction and drop only when nothing is pending; a drop between two predictions can end the phase.
Know when no more work can arrive
An empty queue between bursts is not the end; the scoreboard needs the stimulus owner to say the source is closed.
Never let a forever monitor hold the test
A monitor that raises for each observation keeps the test alive for as long as the bus has traffic.

Whoever owns the objection, keep the final accounting check in check_phase.

Where this scoreboard stops working

Every rule above rests on the assumptions in the first section. Each row is one that fails in real designs, and the lesson that handles it.

Assumption	What breaks	Where it is taught
Results come back in acceptance order	Correct results reorder across tags and pair with the wrong predictions	U8.2: a queue of predictions per tag
A prediction exists before its output arrives	A slow model, or separately scheduled monitors, deliver the output first	U8.1 S4: two analysis FIFOs and a matcher task
Every field counts, the same way in every mode	Time stamps, unspecified bits or a mode bit turn equal results into mismatches	U8.3: what `do_compare` decides
No reset during traffic	Predictions for flushed requests wait forever	U10.2
Only values matter	A correct value that arrives too late passes	U8.4: latency and throughput

The queue scoreboard is correct only while all five assumptions hold.

Try it

At 80 ns the source has finished but prediction 49 remains pending. The third comparison completes at 95 ns. Monitors then sample at 105 and 115 ns, the quiet window, and the test ends at 120 ns.

Fault experiments: change the DUT to add two instead of one; duplicate the monitor's out_ap.write(r); or remove that write. Predict mismatch, unexpected-output and timeout behavior before running each change separately.

Transfer exercise: move comparison into a worker that takes 50 ns after get. At the instant its FIFO empties, which work is still owned by that worker? Rewrite the completion condition, and state how long late output remains observable.

RUNVerilator + UVM 2020.3.1 · simulator service

Edit the code, then run it

Expected among the UVM messages: 35 ns scoreboard y=17 checked, 0 waiting 65 ns scoreboard y=33 checked, 0 waiting 80 ns test stimulus done, 1 waiting 95 ns scoreboard y=49 checked, 0 waiting 120 ns test quiet window closed inputs 3, outputs 3, checked 3, mismatches 0, unexpected 0, waiting 0 UVM_ERROR : 0

Check yourself

Predict the outputThis variant drops its objection immediately after the sequence ends, with a 10 ns drain time. It omits the main example's progress wait and final observation window. What line does the test's check_phase print?

`timescale 1ns/1ns
`include "uvm_macros.svh"
import uvm_pkg::*;

// The incrementer from U4.3 with a result port. It captures x at a rising edge k where valid and
// ready are both 1 and is busy for the next BUSY edges. It drives y = x + 1 and out_valid after
// edge k+1, for one cycle, so a monitor sampling at rising edges sees the result at edge k+2.
interface incr_if (input logic clk);
  logic       valid = 0;
  logic [7:0] x = 0;
  logic       ready;
  logic       out_valid = 0;
  logic [7:0] y = 0;
endinterface

module incr_pipe #(parameter int BUSY = 2) (incr_if bus);
  logic [1:0] busy = 0;
  logic       staged = 0;
  logic [7:0] stage = 0;
  assign bus.ready = (busy == 0);
  always @(posedge bus.clk) begin
    bus.out_valid <= staged;
    bus.y         <= stage;
    staged        <= bus.valid && bus.ready;
    stage         <= bus.x + 1;
    if (bus.valid && bus.ready) busy <= BUSY;
    else if (busy != 0)         busy <= busy - 1;
  end
endmodule

// What the sequence sends, and what the monitor sees going in.
class incr_item extends uvm_sequence_item;
  `uvm_object_utils(incr_item)
  bit [7:0] x;
  function new(string name = "incr_item"); super.new(name); endfunction
endclass

// What the monitor sees coming out.
class incr_result extends uvm_sequence_item;
  `uvm_object_utils(incr_result)
  bit [7:0] y;
  function new(string name = "incr_result"); super.new(name); endfunction
endclass

// Sends the values in the queue, one item each.
class incr_seq extends uvm_sequence #(incr_item);
  `uvm_object_utils(incr_seq)
  bit [7:0] values[$];
  function new(string name = "incr_seq"); super.new(name); values = '{16, 32, 48}; endfunction
  task body();
    foreach (values[i]) begin
      incr_item it = incr_item::type_id::create("it");
      start_item(it);
      it.x = values[i];
      finish_item(it);
    end
  endtask
endclass

// The driver from U4.3: offers an item at a falling edge, holds it until an edge with ready.
class incr_driver extends uvm_driver #(incr_item);
  `uvm_component_utils(incr_driver)
  virtual incr_if vif;
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  task run_phase(uvm_phase phase);
    forever begin
      @(negedge vif.clk);
      seq_item_port.try_next_item(req);
      if (req == null) begin
        vif.valid <= 0;
        continue;
      end
      vif.valid <= 1;
      vif.x     <= req.x;
      do @(posedge vif.clk); while (!vif.ready);
      seq_item_port.item_done();
    end
  endtask
endclass

// Watches the pins and publishes one transaction per transfer. It never drives anything.
class incr_monitor extends uvm_monitor;
  `uvm_component_utils(incr_monitor)
  virtual incr_if vif;
  uvm_analysis_port #(incr_item)   in_ap;
  uvm_analysis_port #(incr_result) out_ap;
  function new(string name, uvm_component parent);
    super.new(name, parent);
    in_ap  = new("in_ap", this);
    out_ap = new("out_ap", this);
  endfunction
  task run_phase(uvm_phase phase);
    forever begin
      @(posedge vif.clk);                      // values just before the edge
      if (vif.valid && vif.ready) begin        // a transfer: the design takes x
        incr_item t = incr_item::type_id::create("t");
        t.x = vif.x;
        in_ap.write(t);
      end
      if (vif.out_valid) begin                 // a result: the design shows y
        incr_result r = incr_result::type_id::create("r");
        r.y = vif.y;
        out_ap.write(r);
      end
    end
  endtask
endclass

`uvm_analysis_imp_decl(_in)
`uvm_analysis_imp_decl(_out)

// Predicts y = x + 1 for each input and checks results in order.
class incr_scoreboard extends uvm_component;
  `uvm_component_utils(incr_scoreboard)
  uvm_analysis_imp_in  #(incr_item, incr_scoreboard)   in_imp;
  uvm_analysis_imp_out #(incr_result, incr_scoreboard) out_imp;
  bit [7:0] expected[$];                 // predictions waiting for their result
  int inputs_seen, outputs_seen, checked, mismatches, unexpected;
  function new(string name, uvm_component parent);
    super.new(name, parent);
    in_imp  = new("in_imp", this);
    out_imp = new("out_imp", this);
  endfunction
  function void write_in(incr_item t);
    inputs_seen++;
    expected.push_back(t.x + 1);         // the reference model
  endfunction
  function void write_out(incr_result r);
    bit [7:0] want;
    outputs_seen++;
    if (expected.size() == 0) begin
      unexpected++;
      `uvm_error("SB", $sformatf("result y=%0d with no input waiting", r.y))
      return;
    end
    want = expected.pop_front();
    if (r.y != want) begin
      mismatches++;
      `uvm_error("SB", $sformatf("y=%0d, expected %0d", r.y, want))
    end
    checked++;
  endfunction
  function void check_phase(uvm_phase phase);
    if (inputs_seen == 0)
      `uvm_error("SB", "no accepted inputs were observed")
    if (inputs_seen != checked + expected.size())
      `uvm_error("SB", "input accounting lost an obligation")
    if (outputs_seen != checked + unexpected)
      `uvm_error("SB", "output accounting lost an observation")
    if (expected.size() != 0)
      `uvm_error("SB", $sformatf("%0d expected results never arrived", expected.size()))
  endfunction
endclass

class predict_test extends uvm_test;
  `uvm_component_utils(predict_test)
  uvm_sequencer #(incr_item) sqr;
  incr_driver  drv;
  incr_monitor mon;
  incr_scoreboard sb;
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  function void build_phase(uvm_phase phase);
    virtual incr_if vif;
    sqr = uvm_sequencer #(incr_item)::type_id::create("sqr", this);
    drv = incr_driver::type_id::create("drv", this);
    mon = incr_monitor::type_id::create("mon", this);
    if (!uvm_config_db #(virtual incr_if)::get(this, "", "vif", vif))
      `uvm_fatal("NOVIF", "no virtual interface")
    drv.vif = vif;
    mon.vif = vif;
    sb = incr_scoreboard::type_id::create("sb", this);
  endfunction
  function void connect_phase(uvm_phase phase);
    drv.seq_item_port.connect(sqr.seq_item_export);
    mon.in_ap.connect(sb.in_imp);
    mon.out_ap.connect(sb.out_imp);
  endfunction
  task run_phase(uvm_phase phase);
    incr_seq seq = incr_seq::type_id::create("seq");
    phase.raise_objection(this);
    seq.start(sqr);
    phase.get_objection().set_drain_time(this, 10ns);
    phase.drop_objection(this);
  endtask
  function void check_phase(uvm_phase phase);
    $display("run ended at %0t: checked %0d, waiting %0d", $time, sb.checked, sb.expected.size());
  endfunction
endclass

module tb;
  bit clk;
  always #5 clk = ~clk;
  incr_if bus (clk);
  incr_pipe dut (bus);
  initial begin
    uvm_root::get().set_timeout(2000ns, 1);
    uvm_config_db #(virtual incr_if)::set(null, "uvm_test_top", "vif", bus);
    run_test("predict_test");
  end
endmodule

Spot the bugThis harness reaches each option at 80 ns, after the sequence and the following falling edge. The final result is sampled at 95 ns. Which ending leaves a result unchecked?

phase.get_objection().set_drain_time(this, 10ns);
phase.drop_objection(this);

#30ns;
phase.drop_objection(this);

phase.get_objection().set_drain_time(this, 30ns);
phase.drop_objection(this);

wait (sb.expected.size() == 0);
phase.drop_objection(this);

ConceptA test finishes with "UVM_ERROR : 0", but its scoreboard has no check_phase. What does the zero prove?

That the design has no bugs on any of the paths the sequence exercisedNo comparison failed and nothing unexpected arrived; missing results went uncheckedThat every input the sequence sent produced a correct result, in orderNothing at all, because UVM only counts errors that are reported in check_phase

Read the waveformThe same in-order pairing rule is used with a memory predictor on the two-bank memory of U8.2. Read tag 1 (bank 1, data 22) is taken before read tag 2 (bank 0, data 11), and both responses are correct. What does the in-order scoreboard report?

Bank 0 overtakes bank 1

The queue pops the oldest prediction whatever tag the response carries.

Nothing: both values appear among the predictions, so both passOne orphan response, because the first response has the wrong tagTwo mismatches, 11 against 22 and then 22 against 11, although both values are rightOne pending prediction at the end, because the tags do not match

ConceptWhy is a guessed drain time insufficient as the only completion rule?

It stops the monitors while the run phase is still activeDrain time is ignored when only one component has raised an objectionIt ends the run phase before the sequence has finished sending itemsIt does not establish which required work completed before its time expired

Remember

Prediction uses observed inputs; comparison checks observed outputs against the specified behavior, with a stated pairing rule.
A single expected queue assumes one result per input, in acceptance order; the closing table lists where each assumption breaks.
Count accepted inputs, checked pairs, pending predictions and unexpected outputs; a mismatch is still a completed comparison.
Close the stimulus source, require planned progress, and account for queued and busy consumer work before ending.
After the last comparison, keep the monitors alive for a quiet window as long as the longest input-to-result latency; a timeout is a failure, not an ending.