Predictor, comparator, scoreboard

Prediction and evaluation are different jobs

A scoreboard answers one question: did the design do what the specification says? The answer takes two steps. First something has to say what the design should have produced for the inputs it saw. Then something has to compare that with what the monitor saw coming out.

The Cookbook names the two parts. The predictor takes the same input stream as the design and produces expected output transactions. The evaluator, usually a comparator, takes the expected stream and the actual stream and decides, pair by pair. The scoreboard is the component that owns both and connects them.

Keeping them apart pays off quickly. The predictor can be a SystemVerilog class today and a C model over DPI next month, with the comparator untouched. The comparator can be replaced by an out-of-order one when the next design reorders, with the model untouched. And the predictor can stand in for the design itself while the RTL is still being written, so the rest of the bench is debugged early.

The whole lesson keeps the U7 incrementer bench: the design takes x, is busy for two edges, and shows y = x + 1 two edges later. The model stays one line. Everything around it changes shape.

A predictor is a subscriber with a port of its own

A predictor listens to one input stream, so the natural base is uvm_subscriber #(incr_item). That base gives it an analysis_export and demands a write function. What it adds is an analysis port of its own, uvm_analysis_port #(incr_result) ap, through which it broadcasts each prediction.

class incr_predictor extends uvm_subscriber #(incr_item);
  uvm_analysis_port #(incr_result) ap;
  int predicted;
  function void write(incr_item t);
    incr_result e = incr_result::type_id::create("e");
    e.y = t.x + 1;                 // the model
    predicted++;
    ap.write(e);                   // whoever wants predictions, listens here
  endfunction
endclass

Any number of components can listen to that port: a comparator, a coverage collector that bins expected values, a logger. In the example two comparators listen at once, and the predictor does not know.

The predicted stream is a transaction stream like any other, so predictors chain. A model of an encoder can feed a model of a scrambler, each a subscriber to the previous one's port, and only the last port reaches the comparator. Each stage stays small enough to test on its own.

The stock in-order comparator

The library ships one evaluator for the simple case: uvm_in_order_class_comparator #(T). It has two analysis exports, before_export for the expected stream and after_export for the actual one, each feeding an internal analysis FIFO. Its run_phase takes one transaction from each FIFO, in arrival order, and calls before.compare(after).

For that to work, T must implement compare() and convert2string(), which incr_result does through do_compare and a one-line convert2string. The result type must also be a uvm_object, since compare lives there. A variant, uvm_in_order_built_in_comparator, uses == for integers and other built-in types.

That last column is the trap. In the quiz program the model is wrong by one, every pair mismatches, and the run still ends with UVM_ERROR : 0. A test that uses the stock comparator must read m_mismatches or count the Comparator Mismatch id in check_phase and raise its own error. The counters are public despite their m_ prefix.

Writing the comparator yourself

The stock comparator is a good first evaluator and a poor last one. What it compares is up to the item: it calls compare(), so a do_compare with masks or a mode (U8.3) works inside it unchanged. What it cannot change is the architecture around the pair: in-order pairing only (U8.2), a mismatch message without the input and its time, and no end-of-test check. Mismatches are warnings, though set_report_severity_id_override(UVM_WARNING, "Comparator Mismatch", UVM_ERROR) fixes that from outside. Write your own when the architecture is the problem. The shape to copy is the Cookbook's: two analysis FIFOs and a run_phase that synchronises them.

The FIFOs matter. Each write runs in the monitor's process and must return at once, so it only drops its transaction into a FIFO; the waiting happens in the comparator's task. Which get comes first does not matter: the loop cannot compare until both have delivered.

stock = uvm_in_order_class_comparator #(incr_result)
          ::type_id::create("stock", this);
// mismatch: UVM_WARNING, counted in m_mismatches
// test must check m_mismatches itself

before_fifo.get(e);
after_fifo.get(a);
if (e.compare(a)) matched++;
else `uvm_error("CMP", $sformatf("%s, predicted %s",
       a.convert2string(), e.convert2string()))

Compare with e.compare(a), never with e == a. Two handles are equal only when they point at the same object, and a prediction and a monitored result never do. A comparison of handles reports every pair as a mismatch, which the spot question shows.

Where the scoreboard went

In the example the test builds the predictor and both comparators and connects them. In a real bench that wiring moves into a uvm_scoreboard subclass or the environment, which exposes the two analysis exports and hides the parts. The drain check from U7.3 moves too, and splits. The test's check_phase compares pred.predicted with cmp.matched + cmp.mismatched and reports any prediction that never met a result, whether it sits in before_fifo or was already taken out into e. The comparator's own check_phase reports the opposite leak: results left in after_fifo at the end, which the design produced and nobody predicted.

When checking may end

U7.3 S4 is the home of the end-of-test rule. Require the planned work, then keep the monitors sampling for a quiet window as long as the design's longest input-to-result latency, 20 ns here. Two things change when prediction and comparison are separate components.

First, the balance is a count across components: the test waits until the comparator has handled as many pairs as the predictor published. Second, an extra result has a place to wait. The comparator loop is blocked on before_fifo.get with no prediction to pair, so the result sits in after_fifo, and only the comparator's leftover check in check_phase reports it.

The example waits #(max_latency) after the balance, with max_latency a time field of the test, and prints the leftover count at 115 ns. Set the bench's fault switch and the design shows a fourth result at 105 ns: it lands in after_fifo and the comparator reports it. Dropping at the balance, 95 ns, ends the run before it arrives; the second spot question shows that.

A drain time set after the balance, set_drain_time(this, 20ns), is the same window written another way. A result the design invents long after it went idle falls outside any window. An assertion on the interface, no result without an outstanding input, catches that while the simulation runs.

Try it

The predictor publishes three predictions at the input edges (15, 45 and 75 ns). Both comparators receive them and the three results, and both report three matches. The stock comparator's matches appear as [Comparator Match] info lines; the hand-written one prints its own.

The counts balance at 95 ns and the test keeps observing for 20 ns. Set .EXTRA(1) on dut in tb and rerun: the quiet-window line reports 1 result left over and the comparator raises a UVM_ERROR for it. Change the model to t.x + 2 instead and the hand-written comparator reports three UVM_ERROR lines with both values; the stock one reports three warnings.

Check yourself

`timescale 1ns/1ns
`include "uvm_macros.svh"
import uvm_pkg::*;

// The incrementer from U7: takes x at a rising edge where valid and ready are 1, is busy for two
// edges, and shows y = x + 1 with out_valid for one cycle so that a monitor sees it two edges later.
interface incr_if (input logic clk);
  logic       valid = 0;
  logic [7:0] x = 0;
  logic       ready;
  logic       out_valid = 0;
  logic [7:0] y = 0;
endinterface

// EXTRA is a fault switch for the bench: 1 makes the design show one result too many, one edge
// after its third result.
module incr_pipe #(parameter int BUSY = 2, parameter bit EXTRA = 0) (incr_if bus);
  logic [1:0] busy = 0;
  logic       staged = 0;
  logic [7:0] stage = 0;
  int         shown = 0;
  bit         extra_sent = 0;
  assign bus.ready = (busy == 0);
  always @(posedge bus.clk) begin
    bus.out_valid <= staged;
    if (staged) shown <= shown + 1;
    if (EXTRA && shown == 3 && !extra_sent) begin
      bus.out_valid <= 1;                  // the fault: a fourth result nobody asked for
      extra_sent    <= 1;
    end
    bus.y         <= stage;
    staged        <= bus.valid && bus.ready;
    stage         <= bus.x + 1;
    if (bus.valid && bus.ready) busy <= BUSY;
    else if (busy != 0)         busy <= busy - 1;
  end
endmodule

class incr_item extends uvm_sequence_item;
  `uvm_object_utils(incr_item)
  bit [7:0] x;
  function new(string name = "incr_item"); super.new(name); endfunction
endclass

// The result. compare() and convert2string() are what a stock comparator calls.
class incr_result extends uvm_sequence_item;
  `uvm_object_utils(incr_result)
  bit [7:0] y;
  function new(string name = "incr_result"); super.new(name); endfunction
  function bit do_compare(uvm_object rhs, uvm_comparer comparer);
    incr_result o;
    if (!$cast(o, rhs)) return 0;
    return comparer.compare_field_int("y", y, o.y, 8);
  endfunction
  function string convert2string(); return $sformatf("y=%0d", y); endfunction
endclass

class incr_seq extends uvm_sequence #(incr_item);
  `uvm_object_utils(incr_seq)
  bit [7:0] values[$];
  function new(string name = "incr_seq"); super.new(name); values = '{16, 32, 48}; endfunction
  task body();
    foreach (values[i]) begin
      incr_item it = incr_item::type_id::create("it");
      start_item(it);
      it.x = values[i];
      finish_item(it);
    end
  endtask
endclass

class incr_driver extends uvm_driver #(incr_item);
  `uvm_component_utils(incr_driver)
  virtual incr_if vif;
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  task run_phase(uvm_phase phase);
    forever begin
      @(negedge vif.clk);
      seq_item_port.try_next_item(req);
      if (req == null) begin
        vif.valid <= 0;
        continue;
      end
      vif.valid <= 1;
      vif.x     <= req.x;
      do @(posedge vif.clk); while (!vif.ready);
      seq_item_port.item_done();
    end
  endtask
endclass

class incr_monitor extends uvm_monitor;
  `uvm_component_utils(incr_monitor)
  virtual incr_if vif;
  uvm_analysis_port #(incr_item)   in_ap;
  uvm_analysis_port #(incr_result) out_ap;
  function new(string name, uvm_component parent);
    super.new(name, parent);
    in_ap  = new("in_ap", this);
    out_ap = new("out_ap", this);
  endfunction
  task run_phase(uvm_phase phase);
    forever begin
      @(posedge vif.clk);
      if (vif.valid && vif.ready) begin
        incr_item t = incr_item::type_id::create("t");
        t.x = vif.x;
        in_ap.write(t);
      end
      if (vif.out_valid) begin
        incr_result r = incr_result::type_id::create("r");
        r.y = vif.y;
        out_ap.write(r);
      end
    end
  endtask
endclass

// Prediction only. It knows the model and nothing about comparing.
class incr_predictor extends uvm_subscriber #(incr_item);
  `uvm_component_utils(incr_predictor)
  uvm_analysis_port #(incr_result) ap;
  int predicted;
  function new(string name, uvm_component parent); super.new(name, parent); ap = new("ap", this); endfunction
  function void write(incr_item t);
    incr_result e = incr_result::type_id::create("e");
    e.y = t.x + 2;
    predicted++;
    ap.write(e);
  endfunction
endclass

class predict_test extends uvm_test;
  `uvm_component_utils(predict_test)
  uvm_sequencer #(incr_item) sqr;
  incr_driver    drv;
  incr_monitor   mon;
  incr_predictor pred;  uvm_in_order_class_comparator #(incr_result) stock;
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  function void build_phase(uvm_phase phase);
    virtual incr_if vif;
    sqr   = uvm_sequencer #(incr_item)::type_id::create("sqr", this);
    drv   = incr_driver::type_id::create("drv", this);
    mon   = incr_monitor::type_id::create("mon", this);
    pred  = incr_predictor::type_id::create("pred", this);
    stock = uvm_in_order_class_comparator #(incr_result)::type_id::create("stock", this);
    if (!uvm_config_db #(virtual incr_if)::get(this, "", "vif", vif))
      `uvm_fatal("NOVIF", "no virtual interface")
    drv.vif = vif;
    mon.vif = vif;
  endfunction
  function void connect_phase(uvm_phase phase);
    drv.seq_item_port.connect(sqr.seq_item_export);
    mon.in_ap.connect(pred.analysis_export);
    pred.ap.connect(stock.before_export);
    mon.out_ap.connect(stock.after_export);
  endfunction
  task run_phase(uvm_phase phase);
    incr_seq seq = incr_seq::type_id::create("seq");
    phase.raise_objection(this);
    seq.start(sqr);
    wait (stock.m_matches + stock.m_mismatches == 3);
    phase.drop_objection(this);
  endtask
  function void report_phase(uvm_phase phase);
    uvm_report_server rs = uvm_report_server::get_server();
    $display("stock comparator: %0d matches, %0d mismatches; errors %0d, warnings %0d",
             stock.m_matches, stock.m_mismatches, rs.get_severity_count(UVM_ERROR), rs.get_severity_count(UVM_WARNING));
  endfunction
endclass

module tb;
  bit clk;
  always #5 clk = ~clk;
  incr_if bus (clk);
  incr_pipe dut (bus);
  initial begin
    uvm_root::get().set_timeout(2000ns, 1);
    uvm_config_db #(virtual incr_if)::set(null, "uvm_test_top", "vif", bus);
    run_test("predict_test");
  end
endmodule

before_fifo.get(e);
after_fifo.get(a);
if (e.compare(a)) matched++;
else begin
  mismatched++;
  `uvm_error("CMP", $sformatf("%s, predicted %s", a.convert2string(), e.convert2string()))
end

before_fifo.get(e);
after_fifo.get(a);
if (e == a) matched++;
else begin
  mismatched++;
  `uvm_error("CMP", $sformatf("%s, predicted %s", a.convert2string(), e.convert2string()))
end

after_fifo.get(a);
before_fifo.get(e);
if (a.y == e.y) matched++;
else begin
  mismatched++;
  `uvm_error("CMP", $sformatf("y=%0d, predicted %0d", a.y, e.y))
end

begin
  uvm_comparer c = new();
  before_fifo.get(e);
  after_fifo.get(a);
  if (e.compare(a, c)) matched++;
  else begin
    mismatched++;
    `uvm_error("CMP", c.get_miscompares())
  end
end

#(max_latency);

phase.get_objection().set_drain_time(this, max_latency);

// nothing: the counts are balanced

#30ns;