An untimed model and the sync point

Feed an untimed SystemVerilog model from the input monitor, choose the event at which its answer meets the design's, and see what goes wrong when a mode change is in flight.

Lesson
U9.2
Time
30 min
Runs on
Verilator with UVM
Check
5 questions

A pipeline passes every directed test. Then a random test changes the operating mode while beats are in flight, and the scoreboard reports two mismatches in a row: y=11, model said 20 and y=21, model said 40. The design is right. The model was asked for its answer when each result came out, three cycles after the beat went in, and by then it had already seen the mode change.

Where the model was asked matters
0123456edge (ns)15253545557585beat taken1020mode 13040-design moderesult y-11216080model modeasked late-20406080result for 10 meets a model already in mode 1

The model applied the mode beat the moment it saw it. The design applies it only to beats taken afterwards, and its results for 10 and 20 arrive after the switch.

A model with no clock

An untimed model computes what the design computes, with none of the design's timing. The alu_model class here is a dozen lines: a mode and a step function that returns the result for one byte. There is no pipeline, no valid, no edge. Call it and the answer is there.

That is its strength. It is written from the specification, not from the micro-architecture, so it survives a redesign of the pipeline. It is small enough to review against the document in an afternoon. And it runs at the speed of a function call, which matters once the design has thousands of cycles of latency.

It is also its weakness. The model has an answer the moment a beat is taken. The design produces its answer three edges later. The bench must choose a prediction point, when the model is asked, and a comparison point, when its answer meets the design's: together, U9.1's sync point. The model has no opinion: it answers whenever it is asked.

Ask at the input, compare at the output

Ask at the input, compare at the outputArrows from the left are monitor writes; the model is a function call; the queue holds predictions.
Ask at the input, compare at the outputArrows from the left are monitor writes; the model is a function call; the queue holds predictions. input Data beat taken 10 at 15 ns, 20 at25 ns input Mode beat taken mode 1 at 35 ns model alu_model step(x) orset_mode(m), no time queue expected[$] 11, 21 waiting output Result seen 11 at 45 ns compare write_out Pop the oldest,compare step(x)set_modepushwritepop
  1. 15 ns

    The data beat 10 is answered in mode 0 at once: 11 waits in the queue.

    More context

    Log line 15 ns model x=10 -> 11, 1 waiting. The model is a function call, so the answer exists in the same time step as the beat.

    Follow the calls, values and waits
    1. call step(10): Data beat taken → alu_model
    2. data prediction 11: alu_model → expected[$]
  2. 25 ns

    The beat 20 is answered the same way: 21 joins 11 in the queue.

    More context

    Log line 25 ns model x=20 -> 21, 2 waiting. Neither result has left the pipeline yet; the latency is 30 ns.

    Follow the calls, values and waits
    1. call step(20): Data beat taken → alu_model
    2. data prediction 21: alu_model → expected[$]
  3. 35 ns

    The mode beat reaches the model at the same event the design acts on. Earlier answers are already queued.

    More context

    Log line 35 ns mode 1. The two predictions in the queue were computed in mode 0 and stay as they are.

    Follow the calls, values and waits
    1. call set_mode(1): Mode beat taken → alu_model
  4. 45 ns

    The result for 10 pops 11, which was computed in mode 0. With late = 1 the model would be asked now, in mode 1, and say 20.

    More context

    Log line 45 ns match y=11 (beat taken at 15 ns). In the same time step the beat 30 is asked in mode 1 and 60 joins the queue.

    Follow the calls, values and waits
    1. data observed y=11: Result seen → write_out
    2. data prediction 11: expected[$] → write_out
Read the diagram as text
  • Data beat taken: 10 at 15 ns, 20 at 25 ns
  • Mode beat taken: mode 1 at 35 ns
  • alu_model: step(x) or set_mode(m), no time
  • expected[$]: 11, 21 waiting
  • Result seen: 11 at 45 ns
  • write_out: Pop the oldest, compare
  • Data beat taken → alu_model: step(x)
  • Mode beat taken → alu_model: set_mode
  • alu_model → expected[$]: push
  • Result seen → write_out: write
  • expected[$] → write_out: pop

Data and mode reach the model in the order the design took them; the queue carries answers across the 30 ns latency.

The scoreboard asks the model when the input monitor reports a beat, and stamps each prediction with the time the beat was taken. write_out pops the oldest prediction when a result appears.

The model is asked at the event the design acts on: the edge that takes the beat.
function void write_in(op_item t);
  if (t.is_mode) begin model.set_mode(t.x[1:0]); return; end
  e = op_result::type_id::create("e");
  e.y  = model.step(t.x);     // the prediction point: the beat is taken, the model answers
  e.at = t.at;
  expected.push_back(e);
endfunction

A mode beat goes to the model at the same event, through set_mode. Data beats and mode beats reach the model in the order the design took them, so the model is in the right mode for every call.

The queue between write_in and write_out is the decoupling. The model runs ahead and its answers wait. The example reports a peak of three predictions waiting. At 45 ns the beat 30 is taken and the result for 10 arrives at the same edge. The monitor publishes the input first, so for a moment three answers are in the queue. For a design with a latency of 300 cycles the queue would hold 300. uvm_tlm_analysis_fifo (U7.2) is the same idea with a get that blocks, for a comparator that runs in run_phase.

The queue also tells the test when it may stop. Results trail their beats by 30 ns, so the test waits 40 ns after the last beat is taken before it drops its objection. Then check_phase reports any prediction still queued: a beat the design took and never answered is an error there, not a silent pass.

The hook, rerun

Set late = 1 in the scoreboard and rerun. Now write_in stores the raw beat, and write_out asks the model when the result arrives. The mode beat still reaches the model at 35 ns, because a mode beat has no result to wait for. So at 45 ns the result for beat 10 meets a model in mode 1, which doubles 10 and says 20. The design, in mode 0 at 15 ns, said 11. Two false mismatches, then two matches once the mode has caught up.

Model asked when the result arrives

function void write_in(op_item t);
  if (t.is_mode) model.set_mode(t.x[1:0]);
  else raw.push_back(t);
endfunction
function void write_out(op_result r);
  op_item t = raw.pop_front();
  compare(r.y, model.step(t.x));
endfunction

Mode changes reach the model at input time and data at output time, so the two drift apart whenever a mode beat has beats in flight around it.

Model asked when the beat is taken

function void write_in(op_item t);
  if (t.is_mode) model.set_mode(t.x[1:0]);
  else expected.push_back(model.step(t.x));
endfunction
function void write_out(op_result r);
  compare(r.y, expected.pop_front());
endfunction

Everything reaches the model in the order the design saw it. The queue carries the answers across the latency.

The lesson is general. A model with state must see every event in the order the design saw it. Each data beat must then be answered in the state in force when the design took it. An arrangement that applies some events early and others late works until state changes while results are in flight. Random tests find that case in minutes.

Directed tests rarely do. They tend to set the mode, wait for the pipeline to empty, then send data. With nothing in flight when the mode changes, the late arrangement gives the right answers. The bug needs a mode beat with data beats on both sides of it inside one latency window, which is exactly what the example script sends.

There are other correct arrangements. The quiz's spot item has a scoreboard that queues every beat, mode beats included, and replays them through the model in order when a result arrives. That is also right, because the replay keeps the order. It must drain its queue at the end too, or a mode beat at the tail is never applied. What fails is applying one kind of event early and the other late.

What a cycle-accurate model would add

The untimed model cannot say when a result should appear. If the pipeline produced 11 at 55 ns instead of 45, or produced it twice, the queue would still pair values in order and the first case would pass. A cycle-accurate model reproduces the design's timing: three stages, one result per taken beat, out_valid on the right edge. Compared against the pins edge by edge, it catches a late result, a dropped beat and a stuck out_valid.

Untimed class modelCycle-accurate model
Written fromThe specificationThe micro-architecture
CatchesWrong values, wrong mode handlingThose, plus wrong timing, duplicates, drops
CostA function per outputA second copy of the pipeline's control
When the pipeline is redesignedUnchangedRewritten with it
Shares assumptions with the designOnly the specificationThe specification and the timing
Prediction and comparison pointsTransaction events, chosen by youEvery clock edge

The cycle-accurate model buys timing checks at the price of tracking the design.

Most benches take the middle road: an untimed model for the values, and separate, cheap timing checks for the rest. One response per request is an assertion or a count in check_phase (U7.3). A latency bound is a meter (U8.4). Those checks do not need a second pipeline, and they do not break when the designer adds a stage.

Try it

The model answers at the edge that takes each beat; the design answers three edges later. Up to three answers wait in the queue, and each result pops the oldest.

Set late = 1 in alu_scoreboard and rerun: the results for 10 and 20 are compared against a model that has already switched to mode 1, and the summary shows mismatches 2. The design did not change.

RUNVerilator + UVM 2020.3.1 · simulator service
Edit the code, then run it
Expected among the UVM messages: 15 ns model x=10 -> 11, 1 waiting 25 ns model x=20 -> 21, 2 waiting 35 ns mode 1 45 ns model x=30 -> 60, 3 waiting 45 ns match y=11 (beat taken at 15 ns) 55 ns model x=40 -> 80, 3 waiting 55 ns match y=21 (beat taken at 25 ns) 75 ns match y=60 (beat taken at 45 ns) 85 ns match y=80 (beat taken at 55 ns) checked 4, mismatches 0, peak waiting 3 UVM_ERROR : 0

Check yourself

Predict the outputThe scoreboard runs with late = 1, asking the model when each result arrives. The script sets mode 2 (halve), sends 8, sets mode 0 (add 1) and sends 8 again. What line does report_phase print?
`timescale 1ns/1ns
`include "uvm_macros.svh"
import uvm_pkg::*;

interface alu_if (input logic clk);
  logic       valid   = 0;
  logic       is_mode = 0;      // 1: this beat sets the mode instead of carrying data
  logic [7:0] x       = 0;
  logic       out_valid = 0;
  logic [7:0] y       = 0;
endinterface

// Takes one beat per rising edge where valid is 1. A mode beat sets the mode for the beats taken
// after it. A data beat produces y three edges later, for one cycle. Bytes are unsigned:
// mode 0 adds 1, mode 1 doubles, mode 2 halves (logical shift right), mode 3 inverts.
module alu_pipe (alu_if bus);
  logic [1:0] mode = 0;
  logic       v1 = 0, v2 = 0;
  logic [7:0] s1 = 0, s2 = 0;
  function automatic logic [7:0] f(logic [1:0] m, logic [7:0] x);
    case (m)
      0: return x + 1;
      1: return x << 1;
      2: return x >> 1;
      default: return ~x;
    endcase
  endfunction
  always @(posedge bus.clk) begin
    if (bus.valid && bus.is_mode) mode <= bus.x[1:0];
    v1 <= bus.valid && !bus.is_mode;
    s1 <= f(mode, bus.x);            // the mode in force at the taking edge
    v2 <= v1;  s2 <= s1;
    bus.out_valid <= v2;  bus.y <= s2;
  end
endmodule

class op_item extends uvm_sequence_item;
  `uvm_object_utils(op_item)
  bit       is_mode;
  bit [7:0] x;
  time      at;
  function new(string name = "op_item"); super.new(name); endfunction
endclass

class op_result extends uvm_sequence_item;
  `uvm_object_utils(op_result)
  bit [7:0] y;
  time      at;
  function new(string name = "op_result"); super.new(name); endfunction
endclass

// One beat per cycle: offered at a falling edge, taken at the next rising edge.
class alu_driver extends uvm_driver #(op_item);
  `uvm_component_utils(alu_driver)
  virtual alu_if vif;
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  task run_phase(uvm_phase phase);
    forever begin
      @(negedge vif.clk);
      seq_item_port.try_next_item(req);
      if (req == null) begin
        vif.valid <= 0;
        continue;
      end
      vif.valid   <= 1;
      vif.is_mode <= req.is_mode;
      vif.x       <= req.x;
      @(posedge vif.clk);
      seq_item_port.item_done();
    end
  endtask
endclass

// Publishes every beat the pipeline takes and every result it shows, stamped with the time.
class alu_monitor extends uvm_monitor;
  `uvm_component_utils(alu_monitor)
  virtual alu_if vif;
  uvm_analysis_port #(op_item)   in_ap;
  uvm_analysis_port #(op_result) out_ap;
  function new(string name, uvm_component parent);
    super.new(name, parent);
    in_ap  = new("in_ap", this);
    out_ap = new("out_ap", this);
  endfunction
  task run_phase(uvm_phase phase);
    forever begin
      @(posedge vif.clk);
      if (vif.valid) begin
        op_item t = op_item::type_id::create("t");
        t.is_mode = vif.is_mode; t.x = vif.x; t.at = $time;
        in_ap.write(t);
      end
      if (vif.out_valid) begin
        op_result r = op_result::type_id::create("r");
        r.y = vif.y; r.at = $time;
        out_ap.write(r);
      end
    end
  endtask
endclass

class alu_model;
  bit [1:0] mode = 0;
  function void set_mode(bit [1:0] m); mode = m; endfunction
  function bit [7:0] step(bit [7:0] x);
    case (mode)
      0: return x + 1;
      1: return x << 1;
      2: return x >> 1;
      default: return ~x;
    endcase
  endfunction
endclass

`uvm_analysis_imp_decl(_in)
`uvm_analysis_imp_decl(_out)

class alu_scoreboard extends uvm_scoreboard;
  `uvm_component_utils(alu_scoreboard)
  uvm_analysis_imp_in  #(op_item, alu_scoreboard)   in_imp;
  uvm_analysis_imp_out #(op_result, alu_scoreboard) out_imp;
  alu_model model = new();
  bit       late = 0;                 // 1: ask the model when the result arrives
  op_result expected[$];              // the model's answers, waiting for the design
  op_item   raw[$];                   // late only: beats waiting to be run through the model
  int       checked, mismatches, peak;
  bit [7:0] first_y, first_want;
  function new(string name, uvm_component parent);
    super.new(name, parent);
    in_imp  = new("in_imp", this);
    out_imp = new("out_imp", this);
  endfunction
  function void write_in(op_item t);
    op_result e;
    if (t.is_mode) begin
      model.set_mode(t.x[1:0]);
      return;
    end
    if (late) begin raw.push_back(t); return; end
    e = op_result::type_id::create("e");
    e.y = model.step(t.x);              // the prediction point: the beat is taken, the model answers
    e.at = t.at;
    expected.push_back(e);
    if (expected.size() > peak) peak = expected.size();
  endfunction
  function void write_out(op_result r);
    bit [7:0] want;
    time      at;
    if (late) begin
      op_item t;
      if (raw.size() == 0) begin `uvm_error("SB", "result with no beat waiting") return; end
      t = raw.pop_front();
      want = model.step(t.x);           // the model's mode may have moved on by now
      at = t.at;
    end else begin
      op_result e;
      if (expected.size() == 0) begin `uvm_error("SB", "result with no prediction waiting") return; end
      e = expected.pop_front();
      want = e.y; at = e.at;
    end
    checked++;
    if (r.y == want) begin
    end else begin
      if (mismatches == 0) begin first_y = r.y; first_want = want; end
      mismatches++;
      `uvm_error("SB", $sformatf("y=%0d, model said %0d for the beat taken at %0t", r.y, want, at))
    end
  endfunction
  function void check_phase(uvm_phase phase);
    if (expected.size() + raw.size() != 0)
      `uvm_error("SB", $sformatf("%0d beat(s) never produced a result", expected.size() + raw.size()))
  endfunction
endclass

class alu_seq extends uvm_sequence #(op_item);
  `uvm_object_utils(alu_seq)
  function new(string name = "alu_seq"); super.new(name); endfunction
  task send(bit is_mode, bit [7:0] x);
    op_item it = op_item::type_id::create("it");
    start_item(it);
    it.is_mode = is_mode; it.x = x;
    finish_item(it);
  endtask
  task body();
    send(1, 2);       // mode 2: halve
    send(0, 8);
    send(1, 0);       // mode 0: add 1
    send(0, 8);
  endtask
endclass

class predict_test extends uvm_test;
  `uvm_component_utils(predict_test)
  uvm_sequencer #(op_item) sqr;
  alu_driver  drv;
  alu_monitor mon;
  alu_scoreboard sb;
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  function void build_phase(uvm_phase phase);
    virtual alu_if vif;
    sqr = uvm_sequencer #(op_item)::type_id::create("sqr", this);
    drv = alu_driver::type_id::create("drv", this);
    mon = alu_monitor::type_id::create("mon", this);
    if (!uvm_config_db #(virtual alu_if)::get(this, "", "vif", vif))
      `uvm_fatal("NOVIF", "no virtual interface")
    drv.vif = vif;
    mon.vif = vif;
    sb = alu_scoreboard::type_id::create("sb", this);
  endfunction
  function void connect_phase(uvm_phase phase);
    drv.seq_item_port.connect(sqr.seq_item_export);
    mon.in_ap.connect(sb.in_imp);
    mon.out_ap.connect(sb.out_imp);
  endfunction
  task run_phase(uvm_phase phase);
    alu_seq seq = alu_seq::type_id::create("seq");
    phase.raise_objection(this);
    sb.late = 1;                        // the model runs when each result arrives
    seq.start(sqr);
    #40;                                // room for the last result: three edges

    phase.drop_objection(this);
  endtask
  function void report_phase(uvm_phase phase);
    $display("checked %0d, mismatches %0d, first mismatch y=%0d predicted %0d",
             sb.checked, sb.mismatches, sb.first_y, sb.first_want);
  endfunction
endclass

module tb;
  bit clk;
  always #5 clk = ~clk;
  alu_if bus (clk);
  alu_pipe dut (bus);
  initial begin
    uvm_root::get().set_timeout(2000ns, 1);
    uvm_config_db #(virtual alu_if)::set(null, "uvm_test_top", "vif", bus);
    run_test("predict_test");
  end
endmodule
Spot the bugThe script sends 10, 20, a mode-1 beat, 30 and 40. Each option is the scoreboard's pair of write_in and write_out. Which one reports false mismatches?
ConceptThe pipeline has a bug: every result comes out one edge later than specified, with the right value. The test waits long enough for all of them. What does the example's scoreboard report?
ConceptA design's latency grows to 300 cycles and the test sends a beat every cycle. What changes in the untimed-model scoreboard?
ConceptThe example asks the model at the input for data beats and mode beats alike. Why does that arrangement give the design's answers?

Remember

  • An untimed model computes values with none of the design's timing; it is written from the specification and survives a pipeline redesign.
  • Choose a prediction point and a comparison point: ask the model at the event the design acts on, and compare when the result appears.
  • A model with state must receive every event, data and control alike, in the order the design saw it; splitting them across events gives false mismatches once state changes in flight.
  • A queue or analysis FIFO between prediction and comparison carries the model's answers across the design's latency; it grows with the beats in flight.
  • A cycle-accurate model adds timing checks at the price of tracking the micro-architecture; most benches pair an untimed model with separate, cheap timing checks.