Metric analyzers and the ledger

Measure the latency of every read and the throughput over a window, decide their bounds in check_phase, report the numbers that make a passing test believable, and keep a ledger that catches a scoreboard losing predictions.

Lesson: U8.4
Time: 30 min
Runs on: Verilator with UVM
Check: 5 questions

The memory passes its functional regression and fails in the lab: under load, a read from bank 1 waits behind a burst of bank 0 reads until the controller times out. No test measured how long a read took. Every value was right, so every test was green.

A bank 0 burst starves bank 1

Tag 1 is ready at 65 ns and loses to bank 0 twice. Its latency is 60 ns instead of 40.

Requirements about time rather than values

A specification promises more than values. It promises that a read completes within so many cycles, that a link sustains so many bytes per second, that an arbiter never starves a port. The scoreboard of U8.2 checks none of that: it would happily match a response that arrived a week late. A design can be functionally perfect and still unusable, and the bench has to be able to say so.

The Cookbook calls the component for this a metric analyzer. It is an ordinary analysis component, fed by the same monitors through the same analysis ports, and it knows nothing about the right data. It knows when things happened. The example builds a latency_meter with two analysis imps, one for requests and one for responses, beside the scoreboard.

Record, compute, decide, tell

The meter spreads its work over four phases, and each phase has a reason.

Phase	What the meter does	Why there
`run_phase` (via `write`)	Records when each read was taken and when it was answered	The data exists only while the bus is running
`extract_phase`	Computes each latency, the maximum, the mean, how many exceed the bound, and the throughput over the busy window	A function, after the last response, before any verdict
`check_phase`	Raises `UVM_ERROR` if a read exceeded the bound, if a read was taken but never answered, or if the throughput fell below its floor	By now every component's numbers exist; a parent can combine them
`report_phase`	Prints the count, maximum, mean and bound	Runs whether or not the check failed, so the log always carries the numbers

The post-run phases run bottom-up, children before parents, one phase at a time across the whole bench.

The split is more than tidiness. Every component finishes extract_phase before anyone starts check_phase. So an environment can read its analyzers' numbers in its own check_phase and decide something none of them can alone, such as a total bandwidth across three ports. And the report phase prints the same numbers after a pass as after a fail, which is what makes a pass believable: "4 reads, max 60 ns" says more than "0 errors".

Three phases, three verbs. The bound belongs to the specification, so it is a field a test can configure.

function void extract_phase(uvm_phase phase);      // compute
  foreach (taken[i]) begin
    time lat = answered[i] - taken[i];
    if (lat > max_latency) max_latency = lat;
    if (lat > bound) late++;
  end
endfunction
function void check_phase(uvm_phase phase);        // decide
  if (late != 0)
    `uvm_error("LATENCY", $sformatf("%0d of %0d reads took longer than %0d ns", late, taken.size(), bound))
endfunction
function void report_phase(uvm_phase phase);       // tell
  $display("latency: %0d reads, max %0d ns, ...", taken.size(), max_latency);
endfunction

Pairing a response with its read

A latency is the time between a read being taken and its response being seen, so the meter must pair the two. Both time stamps come from the monitor, which stamps each transaction at the sampling edge, so the meter never reads the clock itself. The pairing problem is the one U8.2 solved: responses arrive out of order and tags repeat. So the meter keeps time issued[bit [3:0]][$], a queue of issue times per tag, and pops the front when a response with that tag arrives.

The tempting shortcut, "time since the most recent request", measures nothing once two reads overlap. In the example the response to tag 1 arrives at 95 ns; the most recent read was taken at 65 ns, which gives 30 ns, while the true latency is 60 ns. The spot question has that version.

The example's numbers: tags 2, 3 and 4 take 20 ns each, and tag 1 takes 60 ns because bank 0 was ready at the same edge twice and goes first. The maximum sits exactly on the 60 ns bound, so the check passes. One more bank 0 read behind tag 1 pushes it to 70 ns, and the quiz asks for the resulting report line.

A read that is taken and never answered leaves no sample, and a meter that only averages its samples would never notice. So the meter keeps outstanding(), the reads still in its queues, and check_phase reports any that remain. The test waits for both the scoreboard and the meter to drain. Then it keeps observing for one latency bound, 60 ns, before it drops its objection: the U8.1 quiet-window rule with this test's bound. In the example the counts balance at 95 ns and the run phase ends at 155 ns. A meter with work left cannot be cut off by a scoreboard that has none.

Throughput over a declared window

Four latencies, one throughput window

Each read has its own latency bar; throughput is one window over all of them: 4 reads from 35 to 95 ns, 0.67 reads per 10 ns cycle against a floor of 0.50.

Throughput is a count divided by a time, and the time is the whole decision. The meter uses the busy window, from the first read taken to the last response seen. Idle time outside it is left out, because the test, not the design, chose it.

Say which window you measured, in the report line, so the number can be reproduced. A window that starts at the first request includes the ramp while the pipeline fills, which understates a sustained rate; a long test measures a middle stretch instead. A window that ends at the last request rather than the last response leaves reads out of the count while their time is in the window. Whatever the choice, the floor is a specification value and the check lives in check_phase like the latency bound.

The ledger

U7.3 introduced the idea: every input creates an obligation and every checked result discharges one.

Three counters that must balanceArrows show which counters each test in check_phase reads.

The books balance155 ns (illustrative)
After the run phase ends, check_phase reads all three counters: 4 reads = 4 checked + 0 waiting.
More context
The example prints "ledger: 4 reads = 4 checked + 0 waiting; mismatches 0, orphans 0, checks on".
Follow the calls, values and waits
1. data reads = 4: reads → reads == checked + waiting
2. data checked = 4: checked → reads == checked + waiting
3. data waiting = 0: waiting() → reads == checked + waiting
Something was checked155 ns (illustrative)
With checks on, checked must not be 0. Here it is 4, so the run proved something.
More context
A run that compares nothing also balances: 0 reads = 0 checked + 0 waiting. This second test catches it.
Follow the calls, values and waits
1. data checked = 4: checked → checked != 0 with checks on
A lost prediction
The quiz scoreboard keeps one slot per tag and drops a response that finds nothing. It ends with 5 reads, 4 checked and 0 waiting: the rule fails, and only the ledger reports it.
More context
Its own compares report zero errors, zero mismatches and zero orphans. A read still in flight would be waiting; this one was dropped on the floor.
Follow the calls, values and waits
1. data reads = 5: reads → reads == checked + waiting
2. data checked = 4: checked → reads == checked + waiting
3. data waiting = 0: waiting() → reads == checked + waiting
A run that proved nothing
If the scoreboard never compared anything while checks were on, checked is 0 and the second test reports it.
More context
For example, an analysis port left unconnected: the scoreboard sees no traffic, every count stays at 0 and the ledger still balances.
Follow the calls, values and waits
1. data checked = 0: checked → checked != 0 with checks on

Read the diagram as text

reads: write_req, one per READ
checked: write_rsp, one per compared response
waiting(): predictions still in pending
reads == checked + waiting: else: the scoreboard lost a prediction
checked != 0 with checks on: else: a run that proved nothing

reads → reads == checked + waiting
checked → reads == checked + waiting
waiting() → reads == checked + waiting
checked → checked != 0 with checks on

Both tests check the scoreboard's own bookkeeping, not whether the monitor saw every transfer.

What does the ledger catch that the orphan and missing-read reports do not? A scoreboard that is wrong quietly. The quiz program keeps one prediction per tag and ignores a response that finds nothing, two small shortcuts a tired engineer makes. Two reads with tag 3 are issued back to back. The second prediction overwrites the first, the first response matches it, the second response finds nothing and is dropped. Zero errors, zero mismatches, zero orphans, and only four of five reads were checked. The ledger is the only line that notices.

Counts only what it saw

function void check_phase(uvm_phase phase);
  foreach (pending[t]) if (pending[t].size() != 0)
    `uvm_error("SB", "...never answered")
endfunction

Silent when a prediction was lost before it could be waited for.

Balances the books

function void check_phase(uvm_phase phase);
  foreach (pending[t]) if (pending[t].size() != 0)
    `uvm_error("SB", "...never answered")
  if (reads != checked + waiting())
    `uvm_error("SB", $sformatf("ledger: %0d reads, but %0d checked + %0d waiting",
               reads, checked, waiting()))
  if (checks_enable && checked == 0)
    `uvm_error("SB", "no response was checked, so this run proves nothing")
endfunction

A lost prediction and an empty run both fail, whatever the design did.

The last line guards against the oldest false pass of all: zero errors with zero checks. A sequence that sends only writes, a monitor whose virtual interface points at the wrong instance, an analysis port left unconnected. Each produces a clean run that checked nothing. "No response was checked" turns that into a failure with a cause.

Knobs that switch checking off

Some tests must run with the scoreboard quiet. One sends illegal requests to exercise error handling; another resets the design half way and expects responses to vanish. The Cookbook's monitor has checks_enable and coverage_enable for this, and the scoreboard takes the same knob through uvm_config_db.

When checks_enable is 0, write_req and write_rsp return before counting anything, so the ledger stays balanced at zero and the zero-check guard, which tests the knob first, stays silent. A knob that only skips the compare while the counts keep running would make the ledger lie, which is why both sides of the bookkeeping must honour it.

Treat the knob as a build-time choice, set once from the test. Flipping it during the run would leave predictions in the array with no response coming, or let responses arrive for reads that were never predicted. The next read with a reused tag would then pair with stale state. A reset in the middle of traffic needs its own rule for what happens to outstanding work, which U10.2 covers. And a scoreboard with checking off is only one consumer: the latency meter in the example keeps measuring, so the test still has to wait for it.

The ledger is a consistency check among the scoreboard's own counters. It proves that nothing was lost between write_req and check_phase; it cannot prove that the monitor saw every transfer, or that the masked bits it did not compare mattered. Those need the monitor's own counts of both streams and functional coverage of what was compared.

Print the counts in report_phase whether checks were on or off. In the log of a reset test, a line saying that no read was checked and checks were disabled is a fact. The same line in a data test is a bug. Only the printed numbers tell the two apart, and only if they are printed every time.

Try it

The bank 1 read (tag 1) is taken at 35 ns and ready at 65 ns, but bank 0 reads are ready at 65 and 75 ns and go first, so it is seen at 95 ns: 60 ns, on the bound. The three bank 0 reads take 20 ns each. Four reads completed between the first read at 35 ns and the last response at 95 ns: 0.67 reads per cycle, above the floor of 0.50. The meter prints its numbers in report_phase, after check_phase has passed them, and the test prints the balanced ledger with the checking mode.

Set bound = 40ns in the meter and check_phase reports one late read. Add a fourth bank 0 read to the script and tag 1 takes 70 ns.

RUNVerilator + UVM 2020.3.1 · simulator service

Edit the code, then run it

Expected among the UVM messages: 65 ns rsp tag=2 data=11 matches READ tag=2 bank=0 addr=1 taken at 45 ns, waiting: 1 3 4 75 ns rsp tag=3 data=11 matches READ tag=3 bank=0 addr=1 taken at 55 ns, waiting: 1 4 85 ns rsp tag=4 data=11 matches READ tag=4 bank=0 addr=1 taken at 65 ns, waiting: 1 95 ns rsp tag=1 data=22 matches READ tag=1 bank=1 addr=2 taken at 35 ns, waiting: none 155 ns test counts balanced, quiet window over, run phase ends latency: 4 reads, max 60 ns, mean 30.0 ns, bound 60 ns, 0 late throughput: 4 reads in 60 ns (35 to 95 ns) = 0.67 reads per cycle, floor 0.50 ledger: 4 reads = 4 checked + 0 waiting; mismatches 0, orphans 0, checks on UVM_ERROR : 0

Check yourself

Predict the outputThe example's script gains one more read: send(mem_req::READ, 5, 0, 1) after the tag 4 read, so three bank 0 reads follow the bank 1 read. The bound stays at 60 ns. What line does the latency meter print in report_phase?

`timescale 1ns/1ns
`include "uvm_macros.svh"
import uvm_pkg::*;

interface mem_if (input logic clk);
  logic       req_valid = 0;
  logic [1:0] req_kind  = 0;    // 0 read, 1 write, 2 control write
  logic [3:0] req_tag   = 0;
  logic       req_bank  = 0;
  logic [3:0] req_addr  = 0;
  logic [7:0] req_data  = 0;
  logic       rsp_valid = 0;
  logic [3:0] rsp_tag   = 0;
  logic [7:0] rsp_data  = 0;
endinterface

// A two-bank memory with tagged reads. Bank 0 is ready to answer one edge after a read is taken,
// bank 1 three edges after; the response is driven at that edge and a monitor sampling at rising
// edges sees it one edge later (two and four edges after the read, when nothing is in the way).
// One response per edge; bank 0 goes first when both are ready. Reads that share a tag
// answer in request order whatever their bank. A control write sets inv; a read taken while inv
// is 1 returns the stored byte inverted. The power-up contents are unspecified. LOSE_TAG is a
// fault switch for the bench: reads with that tag are taken and never answered (16 = off).
module tag_mem #(parameter int LOSE_TAG = 16) (mem_if bus);
  typedef struct { bit [3:0] tag; bit [7:0] data; int due; int seq; } pend_t;
  bit [7:0] mem0 [16], mem1 [16];
  bit       inv = 0;
  pend_t    q0[$], q1[$];
  int       cycle = 0, seqno = 0;
  initial foreach (mem0[i]) begin mem0[i] = 8'hC0 | i[3:0]; mem1[i] = 8'hD0 | i[3:0]; end
  // An older read with the same tag is still waiting in the other bank.
  function automatic bit held(pend_t head, pend_t other[$]);
    foreach (other[i]) if (other[i].tag == head.tag && other[i].seq < head.seq) return 1;
    return 0;
  endfunction
  always @(posedge bus.clk) begin
    pend_t p;
    cycle++;
    if (bus.req_valid) case (bus.req_kind)
      0: begin
        p.tag  = bus.req_tag;
        p.data = (bus.req_bank ? mem1[bus.req_addr] : mem0[bus.req_addr]) ^ {8{inv}};
        p.due  = cycle + (bus.req_bank ? 3 : 1);
        p.seq  = seqno++;
        if (bus.req_tag != LOSE_TAG[3:0] || LOSE_TAG > 15) begin
          if (bus.req_bank) q1.push_back(p); else q0.push_back(p);
        end
      end
      1: if (bus.req_bank) mem1[bus.req_addr] = bus.req_data; else mem0[bus.req_addr] = bus.req_data;
      2: inv = bus.req_data[0];
      default: ;
    endcase
    bus.rsp_valid <= 0;
    if (q0.size() != 0 && q0[0].due <= cycle && !held(q0[0], q1)) begin
      p = q0.pop_front();
      bus.rsp_valid <= 1; bus.rsp_tag <= p.tag; bus.rsp_data <= p.data;
    end else if (q1.size() != 0 && q1[0].due <= cycle && !held(q1[0], q0)) begin
      p = q1.pop_front();
      bus.rsp_valid <= 1; bus.rsp_tag <= p.tag; bus.rsp_data <= p.data;
    end
  end
endmodule

class mem_req extends uvm_sequence_item;
  `uvm_object_utils(mem_req)
  typedef enum bit [1:0] { READ, WRITE, CTRL } kind_e;
  kind_e    kind;
  bit [3:0] tag;
  bit       bank;
  bit [3:0] addr;
  bit [7:0] data;
  time      at;                // when the monitor saw the request taken
  function new(string name = "mem_req"); super.new(name); endfunction
  function string convert2string();
    if (kind == READ) return $sformatf("READ tag=%0d bank=%0d addr=%0d", tag, bank, addr);
    return $sformatf("%s tag=%0d bank=%0d addr=%0d data=%02h", kind.name(), tag, bank, addr, data);
  endfunction
endclass

class mem_rsp extends uvm_sequence_item;
  `uvm_object_utils(mem_rsp)
  bit [3:0] tag;
  bit [7:0] data;
  bit [7:0] mask = 8'hFF;      // 1 = this data bit is checked (a prediction's mask decides)
  bit       timed_out;         // a prediction the watchdog has already reported
  time      at;                // when the monitor saw it (a prediction: when the read was taken)
  mem_req   req;               // a prediction remembers the read it answers
  function new(string name = "mem_rsp"); super.new(name); endfunction
  function bit do_compare(uvm_object rhs, uvm_comparer comparer);
    mem_rsp o;
    if (!$cast(o, rhs)) return 0;
    return comparer.compare_field_int("tag", tag, o.tag, 4) &
           comparer.compare_field_int("data", data, o.data, 8);
  endfunction
  function string convert2string(); return $sformatf("tag=%0d data=%02h", tag, data); endfunction
endclass

class script_seq extends uvm_sequence #(mem_req);
  `uvm_object_utils(script_seq)
  function new(string name = "script_seq"); super.new(name); endfunction
  task send(mem_req::kind_e kind, bit [3:0] tag, bit bank, bit [3:0] addr, bit [7:0] data = 0);
    mem_req it = mem_req::type_id::create("it");
    start_item(it);
    it.kind = kind; it.tag = tag; it.bank = bank; it.addr = addr; it.data = data;
    finish_item(it);
  endtask
  task body();
    send(mem_req::WRITE, 0, 0, 1, 8'h11);
    send(mem_req::WRITE, 0, 1, 2, 8'h22);
    send(mem_req::READ,  1, 1, 2);          // 35 ns: bank 1, ready at 65
    send(mem_req::READ,  2, 0, 1);          // 45 ns: bank 0, answered at 55
    send(mem_req::READ,  3, 0, 1);          // 55 ns: ready at 65, and bank 0 goes first
    send(mem_req::READ,  4, 0, 1);          // 65 ns: ready at 75, bank 0 goes first again
    send(mem_req::READ,  5, 0, 1);          // 75 ns: a third bank 0 read
  endtask
endclass

// One request per cycle: offered at a falling edge, taken at the next rising edge.
class mem_driver extends uvm_driver #(mem_req);
  `uvm_component_utils(mem_driver)
  virtual mem_if vif;
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  task run_phase(uvm_phase phase);
    forever begin
      @(negedge vif.clk);
      seq_item_port.try_next_item(req);
      if (req == null) begin
        vif.req_valid <= 0;
        continue;
      end
      vif.req_valid <= 1;
      vif.req_kind  <= req.kind;
      vif.req_tag   <= req.tag;
      vif.req_bank  <= req.bank;
      vif.req_addr  <= req.addr;
      vif.req_data  <= req.data;
      @(posedge vif.clk);
      seq_item_port.item_done();
    end
  endtask
endclass

// Publishes every request the memory takes and every response it gives, stamped with the time.
class mem_monitor extends uvm_monitor;
  `uvm_component_utils(mem_monitor)
  virtual mem_if vif;
  uvm_analysis_port #(mem_req) req_ap;
  uvm_analysis_port #(mem_rsp) rsp_ap;
  function new(string name, uvm_component parent);
    super.new(name, parent);
    req_ap = new("req_ap", this);
    rsp_ap = new("rsp_ap", this);
  endfunction
  task run_phase(uvm_phase phase);
    forever begin
      @(posedge vif.clk);
      if (vif.req_valid) begin
        mem_req t = mem_req::type_id::create("t");
        t.kind = mem_req::kind_e'(vif.req_kind);
        t.tag  = vif.req_tag;
        t.bank = vif.req_bank;
        t.addr = vif.req_addr;
        t.data = vif.req_data;
        t.at   = $time;
        req_ap.write(t);
      end
      if (vif.rsp_valid) begin
        mem_rsp r = mem_rsp::type_id::create("r");
        r.tag  = vif.rsp_tag;
        r.data = vif.rsp_data;
        r.at   = $time;
        rsp_ap.write(r);
      end
    end
  endtask
endclass

`uvm_analysis_imp_decl(_req)
`uvm_analysis_imp_decl(_rsp)

class tag_scoreboard extends uvm_scoreboard;
  `uvm_component_utils(tag_scoreboard)
  uvm_analysis_imp_req #(mem_req, tag_scoreboard) req_imp;
  uvm_analysis_imp_rsp #(mem_rsp, tag_scoreboard) rsp_imp;
  bit [7:0] model0 [16], model1 [16];     // the reference model: what each location holds
  mem_rsp   pending[bit [3:0]][$];        // predictions waiting for a response, by tag, oldest first
  int       reads, checked, mismatches, orphans;
  uvm_event changed = new("changed");     // triggered on every request or response it receives
  bit       checks_enable = 1;            // knob: a test may switch comparison off
  function new(string name, uvm_component parent);
    super.new(name, parent);
    req_imp = new("req_imp", this);
    rsp_imp = new("rsp_imp", this);
  endfunction
  function void build_phase(uvm_phase phase);
    void'(uvm_config_db #(bit)::get(this, "", "checks_enable", checks_enable));
  endfunction
  function int waiting();
    int n = 0;
    foreach (pending[t]) n += pending[t].size();
    return n;
  endfunction
  function string waiting_tags();
    string s = "";
    foreach (pending[t]) repeat (pending[t].size()) s = {s, $sformatf(" %0d", t)};
    return s.len() ? s : " none";
  endfunction
  function void write_req(mem_req t);
    changed.trigger();                    // wakes a test that waits for the counts to change
    case (t.kind)
      mem_req::WRITE: begin
        if (t.bank) model1[t.addr] = t.data; else model0[t.addr] = t.data;
      end
      mem_req::CTRL: begin
        
      end
      mem_req::READ: begin
        mem_rsp e = mem_rsp::type_id::create("e");
        e.tag = t.tag;
        e.at  = t.at;
        e.req = t;
        if (!checks_enable) return;
        e.data = t.bank ? model1[t.addr] : model0[t.addr];
        pending[t.tag].push_back(e);
        reads++;
      end
    endcase
  endfunction
  function void write_rsp(mem_rsp r);
    mem_rsp e;
    changed.trigger();
    if (!pending.exists(r.tag) || pending[r.tag].size() == 0) begin
      orphans++;
      `uvm_error("SB", $sformatf("response %s at %0t has no read waiting", r.convert2string(), r.at))
      return;
    end
    e = pending[r.tag].pop_front();
    checked++;
    if (!e.compare(r)) begin
      mismatches++;
      `uvm_error("SB", $sformatf("%s, predicted data=%02h for %s taken at %0t",
                 r.convert2string(), e.data, e.req.convert2string(), e.at))
    end
  endfunction

  // Per tag: every waiting prediction the watchdog has not already reported.
  function void check_phase(uvm_phase phase);
    foreach (pending[t]) begin
      int unreported = 0, oldest = -1;
      foreach (pending[t][i]) if (!pending[t][i].timed_out) begin
        unreported++;
        if (oldest < 0) oldest = i;
      end
      if (unreported != 0)
        `uvm_error("SB", $sformatf("tag %0d: %0d read(s) never answered, first: %s at %0t",
                   t, unreported, pending[t][oldest].req.convert2string(), pending[t][oldest].at))
    end
    if (reads != checked + waiting())
      `uvm_error("SB", $sformatf("ledger: %0d reads, but %0d checked + %0d waiting", reads, checked, waiting()))
    if (checks_enable && checked == 0)
      `uvm_error("SB", "no response was checked, so this run proves nothing")
  endfunction
endclass

`uvm_analysis_imp_decl(_lat_req)
`uvm_analysis_imp_decl(_lat_rsp)

// A metric analyzer: how long each read took. It knows nothing about the right data.
class latency_meter extends uvm_component;
  `uvm_component_utils(latency_meter)
  uvm_analysis_imp_lat_req #(mem_req, latency_meter) req_imp;
  uvm_analysis_imp_lat_rsp #(mem_rsp, latency_meter) rsp_imp;
  time    issued[bit [3:0]][$];     // when each outstanding read was taken, by tag, oldest first
  mem_req issued_req[bit [3:0]][$]; // the same reads as objects
  time    last_issued;              // when the most recent read of any tag was taken
  time    taken[$], answered[$];    // one pair per answered read, recorded during the run
  time    bound = 60ns;             // the latency requirement
  real    floor_rate = 0.5;         // the throughput requirement, reads per cycle over the window
  bit     report_rate;              // the example prints the throughput line as well
  time    max_latency, window_start, window_end;
  real    mean_latency, rate;
  int     late;
  function new(string name, uvm_component parent);
    super.new(name, parent);
    req_imp = new("req_imp", this);
    rsp_imp = new("rsp_imp", this);
  endfunction
  function void write_lat_req(mem_req t);
    if (t.kind != mem_req::READ) return;
    issued[t.tag].push_back(t.at);
    issued_req[t.tag].push_back(t);
    last_issued = t.at;
  endfunction
  function void write_lat_rsp(mem_rsp r);
    if (!issued.exists(r.tag) || issued[r.tag].size() == 0) return;   // the scoreboard reports orphans
    answered.push_back(r.at);
    taken.push_back(issued[r.tag].pop_front());
    void'(issued_req[r.tag].pop_front());
  endfunction
  function int outstanding();                        // reads taken and not yet answered
    int n = 0;
    foreach (issued[t]) n += issued[t].size();
    return n;
  endfunction
  function void extract_phase(uvm_phase phase);      // compute
    real sum = 0;
    foreach (taken[i]) begin
      time lat = answered[i] - taken[i];
      sum += lat;
      if (lat > max_latency) max_latency = lat;
      if (lat > bound) late++;
      if (i == 0 || taken[i] < window_start) window_start = taken[i];
      if (answered[i] > window_end) window_end = answered[i];
    end
    mean_latency = taken.size() ? sum / taken.size() : 0;
    // Throughput over the busy window: first read taken to last response seen, in 10 ns cycles.
    rate = window_end > window_start ? taken.size() * 10.0 / (window_end - window_start) : 0;
  endfunction
  function void check_phase(uvm_phase phase);        // decide
    if (late != 0)
      `uvm_error("LATENCY", $sformatf("%0d of %0d reads took longer than %0d ns (max %0d ns)", late, taken.size(), bound, max_latency))
    if (outstanding() != 0)
      `uvm_error("LATENCY", $sformatf("%0d read(s) taken but never answered: no latency sample for them", outstanding()))
    if (taken.size() != 0 && rate < floor_rate)
      `uvm_error("THROUGHPUT", $sformatf("%0.2f reads per cycle over the window, floor %0.2f", rate, floor_rate))
  endfunction
  function void report_phase(uvm_phase phase);       // tell
    $display("latency: %0d reads, max %0d ns, mean %0.1f ns, bound %0d ns, %0d late",
             taken.size(), max_latency, mean_latency, bound, late);
    if (report_rate)
      $display("throughput: %0d reads in %0d ns (%0d to %0d ns) = %0.2f reads per cycle, floor %0.2f",
               taken.size(), window_end - window_start, window_start, window_end, rate, floor_rate);
  endfunction
endclass

class predict_test extends uvm_test;
  `uvm_component_utils(predict_test)
  uvm_sequencer #(mem_req) sqr;
  mem_driver     drv;
  mem_monitor    mon;
  tag_scoreboard sb;
  latency_meter  lat;
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  function void build_phase(uvm_phase phase);
    virtual mem_if vif;
    sqr = uvm_sequencer #(mem_req)::type_id::create("sqr", this);
    drv = mem_driver::type_id::create("drv", this);
    mon = mem_monitor::type_id::create("mon", this);
    sb  = tag_scoreboard::type_id::create("sb", this);
    if (!uvm_config_db #(virtual mem_if)::get(this, "", "vif", vif))
      `uvm_fatal("NOVIF", "no virtual interface")
    drv.vif = vif;
    mon.vif = vif;
    lat = latency_meter::type_id::create("lat", this);
  endfunction
  function void connect_phase(uvm_phase phase);
    drv.seq_item_port.connect(sqr.seq_item_export);
    mon.req_ap.connect(sb.req_imp);
    mon.rsp_ap.connect(sb.rsp_imp);
    mon.req_ap.connect(lat.req_imp);
    mon.rsp_ap.connect(lat.rsp_imp);
  endfunction
  task run_phase(uvm_phase phase);
    script_seq seq = script_seq::type_id::create("seq");
    phase.raise_objection(this);
    seq.start(sqr);
    begin
      bit balanced = 0;
      fork
        begin
          while (sb.waiting() != 0 || lat.outstanding() != 0) sb.changed.wait_trigger();
          balanced = 1;
        end
        #100ns;
      join_any
      disable fork;
      if (balanced) #60ns;                   // balanced: a quiet window of one latency bound
    end
    phase.drop_objection(this);
  endtask
endclass

module tb;
  bit clk;
  always #5 clk = ~clk;
  mem_if bus (clk);
  tag_mem #(.LOSE_TAG(16)) dut (bus);
  initial begin
    uvm_root::get().set_timeout(2000ns, 1);
    uvm_config_db #(virtual mem_if)::set(null, "uvm_test_top", "vif", bus);
    run_test("predict_test");
  end
endmodule

Predict the outputThis scoreboard keeps one prediction per tag and returns silently when a response finds none. The script writes two locations, then reads tag 1 (bank 1), tag 2 (bank 0), tag 3 twice from the same bank 1 address, and tag 4 (bank 0). What line does report_phase print?

`timescale 1ns/1ns
`include "uvm_macros.svh"
import uvm_pkg::*;

interface mem_if (input logic clk);
  logic       req_valid = 0;
  logic [1:0] req_kind  = 0;    // 0 read, 1 write, 2 control write
  logic [3:0] req_tag   = 0;
  logic       req_bank  = 0;
  logic [3:0] req_addr  = 0;
  logic [7:0] req_data  = 0;
  logic       rsp_valid = 0;
  logic [3:0] rsp_tag   = 0;
  logic [7:0] rsp_data  = 0;
endinterface

// A two-bank memory with tagged reads. Bank 0 is ready to answer one edge after a read is taken,
// bank 1 three edges after; the response is driven at that edge and a monitor sampling at rising
// edges sees it one edge later (two and four edges after the read, when nothing is in the way).
// One response per edge; bank 0 goes first when both are ready. Reads that share a tag
// answer in request order whatever their bank. A control write sets inv; a read taken while inv
// is 1 returns the stored byte inverted. The power-up contents are unspecified. LOSE_TAG is a
// fault switch for the bench: reads with that tag are taken and never answered (16 = off).
module tag_mem #(parameter int LOSE_TAG = 16) (mem_if bus);
  typedef struct { bit [3:0] tag; bit [7:0] data; int due; int seq; } pend_t;
  bit [7:0] mem0 [16], mem1 [16];
  bit       inv = 0;
  pend_t    q0[$], q1[$];
  int       cycle = 0, seqno = 0;
  initial foreach (mem0[i]) begin mem0[i] = 8'hC0 | i[3:0]; mem1[i] = 8'hD0 | i[3:0]; end
  // An older read with the same tag is still waiting in the other bank.
  function automatic bit held(pend_t head, pend_t other[$]);
    foreach (other[i]) if (other[i].tag == head.tag && other[i].seq < head.seq) return 1;
    return 0;
  endfunction
  always @(posedge bus.clk) begin
    pend_t p;
    cycle++;
    if (bus.req_valid) case (bus.req_kind)
      0: begin
        p.tag  = bus.req_tag;
        p.data = (bus.req_bank ? mem1[bus.req_addr] : mem0[bus.req_addr]) ^ {8{inv}};
        p.due  = cycle + (bus.req_bank ? 3 : 1);
        p.seq  = seqno++;
        if (bus.req_tag != LOSE_TAG[3:0] || LOSE_TAG > 15) begin
          if (bus.req_bank) q1.push_back(p); else q0.push_back(p);
        end
      end
      1: if (bus.req_bank) mem1[bus.req_addr] = bus.req_data; else mem0[bus.req_addr] = bus.req_data;
      2: inv = bus.req_data[0];
      default: ;
    endcase
    bus.rsp_valid <= 0;
    if (q0.size() != 0 && q0[0].due <= cycle && !held(q0[0], q1)) begin
      p = q0.pop_front();
      bus.rsp_valid <= 1; bus.rsp_tag <= p.tag; bus.rsp_data <= p.data;
    end else if (q1.size() != 0 && q1[0].due <= cycle && !held(q1[0], q0)) begin
      p = q1.pop_front();
      bus.rsp_valid <= 1; bus.rsp_tag <= p.tag; bus.rsp_data <= p.data;
    end
  end
endmodule

class mem_req extends uvm_sequence_item;
  `uvm_object_utils(mem_req)
  typedef enum bit [1:0] { READ, WRITE, CTRL } kind_e;
  kind_e    kind;
  bit [3:0] tag;
  bit       bank;
  bit [3:0] addr;
  bit [7:0] data;
  time      at;                // when the monitor saw the request taken
  function new(string name = "mem_req"); super.new(name); endfunction
  function string convert2string();
    if (kind == READ) return $sformatf("READ tag=%0d bank=%0d addr=%0d", tag, bank, addr);
    return $sformatf("%s tag=%0d bank=%0d addr=%0d data=%02h", kind.name(), tag, bank, addr, data);
  endfunction
endclass

class mem_rsp extends uvm_sequence_item;
  `uvm_object_utils(mem_rsp)
  bit [3:0] tag;
  bit [7:0] data;
  bit [7:0] mask = 8'hFF;      // 1 = this data bit is checked (a prediction's mask decides)
  bit       timed_out;         // a prediction the watchdog has already reported
  time      at;                // when the monitor saw it (a prediction: when the read was taken)
  mem_req   req;               // a prediction remembers the read it answers
  function new(string name = "mem_rsp"); super.new(name); endfunction
  function bit do_compare(uvm_object rhs, uvm_comparer comparer);
    mem_rsp o;
    if (!$cast(o, rhs)) return 0;
    return comparer.compare_field_int("tag", tag, o.tag, 4) &
           comparer.compare_field_int("data", data, o.data, 8);
  endfunction
  function string convert2string(); return $sformatf("tag=%0d data=%02h", tag, data); endfunction
endclass

class script_seq extends uvm_sequence #(mem_req);
  `uvm_object_utils(script_seq)
  function new(string name = "script_seq"); super.new(name); endfunction
  task send(mem_req::kind_e kind, bit [3:0] tag, bit bank, bit [3:0] addr, bit [7:0] data = 0);
    mem_req it = mem_req::type_id::create("it");
    start_item(it);
    it.kind = kind; it.tag = tag; it.bank = bank; it.addr = addr; it.data = data;
    finish_item(it);
  endtask
  task body();
    send(mem_req::WRITE, 0, 0, 1, 8'h11);
    send(mem_req::WRITE, 0, 1, 2, 8'h22);
    send(mem_req::READ,  1, 1, 2);
    send(mem_req::READ,  2, 0, 1);
    send(mem_req::READ,  3, 1, 2);          // tag 3 ...
    send(mem_req::READ,  3, 1, 2);          // ... twice, same bank and address
    send(mem_req::READ,  4, 0, 1);
  endtask
endclass

// One request per cycle: offered at a falling edge, taken at the next rising edge.
class mem_driver extends uvm_driver #(mem_req);
  `uvm_component_utils(mem_driver)
  virtual mem_if vif;
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  task run_phase(uvm_phase phase);
    forever begin
      @(negedge vif.clk);
      seq_item_port.try_next_item(req);
      if (req == null) begin
        vif.req_valid <= 0;
        continue;
      end
      vif.req_valid <= 1;
      vif.req_kind  <= req.kind;
      vif.req_tag   <= req.tag;
      vif.req_bank  <= req.bank;
      vif.req_addr  <= req.addr;
      vif.req_data  <= req.data;
      @(posedge vif.clk);
      seq_item_port.item_done();
    end
  endtask
endclass

// Publishes every request the memory takes and every response it gives, stamped with the time.
class mem_monitor extends uvm_monitor;
  `uvm_component_utils(mem_monitor)
  virtual mem_if vif;
  uvm_analysis_port #(mem_req) req_ap;
  uvm_analysis_port #(mem_rsp) rsp_ap;
  function new(string name, uvm_component parent);
    super.new(name, parent);
    req_ap = new("req_ap", this);
    rsp_ap = new("rsp_ap", this);
  endfunction
  task run_phase(uvm_phase phase);
    forever begin
      @(posedge vif.clk);
      if (vif.req_valid) begin
        mem_req t = mem_req::type_id::create("t");
        t.kind = mem_req::kind_e'(vif.req_kind);
        t.tag  = vif.req_tag;
        t.bank = vif.req_bank;
        t.addr = vif.req_addr;
        t.data = vif.req_data;
        t.at   = $time;
        req_ap.write(t);
      end
      if (vif.rsp_valid) begin
        mem_rsp r = mem_rsp::type_id::create("r");
        r.tag  = vif.rsp_tag;
        r.data = vif.rsp_data;
        r.at   = $time;
        rsp_ap.write(r);
      end
    end
  endtask
endclass

`uvm_analysis_imp_decl(_req)
`uvm_analysis_imp_decl(_rsp)

class tag_scoreboard extends uvm_scoreboard;
  `uvm_component_utils(tag_scoreboard)
  uvm_analysis_imp_req #(mem_req, tag_scoreboard) req_imp;
  uvm_analysis_imp_rsp #(mem_rsp, tag_scoreboard) rsp_imp;
  bit [7:0] model0 [16], model1 [16];     // the reference model: what each location holds
  mem_rsp   pending[bit [3:0]][$];        // predictions waiting for a response, by tag, oldest first
  int       reads, checked, mismatches, orphans;
  uvm_event changed = new("changed");     // triggered on every request or response it receives
  mem_rsp   slot[bit [3:0]];              // one prediction per tag
  function new(string name, uvm_component parent);
    super.new(name, parent);
    req_imp = new("req_imp", this);
    rsp_imp = new("rsp_imp", this);
  endfunction
  function int waiting();
    int n = 0;
    foreach (pending[t]) n += pending[t].size();
    return n;
  endfunction
  function string waiting_tags();
    string s = "";
    foreach (pending[t]) repeat (pending[t].size()) s = {s, $sformatf(" %0d", t)};
    return s.len() ? s : " none";
  endfunction
  function void write_req(mem_req t);
    changed.trigger();                    // wakes a test that waits for the counts to change
    case (t.kind)
      mem_req::WRITE: begin
        if (t.bank) model1[t.addr] = t.data; else model0[t.addr] = t.data;
      end
      mem_req::CTRL: begin
        
      end
      mem_req::READ: begin
        mem_rsp e = mem_rsp::type_id::create("e");
        e.tag = t.tag;
        e.at  = t.at;
        e.req = t;
        e.data = t.bank ? model1[t.addr] : model0[t.addr];
        slot[t.tag] = e;
        reads++;
      end
    endcase
  endfunction
  function void write_rsp(mem_rsp r);
    mem_rsp e;
    changed.trigger();
    if (!slot.exists(r.tag)) return;      // nothing to compare with
    e = slot[r.tag];
    slot.delete(r.tag);
    checked++;
    if (!e.compare(r)) begin
      mismatches++;
      `uvm_error("SB", $sformatf("%s, predicted data=%02h", r.convert2string(), e.data))
    end
  endfunction

  // Per tag: every waiting prediction the watchdog has not already reported.
  function void check_phase(uvm_phase phase);
    foreach (pending[t]) begin
      int unreported = 0, oldest = -1;
      foreach (pending[t][i]) if (!pending[t][i].timed_out) begin
        unreported++;
        if (oldest < 0) oldest = i;
      end
      if (unreported != 0)
        `uvm_error("SB", $sformatf("tag %0d: %0d read(s) never answered, first: %s at %0t",
                   t, unreported, pending[t][oldest].req.convert2string(), pending[t][oldest].at))
    end
    if (reads != checked + waiting())
      `uvm_error("SB", $sformatf("ledger: %0d reads, but %0d checked + %0d waiting", reads, checked, waiting()))
    if (checked == 0)
      `uvm_error("SB", "no response was checked, so this run proves nothing")
  endfunction
endclass

class predict_test extends uvm_test;
  `uvm_component_utils(predict_test)
  uvm_sequencer #(mem_req) sqr;
  mem_driver     drv;
  mem_monitor    mon;
  tag_scoreboard sb;
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  function void build_phase(uvm_phase phase);
    virtual mem_if vif;
    sqr = uvm_sequencer #(mem_req)::type_id::create("sqr", this);
    drv = mem_driver::type_id::create("drv", this);
    mon = mem_monitor::type_id::create("mon", this);
    sb  = tag_scoreboard::type_id::create("sb", this);
    if (!uvm_config_db #(virtual mem_if)::get(this, "", "vif", vif))
      `uvm_fatal("NOVIF", "no virtual interface")
    drv.vif = vif;
    mon.vif = vif;
  endfunction
  function void connect_phase(uvm_phase phase);
    drv.seq_item_port.connect(sqr.seq_item_export);
    mon.req_ap.connect(sb.req_imp);
    mon.rsp_ap.connect(sb.rsp_imp);
  endfunction
  task run_phase(uvm_phase phase);
    script_seq seq = script_seq::type_id::create("seq");
    phase.raise_objection(this);
    seq.start(sqr);
    #120ns;
    phase.drop_objection(this);
  endtask
  function void report_phase(uvm_phase phase);
    $display("reads %0d, checked %0d, waiting %0d, orphans %0d", sb.reads, sb.checked, sb.waiting(), sb.orphans);
  endfunction
endclass

module tb;
  bit clk;
  always #5 clk = ~clk;
  mem_if bus (clk);
  tag_mem #(.LOSE_TAG(16)) dut (bus);
  initial begin
    uvm_root::get().set_timeout(2000ns, 1);
    uvm_config_db #(virtual mem_if)::set(null, "uvm_test_top", "vif", bus);
    run_test("predict_test");
  end
endmodule

Spot the bugEach option is the body of the meter's write_lat_rsp after the check that a read with this tag is outstanding. The bench is the example's, and the test passes when four latencies are recorded with a maximum of 60 ns and none late. Which body fails?

answered.push_back(r.at);
taken.push_back(issued[r.tag].pop_front());
void'(issued_req[r.tag].pop_front());

answered.push_back($time);
taken.push_back(issued[r.tag].pop_front());
void'(issued_req[r.tag].pop_front());

answered.push_back(r.at);
taken.push_back(last_issued);
void'(issued[r.tag].pop_front());
void'(issued_req[r.tag].pop_front());

begin
  mem_req q = issued_req[r.tag].pop_front();
  answered.push_back(r.at);
  taken.push_back(q.at);
  void'(issued[r.tag].pop_front());
end

ConceptWhy compute the maximum latency in extract_phase and compare it with the bound in check_phase, instead of doing both in check_phase?

Because report_phase can only print values that were computed in extract_phase, so the maximum has to be computed thereBecause extract_phase is a task and can wait for the last response to arrive, while check_phase is a function and cannotBecause a uvm_error raised in extract_phase is not counted towards the final summary; only check_phase errors areBecause the phases run bottom-up, so by check_phase every component's numbers exist and a parent can combine them before deciding

ConceptA scoreboard ends with zero errors and prints reads 5, checked 4, waiting 0, orphans 0. What happened?

The scoreboard lost a prediction: an unanswered read would be waiting, a missed request would lower the read countThe monitor missed one request on the bus, so only four of the five reads were ever predicted and then checkedOne read was still in flight when the run phase ended, and its response came too late to be counted as checkedThe design answered one read twice, and the duplicate response was dropped as an orphan without being counted

Remember

A metric analyzer is an analysis component that measures time or throughput; it knows nothing about the right data.
Record during the run, compute in extract_phase, decide with uvm_error in check_phase, print in report_phase; the phases run bottom-up, so parents can combine children's numbers.
Throughput is a count over a declared window; say which window, and leave the test's idle time out of it.
Pair each response with its read by tag, with a queue per tag; "time since the last request" measures nothing once reads overlap.
The ledger reads == checked + waiting catches a scoreboard that loses predictions silently, checked == 0 catches a run that proved nothing, and the counts printed in report_phase make a pass believable.

Practice

A latency meter with two bounds Implement · Medium