ASIC Point

Performance and scale

Measure what a bench spends per item, remove the usual costs (objections per item, config lookups in loops, formatted messages, recording, object churn), and stop a pending table from growing without bound.

Lesson
U15.3
Time
30 min
Runs on
Verilator with UVM
Check
5 questions

Two small changes made a bench about six times slower per item and made its memory climb on long tests. One change raised an objection for every item in the driver; the other was a scoreboard table that never forgot a request the design had dropped. Nothing failed while the traffic ran, so nothing pointed at either change.

Per item, measured on this machineHost time
Driver loop, no objectionabout 16 us
Same loop, objection per itemabout 100 us
uvm_config_db::get with 41 entries of that name268 to 310 us (five runs)
Reading cfg.gap through a held handleunder 0.1 us

Host times on this machine (Verilator 5.052, -O0 build, a shared host): one run of the example, plus a five-run range for the lookup. Numbers and ratios belong to this build, workload and host; the operation counts are what carry over.

Measure counts before time

Host time per operation changes with the machine, the build and the load. Counts do not. So measure in this order. First make sure two runs are comparable: the same test, seed, simulated time, simulator build and host conditions. Then compare their total host time and simulated time. Then count the expensive events per item, and only then time the few operations the counts point to.

Count per itemHow to get itTarget in this bench
Objection calls reaching the testOverride raised() and dropped() in the test0 during traffic
Config lookups+UVM_CONFIG_DB_TRACE: count GET lines after a marker message at the start of run_phase0 after build
Messages formattedMessages printed per item; calls to $sformatf outside the macrosFew, and only at the verbosity in use
Objects allocatedCount create, new and clone calls, and compare with itemsOne per stored or published transaction
Table and FIFO sizesSample size() at fixed timesFlat after warm-up

The first numbers to collect for a slow or growing run, once two runs are known to be comparable.

The example's stopwatch reads the host clock through a three-line C function, wall_us() in clock.c, built with the program. Its numbers below are one run on this machine; rerun it and they will move.

An objection per item

One objection per item climbs the whole treeThe example's worker at uvm_test_top.e.agt.drv. Arrows are calls made by the objection as it propagates: one raised() per level for each raise, one dropped() per level for each drop.
One objection per item climbs the whole treeThe example's worker at `uvm_test_top.e.agt.drv`. Arrows are calls made by the objection as it propagates: one `raised()` per level for each raise, one `dropped()` per level for each drop. driver e.agt.drv raise, 10 ns of work,drop agent e.agt raised(), dropped() env e raised(), dropped() test uvm_test_top Counts the calls itreceives root uvm_top Total decides whenthe phase may end process Drain process Forked at each levelwhose count reaches0 propagatepropagatepropagatepropagatecount 0: scheduledrop passes up
  1. Raise

    The worker raises for one item. The count is added at each level above it, and each level's raised() runs: the test sees one call.

    More context

    Every level runs its raised() on the way up. With no item objection the test was notified 0 times for 2000 items.

    Follow the calls, values and waits
    1. call raised(): e.agt.drv → e.agt
    2. call raised(): e.agt → e
    3. call raised(): e → uvm_test_top
    4. call raised(): uvm_test_top → uvm_top
  2. Drop to zero

    After 10 ns the worker drops. Its own count is now 0, so the library schedules a drain in a background process instead of propagating at once.

    More context

    A raise in the same time step, before the drain runs, cancels it: 2000 zero-time pairs notify the test once right after the loop (the first raise) and twice 1 ns later.

    Follow the calls, values and waits
    1. call drop_objection: e.agt.drv → Drain process
    2. Waiting: Drain process — drain scheduled, not yet run
  3. The drain propagates

    The background process runs the drain and passes the drop up, level by level; the test sees a second call. Measured: 4000 calls at the test for 2000 items.

    More context

    The example prints "2000 items, objection per item: the test was notified 4000 times": one raise and one drop per item. A #0 after each drop lets every drain run, giving 4000 at once.

    Follow the calls, values and waits
    1. call dropped(): Drain process → e.agt
    2. call dropped(): e.agt → e
    3. call dropped(): e → uvm_test_top
    4. call dropped(): uvm_test_top → uvm_top
    5. Released: Drain process
Read the diagram as text
  • e.agt.drv: raise, 10 ns of work, drop
  • e.agt: raised(), dropped()
  • e: raised(), dropped()
  • uvm_test_top: Counts the calls it receives
  • uvm_top: Total decides when the phase may end
  • Drain process: Forked at each level whose count reaches 0
  • e.agt.drv → e.agt: propagate
  • e.agt → e: propagate
  • e → uvm_test_top: propagate
  • uvm_test_top → uvm_top: propagate
  • e.agt.drv → Drain process: count 0: schedule
  • Drain process → e.agt: drop passes up

One test-level raise and drop around the whole traffic replaces these per-item pairs; it starts at the test, so it never climbs through drv, agt and e.

An objection exists to keep a phase from ending while work is pending. Raising and dropping one per item in a component three levels down repeats the whole propagation for every item. The test was notified 4000 times for 2000 items, and each item cost about six times as much host time as without the objection.

Hold one objection in the test, or in the root sequence's starting phase, for the whole traffic. If per-item objections come from code you cannot change, phase.get_objection().set_propagate_mode(0) sends each raise and drop straight to the top. That skips the calls at the levels between, but it does not remove the per-item drain. For work still in flight when the traffic ends, keep the completion accounting of U10.3. A drain time only adds a grace period, sized from a known latency bound; the final checks still prove that the work completed.

Why a raise right after a drop can look free

Suppose a component drops to zero and raises again in the same time step, before the background process has run. The library finds the pending drain and cancels it, so that pair adds nothing above the component. The example measures it: 2000 zero-time pairs notify the test once at the end of the loop (the first raise) and twice 1 ns later, when the last drop has propagated. Read such a count only after the scheduler has had a chance to run.

A #0 after each drop yields to the background process, which in this bench lets each drain run before the next raise: the example counts 4000 notifications for 2000 pairs. That holds here because no drain time is set and no all_dropped callback waits; with either, the next raise can still cancel a drain in progress. This is a scheduling detail of uvm-core 2020.3.1 (m_raise searches the scheduled, forked and running drains first), not a reason to raise per item.

Configuration lookups belong in build

A get is a search. The library collects every entry with the requested name and matches each entry's scope against the caller's path. The example sets 41 entries named gap at made-up paths, as a large bench with one per agent might. One lookup took 268 to 310 us here over five runs; with one entry, 18 to 22 us. Reading cfg.gap through a held handle took under 0.1 us. A narrower scope does not help, because every entry with that name is still checked. These entries use exact paths; wildcard and regex scopes add pattern matching for each entry, so they are likely to cost more (not measured here).

Lookup per item

task run_phase(uvm_phase phase);
  forever begin
    seq_item_port.get_next_item(req);
    void'(uvm_config_db #(int)::get(this, "", "gap", gap));
    repeat (gap) @(posedge vif.clk);
    ...

One search per item, and the cost grows with every bench that adds an entry with the same name.

Lookup once, keep the handle

function void build_phase(uvm_phase phase);
  if (!uvm_config_db #(agent_cfg)::get(this, "", "cfg", cfg))
    `uvm_fatal("CFG", "no agent_cfg")
endfunction
...
repeat (cfg.gap) @(posedge vif.clk);   // a field read

One search at build. A test that changes cfg.gap mid-run is seen at the next item, because the handle is shared. Setting a new object in the database would not reach this held handle.

If a value really must change through the database at run time, have one thread call uvm_config_db #(int)::wait_modified(this, "", "gap"). It blocks until a set to that name reaches this component; the thread then calls get once and updates a field, instead of polling.

Messages and recording

Operation, at UVM_HIGH, filtered by the current verbosityHost time each
The uvm_info macro with a $sformatf messageabout 0.3 us
uvm_report_info(id, $sformatf(...), UVM_HIGH)about 2 us
begin_tr + end_tr, recording offabout 8 us
begin_tr + end_tr, recording on (text database)about 60 us

Measured on this machine, one run. Neither message call printed anything, because the default verbosity is UVM_MEDIUM.

The macro checks the verbosity before it evaluates its message, so a filtered $sformatf is never formatted. Calling uvm_report_info directly formats the string first and filters it afterwards. Use the macros in hot paths, and keep UVM_HIGH detail there for the run that needs it.

Writing records is opt-in, but the bracketing is not. Every item that passes through a sequencer gets begin_tr and end_tr automatically from start_item and finish_item, and those calls cost time even with recording off (the third row). Enable recording only on the components you need, for the failing seed. Where per-item cost matters, call disable_auto_item_recording() on the sequencer or through the driver's seq_item_port, or define UVM_DISABLE_AUTO_ITEM_RECORDING. A driver that brackets items itself should call it too, so items are not stamped twice (U2.4).

Objects in hot loops

Creating a transaction through the factory cost about 7 us here, new about 5 us (one run on this machine), and rewriting one object's fields almost nothing. The factory's extra cost buys overrides; keep it for objects a test may replace. Field macros do not slow construction: their cost appears in copy, compare, print and record, which U2.4 covers.

Reuse is safe once every holder has released the handle or taken its own copy. The pending table keeps each expectation's handle until the response arrives. If the generator rewrites one object instead of creating one per request, every waiting entry points at the same object and shows the last request's fields. U7.2 states the rule for analysis traffic. A publisher that sends a fresh object and never changes it after write lets listeners keep handles; one that reuses its object forces each keeper to copy. Here, one fresh object per stored expectation is the simple choice.

Memory that grows: an unbounded pending table

An out-of-order checker keeps each expectation until its response arrives (U8.2). The example switches from timing loops to a second experiment: traffic() is a synthetic response schedule, not a design, and it calls both tables directly. Request ids are unique, 0 to 1999, and every eighth response is dropped. Without an age limit, those expectations wait forever, and the table grows with the length of the test. When several requests can share a tag, use U8.2's per-tag queues instead of one entry per id.

Traffic timeUnbounded tableBounded table (100 ns)
5 us654
10 us1274
15 us1904
20 us2524
After the end scan250 never answered0 waiting, 250 missing

Measured entries waiting in each table. The unbounded table grows by one entry per eight requests; the bounded one never held more than 5.

Memory is entries times their size, and each entry here holds a transaction handle. The portable measurement is the entry count at fixed times. A count that rises with test length while traffic is steady is a symptom: either a backlog, where arrivals outpace a working consumer, or retained state that will never retire. Here the dropped responses prove the second. The same check applies to a table, a queue or an unread analysis FIFO.

An age limit detects each lost response close to where it happened and frees its entry. The first was detected and timestamped 180 ns into the traffic; the end scan then reports the total as an error. The limit must come from the specified legal latency, and the sweep must run throughout the test. Set it too small and the checker reports good responses as missing, then as unexpected when they arrive. Without a known latency bound, use a capacity limit, backpressure or a completion rule instead of silently removing outstanding work. Both tables keep the end scan.

Try it

The host lines are this run's stopwatch readings and change on every run; the other lines do not. The two errors are the end scans: 250 expectations never answered in the unbounded table, and 250 missing responses in the bounded one, the first detected 180 ns into the traffic and reported at the end.

Move phase.raise_objection(this) and drop_objection out of items into the caller and the per-item cost falls back to the first line's. In traffic, change bounded.max_age = 100 to 10 and watch good responses turn into missing and unexpected ones.

clock.c, compiled with main.sv. Only main.sv is editable on this page.
// clock.c: host wall-clock time for the measurements, built together with main.sv.
#include <time.h>
#include "svdpi.h"
#include "Vtb__Dpi.h"   // the prototype generated from the import in main.sv

double wall_us(void) {
  struct timespec t;
  clock_gettime(CLOCK_MONOTONIC, &t);
  return t.tv_sec * 1e6 + t.tv_nsec / 1e3;
}
RUNVerilator + UVM 2020.3.1 · simulator service
Edit the code, then run it

        
Expected among the UVM messages: 2000 items, no item objection: the test was notified 0 times 2000 items, objection per item: the test was notified 4000 times 2000 zero-time pairs: notified 1 times at once, 2 after 1 ns 2000 pairs with #0 after each drop: notified 4000 times at once, 4000 after 1 ns unbounded table: entries at 5, 10, 15, 20 us: 65 127 190 252 bounded table:   entries at 5, 10, 15, 20 us: 4 4 4 4 unbounded table: 1750 matched, largest 253, 250 still waiting at the end bounded table:   1750 matched, largest 5, 250 missing, first detected at 180 ns UVM_ERROR :    2

        
        
      
Read included file: clock.c
// clock.c: host wall-clock time for the measurements, built together with main.sv.
#include <time.h>
#include "svdpi.h"
#include "Vtb__Dpi.h"   // the prototype generated from the import in main.sv

double wall_us(void) {
  struct timespec t;
  clock_gettime(CLOCK_MONOTONIC, &t);
  return t.tv_sec * 1e6 + t.tv_nsec / 1e3;
}


    
Check yourself

      

        
Predict the outputThe bounded table's age limit is set to 10 ns. Each 10 ns step adds a new expectation, then delivers the response for the request three steps back, then sweeps entries older than the limit. Every eighth request is dropped, as before. What line does the $display print?
      
`timescale 1ns/1ns
`include "uvm_macros.svh"
import uvm_pkg::*;

class pkt extends uvm_sequence_item;
  `uvm_object_utils_begin(pkt)
    `uvm_field_int(id, UVM_ALL_ON)
    `uvm_field_int(addr, UVM_ALL_ON)
  `uvm_object_utils_end
  int id; bit [15:0] addr;
  function new(string name = "pkt"); super.new(name); endfunction
endclass

// Out-of-order checking: each expectation waits by id for its response (U8.2).
class pending_sb extends uvm_component;
  `uvm_component_utils(pending_sb)
  pkt  pending[int];                        // id -> expected transaction
  time since[int];                          // id -> when it started waiting
  int  matched, missing, unexpected, largest;
  time max_age = 0;                         // 0: wait forever
  time first_missing_at;                   // measured from the start of the traffic
  time started;
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  function void expect_rsp(pkt p);
    pending[p.id] = p; since[p.id] = $time;
    if (pending.size() > largest) largest = pending.size();
  endfunction
  function void response(int id);
    if (!pending.exists(id)) begin unexpected++; return; end
    pending.delete(id); since.delete(id); matched++;
  endfunction
  // With an age limit: an expectation older than max_age is a missing response.
  function void sweep();
    int old[$];
    if (max_age == 0) return;
    foreach (since[id]) if ($time - since[id] > max_age) old.push_back(id);
    foreach (old[i]) begin
      if (missing == 0) first_missing_at = $time - started;
      missing++; pending.delete(old[i]); since.delete(old[i]);
    end
  endfunction
  // End scan: whatever still waits was never answered.
  function void check_phase(uvm_phase phase);
    if (pending.size() > 0)
      `uvm_error("SB", $sformatf("%0d expectations never answered", pending.size()))
    if (missing > 0)
      `uvm_error("SB", $sformatf("%0d responses missing, the first %0t ns into the traffic", missing, first_missing_at))
    if (unexpected > 0)
      `uvm_error("SB", $sformatf("%0d responses arrived with no expectation", unexpected))
  endfunction
endclass

class perf_test extends uvm_test;
  `uvm_component_utils(perf_test)
  pending_sb loose, bounded;
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  function void build_phase(uvm_phase phase);
    loose   = pending_sb::type_id::create("loose", this);
    bounded = pending_sb::type_id::create("bounded", this);
  endfunction
  // A synthetic response schedule standing in for a design: 2000 requests with unique ids,
  // one per 10 ns step, each answered 30 ns later except the dropped ones. Each step: new
  // expectation, then the response for the request three steps back, then the bounded sweep.
  string sizes = "", bsizes = "";             // entries waiting at 5, 10, 15 and 20 us
  task traffic(pending_sb loose, pending_sb bounded);
    pkt p = pkt::type_id::create("p");
    bounded.max_age = 10;
    loose.started = $time; bounded.started = $time;
    for (int t = 0; t < 2003; t++) begin
      if (t < 2000) begin
        p = pkt::type_id::create("p"); p.id = t; p.addr = 16'(4 * t);
        loose.expect_rsp(p); bounded.expect_rsp(p);
      end
      if (t >= 3 && (t - 3) % 8 != 7) begin loose.response(t - 3); bounded.response(t - 3); end
      bounded.sweep();
      #10;
      if ((t + 1) % 500 == 0 && t < 2000) begin
        sizes  = {sizes,  $sformatf(" %0d", loose.pending.size())};
        bsizes = {bsizes, $sformatf(" %0d", bounded.pending.size())};
      end
    end
    #(bounded.max_age + 10); bounded.sweep();
  endtask

  task run_phase(uvm_phase phase);
    phase.raise_objection(this);
    traffic(loose, bounded);
    $display("bounded table: %0d matched, %0d missing, %0d unexpected", bounded.matched, bounded.missing, bounded.unexpected);
    phase.drop_objection(this);
  endtask
endclass

module tb;
  initial run_test("perf_test");
endmodule


      

      

      


Predict the outputThe generator now creates one pkt before the loop and rewrites its id and addr (4 × id) for each request instead of creating a new one. The tables store the handle they receive, keyed by the id at that moment. Request 7 is dropped. What line does the $display print?
      
`timescale 1ns/1ns
`include "uvm_macros.svh"
import uvm_pkg::*;

class pkt extends uvm_sequence_item;
  `uvm_object_utils_begin(pkt)
    `uvm_field_int(id, UVM_ALL_ON)
    `uvm_field_int(addr, UVM_ALL_ON)
  `uvm_object_utils_end
  int id; bit [15:0] addr;
  function new(string name = "pkt"); super.new(name); endfunction
endclass

// Out-of-order checking: each expectation waits by id for its response (U8.2).
class pending_sb extends uvm_component;
  `uvm_component_utils(pending_sb)
  pkt  pending[int];                        // id -> expected transaction
  time since[int];                          // id -> when it started waiting
  int  matched, missing, unexpected, largest;
  time max_age = 0;                         // 0: wait forever
  time first_missing_at;                   // measured from the start of the traffic
  time started;
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  function void expect_rsp(pkt p);
    pending[p.id] = p; since[p.id] = $time;
    if (pending.size() > largest) largest = pending.size();
  endfunction
  function void response(int id);
    if (!pending.exists(id)) begin unexpected++; return; end
    pending.delete(id); since.delete(id); matched++;
  endfunction
  // With an age limit: an expectation older than max_age is a missing response.
  function void sweep();
    int old[$];
    if (max_age == 0) return;
    foreach (since[id]) if ($time - since[id] > max_age) old.push_back(id);
    foreach (old[i]) begin
      if (missing == 0) first_missing_at = $time - started;
      missing++; pending.delete(old[i]); since.delete(old[i]);
    end
  endfunction
  // End scan: whatever still waits was never answered.
  function void check_phase(uvm_phase phase);
    if (pending.size() > 0)
      `uvm_error("SB", $sformatf("%0d expectations never answered", pending.size()))
    if (missing > 0)
      `uvm_error("SB", $sformatf("%0d responses missing, the first %0t ns into the traffic", missing, first_missing_at))
    if (unexpected > 0)
      `uvm_error("SB", $sformatf("%0d responses arrived with no expectation", unexpected))
  endfunction
endclass

class perf_test extends uvm_test;
  `uvm_component_utils(perf_test)
  pending_sb loose, bounded;
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  function void build_phase(uvm_phase phase);
    loose   = pending_sb::type_id::create("loose", this);
    bounded = pending_sb::type_id::create("bounded", this);
  endfunction
  // A synthetic response schedule standing in for a design: 2000 requests with unique ids,
  // one per 10 ns step, each answered 30 ns later except the dropped ones. Each step: new
  // expectation, then the response for the request three steps back, then the bounded sweep.
  string sizes = "", bsizes = "";             // entries waiting at 5, 10, 15 and 20 us
  task traffic(pending_sb loose, pending_sb bounded);
    pkt p = pkt::type_id::create("p");
    bounded.max_age = 100;
    loose.started = $time; bounded.started = $time;
    for (int t = 0; t < 2003; t++) begin
      if (t < 2000) begin
        p.id = t; p.addr = 16'(4 * t);                                    // one object, rewritten
        loose.expect_rsp(p); bounded.expect_rsp(p);
      end
      if (t >= 3 && (t - 3) % 8 != 7) begin loose.response(t - 3); bounded.response(t - 3); end
      bounded.sweep();
      #10;
      if ((t + 1) % 500 == 0 && t < 2000) begin
        sizes  = {sizes,  $sformatf(" %0d", loose.pending.size())};
        bsizes = {bsizes, $sformatf(" %0d", bounded.pending.size())};
      end
    end
    #(bounded.max_age + 10); bounded.sweep();
  endtask

  task run_phase(uvm_phase phase);
    phase.raise_objection(this);
    traffic(loose, bounded);
    $display("entry 7 holds id %0d, addr %0d", loose.pending[7].id, loose.pending[7].addr);
    phase.drop_objection(this);
  endtask
endclass

module tb;
  initial run_test("perf_test");
endmodule


      

      

      


Spot the bugEach option is the body of pending_sb::response(int id). Every request is answered here, so both tables should end empty. Which option leaks memory?


ConceptA driver reads its inter-item gap with uvm_config_db #(int)::get at the start of every item, so that a test can change it mid-run. Which change keeps that ability and removes the per-item search?


ConceptA regression became three times slower after a week of changes, with no failures. You have confirmed that the old and new runs use the same test, seed, simulated time, simulator build and host conditions. What do you measure next?

        

        
      

    

    
Remember
  • Measure counts per item before host time: they are stable and point at the mechanism that multiplied.
  • Hold one objection for the whole traffic; an objection per item propagates up every level and cost about six times the host time per item here.
  • Look up configuration in build and keep a handle; a get searches every entry with that name.
  • Use the message macros in hot paths, keep recording off in regressions, and reuse an object only after every holder has released or copied it.
  • Sample table sizes over time; bound a pending table by its specified legal latency (or capacity), and keep the end scan.