Reports that support a verdict

Severity and verbosity answer different questions

Every UVM report has a severity: info, warning, error or fatal. Severity says what kind of event it is. An info message is a progress note, and an error means a check failed. A fatal error ends the test at once.

Info messages also carry a verbosity, from UVM_NONE up through UVM_LOW, UVM_MEDIUM, UVM_HIGH, UVM_FULL and UVM_DEBUG. Each component has a verbosity setting, UVM_MEDIUM by default. An info message prints only if its verbosity is at or below that setting, so UVM_HIGH detail stays hidden unless you ask for it.

The macros issue warnings, errors and fatals at UVM_NONE, so the verbosity setting never hides them. A filtered info message is dropped before it reaches the report server, and it is not even counted. Lowering verbosity makes a log shorter; it can never turn a failing test into a passing one.

What a report does: actions

Once a report gets through, its action decides what happens. The action is a set of flags, and each severity has a default set.

UVM_COUNT is easy to misread. It feeds the quit count, which ends the test after a set number of errors. The limit is 0 by default, which means no limit. The severity count is a different counter: it grows for every report that is executed, whatever its action. Strip UVM_COUNT from errors and they still show up in the error total.

You can change reports per component and per ID. set_report_verbosity_level changes one component's filter, and the _hier versions apply it to a whole subtree. set_report_severity_id_override(UVM_ERROR, "RETRY", UVM_WARNING) turns one ID's errors into warnings. set_report_id_action changes what one ID does. Each of these tools can hide a real failure, so keep them scoped to an ID you understand.

The report server keeps the score

All reports end up in one report server, uvm_report_server::get_server(). It keeps a count per severity and a count per ID. At the end of the run it prints both, in the report summary. The summary is the first thing to read in a failing log.

Read the counts by ID first. Forty DATA errors after one FLAGS error usually means one root cause: the flags went wrong, and every later read followed. Fix the first error in time, then rerun, before reading anything after it.

A quit limit helps with that kind of flood. uvm_report_server::get_server().set_max_quit_count(10) ends the test after the tenth counted error, so a broken run stops early with a short log. While a limit is set, the summary adds a Quit count line, such as 2 of 10 for a run with two counted errors.

get_severity_count(UVM_ERROR) and get_id_count("DATA") let a test read those numbers itself. A verdict built on them belongs in report_phase, after check_phase has run the end-of-test checks, such as the scoreboard's check for values never read.

Expecting an error on purpose

An error-injection test breaks something on purpose and expects the bench to complain. If the complaint is a UVM_ERROR, the run must not fail because of it. A report catcher handles this case: a callback object that sees each report before it is executed.

A catcher extends uvm_report_catcher and overrides catch(). It reads the report with get_severity(), get_id() and get_message(). It may change it with set_severity() or set_message(). It returns THROW to pass the report on, or CAUGHT to drop it without printing or counting it.

Three choices keep a catcher honest. Match narrowly: the severity and the ID, and attach it only to the component that reports the fault. Demote rather than drop: an expected error demoted to info still prints, and the catcher summary at the end counts demoted reports. Count: the test knows how many faults it injected, and the verdict requires that exact number.

function action_e catch();
  if (get_severity() == UVM_ERROR)
    set_severity(UVM_INFO);
  return THROW;
endfunction
// installed with uvm_report_cb::add(null, c)

function action_e catch();
  if (get_severity() == UVM_ERROR && get_id() == "DATA") begin
    seen++;
    set_severity(UVM_INFO);
  end
  return THROW;
endfunction
// installed with uvm_report_cb::add(chk, c)

Where you attach the catcher matters. uvm_report_cb::add(obj, c) covers reports issued by obj alone, not by its children. A catcher added to the environment never sees the scoreboard inside it. add(null, c) covers every reporter in the bench. add_by_name("*.sb", c, root) covers every component whose full name matches the pattern.

Keep the catcher only as long as the fault lasts. When the injected window ends, remove it with uvm_report_cb::delete(obj, c). A real DATA error later in the same test then fails the run as it should.

Zero errors can mean zero checks

A summary with UVM_ERROR : 0 says that no check failed. It does not say that any check ran. A driver that never got an item, or a monitor wired to the wrong interface, also produces zero errors.

So a verdict needs positive evidence as well as an empty error count. The example prints TEST PASSED only when the checker compared at least one read, the catcher saw exactly the injected faults, and no error or fatal was counted. U16.3 extends the same idea to a full ledger of writes, reads and canceled values.

Try it

The two injected faults appear as UVM_INFO lines starting with expected:, and the catcher summary counts 2 demoted errors. The two correct reads print nothing, because their messages are UVM_HIGH and the default setting is UVM_MEDIUM.

Remove the got ^= 8'h01; line and run again: the catcher sees 0 errors, so the verdict reports VERDICT instead of passing. Then attach the catcher to this instead of chk and watch the two DATA errors reach the summary.

Check yourself

`timescale 1ns/1ns
`include "uvm_macros.svh"
import uvm_pkg::*;

class report_test extends uvm_test;
  `uvm_component_utils(report_test)
  function new(string name, uvm_component parent); super.new(name, parent); endfunction
  task run_phase(uvm_phase phase);
    uvm_report_server srv = uvm_report_server::get_server();
    int info0 = srv.get_severity_count(UVM_INFO);
    int warn0 = srv.get_severity_count(UVM_WARNING);
    int err0  = srv.get_severity_count(UVM_ERROR);
    phase.raise_objection(this);
    set_report_verbosity_level(UVM_LOW);
    `uvm_info("STEP", "low detail", UVM_LOW)
    `uvm_info("STEP", "medium detail", UVM_MEDIUM)
    set_report_severity_id_override(UVM_ERROR, "RETRY", UVM_WARNING);
    `uvm_error("RETRY", "slave asked for a retry")
    `uvm_warning("SLOW", "response took 9 cycles")
    set_report_severity_action(UVM_ERROR, UVM_DISPLAY);
    `uvm_error("DATA", "read 2: got 21, expected 20")
    $display("info %0d, warning %0d, error %0d",
             srv.get_severity_count(UVM_INFO) - info0,
             srv.get_severity_count(UVM_WARNING) - warn0,
             srv.get_severity_count(UVM_ERROR) - err0);
    phase.drop_objection(this);
  endtask
endclass
module tb;
  initial run_test("report_test");
endmodule

uvm_report_cb::add(env, catcher);

uvm_report_cb::add(null, catcher);

uvm_report_cb::add_by_name("*.sb", catcher, this);

uvm_report_cb::add(env.sb, catcher);