← Design Verification Interviews
Interview Question · Debug
A Test That Passed with Its Errors Hidden
The Question
The interviewer asks
An error-injection test passes with zero UVM errors, as in this log. A colleague then finds a real data bug in the same run. What happened?
UVM_INFO @ 0: reporter [RNTST] Running test inject_test...
TEST PASSED
--- UVM Report catcher Summary ---
Number of demoted UVM_FATAL reports : 0
Number of demoted UVM_ERROR reports : 2
Number of demoted UVM_WARNING reports: 0
--- UVM Report Summary ---
UVM_ERROR : 0
UVM_FATAL : 0You have two and a half minutes. Speak or type, then press Get Feedback under your answer. No credits needed.
Speech-to-text runs in your browser. Chrome and Edge may send audio to their own speech service.
You can edit your answer before you send it. Feedback uses AI and checks your answer only against our written answer. No credits are used and your answer is not saved.
Reveal the Model Answer
A report catcher turned every SB error into an info message. It was meant to hide only the one fault the test injects. The real mismatch was demoted too, so the error count stayed at 0. The catcher summary gives it away: 2 demoted errors for 1 injected fault. A catcher must match the exact expected message and count what it demotes.
- Read 1 matches10 ns (illustrative)
The values agree, so
chkcallsuvm_infoatUVM_HIGH. Its setting isUVM_MEDIUM, so no report is issued: nothing is printed or counted.Follow the calls, values and waits
- call read 1: 10, 10: corrupt_read_test → chk
- data info, UVM_HIGH: chk → Verbosity check
- data info, UVM_HIGH: Verbosity check → Not issued
- Read 2 carries a fault20 ns (illustrative)
chkreportsUVM_ERROR [DATA] read 2: got 21, expected 20. Errors pass any verbosity setting, so the report reaches the catcher.Follow the calls, values and waits
- call read 2: 21, 20: corrupt_read_test → chk
- data UVM_ERROR [DATA]: chk → Verbosity check
- data UVM_ERROR [DATA]: Verbosity check → expect_data
- The catcher demotes it20 ns (illustrative)
Severity and ID match, so the catcher counts the report, makes it
UVM_INFOand putsexpected:in front of the message. It returnsTHROW, so the server prints it.Follow the calls, values and waits
- data UVM_INFO, expected: ...: expect_data → Report server
- Reads 3 and 430 ns, 40 ns (illustrative)
Read 3 matches and goes nowhere, like read 1. Read 4, got 41 and expected 40, takes the path of read 2: the catcher's count reaches 2.
Follow the calls, values and waits
- call read 4: 41, 40: corrupt_read_test → chk
- data UVM_ERROR [DATA]: chk → Verbosity check
- data UVM_ERROR [DATA]: Verbosity check → expect_data
- data UVM_INFO, expected: ...: expect_data → Report server
- The verdictreport phase (illustrative)
The test reads the counts: 4 reads checked, 2 faults injected, 2 expected errors seen, and no error or fatal on the server. It prints
TEST PASSED.
Read the diagram as text
- corrupt_read_test: flips bit 0 of reads 2 and 4
- chk: fifo_checker
- Verbosity check: chk at UVM_MEDIUM
- Not issued: not printed, not counted
- expect_data: catcher on chk only
- Report server: prints and counts
- corrupt_read_test → chk: check_read
- chk → Verbosity check: report
- Verbosity check → Not issued: UVM_HIGH
- Verbosity check → expect_data: passes
- expect_data → Report server: THROW
This catcher touches only DATA errors from chk, and the test checks that it saw exactly the injected faults. A catcher that matches more would hide a real error.
function action_e catch();
// Demote only the one fault this test injects, and count it.
if (get_severity() == UVM_ERROR && get_id() == "SB" &&
get_message() == "read 2: got ee, expected 22") begin
set_severity(UVM_INFO);
expected_seen++;
end
return THROW;
endfunction
// report_phase: fail unless expected_seen == injectedSelf-Check
Tick each key point you covered in your answer.
What Interviewers Listen For (Junior)
A good junior answer reads the summary lines, not only the pass line. It spots that the catcher demoted more errors than the test injected.
What Interviewers Listen For (Mid-Level)
A good mid-level answer narrows the catcher to the exact injected message. It fails the test when the demoted count differs from the injected count.
Cross-Questions
Where the interviewer goes next. Open a node to practise that follow-up.
What IfProve ItBreak ItCompareScale It
- A Test That Passed with Its Errors Hidden
-
CompareReport Catcher vs Severity OverrideL2 Mid-Level
Compare a report catcher with
set_report_severity_id_override().2:30You have two and a half minutes. Speak or type, then press Get Feedback under your answer. No credits needed.
Speech-to-text runs in your browser. Chrome and Edge may send audio to their own speech service.
You can edit your answer before you send it. Feedback uses AI and checks your answer only against our written answer. No credits are used and your answer is not saved.
Reveal the Model Answer
A severity override changes every message with that ID from one component, with no conditions. A catcher runs code for each message, so it can match the text, the values and a count. Use an override only for a whole check you have decided to waive. Use a catcher for one known, specific message.
Self-Check
Tick each key point you covered in your answer.
-
What IfChecking That an Injected Fault Was CaughtL2 Mid-Level
How should an error-injection test check that the bench caught the injected fault?
2:30You have two and a half minutes. Speak or type, then press Get Feedback under your answer. No credits needed.
Speech-to-text runs in your browser. Chrome and Edge may send audio to their own speech service.
You can edit your answer before you send it. Feedback uses AI and checks your answer only against our written answer. No credits are used and your answer is not saved.
Reveal the Model Answer
Count the expected errors the catcher demoted, and compare that count with the faults the test injected. Equal counts pass; anything else fails. Every other error stays an error. A test that expects an error and sees none must fail too.
Self-Check
Tick each key point you covered in your answer.
-
Scale ItWhat Makes a Pass TrustworthyL3 Senior
Scale it: what makes a regression's pass or fail decision trustworthy?
2:30You have two and a half minutes. Speak or type, then press Get Feedback under your answer. No credits needed.
Speech-to-text runs in your browser. Chrome and Edge may send audio to their own speech service.
You can edit your answer before you send it. Feedback uses AI and checks your answer only against our written answer. No credits are used and your answer is not saved.
Reveal the Model Answer
A test passes only when three things hold. It prints its own pass line, the summary shows zero errors and fatals, and its checkers report real work. Waivers live in one reviewed file, each with an owner and an end date. The regression also tracks demoted messages per test and flags any change in that count.
Self-Check
Tick each key point you covered in your answer.
What Interviewers Listen For (Senior)
A good senior answer defines a pass from several independent signs, not just one. It puts every waiver under review, with an owner.
-
Break ItA Pass with Nothing CheckedL3 Senior
How can a test pass with zero errors and still check nothing?
2:30You have two and a half minutes. Speak or type, then press Get Feedback under your answer. No credits needed.
Speech-to-text runs in your browser. Chrome and Edge may send audio to their own speech service.
You can edit your answer before you send it. Feedback uses AI and checks your answer only against our written answer. No credits are used and your answer is not saved.
Reveal the Model Answer
If the monitor is not connected, or the scoreboard never gets items, nothing is compared and nothing fails. A final check that the scoreboard compared at least the expected number of items catches this. Coverage that stays empty points the same way.
Two Runs with the Same Summary All cycles- Cycle 0
The healthy run drives 12 items, checks 4 reads and reports no errors.
- Cycle 1
The broken run drives the same 12 items. Its monitor sees nothing, so no read is checked, and it also reports no errors.
Both runs report no errors. Only the activity counts show that the second one checked nothing.
Self-Check
Tick each key point you covered in your answer.
- Cycle 0
-
Prove ItSolve U7.4.1: The Checker That Fails NothingSimulator-graded
Show it, don't just say it: this Silicon Lab problem is graded by the simulator.
What People Get Wrong
Common wrong answers and why they fail. Run It shows the wrong belief failing on a real simulator.
Misconception
"A UVM_ERROR count of 0 means no check failed."
A catcher, a severity override or a changed action can stop an error being counted. The twin demotes both scoreboard errors, and the test prints TEST PASSED.
Run It
A catcher meant for one injected fault, and a real mismatch in the same run.
UVM_ERROR count 0, scoreboard mismatches 2, injected 1 TEST PASSED FAIL: the catcher demoted every SB error, so a real mismatch was counted as a pass Number of demoted UVM_ERROR reports : 2 UVM_ERROR : 0Half-Truth
"Demote by message ID; the ID is specific enough."
One ID usually covers every failure of that check. Matching the ID alone hides real failures of the same kind. Match the expected values too, and limit the count.
Tool as Truth
"The simulator exits with an error code when a UVM test fails."
A UVM_ERROR does not change the exit status by itself. The regression must read the report summary or a pass line.
Source Measured: the mid-test reset twin in this bank prints two UVM_ERROR reports and still exits with status 0 (uvm-core 2020.3.1 on Verilator 5.052)