Deciding That Verification Is Done

Where the interviewer goes next. Open a node to practise that follow-up.

What IfProve ItBreak ItCompareScale It

1. Deciding That Verification Is Done
2. Break ItHow You Know a Checker Can FailL2 Mid-LevelL3 Senior

   How could your green regression be hiding a broken checker, and how do you rule that out?

   Reveal the Model Answer

   A checker can be disconnected, demoted to a warning, or even compare a value with itself. Then every test passes. Prove each checker by injecting a known bug, such as a wrong data bit or a dropped response. Then confirm that the right check fails. Keep a few such runs in the regression.

   1. Fault 0: the correct design

      With fault at 0, the seeded stimulus runs through the correct FIFO. The scoreboard finds 0 errors, so the result is clean.

      Follow the calls, values and waits

      1. data fault = 0: Fault loop → fifo
      2. data seed 7 operations: fifo_generator → fifo
      3. data fifo_obs: fifo → fifo_monitor
      4. data fifo_obs: fifo_monitor → fifo_scoreboard
      5. data clean (0 errors): fifo_scoreboard → Result line
   2. Measure the stimulus once

      Only in this run, each observation also goes to fifo_coverage. Coverage measures the stimulus once, not eight times, and reaches 100% for both covergroups.

      Follow the calls, values and waits

      1. data fifo_obs: fifo_monitor → fifo_coverage
   3. Fault 1: one flipped bit

      The loop sets fault to 1, builds a fresh scoreboard and replays seed 7. Read data comes back with bit 0 flipped, and the scoreboard reports 25 errors.

      Follow the calls, values and waits

      1. data fault = 1: Fault loop → fifo
      2. data seed 7 operations: Fault loop → fifo_generator
      3. data seed 7 operations: fifo_generator → fifo
      4. data fifo_obs: fifo → fifo_monitor
      5. data fifo_obs: fifo_monitor → fifo_scoreboard
      6. data caught (25 errors): fifo_scoreboard → Result line
   4. Fault 5: a rarer situation

      Fault 5 shows only when a read and a write meet with items stored. The same stimulus produces that often enough for 8 errors.

      Follow the calls, values and waits

      1. data fault = 5: Fault loop → fifo
      2. data caught (8 errors): fifo_scoreboard → Result line
   5. What would count as failure

      A broken design with 0 errors would print MISSED, and an error on fault 0 would print FALSE ALARM. In this run every fault is caught.

   Read the diagram as text

   • Fault loop: sets dut.fault, builds a new scoreboard
   • fifo_generator: seed 7: the same operations each run
   • fifo: fault 0, 1, 2, 3, 5, 6, 8 or 9
   • fifo_monitor: one fifo_obs per edge
   • fifo_coverage: sampled only while fault is 0
   • fifo_scoreboard: counts errors
   • Result line: clean, caught, MISSED or FALSE ALARM

   • Fault loop → fifo: fault = k
   • Fault loop → fifo_generator: new(req, 60, 7)
   • fifo_generator → fifo: operations
   • fifo → fifo_monitor: pins
   • fifo_monitor → fifo_coverage: fifo_obs
   • fifo_monitor → fifo_scoreboard: fifo_obs
   • fifo_scoreboard → Result line: errors

   Only the design changes between runs. The stimulus, the monitor and the scoreboard stay the same, so every difference in the result comes from the fault.

   Self-Check

   Tick each key point you covered in your answer.

   Key point 1 A checker can be disconnected, demoted to a warning, or even compare a value with itself. Key point 2 Then every test passes. Key point 3 Prove each checker by injecting a known bug, such as a wrong data bit or a dropped response. Key point 4 Then confirm that the right check fails. Key point 5 Keep a few such runs in the regression.

   Misconception

   "Our assertions never fire, so the design is clean."

   If a test fails and no assertion fires, the assertions are not on the failing path. Silence is not proof.

   Source Mehta, ASIC/SoC Functional Design Verification, §2.2 item 8 (PDF p36)

   • Lesson U7.4Checks in Interfaces: SVA Inside a UVM Bench
   • Lesson U16.1Reports That Support a Verdict
   • Problem U7.4.1The Checker That Fails Nothing
3. Prove ItSolve U7.4.1: The Checker That Fails NothingSimulator-graded

   Show it, don't just say it: this Silicon Lab problem is graded by the simulator.

   Open Problem U7.4.1

4. What IfA Planted Bug Caught by the Wrong CheckL2 Mid-LevelL3 Senior

   What if the planted bug makes the run fail, but not through the check you meant to prove?

   Reveal the Model Answer

   Then the planned check is still unproven, even though the run fails. Find out why it stayed quiet: it may be disabled, wired to the wrong signal, or compare too late. Make the planted-bug run match the expected message, not just "some error".

   Self-Check

   Tick each key point you covered in your answer.

   Key point 1 Then the planned check is still unproven, even though the run fails. Key point 2 Find out why it stayed quiet: it may be disabled, wired to the wrong signal, or compare too late. Key point 3 Make the planted-bug run match the expected message, not just "some error".

   Half-Truth

   "The planted-bug run failed, so the checker works."

   Another check may have caught it. Only the expected message shows that the checker you are proving fired.

   • Lesson U7.4Checks in Interfaces: SVA Inside a UVM Bench
   • Lesson U16.1Reports That Support a Verdict
   • Problem U7.4.1The Checker That Fails Nothing
5. CompareBug Rate vs Coverage as a Sign-Off SignalL3 Senior

   Compare the bug-discovery trend with coverage as a sign-off signal. Which do you trust more?

   Reveal the Model Answer

   Neither alone. Coverage says what the tests exercised, as far as the model goes. The bug trend says whether new tests still find problems. A flat bug curve with low coverage means the tests repeat themselves. High coverage with bugs still arriving means the model or the checkers miss something.

   Self-Check

   Tick each key point you covered in your answer.

   Key point 1 Neither alone. Key point 2 Coverage says what the tests exercised, as far as the model goes. Key point 3 The bug trend says whether new tests still find problems. Key point 4 A flat bug curve with low coverage means the tests repeat themselves. Key point 5 High coverage with bugs still arriving means the model or the checkers miss something.

   Half-Truth

   "No new bugs this month, so we are done."

   A flat curve can mean the tests stopped reaching new logic. Coverage tells you which.

   Source Mehta, ASIC/SoC Functional Design Verification, §7.3.1 (PDF p158)

   • Lesson U16.2Seeds, Coverage and Directed Closure
   • Lesson C7Closing Holes
6. Scale ItSign-Off for a Derivative ChipL4 Staff

   A derivative chip changes two blocks and keeps twenty. How does your sign-off change?

   Reveal the Model Answer

   Spend the effort where risk changed. The two changed blocks get full block-level closure. Unchanged blocks re-run their existing regressions on the new RTL to show nothing moved. Integration gets the new work: connectivity, address maps and the paths that touch the changed blocks. Write down what you chose not to re-verify, so the risk is visible.

   Self-Check

   Tick each key point you covered in your answer.

   Key point 1 Spend the effort where risk changed. Key point 2 The two changed blocks get full block-level closure. Key point 3 Unchanged blocks re-run their existing regressions on the new RTL to show nothing moved. Key point 4 Integration gets the new work: connectivity, address maps and the paths that touch the changed blocks. Key point 5 Write down what you chose not to re-verify, so the risk is visible.

   Missed Corner

   "Unchanged blocks need nothing."

   Their neighbours, parameters or clocks may have changed. A re-run of the existing regression is cheap proof.

   Source Mehta, ASIC/SoC Functional Design Verification, §2.2 item 1: block verification is imported into subsystem verification (PDF p34)

   • Lesson U12.2Vertical Reuse: A Block Environment Inside a Subsystem
   • Lesson U15.3Performance and Scale
7. What IfTape-Out While Coverage Is Still OpenL4 Staff

   What if tape-out is in a week and 3% of functional coverage is still open?

   Reveal the Model Answer

   Rank the holes by risk, not by count. Close the ones on paths that real software uses, if a directed test can reach them in time. Review the rest with design: some are unreachable and get waived, others become known risks with a plan for post-silicon tests. Record every decision, because someone will ask about it after silicon returns.

   Self-Check

   Tick each key point you covered in your answer.

   Key point 1 Rank the holes by risk, not by count. Key point 2 Close the ones on paths that real software uses, if a directed test can reach them in time. Key point 3 Review the rest with design: some are unreachable and get waived, others become known risks with a plan for post-silicon tests. Key point 4 Record every decision, because someone will ask about it after silicon returns.

   Wrong Altitude

   "Waive whatever is left; 97% is good enough."

   The number says nothing about which 3% is open. One hole can be the error path that every customer hits.

   Source Mehta, ASIC/SoC Functional Design Verification, §7.3: measure coverage against the functional plan (PDF p155-156)

   • Lesson C7Closing Holes
   • Lesson U16.2Seeds, Coverage and Directed Closure
8. Prove ItSolve U16.4.1: The Interrupt That Clears ItselfSimulator-graded

   Show it, don't just say it: this Silicon Lab problem is graded by the simulator.

   Open Problem U16.4.1