Merging Coverage and Ranking Tests

Where the interviewer goes next. Open a node to practise that follow-up.

WhyWhat IfCompareScale It

1. Merging Coverage and Ranking Tests
2. WhyWhy Failing Runs Don't CountL2 Mid-Level

   Why is coverage from failing runs thrown away before merging?

   Reveal the Model Answer

   A failing run shows the design did not behave as the spec says in that run. The bins it hit describe cases that ran, but were not shown to work. Counting them would mark features as verified while their bug is still open. Once the bug is fixed, rerun the test and count the new, passing coverage.

   Self-Check

   Tick each key point you covered in your answer.

   Key point 1 A failing run shows the design did not behave as the spec says in that run. Key point 2 The bins it hit describe cases that ran, but were not shown to work. Key point 3 Counting them would mark features as verified while their bug is still open. Key point 4 Once the bug is fixed, rerun the test and count the new, passing coverage.

   Half-Truth

   "Only the failing check is affected; the rest of that run's coverage is fine."

   After the first error, the design state may already be wrong, so later bins were reached from a broken state.

   Source Spear & Tumbush, SystemVerilog for Verification (3rd ed.), §9.1 (PDF p362)

   • Lesson C7Closing Holes
3. CompareA Ranked Test Set vs the Full RegressionL3 Senior

   Compare running the ranked set with running the full regression. When do you use each?

   Reveal the Model Answer

   The ranked set reaches the same coverage in a fraction of the time, so it suits every check-in and quick bug-fix checks. Its seeds are fixed, so it repeats the same stimulus every time. The full regression keeps exploring with new seeds and finds bugs the coverage model never described. Run the ranked set often and the full regression nightly or weekly.

   	Ranked set	Full regression
   Run time	A fraction of the full run	Hours
   Seeds	Fixed, so the same stimulus every time	New seeds every run
   Finds	Breaks in what was already covered	New bugs the model never described
   Run it	At every check-in	Nightly or weekly

   Self-Check

   Tick each key point you covered in your answer.

   Key point 1 The ranked set reaches the same coverage in a fraction of the time, so it suits every check-in and quick bug-fix checks. Key point 2 Its seeds are fixed, so it repeats the same stimulus every time. Key point 3 The full regression keeps exploring with new seeds and finds bugs the coverage model never described. Key point 4 Run the ranked set often and the full regression nightly or weekly.

   What Interviewers Listen For (Senior)

   A good senior answer re-ranks the tests after every change to the coverage model. It still tracks the bugs that only the full regression finds.

   Half-Truth

   "The ranked set is enough once it reaches full coverage."

   It repeats the same seeds, so it explores nothing new. Bugs outside the model's bins stay hidden.

   Source Spear & Tumbush, SystemVerilog for Verification (3rd ed.), §9.1 (PDF p361)

   • Lesson U16.2Seeds, Coverage and Directed Closure
4. What IfMerged Coverage After an RTL ChangeL3 Senior

   What if one block's RTL changes after coverage has been merged? Do you keep the old data?

   Reveal the Model Answer

   Look at the size of the change first. For a local fix, rerun the tests that touch the changed logic, and replace that block's coverage with the new runs. For a broad change, such as a new feature or changed timing, start that block's merge from scratch. Sign-off coverage must come from runs on the RTL that tapes out.

   Self-Check

   Tick each key point you covered in your answer.

   Key point 1 Look at the size of the change first. Key point 2 For a local fix, rerun the tests that touch the changed logic, and replace that block's coverage with the new runs. Key point 3 For a broad change, such as a new feature or changed timing, start that block's merge from scratch. Key point 4 Sign-off coverage must come from runs on the RTL that tapes out.

   Misconception

   "Coverage is about the tests, so it survives RTL changes."

   A bin records what the design did in a run. Change the design, and the old record no longer describes it.

   Source Spear & Tumbush, SystemVerilog for Verification (3rd ed.), §9.1 (PDF p362)

   • Lesson C7Closing Holes
5. Scale ItCoverage Across IP ReleasesL4 Staff

   An IP block is reused in three chips, each with a new release. How do you manage its coverage?

   Reveal the Model Answer

   Sign off coverage per IP release, stored with the release, on the configurations each chip uses. A new chip reuses that result for unchanged logic, and closes coverage only for new configurations and changed code. At chip level, cover the integration, not the IP's internals again. Keep a list of which configurations were never verified, so nobody assumes they were.

   Self-Check

   Tick each key point you covered in your answer.

   Key point 1 Sign off coverage per IP release, stored with the release, on the configurations each chip uses. Key point 2 A new chip reuses that result for unchanged logic, and closes coverage only for new configurations and changed code. Key point 3 At chip level, cover the integration, not the IP's internals again. Key point 4 Keep a list of which configurations were never verified, so nobody assumes they were.

   What Interviewers Listen For (Staff)

   A good staff answer ties each coverage result to one IP release and one configuration. It makes the configurations nobody verified visible to every chip team.

   Wrong Altitude

   "The IP was verified once; reuse needs no coverage work."

   A new parameter set or a new mode is a new design as far as coverage is concerned. Only the configurations that were closed are verified.

   Source Mehta, ASIC/SoC Functional Design Verification, §4.9 (PDF p87)

   • Lesson U12.2Vertical Reuse: A Block Environment Inside a Subsystem