ASIC Point Explore

← Design Verification Interviews

Interview Question · Debug

A Random Test That Fails on Some Seeds

L2 Mid-Leveluvmdebugrandomization

The interviewer asks

A random test passes on most seeds but fails on about one seed in 25. Here are two runs of the same build. How do you reproduce the failure and find its cause?

log
# run 1: +SEED=3
seed 3
UVM_INFO @ 0: reporter [RNTST] Running test smoke_test...
checked 12 results
--- UVM Report Summary ---
UVM_ERROR :    0

# run 2: +SEED=65
seed 65
UVM_INFO @ 0: reporter [RNTST] Running test smoke_test...
UVM_ERROR main.sv(163) @ 365: uvm_test_top.sb [SB] y=0, expected 255
checked 12 results
--- UVM Report Summary ---
UVM_ERROR :    1
Reveal the Model Answer

Rerun the failing seed on the same build; with the same code and seed, the run repeats exactly. The error shows y=0 where 255 was expected, and only the input 255 gives that pair. The spec says the output saturates at 255, but the DUT wrapped to 0. Uniform 8-bit stimulus picks 255 rarely, so most seeds never test it. Fix the DUT, then add weighted stimulus and a coverage bin for 255.

Random Stages, Then Six Aimed ItemsArrows show data. Items go from two sequences to the driver, pins to the monitor, and one sample per edge to both subscribers.
Random stages, then six aimed itemsArrows show data. Items go from two sequences to the driver, pins to the monitor, and one sample per edge to both subscribers. sequence random_seq 32 items, run twice sequence closure_seq 5 writes, then areset component Driver and FIFO one item per clock component Monitor one sample per edge component fifo_coverage 9 bins component fifo_scoreboard checks every read itemsitemspinswrite(s)write(s)
  1. Record the run0 ns (illustrative)

    The test prints its name and seed first: closure_test, seed 3271625496. With the same code and tools, that seed replays every random item.

  2. 32 random items

    The monitor sends a sample for every edge to both subscribers. 6 of 9 bins: level 4, write when full and reset with data are holes.

    Follow the calls, values and waits
    1. data random items: random_seq → Driver and FIFO
    2. data pins: Driver and FIFO → Monitor
    3. data samples: Monitor → fifo_coverage
    4. data samples: Monitor → fifo_scoreboard
  3. 32 more

    7 of 9: level 4 is hit. Write when full is an unlucky hole; reset with data is unreachable, because c_no_reset holds for every random item.

    Follow the calls, values and waits
    1. data random items: random_seq → Driver and FIFO
    2. data pins: Driver and FIFO → Monitor
    3. data samples: Monitor → fifo_coverage
  4. Five directed writes

    Whatever level the random stages left, the last of five writes meets a full FIFO. The design refuses it, and write when full is hit.

    Follow the calls, values and waits
    1. data write x5: closure_seq → Driver and FIFO
    2. data wr while full: Driver and FIFO → Monitor
    3. data write when full: Monitor → fifo_coverage
  5. One directed reset

    With c_no_reset off for this one item, the reset meets stored data. Coverage counts reset with data, and the scoreboard cancels what it held.

    Follow the calls, values and waits
    1. data reset: closure_seq → Driver and FIFO
    2. data rst: Driver and FIFO → Monitor
    3. data reset with data: Monitor → fifo_coverage
    4. data reset sample: Monitor → fifo_scoreboard
  6. Both questions answered

    9 of 9 bins, and the scoreboard checked the same run: 24 writes, 20 reads, 4 canceled, 0 mismatches.

Read the diagram as text
  • random_seq: 32 items, run twice
  • closure_seq: 5 writes, then a reset
  • Driver and FIFO: one item per clock
  • Monitor: one sample per edge
  • fifo_coverage: 9 bins
  • fifo_scoreboard: checks every read
  • random_seq → Driver and FIFO: items
  • closure_seq → Driver and FIFO: items
  • Driver and FIFO → Monitor: pins
  • Monitor → fifo_coverage: write(s)
  • Monitor → fifo_scoreboard: write(s)

Random stages explore and leave holes; a few aimed items close them. A rare value such as 255 needs the same treatment: a coverage bin, then stimulus that reaches it.

Self-Check

Tick each key point you covered in your answer.

What Interviewers Listen For (Mid-Level)

A good mid-level answer asks for the seed and the build first. It reruns the failing seed and reads the failing values before opening waves.

Cross-Questions

Where the interviewer goes next. Open a node to practise that follow-up.

WhyWhat IfBreak ItCompareScale It

  1. A Random Test That Fails on Some Seeds
  2. WhyWhy the Same Seed Repeats the RunL1 JuniorL2 Mid-Level

    Why does the same seed reproduce the run exactly?

    Reveal the Model Answer

    Every random value comes from generators that start from the seed. Each thread and object gets its own generator, seeded in a fixed order. With the same code, build and seed, every call returns the same value. So the failure happens again at the same time.

    Self-Check

    Tick each key point you covered in your answer.

  3. What IfA Failing Seed That Now PassesL2 Mid-LevelL3 Senior

    Asked if they say the seed does not reproduce it

    What if the failing seed passes when you rerun it?

    Reveal the Model Answer

    Then something besides the seed changed. Check the build: a newer RTL or bench commit, other compile options, or another simulator version. Check the plusargs and any files the test reads. Also suspect a race between processes, which can resolve differently after a small code change. Record the commit, seed and command line with every failure.

    Self-Check

    Tick each key point you covered in your answer.

    What Interviewers Listen For (Senior)

    A good senior answer makes saving the commit, seed and command line a job for the regression system. It does not leave it to one engineer's habit.

  4. Break ItA Race That Moves with Unrelated EditsL3 Senior

    How can a race make a failure depend on an unrelated change?

    Reveal the Model Answer

    When two processes read and write one variable in the same time step, the language allows either order. A small edit can change the order a simulator picks. The failure then moves with the edit, not with the seed. Clocking blocks and nonblocking writes remove such races.

    Self-Check

    Tick each key point you covered in your answer.

    Tool as Truth

    "Processes in one time step always run in source order."

    The standard leaves the order of processes in the active region open. One simulator's order is not a rule.

    Source IEEE 1800-2017 §4.7

  5. CompareUVM Seeding vs Thread StabilityL3 Senior

    Compare how UVM seeds its objects with plain SystemVerilog thread stability.

    Reveal the Model Answer

    In plain SystemVerilog, a new object's generator is seeded from the thread that creates it, in creation order. Creating one extra object earlier shifts every later one. UVM reseeds components and sequence items from their type name, full name and a global seed. So adding an unrelated component leaves other items' values alone, but renaming a sequence changes them.

    Creation Order Hands Out the SeedsWhere each object's seed comes from in plain SystemVerilog, in two runs. Arrows show seeds handed to new objects; the dashed pulse is a call to srandom.
    Creation order hands out the seedsWhere each object's seed comes from in plain SystemVerilog, in two runs. Arrows show seeds handed to new objects; the dashed pulse is a call to `srandom`. process Run 1: creatingprocess its generator handsout seeds in order object Burst generator created first: firstseed process Run 2: creatingprocess the same run seed object Debug object created first: firstseed object Burst generator created second:second seed call srandom(seed +1) called right after new newnewnewsets the seed
    1. Run 1

      An object's generator is seeded when the object is created, from the generator of the process that creates it. In run 1 the burst generator is created first and takes the first seed.

      Follow the calls, values and waits
      1. data first seed: Run 1: creating process → Burst generator
    2. Run 2

      In run 2 a debug object is created first and takes that seed. The burst generator gets the next one, so its whole sequence of bursts changes.

      Follow the calls, values and waits
      1. data first seed: Run 2: creating process → Debug object
      2. data second seed: Run 2: creating process → Burst generator
    3. Seed it on purpose

      Calling srandom right after new, with a seed derived from the run's seed, fixes the generator's stream. Adding or reordering other objects no longer shifts it.

      Follow the calls, values and waits
      1. call run seed + 1: srandom(seed + 1) → Burst generator
    Read the diagram as text
    • Run 1: creating process: its generator hands out seeds in order
    • Burst generator: created first: first seed
    • Run 2: creating process: the same run seed
    • Debug object: created first: first seed
    • Burst generator: created second: second seed
    • srandom(seed + 1): called right after new
    • Run 1: creating process → Burst generator: new
    • Run 2: creating process → Debug object: new
    • Run 2: creating process → Burst generator: new
    • srandom(seed + 1) → Burst generator: sets the seed

    Calls on one object never disturb another object's stream; only the order of creation hands out different seeds.

    Self-Check

    Tick each key point you covered in your answer.

    Misconception

    "Renaming a sequence is harmless; names are only for messages."

    UVM uses the full name to seed each item. A new name gives the items new random values, so an old failing seed no longer reproduces.

    Source uvm-core 2020.3.1 (IEEE 1800.2 library) source, base/uvm_object.svh (reseed uses type name and full name)

  6. Scale ItSeeds in a Regression SystemL3 SeniorL4 Staff

    Scale it: a test fails once in 500 nightly runs. How should the regression system handle seeds so that such failures are caught and kept?

    Reveal the Model Answer

    Run new random seeds every night, and keep a list of every seed that once failed. Rerun that list on every commit as a fixed guard set. Store the seed, commit and command line with each result. Rank seeds by the coverage they add, and keep the ones that add most.

    Self-Check

    Tick each key point you covered in your answer.

    What Interviewers Listen For (Staff)

    A good staff answer keeps two kinds of seeds apart: new seeds that explore, and saved seeds that guard. It ranks seeds by the coverage they add.

What People Get Wrong

Common wrong answers and why they fail.

Misconception

"Add debug code and rerun the same seed; it will fail the same way."

Code changes can shift which random values each thread or object gets. A new object or randomize call earlier in a thread changes the values after it.

Source IEEE 1800-2017 §18.14

Delivery

Calls it a flaky test and reruns until it passes.

A failure that repeats with its seed is not flaky; it is a rare input. Rerunning hides a real bug that silicon will hit far more often.

Missed Corner

"Random stimulus will hit 255 sooner or later."

Uniform 8-bit values hit one value about once in 256 draws. A boundary that matters needs weight in the constraint, such as a dist on 0 and 255. Add a coverage bin to prove it ran.

Source IEEE 1800-2017 §18.5.4

Learn It, Then Prove It