Random stability and callbacks

Make a failing random test reproducible, know what a seed does and does not promise, and use pre_randomize and post_randomize without breaking the solved values.

Lesson
5.6
Time
25 min
Runs on
Verilator
Check
5 questions

A nightly regression fails at burst 4,512 with seed 1234, and the rerun with the same seed passes. Between the two runs, someone added a debug object that is created before the burst generator. That one extra object changed the seed the generator received, so every burst after it changed too, and the stimulus that exposed a real bug was gone.

Same seed, one extra object created first
0123run 1: addr3c8018fcrun 1: resultokERRORrun 2: addr9408c430run 2: resultokburst 4,512

The last four bursts before the failure. With a different object order, run 2 never builds the failing burst.

Every object has its own random stream

Each process and each object has its own random number generator (RNG). An object's RNG gets its seed when the object is created, from the RNG of the process that creates it. The standard calls the result random stability.

Two things follow. First, calls on one object never disturb another object's sequence. In the example, two objects with the same seed produce 10 identical bursts, although a third object is randomized between them. Second, creating objects in a different order hands out different seeds, which is exactly what broke the rerun in the opening story.

So keep the order in which a test builds its objects stable. Add new objects after the existing ones, or give each important object its own seed, and a debug object will no longer reshuffle the stimulus.

A simple scheme is to derive each important object's seed from the run's seed plus a fixed number for that object, and set it with srandom right after new. Adding or reordering objects then never shifts anyone else's sequence.

Seeding on purpose with srandom

t.srandom(seed) sets the seed of one object's RNG. Call it once, when you build the object, and the object's sequence then depends only on that seed and on the calls made on that object.

Calling srandom with the same seed before every randomize() restarts the stream each time, so every call produces the same transaction. The example shows it: seed 5 twice gives the same burst twice.

The simulator also has a seed option for the whole run, under a different name in each tool. Print the seed, the simulator version and the command-line options at the top of every log, because a seed alone does not reproduce a run.

pre_randomize and post_randomize

Every class has two empty callback functions that randomize() calls for you, and you can override them. Here is the order in which one call runs.

  1. pre_randomize()

    Runs first. Use it to adjust settings that the constraints read, such as a limit for this call.

  2. Solve

    The solver looks for values of all active random fields that satisfy all active constraints.

  3. Success: assign, then post_randomize()

    The new values are assigned, randomize() calls post_randomize() and then returns 1.

  4. Failure: return 0

    No field changes, the standard says post_randomize() is not called, and randomize() returns 0.

Use post_randomize() to compute fields that are not random from the solved ones, such as the end address or a checksum. Never change a random field there: the solver has finished, so nothing checks the new value against the constraints.

pre_randomize() suits the opposite job. It can copy test settings that change between calls, such as the current number of free credits, into the non-random fields that the constraints read.

What a seed promises

The same seed, the same code, the same simulator version and the same options give the same stimulus. Change any one of them and the sequence may change.

A different simulator, or a new version of the same one, may produce a different sequence from the same seed, because the standard does not specify the generator. To reproduce a failure there, you need the transaction itself, so log the failing burst's fields as well as the seed.

Log the bursts just before the failure too. A bug often depends on state left by earlier traffic, such as a full buffer, and the failing burst alone may pass when it is sent on its own.

Try it

Objects a and b get the same seed, and other is randomized between them on every pass, yet all 10 pairs match. Reseeding a with 5 before each call gives the same burst both times, and post_randomize() has filled in end_addr from the solved fields.

Create other before a in the declaration and predict whether the first line changes. Then remove the two srandom calls and predict it again.

RUNVerilator · simulator service
Edit the code, then run it
Expected: same seed, 10 calls each: 10 identical seed 5: addr=78 beats=4 end=136 seed 5: addr=78 beats=4 end=136

Check yourself

Predict the outputPredict the exact output.
class item;
  rand bit [7:0] v;
  int pre_n, post_n;
  constraint c { v inside {[1:200]}; }
  function void pre_randomize();  pre_n++;  endfunction
  function void post_randomize(); post_n++; endfunction
endclass

module tb;
  item a = new;
  bit [7:0] v1;
  bit same;
  initial begin
    a.srandom(9);
    if (!a.randomize()) $fatal(1, "failed");
    v1 = a.v;
    a.srandom(9);
    if (!a.randomize()) $fatal(1, "failed");
    same = (v1 == a.v);
    if (!a.randomize()) $fatal(1, "failed");
    $display("pre=%0d post=%0d same=%0d", a.pre_n, a.post_n, same);
  end
endmodule
Spot the bugEach option is the body of post_randomize() in the burst class. end_addr must equal addr + 4 * beats, and every burst must still satisfy the class rules. Which option breaks a rule?
ConceptTwo objects are seeded with srandom(7). A third object is randomized between each pair of calls. Do the first two still produce identical sequences?
ConceptWhy does creating a new debug object at the start of the test change the bursts a generator object produces later?
ConceptWhat should a bug report contain so that someone else can rerun a failing random test?

Remember

  • Each object has its own RNG, seeded from the process that creates it, in creation order.
  • Calls on one object never disturb another; changing the creation order changes the seeds.
  • Call srandom once per object; reseeding before every call repeats the same values.
  • post_randomize() computes derived fields; never change a random field there.
  • A seed reproduces a run only with the same code, tool version and options, so log all of them.

Practice