Random stability and callbacks
Make a failing random test reproducible, know what a seed does and does not promise, and use pre_randomize and post_randomize without breaking the solved values.
- Lesson
- 5.6
- Time
- 25 min
- Runs on
- Verilator
- Check
- 5 questions
A nightly regression fails at burst 4,512 with seed 1234, and the rerun with the same seed passes. Between the two runs, someone added a debug object that is created before the burst generator. That one extra object changed the seed the generator received, so every burst after it changed too, and the stimulus that exposed a real bug was gone.
The last four bursts before the failure. With a different object order, run 2 never builds the failing burst.
Every object has its own random stream
Each process and each object has its own random number generator (RNG). An object's RNG gets its seed when the object is created, from the RNG of the process that creates it. The standard calls the result random stability.
Two things follow. First, calls on one object never disturb another object's sequence. In the example, two objects with the same seed produce 10 identical bursts, although a third object is randomized between them. Second, creating objects in a different order hands out different seeds, which is exactly what broke the rerun in the opening story.
So keep the order in which a test builds its objects stable. Add new objects after the existing ones, or give each important object its own seed, and a debug object will no longer reshuffle the stimulus.
A simple scheme is to derive each important object's seed from the run's seed plus a fixed number for that object, and set it with srandom right after new. Adding or reordering objects then never shifts anyone else's sequence.
Seeding on purpose with srandom
t.srandom(seed) sets the seed of one object's RNG. Call it once, when you build the object, and the object's sequence then depends only on that seed and on the calls made on that object.
Calling srandom with the same seed before every randomize() restarts the stream each time, so every call produces the same transaction. The example shows it: seed 5 twice gives the same burst twice.
The simulator also has a seed option for the whole run, under a different name in each tool. Print the seed, the simulator version and the command-line options at the top of every log, because a seed alone does not reproduce a run.
pre_randomize and post_randomize
Every class has two empty callback functions that randomize() calls for you, and you can override them. Here is the order in which one call runs.
pre_randomize()
Runs first. Use it to adjust settings that the constraints read, such as a limit for this call.
Solve
The solver looks for values of all active random fields that satisfy all active constraints.
Success: assign, then post_randomize()
The new values are assigned,
randomize()callspost_randomize()and then returns 1.Failure: return 0
No field changes, the standard says
post_randomize()is not called, andrandomize()returns 0.
Use post_randomize() to compute fields that are not random from the solved ones, such as the end address or a checksum. Never change a random field there: the solver has finished, so nothing checks the new value against the constraints.
pre_randomize() suits the opposite job. It can copy test settings that change between calls, such as the current number of free credits, into the non-random fields that the constraints read.
What a seed promises
The same seed, the same code, the same simulator version and the same options give the same stimulus. Change any one of them and the sequence may change.
A different simulator, or a new version of the same one, may produce a different sequence from the same seed, because the standard does not specify the generator. To reproduce a failure there, you need the transaction itself, so log the failing burst's fields as well as the seed.
Log the bursts just before the failure too. A bug often depends on state left by earlier traffic, such as a full buffer, and the failing burst alone may pass when it is sent on its own.
Try it
Objects a and b get the same seed, and other is randomized between them on every pass, yet all 10 pairs match. Reseeding a with 5 before each call gives the same burst both times, and post_randomize() has filled in end_addr from the solved fields.
Create other before a in the declaration and predict whether the first line changes. Then remove the two srandom calls and predict it again.
same seed, 10 calls each: 10 identical seed 5: addr=78 beats=4 end=136 seed 5: addr=78 beats=4 end=136Check yourself
Remember
- Each object has its own RNG, seeded from the process that creates it, in creation order.
- Calls on one object never disturb another; changing the creation order changes the seeds.
- Call
srandomonce per object; reseeding before every call repeats the same values. post_randomize()computes derived fields; never change a random field there.- A seed reproduces a run only with the same code, tool version and options, so log all of them.
Practice
- Replay one object stream Implement · Easy
- The burst that changed after the solve Debug · Medium