Reading a Counterexample
Walk a formal trace backwards from the failing step, tell a real bug from a missing reset or assumption, and run covers before asserts.
- Lesson
- F1.4
- Time
- 20 min
- Runs on
- SymbiYosys
- Check
- 4 questions
A formal counterexample showed ack high with no request pending, and the designer spent a day in the RTL looking for the bug. The trace started with the pending flop already set in step 0, before any request: the bench had no reset assumption. The real bug was found the next morning, in a different trace, after the reset was assumed. Both traces failed on the same signal; the step-0 state told them apart.
Measured: FAIL at reported step 7, 629 ms, depth 20, smtbmc yices. Reset is assumed in step 0.
Walk the Trace Backwards
Step 6: the wrong signal
ackis 1 whilepending_q(the value ofpendingone step earlier) is 0. The check asks for a pending request behind every ack. There is none.Step 5: what made ack rise
ackis registered, so look one step back.busy_cntis 3 andpending_dis 1, butpendingis already 0. The ack path readpending_d, a one-cycle-old copy.Step 4: the cause
The shaded
abortis 1 on the last busy cycle. The spec allows an abort at any time, so this input is legal. The design was right until here and an input turned it wrong: that is the cause. The fix is in the RTL,ack <= pending && busy_cnt == 3.
A counterexample reads backwards. Start at the marked step: which signal is wrong? Go one step back: which state bit or input made it so? Repeat until you reach a step where the design was right and a value turned it wrong. The shaded values are the solver's choices; if a shaded value is something the environment would never do, the fix is an assumption, not an RTL change.
Four Causes of a FAIL
Read the diagram as text
- FAIL at step n: walk back to the cause
- Step 0 legal?: could a reset produce it
- Shaded inputs legal?: would the spec allow them
- Check matches the spec?: is the property too strict
- Model the reset: assume rst in step 0
- Add an assumption: on the input, from the spec
- Fix the property: the spec row, as written
- RTL bug: legal start, legal inputs, right check
- FAIL at step n → Step 0 legal?
- Step 0 legal? → Shaded inputs legal?: yes
- Shaded inputs legal? → Check matches the spec?: yes
- Step 0 legal? → Model the reset: no
- Shaded inputs legal? → Add an assumption: no
- Check matches the spec? → Fix the property: no
- Check matches the spec? → RTL bug: yes
Only when the start state, the inputs and the check are all right is the counterexample a bug in the RTL. F3.3 turns this into a flow for protocol benches.
| Cause | The tell in the trace | On this page |
|---|---|---|
| A bug in the RTL | A legal start, legal inputs, a check that matches the spec row. | Trace A: req, then abort on the last busy cycle, then a stale ack. |
| A missing assumption | A shaded input value the environment would never produce. | Trace A again, if the spec forbade abort during busy. The diagnosis depends on the spec, not the trace. |
| A wrong property | The check is stricter than the spec. | Asking for ack exactly one cycle after busy_cnt reaches 3 when the spec allows two. |
| Reset not modelled | Step 0 holds a state no reset produces. | Trace B: ack 1 and pending_q 0 at step 0, fail reported at 1. |
Two traces that fail on the same signal can need opposite fixes. The first step and the shaded rows decide.
Measured: FAIL at reported step 1, 601 ms. The failing state is step 0 itself. F1.6 is about this.
Wiggle Before You Assert
req ##[1:5] ack, before any assertMeasured: Cover reached, reported step 7, 602 ms. The sequence looks like the spec: the environment is sane enough to assert against.
The habit for a new bench: write a cover for each behaviour you expect, run, and read the traces before you write a single assert. Short, odd-looking traces at this stage are normal; they show which flops the reset missed and which inputs still need an assumption. Here the cover shows a request, four busy cycles and an acknowledge, so the environment is fit to assert against. Only then run the assert and read its trace the same way. The $past in the property needs a value at the first tick (A1.2 Clocks and sampling); the engine's form keeps that value in pending_q, which starts at 0.
Asked in Interviews
| Question | Answer |
|---|---|
| Is the counterexample a real bug? | Walk it back. Check the start state against the reset, the shaded inputs against the spec, the check against the spec row; only then the RTL. |
| What is wiggling? | Running covers first to see the design move under the tool's inputs before asserting anything. |
| What would make you add an assumption instead of fixing RTL? | A shaded input value the real environment cannot produce. |
Try It
Concurrent SVA
assert property (@(posedge clk) disable iff (rst) ack |-> $past(pending));
cover property (@(posedge clk) disable iff (rst) req ##[1:5] ack);
Every ack had a pending request one cycle earlier; a request can reach an ack within five cycles.
As this engine runs it
reg pending_q=0;
always @(posedge clk) begin
pending_q <= pending;
if (!rst && ack) assert(pending_q);
end
reg [2:0] win=0; // cycles since req, 1..5
always @(posedge clk) begin
win <= rst ? 0 : req ? 1 : (win && win < 5) ? win + 1 : 0;
if (!rst) cover(ack && win != 0);
end
pending_q is the $past; win is the ##[1:5] window. F3.1 teaches the translation.
[options]
mode bmc
depth 20
[engines]
smtbmc yices
[script]
read_verilog -formal -sv design.sv
prep -top hs_ctrl
[files]
design.svWiggle
Set the mode to cover and run. Read the trace: a request, busy_cnt 1, 2, 3, then ack. The cover is reported at step 7.
The bug trace
Set the mode to bmc and run: FAIL at reported step 7. Walk back from step 6 to the shaded abort at step 4. RTL, assumption, property or reset?
The reset trace
Fix the RTL (
ack <= pending && busy_cnt == 2'd3), delete the two assume lines and every=0initial value, and run. FAIL at reported step 1 withackalready high at step 0. RTL, assumption, property or reset?Both green
Put the reset assumption back (or the initial values): BMC PASS to depth 20.
design.sv
props.sv
Use assert property (@(posedge clk) ...);. Leave blank for immediate assertions in the design.
Inputs are free unless constrained. Run to find a counterexample or check a proof.
Model: one positive-edge clock, binary data. Sampled history requires declaration-initialized variables; primary-input history is unsupported.
SBY summary and log
Check Yourself
Remember
- Read a counterexample backwards from the failing step to the first value that turned the design wrong.
- Four causes of a FAIL: RTL bug, missing assumption, wrong property, reset not modelled. The step-0 state and the shaded inputs decide.
- A shaded value the environment could never produce calls for an assumption, not an RTL change.
- Wiggle first: covers show the design moving under free inputs before any assert can mislead you.