What coverage measures

Passing and exercising are different claims

A passing test says that whatever happened was correct. It says nothing about what did not happen. A scoreboard can only find bugs in traffic that reaches the design, so a regression is only as good as the situations its stimulus produces.

Functional coverage measures those situations directly. You list the cases the design must be tested in, such as every packet kind at every length, and the testbench counts how often each one occurred. A case that never occurred is a hole: a part of the design that the regression has not tested, however green it looks.

The gap is easy to miss because a passing regression produces nothing worth reading. Failures demand attention, but a test that never sends a long write prints nothing at all, which looks exactly like success.

Code coverage and functional coverage

Simulators can also measure code coverage: which lines, branches and toggles of the design ran. It needs no extra work and finds dead code, but it only knows the code that exists. It cannot tell you that a combination of inputs never arrived, or that a feature the specification requires was never written at all.

In the opening story, code coverage could well have been 100%. The write path and the long-packet path each ran; only their combination, a long write, never did. Functional coverage names that combination as a case, so its absence shows up as a hole.

A coverage model comes from the specification

The list of cases is called a coverage model. Build it from the specification and the test plan, never from the design's code. The reason is the same as for a scoreboard's reference model: a model copied from the code would only ask about cases the code already handles.

For the packet link, the specification says there are three kinds of packet, lengths from 1 to 8 beats, and CONFIG packets of exactly one beat. Lengths fall into three classes that matter to the design: a single beat, a short burst of 2 to 4, and a long burst of 5 to 8. The model asks for every kind in every legal length class.

Review the model with the people who wrote the specification. They know which combinations matter, such as long writes, and which can never happen, such as long CONFIG packets.

Writing the model down before running anything also catches gaps in the specification. If nobody can say whether a CONFIG packet may be long, the model forces that question early.

Counting by hand, then with covergroups

The example counts the model's cells by hand, with an array, for 200 random packets. Every packet passes, and the table shows a zero in the WRITE row under 5 to 8 beats: the hole from the opening story.

A hole is information, not a failure. Here it points straight at the leftover constraint, the very bug the regression had been hiding for a quarter.

Counting by hand works for one table, but it grows quickly and every testbench would invent its own format. SystemVerilog has a language feature for this job, the covergroup, which counts, groups values into bins and reports a percentage. Lesson C2 replaces the array with one.

A coverage number is only meaningful next to the model it measures. A model that leaves out long writes can reach 100% and say nothing about them, so review the model itself as carefully as the tests. The rest of this track builds such models with the language's own tools, and ends by testing a model itself.

Try it

All 200 packets pass, and the table makes the hole obvious: no WRITE packet ever had 5 to 8 beats. The inline constraint left from bring-up removed them, and no check could notice, because nothing was wrong with the packets that were sent.

Delete the with { ... } part of the randomize() call and predict which cells change. The counts per cell depend on the simulator and seed; whether a cell is zero does not.

Check yourself

typedef enum bit [1:0] {READ, WRITE, CONFIG} kind_e;

// One packet on the link. CONFIG packets carry a single beat.
class packet;
  rand kind_e    kind;
  rand bit [3:0] len;              // payload beats, 1 to 8
  constraint legal {
    kind inside {READ, WRITE, CONFIG};
    len inside {[1:8]};
    kind == CONFIG -> len == 1;
  }
endclass

module tb;
  bit took [3];
  bit seen [3][3];
  function automatic void handle(kind_e k, bit [3:0] len);
    if (k == CONFIG) took[0] = 1;
    else if (len > 4) took[1] = 1;
    else took[2] = 1;
    seen[k][len == 1 ? 0 : len <= 4 ? 1 : 2] = 1;
  endfunction
  initial begin
    int branches, cells;
    handle(READ, 6);
    handle(WRITE, 2);
    handle(CONFIG, 1);
    foreach (took[i]) branches += took[i];
    foreach (seen[k, c]) cells += seen[k][c];
    $display("branches %0d of 3, model cells %0d of 7", branches, cells);
  end
endmodule

int n = 0;
foreach (seen[k, c])
  if (!(k == CONFIG && c != 0) && seen[k][c] > 0) n++;
return n == 7;

foreach (seen[k, c])
  if (!(k == CONFIG && c != 0) && seen[k][c] == 0) return 0;
return 1;

return seen[READ][0] && seen[READ][1] && seen[READ][2]
    && seen[WRITE][0] && seen[WRITE][1] && seen[WRITE][2]
    && seen[CONFIG][0];

foreach (seen[k, c])
  if (seen[k][c] == 0) return 0;
return 1;