Latency, throughput and registered outputs

Two numbers for speed

Latency is the number of clock edges an input passes before its result is at the output. The lab block registers its inputs and computes the result from those registers, so its latency is 1. Throughput is how many results come out per cycle when inputs arrive back to back.

So a register on the output adds one to the latency and leaves the throughput alone. The clock may rise, because the paths got shorter, and results per second rise with it, as long as the user can take a result every cycle.

Who feels the latency

Some users cannot keep a block busy. A processor that needs each answer before it can form the next question sends one input and waits. Each round trip then costs the full latency, and throughput becomes one result per latency. The second half of the bench is that user.

Look at the levels column. Registering the output barely changed the block's own count. The longest path still runs from a_q through the multiplier and the adder, now into a flop: 31 levels against 32. Cutting between the multiplier and the adder saves only 2 levels, because the multiplier is the long part. R4.5 deals with that.

Where a registered output pays off

The gain shows where one module meets the next. A combinational output path does not end at the port; it continues into the next module's logic. Join madd to a user that adds each result into a 24-bit running total, and synthesize the two together.

That is why the house rule registers every module output. Each module's levels count is then its own, and a team can meet its clock without knowing the logic on the other side. Bus specifications such as AXI go further and forbid any combinational path from an input to an output of an interface.

assign y         = a_q * b_q + 16'(c_q);
assign out_valid = v_q;

always_ff @(posedge clk or negedge rst_n)
  if (!rst_n) begin out_valid <= 1'b0; y <= '0; end
  else begin
    out_valid <= v_q;                    // valid moves with y
    y <= a_q * b_q + 16'(c_q);
  end

The lab ships madd with its output straight from the logic. Run it: results at e2 to e9, and 8 edges for the request-response loop. Register y and out_valid as in the right panel. Run again: the burst moves one edge later and the loop takes 16 edges. Synthesize: 42 flops and 31 levels.

Try it

Each line shows the input taken at that edge and the output right after it. The shipped design shows the result on the same line as its input: one edge of latency. The request-response loop sends one input per edge, because each result is there before the next edge.

With the output registered, the burst results move to e3-e10 and the loop needs 16 edges, e12-e27. Both halves still report 0 wrong. Synthesize shows 424 cells, 42 flops and 31 levels.

`timescale 1ns/1ps
module tb;
  logic clk = 0;
  always #5 clk = ~clk;                   // rising edges at 5, 15, 25, ...
  logic rst_n = 1;                        // falls at 1 ns: a real edge, so an asynchronous reset acts before any clock
  int edge_n = -1;
  always @(posedge clk) edge_n = edge_n + 1;
  logic        in_valid = 0;
  logic [7:0]  a = 0, b = 0, c = 0;
  logic        out_valid;
  logic [15:0] y;
  int want[$];                            // the model's results, oldest first
  int results = 0, wrong = 0, first = -1, last = -1, start;
  bit show = 0;
  // Offer one input before the next edge; the model computes its result now.
  task automatic offer(input int x, input int z, input int w);
    in_valid = 1; a = x; b = z; c = w;
    want.push_back(x * z + w);
  endtask
  madd dut (.clk(clk), .rst_n(rst_n), .in_valid(in_valid), .a(a), .b(b), .c(c),
            .out_valid(out_valid), .y(y));
  // Advance to the next rising edge, wait for the design to settle, then print one line.
  task automatic tick();
    @(posedge clk); #1;
    if (out_valid) begin                  // a result: check it against the oldest model value
      results++;
      if (y != want.pop_front()) wrong++;
      if (first < 0) first = edge_n;
      last = edge_n;
    end
    if (show) $display("e%0d in_valid=%b a=%0d b=%0d c=%0d out_valid=%b y=%0d", edge_n, in_valid, a, b, c, out_valid, y);
  endtask
  initial begin
    #1 rst_n = 0; tick(); tick(); rst_n = 1;   // asserted at 1 ns, edges 0 and 1 in reset, released 1 ns after edge 1
    // Burst: eight inputs on eight edges in a row, every line printed.
    show = 1;
    for (int i = 0; i < 8; i++) begin offer(i + 1, 10, i); tick(); end
    in_valid = 0;
    repeat (2) tick();
    $display("burst: 8 inputs at e2-e9, results at e%0d-e%0d, %0d wrong", first, last, wrong);
    // Request and response: the next input waits until the previous result has come out.
    show = 0; results = 0; first = -1; start = edge_n + 1;
    for (int i = 0; i < 8; i++) begin
      offer(20 + i, 3, 1); tick();
      in_valid = 0;
      while (results < i + 1) tick();
    end
    $display("request-response: 8 results in %0d edges (e%0d-e%0d), %0d wrong", last - start + 1, start, last, wrong);
    $finish(0);                             // quiet: no "$finish called" line
  end
endmodule

Check yourself

// lat2: a block with a latency of two edges (an input and an output register).
module lat2 (input logic clk, rst_n, in_valid, output logic out_valid);
  logic v1;
  always_ff @(posedge clk or negedge rst_n)
    if (!rst_n) begin v1 <= 1'b0; out_valid <= 1'b0; end
    else        begin v1 <= in_valid; out_valid <= v1; end
endmodule
module tb;
  logic clk = 0, rst_n = 1, in_valid = 0, out_valid;
  int e = 0, first = 0, last = 0, results = 0;
  lat2 dut (.clk(clk), .rst_n(rst_n), .in_valid(in_valid), .out_valid(out_valid));
  always #5 clk = ~clk;
  initial begin
    #1 rst_n = 0; #1 rst_n = 1;                 // reset pulse before the first edge
    // Request and response: one request, then wait for its result before the next.
    for (int i = 0; i < 5; i++) begin
      in_valid = 1;
      @(posedge clk); e++; #1 in_valid = 0;      // the request is taken at edge e
      if (i == 0) first = e;
      while (!out_valid) begin @(posedge clk); e++; #1; end
      results++; last = e;
    end
    $display("%0d results in %0d edges", results, last - first + 1);
    $finish(0);
  end
endmodule

logic [15:0] y_d;
always_comb y_d = a_q * b_q + 16'(c_q);
always_ff @(posedge clk or negedge rst_n)
  if (!rst_n) begin out_valid <= 1'b0; y <= '0; end
  else        begin out_valid <= v_q; y <= y_d; end

always_ff @(posedge clk or negedge rst_n)
  if (!rst_n) begin out_valid <= 1'b0; y <= '0; end
  else        begin out_valid <= v_q; y <= a_q * b_q + 16'(c_q); end

logic [15:0] y_d;
always_ff @(posedge clk or negedge rst_n)
  if (!rst_n) begin out_valid <= 1'b0; y_d <= '0; y <= '0; end
  else        begin out_valid <= v_q; y_d <= a_q * b_q + 16'(c_q); y <= y_d; end

always_ff @(posedge clk or negedge rst_n)
  if (!rst_n) out_valid <= 1'b0;
  else        out_valid <= v_q;
always_ff @(posedge clk)
  if (v_q) y <= a_q * b_q + 16'(c_q);