Ordering, tags and out-of-order completion
Explain why responses come back in a different order from the requests, what a tag is for, which order a bus still promises, and what a scoreboard must key on.
- Lesson
- P1.7
- Time
- 30 min
- Runs on
- Your browser
- Check
- 5 questions
A chip had a fast on-die SRAM and a slow path to external memory, both behind one bus. The scoreboard kept the expected read data in one queue, in request order, and compared each response with the head of that queue. On the first test that mixed the two paths it reported hundreds of data errors. Every one of them was the scoreboard's own fault: the bus had returned the SRAM reads first, exactly as its rules allow.
A pill going right is a request, going left a response; the slots are requests still inside a target.
- Cycle 0
Tag 1 goes to the slow target. It will sit there for four cycles.
- Cycle 1
Tag 2 goes to the fast target, one cycle later.
- Cycle 3
Tag 2 is answered first. Different tags, different targets: this reorder is allowed.
- Cycle 4
Tag 1 comes back. A scoreboard that expected it first was wrong; the bus was right.
- Cycle 5
Tag 7 was sent to the slow target in cycle 2 and to the fast one in cycle 3. The fast copy answers first: the same tag has overtaken itself.
- Cycle 6
The second tag 7 arrives. The initiator cannot tell which answer belongs to which request. This reorder is forbidden.
Two reorders, one legal and one illegal. The tag is what makes the difference visible.
Why responses change order
Targets differ in speed. In this lesson the fast target answers two cycles after it accepts a request and the slow one takes four. If the bus insisted on answering in request order, a fast read behind a slow one would wait two extra cycles for nothing. Every initiator queued behind it would wait too.
So most buses let a response return as soon as its target has it. The initiator must then recognise which request each response answers, because position in the stream no longer says so. That is the job of the tag.
A tag names the request
A tag is a small number the initiator attaches to a request. The target, or the interconnect on its behalf, copies the tag onto the response. The initiator matches the two by tag and never by arrival order. AXI calls it an ID, PCIe calls it a tag in the packet header, and TileLink calls it a source.
A tag is a name for a stream of requests, not for one request. An initiator that must keep ten reads in flight needs enough tags to tell them apart. Otherwise it reuses a tag and relies on the rule below to keep that tag's answers in order.
| Bus | What the tag is called | Carried on |
|---|---|---|
| AXI | ID, one per channel pair | the address, data and response channels |
| PCIe | Tag, with the requester ID | the request and completion packets |
| TileLink | source on A, sink on D | every message of a transaction |
| Wishbone, APB | none: one request at a time | nothing; order is position |
Buses without tags allow one outstanding transaction, so order and position are the same thing.
What must stay in order
The rule has two halves. Responses that carry different tags may return in any order. Responses that carry the same tag must return in the order of their requests. A target that answers in order keeps the second half by itself; the interconnect must keep it when one tag is sent to two targets of different speed.
In the figure, tag 7 went to the slow target and then to the fast one, and the fast answer came back first. An interconnect that allows this has a bug: it must hold the second request until the first has been answered, as the bench does when its SAME_TAG_IN_ORDER parameter is 1. Holding costs cycles, which is why an initiator that wants speed spreads its requests over several tags.
Allowed
Tags 1 and 2 return as 2, 1. Different tags, so any order is correct.
Forbidden
Two requests with tag 7 return in the wrong order. The initiator would pair the data with the wrong request.
Who fixes it
The interconnect, by holding a request whose tag is still outstanding at another target. The initiator may also wait for the first response itself.
What a scoreboard keys on
A scoreboard for a reordering bus keys its expectations by tag, with one small queue per tag in request order. A response is compared with the head of its own tag's queue. One global queue is wrong on every legal reorder, and the opening story shows how loudly.
One queue for everything
// at a response
exp = order_q[head++];
if (rsp.data !== exp.data) error();
Complains on the legal reorder of tags 1 and 2. The checker, not the design, is wrong.
One queue per tag
// at a response
exp = exp_q[rsp.tag][head[rsp.tag]++];
if (rsp.data !== exp) error();
Silent on the legal reorder. Complains only when tag 7 overtakes itself, which is the bug.
The per-tag queue relies on the ordering rule: because a tag's answers come back in request order, the head of its queue is always the next expectation. Lesson U8.2 builds the same idea into a UVM scoreboard.
A read after a write
Reordering also reaches across request types. If an initiator writes a location and then reads it through a different path, the read may overtake the write and return stale data. The initiator must wait for the write's response before issuing the read, or send both on the same tag through the same target, where order is kept.
Asked in interviews: ordering and tags
When is out-of-order completion allowed?
Between responses with different tags. It lets a fast target answer without waiting behind a slow one.
What must stay in order?
Responses with the same tag, in request order. The interconnect holds a request whose tag is still outstanding at another target.
What is a tag for?
To match a response with its request when arrival order no longer says so. It names a stream, and reusing it promises in-order answers within that stream.
What does a scoreboard key on?
On the tag, with one queue per tag in request order. One queue for everything reports legal reorders as errors.
How do you keep a read after a write safe?
Wait for the write's response before issuing the read, or send both on one tag through one target. A read through another path may overtake the write.
Try it
Two scoreboards watch the same responses. The one that expects request order complains about tags 1 and 2, which is its own mistake. The one keyed by tag complains only about tag 7, which the interconnect answered out of order.
Now set SAME_TAG_IN_ORDER(1) in the instantiation and run again. The second tag 7 request is held until edge 14, its answer arrives at edge 16, and the tag-keyed scoreboard is silent. Then give the fourth request tag 8 instead and watch both runs.
edge 3: request tag 1 to the slow target edge 4: request tag 2 to the fast target edge 6: response tag 2 data 11 scoreboard in order: expected tag 1, got tag 2 edge 7: response tag 1 data 81 scoreboard in order: expected tag 2, got tag 1 edge 9: request tag 7 to the slow target edge 10: request tag 7 to the fast target edge 12: response tag 7 data 21 scoreboard by tag: tag 7 expected 91, got 21 edge 13: response tag 7 data 91 scoreboard by tag: tag 7 expected 21, got 91 4 responses, 2 mismatches by tag, 2 complaints in request orderRead included file: two_targets.sv
// two_targets.sv: a tagged request/response link with two targets (read-only in this lesson).
// A request carries a tag and an address. Addresses below 128 go to the fast target, which answers
// two edges after accepting; addresses from 128 go to the slow target, which answers four edges
// after accepting. Every response carries its request's tag and data = address + 1. The two
// response streams are merged into one: when both targets have a response waiting, the slow one
// leaves first and the fast one waits a cycle. With SAME_TAG_IN_ORDER = 1 the interconnect holds
// a request whose tag is still outstanding at the other target (req_ready stays 0), so responses
// with one tag always return in request order.
module two_targets #(parameter bit SAME_TAG_IN_ORDER = 0) (
input logic clk,
input logic rst_n,
input logic req_valid,
input logic [3:0] req_tag,
input logic [7:0] req_addr,
output logic req_ready,
output logic rsp_valid,
output logic [3:0] rsp_tag,
output logic [7:0] rsp_data
);
wire to_slow = req_addr[7];
wire [3:0] tag_i = req_valid ? req_tag : 4'd0; // a safe index while nothing is offered
// Outstanding requests per tag (at most 15), and which target holds them.
logic [3:0] outstanding [16];
logic dest [16];
wire blocked = SAME_TAG_IN_ORDER && req_valid && outstanding[tag_i] != 4'd0 && dest[tag_i] != to_slow;
assign req_ready = !blocked;
wire accept = req_valid && req_ready;
// The fast target: one pipeline stage. The slow target: three.
logic fv; logic [3:0] ft; logic [7:0] fd;
logic sv [3]; logic [3:0] st [3]; logic [7:0] sd [3];
// Each target's responses wait in a small queue; the merged output shows the slow queue's head
// when it has one, else the fast queue's head, and the initiator takes a response every edge.
logic [11:0] fq [4], sq [4];
int fw = 0, fr = 0, fn = 0, sw = 0, sr = 0, sn = 0;
wire take_slow = sn > 0, take_fast = !take_slow && fn > 0;
assign rsp_valid = take_slow || take_fast;
assign {rsp_tag, rsp_data} = take_slow ? sq[sr] : take_fast ? fq[fr] : 12'bx;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
fv <= 0; sv[0] <= 0; sv[1] <= 0; sv[2] <= 0;
fw <= 0; fr <= 0; fn <= 0; sw <= 0; sr <= 0; sn <= 0;
for (int i = 0; i < 16; i++) begin outstanding[i] <= 0; dest[i] <= 0; end
end else begin
// Accept a request into one of the pipelines.
fv <= accept && !to_slow; ft <= req_tag; fd <= req_addr + 8'd1;
sv[0] <= accept && to_slow; st[0] <= req_tag; sd[0] <= req_addr + 8'd1;
sv[1] <= sv[0]; st[1] <= st[0]; sd[1] <= sd[0];
sv[2] <= sv[1]; st[2] <= st[1]; sd[2] <= sd[1];
if (accept) begin outstanding[tag_i] <= outstanding[tag_i] + 4'd1; dest[tag_i] <= to_slow; end
// Responses leaving the pipelines wait in their target's queue.
if (fv) begin fq[fw] <= {ft, fd}; fw <= (fw + 1) % 4; end
if (sv[2]) begin sq[sw] <= {st[2], sd[2]}; sw <= (sw + 1) % 4; end
// The response at the output is taken at this edge.
if (take_slow) sr <= (sr + 1) % 4;
else if (take_fast) fr <= (fr + 1) % 4;
fn <= fn + (fv ? 1 : 0) - (take_fast ? 1 : 0);
sn <= sn + (sv[2] ? 1 : 0) - (take_slow ? 1 : 0);
if (rsp_valid) outstanding[rsp_tag] <= outstanding[rsp_tag] - 4'd1 + ((accept && tag_i == rsp_tag) ? 4'd1 : 4'd0);
end
end
endmodule
Read included file: tag_tasks.svh
// tag_tasks.svh: the initiator's tasks, the response monitor and two scoreboards (read-only).
// `cycle` counts rising edges from 0. A request is accepted at an edge where req_valid and
// req_ready are both 1; a response arrives at an edge where rsp_valid is 1.
int cycle = -1;
always @(posedge clk) cycle = cycle + 1;
// Scoreboard A keys by tag: one small queue of expected data per tag, in request order.
logic [7:0] exp_q [16][8];
int exp_w [16], exp_r [16];
// Scoreboard B assumes responses return in request order: one queue for everything.
logic [3:0] order_q [32];
int order_w = 0, order_r = 0;
int responses = 0, tag_mismatches = 0, order_complaints = 0;
bit quiet = 0; // 1: print the summary only
initial for (int t = 0; t < 16; t++) begin exp_w[t] = 0; exp_r[t] = 0; end
// Issue one request and hold it until the interconnect accepts it.
task automatic request(input logic [3:0] tag, input logic [7:0] addr);
req_valid = 1; req_tag = tag; req_addr = addr;
exp_q[tag][exp_w[tag]] = addr + 8'd1; exp_w[tag] = (exp_w[tag] + 1) % 8;
order_q[order_w] = tag; order_w = (order_w + 1) % 32;
do @(posedge clk); while (!req_ready);
#1 if (!quiet) $display("edge %0d: request tag %0d to the %s target", cycle, tag, addr[7] ? "slow" : "fast");
req_valid = 0; req_tag = 'x; req_addr = 'x;
endtask
// Wait n edges.
task automatic idle(input int n);
repeat (n) @(posedge clk);
#1;
endtask
// Every response is printed and checked by both scoreboards.
always @(posedge clk) begin
logic v; logic [3:0] t; logic [7:0] d;
v = rsp_valid; t = rsp_tag; d = rsp_data;
#1 if (v) begin
responses++;
if (!quiet) $display("edge %0d: response tag %0d data %02h", cycle, t, d);
if (d !== exp_q[t][exp_r[t]]) begin
tag_mismatches++;
if (!quiet) $display(" scoreboard by tag: tag %0d expected %02h, got %02h", t, exp_q[t][exp_r[t]], d);
end
exp_r[t] = (exp_r[t] + 1) % 8;
if (t !== order_q[order_r]) begin
order_complaints++;
if (!quiet) $display(" scoreboard in order: expected tag %0d, got tag %0d", order_q[order_r], t);
end
order_r = (order_r + 1) % 32;
end
end
final $display("%0d responses, %0d mismatches by tag, %0d complaints in request order", responses, tag_mismatches, order_complaints);
Check yourself
Remember
- Responses may return in a different order from their requests, because targets differ in speed and nobody should wait behind a slow one.
- A tag travels with the request and comes back on the response; the initiator matches by tag, never by position.
- Responses with different tags may return in any order. Responses with the same tag return in request order, and the interconnect must enforce that across targets.
- A scoreboard keys by tag with one queue per tag. One global queue reports legal reorders as errors.
- A read after a write on different tags or paths may overtake the write; wait for the write response first.
Practice
- A scoreboard keyed by tag Implement · Medium