tt_dispatch_dispatch_waiting_on_upstream
Name
Prometheus Metric Name
tt_dispatch_dispatch_waiting_on_upstream
Metric Path (tt-telemetry)
Schema:
{hostname}/tray{tray}/chip{chip}/cq{cq}/tt_dispatch_dispatch_waiting_on_upstream
Example path:
bh-glx-c09u02/tray1/chip2/cq0/tt_dispatch_dispatch_waiting_on_upstream
Description
A point-in-time flag indicating whether, at the moment of the telemetry read, the dispatch core for
this command queue (CQ) was stalled waiting on its upstream. In the fast-dispatch pipeline (host →
prefetch core → dispatch core → worker cores) the dispatch core’s upstream is the prefetch core, so
a true value means the dispatcher has consumed everything the prefetcher handed it and is waiting
for more commands.
This points one stage further down the pipeline than tt_dispatch_prefetch_waiting_on_upstream: the prefetcher is not keeping the dispatcher fed. That can be because the prefetcher is itself starved by the host, or because prefetching is the bottleneck. Because it is sampled instantaneously once per interval, pair it with tt_dispatch_dispatch_blocked_count to gauge how often the stall occurs.
A separate metric is emitted for every command queue the chip can support (cq0, cq1, …). CQs
absent from a given read, and chips whose firmware bundle is below version 19.12.0, report the
default value of 0.
This is an integer flag (only ever 0 or 1) rather than a boolean so that it does not contribute
to the overall health status.
Values
Type: Integer
Units: None
Allowable values:
1: The dispatch core was waiting on the prefetch core for more work at read time.
0: The dispatch core was not waiting (also reported when the CQ was absent from the read or dispatch telemetry was unavailable).
Prometheus Labels
Label Name |
Value |
|---|---|
hostname |
The host from which the metric was collected. |
hall |
The datacenter hall where the host is located. Sourced from the Factory System Descriptor (FSD). |
aisle |
The datacenter aisle where the host is located. Sourced from the Factory System Descriptor (FSD). |
rack |
The rack number where the host is located. Sourced from the Factory System Descriptor (FSD). |
shelf_u |
The shelf U position in the rack where the host is located. Sourced from the Factory System Descriptor (FSD). |
tray |
The tray (UBB) that the device is located on. |
chip |
The ASIC location within the tray. |
cq |
The hardware command queue index ( |