# tt_process_count

[<< Home](../index.md) | [<< Metrics](../metrics.md)

## Name

### Prometheus Metric Name

```
tt_process_count
```

### Metric Path (tt-telemetry)

Schema:

```
{hostname}/processes/tt_process_count
```

Example path:

```
bh-glx-c09u02/processes/tt_process_count
```

## Description

The number of distinct host processes *other than the telemetry collector itself* that currently hold an open handle to a Tenstorrent kernel-mode driver (KMD) device. It is computed by scanning each per-device directory under `/proc/driver/tenstorrent/`, reading the `pids` file in each, and counting the union of all PIDs. A process attached to more than one device is counted only once (duplicate PIDs are removed).

The collector opens the devices it monitors, so its own PID is removed from the union before counting. A value of `0` therefore means no process besides the telemetry collector is using the devices, rather than that the devices are entirely unused.

The individual processes behind this count are reported by command line by [tt_process](tt_process.md), one metric per PID. Both come from the same scan, performed once per collection cycle. This count can nonetheless be higher than the number of `tt_process` metrics: it counts everything the driver reports, including entries the collector cannot resolve to a specific process (see below), whereas a `tt_process` metric can only be created for one it can.

This is a host-level metric (not scoped to a tray or chip path). It is driver-independent: it reads procfs directly and does not require UMD/driver initialization, so it persists across warm resets. It is only created when device telemetry collection is enabled, and its value is refreshed on every collection cycle. If no Tenstorrent device directories are found under `/proc/driver/tenstorrent/`, a warning is logged and the value is reported as `0`. If the scan itself fails, the last known count is held rather than being reported as `0`.

### Containers and PID Namespaces

This count remains meaningful for a collector running in its own PID namespace — the default for a container — but it becomes a **lower bound** rather than an exact figure.

The kernel renders the `pids` files into the PID namespace of whichever process reads them, so a process the collector cannot see is reported as `0`. Those entries are counted: the driver is saying a process holds the device open, and reporting `0` because the collector cannot put a name to it would assert the devices are idle when they are not. But every such entry is the same `0`, so they collapse into a single one — three invisible processes and one invisible process both add exactly 1 here.

In practice this makes the value **either 0 or 1** on a containerized collector, since nothing else on the node shares its PID namespace and every device user therefore collapses into that one entry. It still distinguishes "something is using the devices" from "nothing is", but it carries no magnitude: one workload and twenty both report `1`. Treat it as a boolean under these conditions, and do not alert on its value.

Processes that *do* share the collector's PID namespace are the exception — one the collector forked, or one started with `kubectl exec` into its own pod — since those render real PIDs. They are counted individually and get a `tt_process` metric each. Processes in other pods do not: those namespaces are siblings, not ancestors.

The telltale is that no [tt_process](tt_process.md) metric accompanies the count, since an unresolvable entry cannot be named or given a metric path. To get an exact count and per-process names, give the collector the host PID namespace: `hostPID: true` (Helm: `daemonset.hostPID`, default `true`) or `--pid=host` under plain Docker. A collector running directly on the host, rather than in a container, is already in that namespace and needs no such setting. See [tt_process](tt_process.md) for the full explanation and how to confirm it.

## Values

**Type:** Unsigned Integer

**Units:** None

**Allowable values:** Non-negative integer representing the number of distinct processes, excluding the telemetry collector, holding a handle to any Tenstorrent device (for example `0` when only the collector has the devices open, or `3` when three other distinct processes hold handles).

## Related Metrics

|Metric|Description|
|---|---|
|[tt_process](tt_process.md)|Command line of an individual host process holding a handle to a Tenstorrent device. One per PID counted here that the collector can resolve to a specific process.|
|[tt_chip_count](tt_chip_count.md)|Number of Tenstorrent chips discovered on the host via PCI device enumeration.|
|[tt_driver_initialized](tt_driver_initialized.md)|Whether the device driver is initialized. `tt_process_count` is assessed independently of driver initialization.|

## Prometheus Labels

|Label Name|Value|
|---|---|
|hostname|The host from which the metric was collected.|