# CLI Reference `tt-fabric-manager-cli` is a thin client over the controller's [`FabricManagerService`](api-overview.md). ``` tt-fabric-manager-cli [OPTIONS] [COMMAND_ARGS] ``` ## Running CLI in Docker Currently the most convenient way to obtain a pre-built FM CLI is via the official FM image. For example: ```bash alias tt-fabric-manager-cli='docker run --rm --net=host \ -e FABRIC_MANAGER_ENDPOINT=ttfm.example.com:80 \ -v $(pwd):$(pwd) \ -w $(pwd) \ ghcr.io/tenstorrent/tt-fabric-manager:latest-cli tt-fabric-manager-cli' ``` ## Connecting to a controller The endpoint is resolved from the following sources, highest priority first: 1. `--endpoint ` 2. `FABRIC_MANAGER_ENDPOINT` environment variable 3. `/etc/fabric-manager/config.yaml` (`controller.listen_address` or `controller.endpoint`) 4. `~/.fabric-manager/config` 5. `/etc/fabric-manager/client.conf` 6. Built-in default, `localhost:50051` Config files are YAML. A lightweight client-only file needs just two keys: ```yaml endpoint: localhost:50052 timeout_seconds: 30 ``` The full server config is also accepted, in which case the endpoint is read from `controller.endpoint` or `controller.listen_address` and the timeout from `controller.timeout_seconds`. ## Global options | Option | Default | Description | |--------|---------|-------------| | `--endpoint ` | `localhost:50051` | Controller address | | `--timeout ` | `60` | gRPC deadline applied to each individual RPC | | `--verbose`, `-v` | off | Diagnostics to stderr, prefixed `[tt-fabric-manager-cli]` | | `--help`, `-h` | | Print usage and exit | `--timeout` used to default to 600 seconds because a placement on a large fabric can run for minutes and the CLI waited for it in a single call. It no longer does: placement runs as an operation on the controller and the CLI polls it, so every RPC returns promptly and the deadline only has to cover the heaviest single query. Use `--placement-budget` to bound how long the CLI waits for a placement, and `--sync` if you need the old single-call behaviour back. Several options are shared by more than one command: `--host-id ` (repeatable, filters by host), `--output ` / `-o` (write payload to a file instead of inlining it in the JSON), and `--source ` (see [topology sources](api-overview.md#topology-sources-psd-and-fsd)). ## Commands ### `query-physical-topology` Fetches the aggregated `PhysicalSystemDescriptor` via `QueryPhysicalTopology`. | Option | Description | |--------|-------------| | `--host-id ` | Restrict to these hosts (repeatable) | | `--source ` | Connectivity source. Default `psd` | | `--output ` | Write the descriptor as textproto to `path` instead of embedding it in the JSON | ```bash # Whole fabric, as JSON tt-fabric-manager-cli query-physical-topology # Two hosts, golden-state wiring, saved as textproto tt-fabric-manager-cli query-physical-topology \ --host-id host-a --host-id host-b --source fsd -o topology.textproto ``` Exits non-zero if the query fails. The `trim_links_outside_filter` request field is not exposed by the CLI; use the [HTTP API](web-ui.md) or the SDK for that. ### `query-factory-descriptor` Fetches the controller's configured FSD files via `QueryFactorySystemDescriptor`. | Option | Description | |--------|-------------| | `--host-id ` | Filter the FSD to these hostnames (repeatable) | | `--output ` | Write the serialized FSD to a file. With multiple files, they are suffixed `path.0`, `path.1`, … | Requires the controller to be configured with `controller.factory_system_descriptor_search_path`. ### `get-valid-placements-mgd` Maps a Mesh Graph Descriptor onto the fabric, and can emit launcher configuration for a chosen placement. By default this starts a [placement operation](api-overview.md#placement-operations) on the controller and polls it to completion, printing the same JSON as before plus the `operation_id` and `operation_state` it ran as. Ctrl-C cancels the operation on the controller rather than merely walking away from it, and exits 130. Progress goes to stderr when it is a terminal, or with `--verbose`. | Option | Default | Description | |--------|---------|-------------| | `--mgd-file ` | **required** | MGD in textproto form | | `--host-id ` | | Restrict placement to these hosts (repeatable) | | `--source ` | `psd` | Connectivity source | | `--placement-budget ` | `0` | How long to wait. `0` waits as long as the controller runs the operation, which it already bounds | | `--max-runtime ` | controller's maximum | Budget requested for the operation on the controller, clamped down to that maximum | | `--idempotency-key ` | derived from the request | Makes retrying safe: a retry with the same key joins the running operation instead of starting a second solve | | `--async` | off | Start the placement, print its operation id, and exit without waiting | | `--sync` | off | Use the single blocking `GetValidPlacementsMGD` call, which needs `--timeout` to cover the whole solve | | `--placement-index ` | `0` | Which returned placement the launcher outputs describe | | `--rank-bindings-out ` | | Also write a rank bindings YAML (`TT_VISIBLE_DEVICES` built from chip IDs) | | `--rankfile-out ` | | Also write an OpenMPI rankfile | | `--mesh-graph-desc-path ` | value of `--mgd-file` | Value written to the rank bindings file's `mesh_graph_desc_path` field | ```bash tt-fabric-manager-cli get-valid-placements-mgd \ --mgd-file examples/mgd.2x4.textproto \ --rank-bindings-out rank_bindings.yaml \ --rankfile-out rankfile ``` The command exits non-zero unless the status is `PLACEMENT_MGD_SUCCESS`, so it can gate a job launch directly. When the wait ends without a placement — cancelled, out of budget on the controller, or interrupted — `status` carries the operation's state instead of a placement status, `placements` is empty, and `operation_id` identifies the operation to follow up on. Against a controller too old to know about placement operations, the command falls back to the blocking call automatically (visible with `--verbose`), in which case `--timeout` has to cover the solve again. ```{warning} Only textproto MGDs are accepted — the file must end in `.textproto` or `.txt`. Binary `.pb` files are rejected with a message pointing at `protoc`; convert first. (The built-in `--help` text still claims `.pb` is supported.) ``` ### `placement-operation` Shows one placement operation, and optionally waits for it. The counterpart to `--async`: start a placement in one job step and collect it in another, without either holding a connection open for the duration. | Option | Default | Description | |--------|---------|-------------| | `--operation-id ` | **required** | Operation to show | | `--wait` | off | Poll until the operation finishes, then print it with its result | | `--placement-budget ` | `0` | How long to wait when using `--wait` | ```bash OP=$(tt-fabric-manager-cli get-valid-placements-mgd --mgd-file mesh.textproto --async | jq -r .operation_id) tt-fabric-manager-cli placement-operation --operation-id "$OP" --wait ``` Exits zero whenever the operation could be read, including while it is still running: `state` and `result.status` carry the outcome. An id the controller does not know is an error with `error_code: NOT_FOUND`, which also covers results past their retention window and ids from a previous controller process. ### `cancel-placement-operation` Stops a placement operation on the controller. Cancellation is cooperative, so the reported state may still be `SOLVING` and reach `CANCELLED` shortly after. Cancelling an operation that already finished changes nothing and keeps its result. | Option | Default | Description | |--------|---------|-------------| | `--operation-id ` | **required** | Operation to stop | ### `list-placement-operations` Lists the placements the controller is running or still retains results for, most recently created first — including those started by other clients and by the web UI. Useful for answering "what is this controller busy with?". | Option | Default | Description | |--------|---------|-------------| | `--include-results` | off | Attach each finished operation's placement | ### `show-topology` The one human-oriented command: renders an ASCII summary of hosts and cross-host connectivity from `QueryTopologySummary`. Output width follows the terminal, or `COLUMNS` when it cannot be detected. Errors go to stderr. | Option | Description | |--------|-------------| | `--source ` | Build the view from the discovered PSD (default), the FSD, or the union of both. `combined` also tags each host with its source and FSD data-centre location | | `--validate` | Ask the controller to diff PSD against FSD and report missing and unexpected cross-host links per host. Requires a configured FSD | ```bash tt-fabric-manager-cli show-topology --source combined --validate ```