What to Know About Your Workstation

Your workstation is a Tenstorrent Quietbox 2: four AI accelerators inside, an operating system you may not have used before, and the software stack already configured and waiting. The machine is ready to go — what’s left is knowing what you’ve got.

This guide doesn’t assume you know Linux, or Python, or what a PCIe slot is. It assumes you’re curious, and that curiosity is enough.

What’s Inside

The Tenstorrent Quietbox 2 (QB2) is a workstation with two Blackhole p300c cards — four Blackhole chips in total — on PCIe. Each p300c is a dual-chip card, and each chip is independent — four separate devices from the software’s point of view, connected to a standard CPU running Ubuntu 24.04 LTS.

The chips don’t share memory. When you open device 0, you’re talking to one Blackhole chip. To use all four together, you use ttnn.CreateDevices({0, 1, 2, 3}) — not four separate open_device() calls.

What Ships Pre-Installed

Tenstorrent ships the QB2 ready to serve models. You don’t install drivers. You don’t compile anything. The full stack is already there:

• Kernel driver — loaded automatically at boot, makes the chips visible to software
• tt — the Tenstorrent CLI. One command for updating the stack, checking hardware, serving models and launching apps (~/.local/bin/tt)
• tt-smi — hardware monitoring tool, lives at /usr/bin/tt-smi
• TTNN / TT-Metalium — as a container, started by the tt-metalium wrapper in ~/.local/bin/
• Hardware tooling venv — ~/.tenstorrent-venv/, holding tt-smi and tt-flash
• tt-inference-server — what tt serve runs to serve a model with vLLM in a container
• TT-Forge/XLA — container wrapper at ~/.local/bin/tt-forge, if it was installed (it’s opt-in)
• tt-studio — the no-code web UI for serving models (tt serve <model> --studio)
• A ready-to-run model — Qwen3-32B, weights pre-cached on disk, served with tt serve Qwen3-32B and no download (your fastest path to a first token)
• Firmware — already flashed to all four chips

What’s intentionally absent: ~/tt-metal — not just the source code, the whole directory. TT-Metalium reaches you as a container image instead, and tt-metalium is how you get inside it. You can build models, run inference, and work with the full API stack without ever having a checkout. Building from source is a later chapter — a much later chapter.

Physical Tour

The QB2 looks like a standard tower workstation. On the inside:

• CPU and motherboard running Ubuntu 24.04 LTS
• Two Blackhole p300c cards (four Blackhole chips total)
• RAM sized for production inference workloads
• Storage for model weights — but watch it carefully (more on that in Chapter 2)

The chips run warm under load. Fans will get louder when you run inference. This is correct. The cooling is designed for sustained operation at full chip temperature.

One Blackhole chip. You have four, on two p300c cards.

Next: First Boot →

First Boot

Power on. Ubuntu loads. You log in. Now what?

Everything from here happens in a terminal. That’s the command line — a text window where you type instructions and the machine responds. On a QB2, the terminal is your instrument panel. Learning its three or four most-used commands will get you surprisingly far.

Finding a Terminal

If you’re looking at the GNOME desktop:

• Press Ctrl+Alt+T — opens a terminal on most Ubuntu setups
• Or press the Super key (Windows key), type terminal, press Enter
• Or right-click the desktop and choose “Open Terminal”

Once a terminal window is open, you’re in the right place. It shows a prompt ending in $ — everything you type goes after that.

The Three Commands You Need Right Now

Check disk space first. Models are large. This is non-negotiable to understand before you do anything else:

df -h ~

This shows your home directory’s disk usage. The Size column is total, Avail is what’s free. You need room — at minimum 3 GB for a small model (Qwen3-0.6B), 20+ GB for anything like Llama-3.1-8B. If you’re under 5 GB free, stop here and figure out where the space went before continuing.

Check internet connectivity:

ping -c 3 google.com

If this fails, check your network cable or go to Settings → Network. Everything else in this guide requires internet access for model downloads.

Update the package list (do this once after first boot):

sudo apt update

sudo means “run as administrator.” Ubuntu will ask for your password. This doesn’t install or change anything — it just refreshes the list of what’s available. You’ll see a lot of text scroll by. That’s normal.

Ubuntu: What You Should Know

The QB2 runs Ubuntu 24.04 LTS. If this is your first time with it:

• Package manager is apt — install things with sudo apt install <name>
• Files are case-sensitive: Model.py and model.py are different files
• Your home directory is ~ — short for /home/yourusername
• sudo runs a command as administrator — use it only when a command tells you to

Your Login, Password, and SSH

Many QB2 units ship with a default login — username ttuser, password ttuser. If that’s how yours arrived, change the password the moment you’re in, before the machine is reachable on a shared network:

passwd

It asks for the current password (ttuser), then a new one twice.

Turn on SSH

Later in this guide — and on every other path — you reach the QB2 from your own laptop over SSH: forwarding a model server’s port back to your machine, copying files, running commands remotely. SSH isn’t always running on a fresh box, so turn it on once:

# Install and enable the SSH server
sudo apt install -y openssh-server
sudo systemctl enable --now ssh

# Confirm it's listening
systemctl status ssh

Then find the address other machines use to reach you:

hostname -I     # the QB2's IP address on your network
hostname        # its name — often <name>.local

From your laptop you can now run ssh ttuser@<that-ip>. This is what makes the remote-access steps in Serving Models on QB2 — and bringing tt-studio’s web UI to your own browser — work.

Python: A Field Guide to the Confusion

This is where new Linux users often hit a wall. Ubuntu ships with its own Python. The Tenstorrent software has its own Python environments. These are separate and don’t mix. Here’s the landscape:

What exists on your system

The important surprise is the third row. There is no TTNN environment on the host. You may find older notes pointing at ~/tt-metal/python_env/ — that path doesn’t exist on a QB2 and the installer never creates it. TT-Metalium arrives as a container image, and tt-metalium is the wrapper that puts you inside it with your home directory mounted. import ttnn works in there and nowhere else.

Why does this matter?

Ubuntu 24.04 enforces what’s called externally-managed Python — the system Python is protected. If you try to pip install something directly, Ubuntu will refuse with an error about breaking system packages. This is intentional. It protects you.

So when you need a Python package of your own, make it a new venv (python3 -m venv ~/.venvs/mine) or let uv tool / pipx make one for you. What you should not do is install into ~/.tenstorrent-venv to dodge the error. That venv exists to hold tt-smi and tt-flash, your QB2 activates it for you at login, and a dependency resolution gone wrong in there takes out the tools you’d use to work out what broke.

What which python3 tells you

Before running any Python code, check which Python is active:

which python3

If you see /usr/bin/python3 — you’re on the host, using Ubuntu’s Python. Tenstorrent imports will fail.

If you see /home/yourname/.tenstorrent-venv/bin/python3 — you’re in the tooling venv. tt-smi works; import ttnn still won’t.

If you see /opt/venv/bin/python3 — you’re inside the tt-metalium container, which is where TTNN lives. Go ahead.

pip, pyenv, uv — a brief map

You may encounter other Python tools in documentation or online:

• pip — Python package installer. Works inside a venv. Fine to use there.
• pyenv — manages multiple Python versions (3.10, 3.11, etc.). The QB2 doesn’t need it — the venvs handle version isolation.
• virtualenv / python -m venv — creates isolated environments. The Tenstorrent venvs were built this way.
• uv — a fast, modern alternative to pip and virtualenv. uv tool install <pkg> is the tidiest way to add a CLI (like hf) without touching a managed environment.

For this guide: ignore pyenv. Reach for uv or pipx when you want a CLI of your own, and use tt-metalium when you want TTNN.

Activating and deactivating

# Enter the TTNN environment — a container, not a venv
tt-metalium

# You're now in a shell inside the container, home directory mounted.
which python3          # → /opt/venv/bin/python3
python3 -c "import ttnn; print('TTNN ready')"

# Leave it the way you leave any shell
exit

There’s no (python_env) prefix to look for, because nothing is being activated — you’re in a different shell on a different filesystem. which python3 is the signal: /opt/venv/bin/python3 means you’re inside.

Next: Is This Thing On? →

Is This Thing On?

Before running a model, confirm the hardware is alive and the software can see it. One command, four chips, zero guessing.

Ask tt

tt device status

                 Tenstorrent devices (driver: TT-KMD 2.11.0)
┏━━━┳━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━┓
┃ # ┃ Board ┃ Bus ID       ┃ Temp    ┃ Power  ┃ AIClk      ┃ Voltage ┃ DRAM ┃
┡━━━╇━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━┩
│ 0 │ p300c │ 0000:01:00.0 │ 55.0 °C │ 74.0 W │ 1337.0 MHz │ 0.81 V  │ True │
│ 1 │ p300c │ 0000:02:00.0 │ 59.9 °C │ 49.0 W │ 1343.0 MHz │ 0.82 V  │ True │
│ 2 │ p300c │ 0000:03:00.0 │ 45.9 °C │ 13.0 W │ 800.0 MHz  │ 0.73 V  │ True │
│ 3 │ p300c │ 0000:04:00.0 │ 47.9 °C │ 18.0 W │ 800.0 MHz  │ 0.73 V  │ True │
└───┴───────┴──────────────┴─────────┴────────┴────────────┴─────────┴──────┘

Four rows, all p300c, all DRAM True: the machine is healthy. (If you get tt: command not found, see the PATH note in Install and Upgrade with tt.)

Two more tt commands answer the next questions:

tt device info      # board ids, PCIe speed and width, firmware versions, one column per chip
tt smi              # hands the terminal to the live tt-smi dashboard (q to quit)

tt device info shows the firmware bundle on each chip (fw.fw_bundle_version), which is the first thing support asks for. tt device status --json gives the same data as JSON, plus the host’s OS, kernel, memory and driver version, so scripts do not have to parse a table.

The rest of this chapter looks one level lower, at tt-smi, which tt runs for you. You need it for the raw fields, and when something is wrong and you are diagnosing.

Reading Your Hardware with tt-smi

tt-smi is the Tenstorrent System Management Interface. Your window into the chips. Run it in snapshot mode to get JSON instead of the interactive TUI:

tt-smi -s

A healthy QB2 returns four entries — one per Blackhole chip. The real shape is nested, not flat — board_info and telemetry are their own sub-objects:

{
  "device_info": [
    {
      "board_info": {
        "bus_id": "0000:01:00.0",
        "board_type": "p300c",
        "board_id": "0000046131924062",
        "dram_status": true,
        "pcie_speed": 4,
        "pcie_width": "4"
      },
      "telemetry": {
        "voltage": "0.72",
        "current": "23.0",
        "power": "16.0",
        "aiclk": "800",
        "asic_temperature": "40.3"
      },
      "firmwares": {
        "fw_bundle_version": "19.13.1.0"
      }
    }
  ]
}

board_type is the board SKU (p300c on a QB2), not a chip-family string — Blackhole doesn’t appear literally anywhere in the output. Four entries in device_info means four chips, all alive. Check it directly:

tt-smi -s | python3 -m json.tool | grep board_type

You should see "p300c" printed four times.

Reading the Output

A healthy QB2 shows four entries in device_info. Look at each one for:

• board_info.board_type: "p300c" — the board SKU for a QB2 (two chips per card, two cards). “BLACKHOLE” doesn’t appear literally anywhere in the output — that’s the chip family the p300c board carries.
• board_info.pcie_speed: 4 — an integer PCIe generation, not the string "GEN4". 4 is full speed; 3 would mean a slot compatibility issue.
• board_info.pcie_width: "4" — the link width in lanes (as a string). Narrower (lower number) means lower bandwidth.
• telemetry.asic_temperature in the 35–55°C range — normal at idle. Higher under load is fine. board_info and telemetry are nested sub-objects on each device entry, not flat fields — see the JSON sample above.

Count the entries:

tt-smi -s | python3 -c "import sys,json; d=json.load(sys.stdin); print(len(d['device_info']), 'devices')"

If you see 4 devices, move on. The QB2 is ready.

If You See Fewer Than Four

A missing device usually means one of three things:

PCIe link not established:

sudo dmesg | grep -i tenstorrent | tail -20

sudo is required here — Ubuntu 24.04 restricts the kernel log buffer by default, and a plain dmesg fails with “read kernel buffer failed: Operation not permitted.” Look for errors about PCIe enumeration or firmware loading failure. A loose card is possible — the QB2 ships with cards seated, but transit happens.

Firmware mismatch:

tt-smi -s | python3 -m json.tool | grep -i fw_bundle_version

If firmware versions differ across devices, or show 0.0.0, you may need to reflash. See the tt-flash documentation for instructions.

Driver not loaded:

lsmod | grep tenstorrent

If nothing prints, the kernel driver isn’t loaded. This shouldn’t happen on a stock QB2, but if it does:

sudo modprobe tenstorrent

Watching in Real Time

For a live view of all four chips while running inference:

tt-smi

This opens the interactive TUI — press q to quit. You’ll see per-chip utilization, temperature, and memory usage update live. Useful when a model is running and you want to see all four chips light up.

For something richer than the built-in TUI, tt-toplike renders the same telemetry as live ASCII art — every chip’s power, temperature, and DRAM state, animated:

This is what active inference looks like inside one chip. Four of these run in parallel on a QB2.

Next: Installing the Stack →

Install and Upgrade with tt

tt is the Tenstorrent command line. It is the single tool you use to install the software stack, keep it current, look at your hardware, serve models and connect apps to them. Under the hood it runs tt-installer, tt-smi, tt-flash, tt-inference-server and tt-studio for you, each at a version Tenstorrent has tested together.

A QB2 ships with the software already installed, so this chapter is mostly about upgrading. The same two commands install everything on a fresh Ubuntu machine.

See What You Have

tt --version
tt device status

If you get tt: command not found, ~/.local/bin is not on your PATH in this shell. Run uv tool update-shell (or add export PATH="$HOME/.local/bin:$PATH" to ~/.bashrc) and open a new terminal.

If tt --help does not list device, model, serve and update, another program named tt is earlier on your PATH. which -a tt lists every match in the order the shell tries them. Put ~/.local/bin first (export PATH="$HOME/.local/bin:$PATH"), or start a login shell with bash -l. A non-login shell, such as a plain docker run <image> bash -c '...', skips ~/.profile and can leave the other program in place.

tt device status lists one row per chip. A QB2 has four, on two p300 boards:

                 Tenstorrent devices (driver: TT-KMD 2.11.0)
┏━━━┳━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━┓
┃ # ┃ Board ┃ Bus ID       ┃ Temp    ┃ Power  ┃ AIClk      ┃ Voltage ┃ DRAM ┃
┡━━━╇━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━┩
│ 0 │ p300c │ 0000:01:00.0 │ 55.0 °C │ 74.0 W │ 1337.0 MHz │ 0.81 V  │ True │
│ 1 │ p300c │ 0000:02:00.0 │ 59.9 °C │ 49.0 W │ 1343.0 MHz │ 0.82 V  │ True │
│ 2 │ p300c │ 0000:03:00.0 │ 45.9 °C │ 13.0 W │ 800.0 MHz  │ 0.73 V  │ True │
│ 3 │ p300c │ 0000:04:00.0 │ 47.9 °C │ 18.0 W │ 800.0 MHz  │ 0.73 V  │ True │
└───┴───────┴──────────────┴─────────┴────────┴────────────┴─────────┴──────┘

In that capture, chips 0 and 1 were running a model (high power, 1.3 GHz clock) and chips 2 and 3 were idle (800 MHz). The clock column is the quickest way to see which chips are in use.

Preview an Upgrade

Tenstorrent publishes a golden set: the versions of the kernel driver, firmware, HugePages setup, tt-smi, tt-flash, tt-inference-server and tt-studio that have been tested together. tt update moves your machine onto it. Preview first. This changes nothing:

tt update --dry-run

  Update plan (goldens: bundled supplement.toml + golden.json v3.0.0 (cached))
┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ component       ┃ kind     ┃ installed       ┃ golden           ┃ action     ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ tt-flash        │ uv-tool  │ 4.0.0           │ 4.0.0            │ up-to-date │
│ tt-inference-s… │ git-venv │ v0.25.0         │ v0.25.0          │ up-to-date │
│ tt-model        │ uv-tool  │ a7aa9cefdd3086… │ a7aa9cefdd3086b… │ up-to-date │
│ tt-smi          │ uv-tool  │ 6.6.0           │ 6.6.0            │ up-to-date │
│ tt-studio       │ git-venv │ —               │ v2.12.0          │ optional   │
│ system-stack    │ system   │ —               │ hugepages 1.4.1, │ converge   │
│                 │          │                 │ kmd 2.11.0, ...  │            │
└─────────────────┴──────────┴─────────────────┴──────────────────┴────────────┘

Read the action column:

Upgrade

tt update

tt update runs in three phases: it checks the golden manifest, installs or upgrades the tools into isolated environments under ~/.local/share/tenstorrent/, and then downloads the matching tt-installer release and runs it. The last phase uses sudo, so it asks for your password. It also asks before flashing firmware, because flashing resets the devices. Stop any model you are serving first (tt model ps shows what is running).

Upgrade the tt command itself separately:

tt self update --check    # is there a newer release?
tt self update

On the QB2 used to test this guide, tt self update --check reported tt 1.2.1 is available (you have 1.2.0). tt update brings the system and its tools onto the golden set, and tt self update upgrades tt. Run both when you want to be current.

Installing on a Fresh Machine

You need Ubuntu 22.04 or 24.04, sudo, and a network connection. Install uv, then tt:

sudo apt update && sudo apt install -y python3 python3-venv python3-pip git curl ca-certificates
curl -LsSf https://astral.sh/uv/install.sh | sh
export PATH="$HOME/.local/bin:$PATH"   # uv and tt install here; this makes them work in this terminal
uv tool install tenstorrent
uv tool update-shell        # puts ~/.local/bin on PATH for future terminals
tt update

The PyPI package is named tenstorrent. The unrelated tt-cli package on PyPI is a time tracker. tt update installs the kernel driver, firmware tooling and HugePages configuration as root, so reboot afterwards.

Using tt-installer Directly

tt update runs tt-installer for you, and for almost everyone that is the right way to use it. tt update <VERSION> runs a specific installer release instead (and implies --force, which allows downgrades). The installer’s own options, and the manual path it replaces, are covered in Installing by hand.

Check Your Work

tt --version
tt update --dry-run      # every row should say up-to-date
tt device status         # four rows, all DRAM True
tt model list            # models for your hardware

If tt device status shows fewer than four rows, go back to Is This Thing On? for the diagnosis steps.

Next: Serve a Model →

Serve a Model

Four chips, one command. tt serve downloads or finds the weights, starts the model in a container, waits until it answers, and tells you the address. What you get is an OpenAI-compatible HTTP API running on your own hardware, which every chat app, coding agent and SDK already knows how to talk to.

This chapter serves Qwen3-32B, the model your QB2 shipped with. Its weights and compiled kernels are already on disk, so the first serve does not download 62 GB.

Pick the Model

tt knows which models run on your hardware:

tt model list

On a QB2 it detects a p300x2 configuration (two p300 boards, four chips) and lists the models that have a serving profile for it. tt model info tells you what a model needs and how well it is validated:

tt model info Qwen3-32B

┌─────────────────────┬────────────────────────────────────────────────────────┐
│ hf_repo             │ Qwen/Qwen3-32B                                         │
│ type                │ llm                                                    │
│ engines             │ vLLM                                                   │
│ via                 │ inference-server, studio                               │
│ servable            │ yes — `tt serve Qwen3-32B` (`--studio` deploys it with │
│                     │ TT-Studio)                                             │
│ parameters          │ 32B                                                    │
│ min disk            │ 84 GB                                                  │
│ min ram             │ 80 GB                                                  │
│ on p300x2           │ vLLM — FUNCTIONAL, max context 131072                  │
│ ...                 │                                                        │
└─────────────────────┴────────────────────────────────────────────────────────┘

Validation status goes EXPERIMENTAL, FUNCTIONAL, COMPLETE. FUNCTIONAL means it runs correctly and is not fully tuned. The status is per board, so check the p300x2 row.

Models worth starting with on a QB2

These have a p300x2 profile in the released catalog, from tt model list --catalog:

Gated models (Llama, Gemma) need you to accept the license on the model’s Hugging Face page first. The hf_repo line in tt model info names the page.

Log In to Hugging Face

Serving needs a Hugging Face token, even for models that are free to download, because tt-inference-server checks it before it starts. Log in once:

tt model login

Choose Log in with your browser (it shows a URL and a short code) or paste a read token from huggingface.co/settings/tokens. tt finds the token again on every later run, and tt serve --dry-run tells you where it came from without printing it.

Check That tt Will Reuse the Shipped Weights

Preview the serve before you run it. --dry-run starts nothing:

tt serve Qwen3-32B --dry-run

✓ Detecting device configuration  p300x2 (override with --device)
                    tt serve Qwen3-32B — dry run
┌──────────────────┬──────────────────────────────────────────────────────────┐
│ backend          │ tt-inference-server                                      │
│ workflow         │ server                                                   │
│ device           │ p300x2                                                   │
│ engines          │ vLLM                                                     │
│ status           │ FUNCTIONAL                                               │
│ docker image     │ ghcr.io/tenstorrent/tt-inference-server/vllm-tt-metal-…  │
│ tool-call parser │ hermes                                                   │
│ reasoning parser │ qwen3                                                    │
│ weights          │ /home/ttuser/data/tt-cache                               │
│                  │ pre-seeded volume — weights and tt_metal_cache reused;   │
│                  │ the HF cache is not consulted                            │
│ HF token         │ from the Hugging Face login store                        │
│ port             │ 20000  (server default)                                  │
└──────────────────┴──────────────────────────────────────────────────────────┘

Look at the weights row. pre-seeded volume means tt found the shipped weights and the compiled kernel cache, so the first start takes minutes and downloads nothing. If it says ~/.cache/huggingface instead, tt did not recognize the shipped directory and a real serve would download 62 GB. The fix is one symbolic link:

ln -s ~/data/tt-cache/volume_id_tt_transformers-Qwen3-32B-vqb2_launch \
      ~/data/tt-cache/volume_id_tt_transformers-Qwen3-32B-v0.17.0

Serve It

tt serve Qwen3-32B

tt serve shows the boot as a live checklist instead of thousands of log lines: host ready, weights ready, image ready, container started, engine started, device opened, KV cache configured, model warmed up, endpoint answering. It returns only once the endpoint answers, so whatever you run next will work. When it finishes you get a box with the endpoint (http://127.0.0.1:20000/v1) and the commands to look at logs and stop the model.

To watch the raw server log while it boots, run this in a second terminal:

tt model logs Qwen3-32B --follow

Pressing Ctrl-C in the tt serve terminal stops watching. The server is a separate container and keeps booting, and tt model stop Qwen3-32B ends it.

Ask It a Question

tt model curl "Explain what an API is in one sentence." --max-tokens 2048

tt model curl asks the server which model it is serving, sends your prompt as a chat completion, and prints the JSON reply. The text is in choices[0].message.content. Unknown options become fields of the request, so --max-tokens 2048 sets max_tokens. Add --print to see the equivalent curl command.

Qwen3-32B is a reasoning model (the dry run above shows a qwen3 reasoning parser): it thinks first, in a separate reasoning field, and then answers in content. A small token budget can be used up by the thinking, so give it room when content comes back empty.

The same endpoint speaks the standard OpenAI API, so any client works. Note the model name: the server reports the full Hugging Face id, which differs from the short name you gave tt serve (for Llama-3.1-8B-Instruct we saw meta-llama/Llama-3.1-8B-Instruct).

curl -s http://localhost:20000/v1/models | python3 -m json.tool | grep '"id"'

curl -s http://localhost:20000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-32B",
    "messages": [{"role": "user", "content": "Explain what an API is in one sentence."}],
    "max_tokens": 2048
  }' | python3 -m json.tool

Manage What Is Running

tt model ps                    # what is served, on which port, and whether it is healthy
tt model logs Qwen3-32B        # the newest server log

Keep the model running for the next chapter. When you are finished with it, tt model stop Qwen3-32B stops it.

tt model ps shows every server tt started, including ones on other ports:

┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━┓
┃ name                  ┃ backend          ┃ container                    ┃ port  ┃ health  ┃ uptime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━┩
│ Llama-3.1-8B-Instruct │ inference-server │ tt-inference-server-162a825f │ 20000 │ healthy │ 5m     │
└───────────────────────┴──────────────────┴──────────────────────────────┴───────┴─────────┴────────┘

healthy means the server answers GET /v1/models. starting means the container is running and not answering yet. tt model stop stops only the named model’s container.

Serving on Half the Machine

A QB2 is two p300 boards. A model with a p300 profile uses one board (two chips). For example, Llama-3.1-8B-Instruct has a p300 profile:

tt serve Llama-3.1-8B-Instruct --device p300 --dry-run

Use --device to override what tt detected. tt serve does not let you choose which board: the container is given every Tenstorrent device node on the machine, so run a half-machine model only when nothing else is using the chips. How to pin a server to specific chips is in Serving Models on QB2.

What Just Happened

tt serve ran tt-inference-server, which started a container holding vLLM and the Tenstorrent plugin. The container opened a mesh across your chips, loaded the weights into device memory, compiled and warmed up the kernels, and published the API. The detail of each stage, and how to run tt-inference-server yourself, is in Serving Models on QB2.

Next: Launch Software →

Launch Software

An API endpoint is the foundation. tt launch connects a ready-made application to the model you are already serving, so you can use it without copying a URL into a settings page. You do not tell the app where the model is: tt asks every server it started what it serves (GET /v1/models) and configures the client from the answer.

Keep the model from the last chapter running, and see what you can connect:

tt launch list

┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ client      ┃ kind        ┃ model          ┃ status         ┃ configures                           ┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ opencode    │ terminal    │ tool calling   │ needs opencode │ ~/.config/opencode/opencode.json     │
│ pi          │ terminal    │ tool calling   │ needs pi       │ ~/.pi/agent/models.json              │
│ aider       │ terminal    │ tool calling   │ needs aider    │ environment (this run only)          │
│ qwencode    │ terminal    │ tool calling   │ ready          │ command-line arguments (this run…)   │
│ hermes      │ terminal    │ tool calling   │ ready          │ environment (this run only)          │
│ openwebui   │ web service │ any chat model │ stopped        │ container tt-open-webui              │
│ anythingllm │ web service │ any chat model │ ready          │ container tt-anythingllm             │
└─────────────┴─────────────┴────────────────┴────────────────┴──────────────────────────────────────┘

There are two kinds of client, and the difference is who installs them:

• Web UIs (Open WebUI, AnythingLLM). tt pulls and runs these in a container for you, after asking once. You only need Docker.
• Terminal coding agents (OpenCode, pi, Aider, Qwen Code, Hermes Agent). You install the agent. tt launch writes the small config it needs to reach your model and hands your terminal to it. These need a model that supports tool calling; tt checks, and stops with a suggestion if the model cannot.

The status column says what is missing: needs opencode means the agent is not installed, ready means it can launch now, and stopped is a web UI container that exists and is not running.

A Chat Window in Your Browser

tt launch openwebui --dry-run     # show what would happen, change nothing
tt launch openwebui

The dry run prints what tt found and what it will do:

┌───────────────────────┬───────────────────────────────────────────────────────────┐
│ model                 │ meta-llama/Llama-3.1-8B-Instruct  (Llama-3.1-8B-Instruct) │
│ endpoint              │ http://127.0.0.1:20000/v1                                 │
│ tool-call parser      │ llama3_json                                               │
│ container             │ tt-open-webui (stopped)                                   │
│ image                 │ ghcr.io/open-webui/open-webui:main                        │
│ endpoint in container │ http://host.docker.internal:20000/v1                      │
│ open                  │ http://localhost:3000                                     │
└───────────────────────┴───────────────────────────────────────────────────────────┘

The first time, tt asks before it pulls the image (a few GB) and runs it as tt-open-webui. It waits until the page answers and prints tt-open-webui is ready at http://localhost:3000. Open that address in a browser. The container is configured with your endpoint (OPENAI_API_BASE_URL=http://host.docker.internal:20000/v1), so its model list comes from your server.

If two models are being served, say which one: tt launch openwebui --model Qwen3-32B. Web UIs start on port 3000, or the next free port above it; --web-port 3080 asks for another. When you are done:

tt launch stop openwebui      # stops the container, keeps your chats and settings

A Coding Agent in Your Terminal

Terminal agents work on your files, so they need a model that can call tools. Qwen3-32B can, with one caveat:

Faster agent models come from the community catalog. The fastest tool-calling models on a QB2 are mixture-of-experts coders with 3B parameters active per token, published as community bundles: raahemnabeel/qwen3-coder-30b-a3b and raahemnabeel/qwen3-coder-next, both with the qwen3_coder tool parser and no thinking mode. Measured on 2026-10-30 on one P300 board: 80 tok/s and 107 tok/s on a real prompt, against about 8 seconds a response for As an alternative to the model from the last chapter, stop it first (tt model stop Qwen3-32B), then serve one with tt serve raahemnabeel/qwen3-coder-30b-a3b --profile p300x2 (see The Model Zoo for the warning that applies to every bundle). tt launch finds it the same way. It prints not in the model catalog and shows tool-call parser — because it has no catalog row to check; the bundle’s card and tt serve <bundle> --dry-run show the parser is set. We ran tt launch qwencode --dry-run and a smolagents task against Qwen3-Coder-Next this way; the task was answered in one 2 s step.

Install one agent, then launch it. Qwen Code and Hermes Agent were ready in the listing above:

tt launch qwencode --dry-run

┌──────────────────┬───────────────────────────────────────────────────────────┐
│ model            │ meta-llama/Llama-3.1-8B-Instruct  (Llama-3.1-8B-Instruct) │
│ endpoint         │ http://127.0.0.1:20000/v1                                 │
│ tool-call parser │ llama3_json                                               │
└──────────────────┴───────────────────────────────────────────────────────────┘

commands:
qwen --auth-type openai --openai-api-key tt-local --openai-base-url http://127.0.0.1:20000/v1 --model meta-llama/Llama-3.1-8B-Instruct

The dry run shows the exact command tt launch runs: your agent, pointed at your endpoint, with no cloud account. Run it without --dry-run to start the agent.

tt shows what it will change and asks before writing a config file. tt launch disconnect opencode removes what it wrote.

Claude Code with Tenstorrent skills

The development version of tt (after release 1.2.1) adds tt agent, which sets up Claude Code with the plugins from the tenstorrent/skills marketplace for what you are doing today (deploy, bringup or develop) and starts it. Preview with tt agent deploy --dry-run. Check tt --help for an agent entry to see whether your release has it.

Prefer Point and Click? TT-Studio

TT-Studio serves the model and provides its own web interface. You start it with tt serve --studio, not tt launch:

tt model stop Qwen3-32B        # TT-Studio takes the chips and port 3000 for itself
tt serve Qwen3-32B --studio

On first use tt fetches the latest TT-Studio release, and TT-Studio asks you to accept its terms. It prints a TT Studio is ready box with its address, http://localhost:3000. Stopping the model with tt model stop Qwen3-32B also takes TT-Studio’s services down and keeps your settings and chat history. Start TT-Studio before any tt launch web UI, since both want port 3000 and only the web UIs move when it is taken.

Check Your Work

tt model ps               # your model is healthy
tt launch list            # your chosen client is ready or connected
tt launch stop openwebui  # clean up the web UI when you are done
tt model stop Qwen3-32B   # and the model

Next: What Comes Next →

What Comes Next

You unboxed a machine that most people have never touched. You confirmed four Blackhole chips were alive, upgraded the software with tt update, served a model with tt serve, asked it a question, and connected an app to it with tt launch. That is the whole loop, and you ran it on your own hardware.

The rest is up to you.

Tools in Your World

The QB2 ships with a full stack, but the ecosystem is bigger. Start with tt-toplike — htop for your chips, except the telemetry comes alive as ASCII art:

Where to Go From Here

Pick a thing you want to do and jump straight in.

Choose Your Next Track

The QB2 is a beginning. There’s a lot of surface area here, and you’ve only scratched it.

← Back to Explore