p300×2 n150 45 min Validated

OpenClaw AI Assistant on TT-QuietBox® 2

Transform your TT-QuietBox 2 into an expert AI assistant that knows everything about Tenstorrent hardware, deployment, and programming - with citations from 45+ interactive lessons and official documentation.

What You'll Build

OpenClaw is an enterprise-grade AI agent framework that runs entirely on your TT-QuietBox 2. Once configured, you get:

What makes this special:

Architecture

┌─────────────┐
│  User (TUI) │ Ask: "What is TT-QuietBox 2?"
└──────┬──────┘
       │ WebSocket to ws://127.0.0.1:18789
       ▼
┌──────────────────┐
│ OpenClaw Gateway │ Process query, search memory, call LLM
└────────┬─────────┘
         │ HTTP POST to http://127.0.0.1:8000/v1/chat/completions
         ▼
┌──────────────────┐
│ vLLM :8000       │ Llama-3.3-70B-Instruct with tool calling
│ (Docker)         │ (accepts all OpenClaw API fields natively)
└────────┬─────────┘
         │ Inference with TT-Metalium + vLLM optimizations
         ▼
┌──────────────────┐
│ 4x p300c Chips   │ 480 Tensix cores, 2,654 TFLOPS, 128K context
│ (Blackhole)      │
└──────────────────┘

Performance on TT-QuietBox 2 (70B model):

Prerequisites

Hardware Requirements

For TT-QuietBox 2 (P300X2) - 70B Model:

Verify your TT-QuietBox 2:

# Check hardware detection
tt-smi -s
# Should show: 4 Blackhole chips, all healthy

# Check available RAM
free -h
# Should show: ~236 GB total

🔍 Check TT-QuietBox 2 Hardware

Software Requirements

Quick verification:

# TT-Inference-Server available?
tt --version    # tt installed (Install the Tenstorrent Software lesson)

# OpenClaw installed?
which openclaw || ls ~/openclaw/openclaw.sh

# Docker working?
docker --version

Step 1: Deploy vLLM with 70B Model

The first step is getting Llama-3.3-70B-Instruct running on your TT-QuietBox 2 with tool calling support (required for OpenClaw agents).

Understanding the Deployment

What happens during deployment:

  1. Environment preparation (30 seconds) - Reset hardware, stop existing containers
  2. Model download (10-30 minutes, one-time) - Downloads 140 GB from HuggingFace
  3. Docker startup (2 minutes) - Initializes container with TT-Metalium™ environment
  4. Model loading (10-20 minutes) - Loads weights to TT hardware (silent phase)
  5. Warmup (5 minutes) - Compiles kernels for P300X2 configuration
  6. Ready - Health endpoint responds, ready for inference

Total time: 15-60 minutes (depending on if model is cached)

The Deployment Command

tt serve Llama-3.3-70B-Instruct --port 8000

🚀 Serve Llama-3.3-70B-Instruct on port 8000
VS Code
tt serve Llama-3.3-70B-Instruct --port 8000

tt serve runs tt-inference-server in a container with the settings OpenClaw needs, all of them read from the model catalog:

If the board is busy or faulting, tt model stop first and, if the fabric does not cycle cleanly, tt-smi -r before serving again.

Faster: Qwen3-Coder-Next from the community catalog

The 70B model is the quality choice and the slow one: 14 seconds a turn. The community catalog (tt model list --community --hw p300x2) has mixture-of-experts coding models with 3B parameters active per token that answer in two to four seconds and call tools with the qwen3_coder parser. Qwen3-Coder-Next (80B total, 3B active, 262K context) measured 107 tok/s on two dies on 2026-10-30; its card reports 178 tok/s on four.

tt serve raahemnabeel/qwen3-coder-next --profile p300x2 --port 8000

🚀 Serve Qwen3-Coder-Next on port 8000
VS Code
tt serve raahemnabeel/qwen3-coder-next --profile p300x2 --port 8000

The first start downloads 148 GB of weights. The smaller raahemnabeel/qwen3-coder-30b-a3b (60 GB, 256K context) is the same shape and the one the Local AI Agents lesson validates. tt serve and tt launch both say the model is not in the model catalog; that is what a community bundle is, and the dry run shows the parser is set.

OpenClaw needs to know the model, so add it to the vllm provider in ~/tt-claw/runtime/openclaw.json and make it the primary:

{
  "models": {
    "providers": {
      "vllm": {
        "baseUrl": "http://127.0.0.1:8000/v1",
        "api": "openai-completions",
        "models": [{
          "id": "Qwen/Qwen3-Coder-Next",
          "name": "Qwen3 Coder Next",
          "reasoning": false,
          "input": ["text"],
          "cost": {"input": 0, "output": 0},
          "contextWindow": 262144,
          "maxTokens": 4096
        }]
      }
    }
  },
  "agents": {"defaults": {"model": {"primary": "vllm/Qwen/Qwen3-Coder-Next"}}}
}

The id is the weights id the server reports in /v1/models, which is what OpenClaw sends back in every request.

Logging In

The first serve asks for a Hugging Face token when none is stored (tt model login stores one; the model is gated, so accept Meta's license on its Hugging Face page first). There is no JWT prompt: local serving through tt needs no API key.

Monitoring Deployment

Watch for these phases:

# tt serve shows the boot as a checklist; for the server's own output:
tt model logs

# Key messages to watch for:
# "Fabric initialized" - Hardware ready
# "Loading checkpoint shards" - Model loading started
# (10-20 min silence) - Weight loading (normal!)
# "Allocated TT KV caches" - Compilation starting
# "Readiness file created" - Ready for inference!

Silent phases are normal! Weight loading takes 10-20 minutes with no log output.

Verify Deployment

tt serve returns when the endpoint answers (tt model ps shows healthy). Test it:

# Check model is loaded
curl http://localhost:8000/v1/models

# Expected output:
# {"object":"list","data":[{"id":"meta-llama/Llama-3.3-70B-Instruct",...}]}

# Test inference (30-second timeout)
curl http://localhost:8000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.3-70B-Instruct",
    "prompt": "Hello",
    "max_tokens": 10
  }'

✅ Check the endpoint
VS Code
curl -s http://localhost:8000/v1/models | python3 -m json.tool 2>/dev/null || curl http://localhost:8000/v1/models

Step 2: Clone tt-claw Repository

The tt-claw repository provides the unified OpenClaw configuration and helper scripts.

# Clone the repository
git clone https://github.com/tsingletaryTT/tt-claw.git ~/tt-claw

# Review structure
cd ~/tt-claw
ls -la

# Expected directories:
# bin/        - Wrapper scripts (openclaw, services, adventure-menu)
# runtime/    - Configuration and data (openclaw.json, agents/, memory/)
# proxy/      - vLLM compatibility proxy (vllm-proxy.py)
# docs/       - Documentation and troubleshooting guides

📦 Clone tt-claw

Understanding the Structure

bin/openclaw

runtime/openclaw.json

runtime/agents/

proxy/vllm-proxy.py (legacy, no longer needed)

Step 3: Install OpenClaw

If OpenClaw isn't already installed:

# Download and install OpenClaw v2026.3.2
npm install -g openclaw@2026.3.2

# Verify installation
openclaw --version
# Should show: 2026.3.2

# Create wrapper script (optional)
mkdir -p ~/openclaw
cat > ~/openclaw/openclaw.sh << 'EOF'
#!/bin/bash
# OpenClaw wrapper with tt-claw runtime config
export OPENCLAW_STATE_DIR="$HOME/tt-claw/runtime"
export OPENCLAW_CONFIG_PATH="$HOME/tt-claw/runtime/openclaw.json"
exec openclaw "$@"
EOF
chmod +x ~/openclaw/openclaw.sh

📥 Install OpenClaw

The configuration is already set up in ~/tt-claw/runtime/openclaw.json, but let's verify the paths exist:

# Check that indexed documentation exists
ls ~/code/tt-vscode-toolkit/content/lessons/*.md | wc -l
# Should show: 45+ lessons

ls ~/tt-metal/METALIUM_GUIDE.md
# Should exist

ls ~/code/tt-inference-server/README.md
# Should exist

What gets indexed:

  1. 45+ Interactive Lessons (1.1 MB markdown)

    • Hardware detection and setup
    • Model deployment and optimization
    • Cookbook examples (Game of Life, Mandelbrot, audio)
    • TT-Forge™, TT-XLA, TT-Metalium frameworks
    • API servers, chat interfaces

  2. TT-Metalium Documentation

    • METALIUM_GUIDE.md - Core framework documentation
    • Release notes and version history
    • Contributing and development guides

  3. TT-Inference-Server Docs

    • Deployment procedures
    • Model bringup guides
    • Workflow documentation

  4. OpenClaw Journey (CLAUDE.md)

    • Complete installation story
    • 70B deployment process
    • vLLM compatibility research
    • Troubleshooting database

Memory search configuration (already in openclaw.json):

{
  "agents": {
    "defaults": {
      "memorySearch": {
        "extraPaths": [
          "$HOME/code/tt-vscode-toolkit/content/lessons",
          "$HOME/tt-metal/METALIUM_GUIDE.md",
          "$HOME/tt-metal/releases",
          "$HOME/code/tt-inference-server/docs",
          "$HOME/tt-claw/CLAUDE.md"
        ],
        "provider": "local",
        "fallback": "none"
      }
    }
  }
}

How it works:

Step 5: Start OpenClaw Gateway

The gateway is the WebSocket server that manages agents, memory, and LLM interactions.

# Set environment variables (if not using wrapper)
export OPENCLAW_STATE_DIR="$HOME/tt-claw/runtime"
export OPENCLAW_CONFIG_PATH="$HOME/tt-claw/runtime/openclaw.json"

# Start gateway
cd ~/openclaw
./openclaw.sh gateway run

# Wait for startup message:
# [gateway] listening on ws://127.0.0.1:18789

⚡ Start Gateway

First startup takes 1-2 minutes:

Watch for these messages:

✓ Loading configuration from ~/tt-claw/runtime/openclaw.json
✓ Indexing documentation: 45 lessons + TT docs
✓ Memory databases created: main.sqlite (50 MB)
✓ Provider "vllm" ready: http://127.0.0.1:8000/v1
✓ Gateway listening on ws://127.0.0.1:18789

Step 7: Launch Terminal UI

Now connect to the gateway with the Terminal UI:

# In a new terminal window
cd ~/openclaw
./openclaw.sh tui

# TUI interface should appear

💬 Launch TUI

Step 8: Test TT Expert Mode

Try these queries to verify everything works:

Basic Hardware Question

You ask:

What is TT-QuietBox 2?

Expected response (with citations):

TT-QuietBox™ 2 is a liquid-cooled, desk-friendly AI
workstation that runs models up to 120 billion parameters locally with a
fully open source software stack. It features:

- 4x Blackhole ASICs (p300c boards)
- 480 Tensix cores (120 cores per chip)
- 2,654 TFLOPS of compute (663 TFLOPS per chip)
- 200 GB/s memory bandwidth per chip
- 120+ billion parameter model support

It's the industry's first desktop AI workstation built on RISC-V
architecture, designed for AI researchers and developers who want
local inference without cloud dependencies.

Source: ../../code/tt-vscode-toolkit/content/lessons/hardware-detection.md#L45-L62

Deployment Question

You ask:

How do I deploy vLLM on Tenstorrent?

Expected response:

To deploy vLLM on Tenstorrent hardware:

1. Ensure TT-Inference-Server is installed:
   cd ~/code/tt-inference-server

2. Run the deployment script with your model and device:
   python3 run.py \
     --model Llama-3.1-8B-Instruct \
     --tt-device n300 \
     --workflow server \
     --docker-server \
     --no-auth

3. Wait for warmup (5-15 minutes for model loading)

4. Test the endpoint:
   curl http://localhost:8000/v1/models

For larger models (70B) on P300X2, add tool calling support:
   --vllm-override-args '{"enable_auto_tool_choice": true, "tool_call_parser": "llama3_json"}'

The deployment process handles Docker containerization, TT-Metalium
environment setup, and hardware initialization automatically.

Source: ../../code/tt-vscode-toolkit/content/lessons/api-server.md#L102-L125

Technical Deep Dive

You ask:

What cookbook examples can I run on my TT-QuietBox 2?

Expected response:

The TT-VSCode-Toolkit includes several cookbook examples you can run:

1. **Game of Life** - Conway's cellular automaton on TT hardware
   - Demonstrates matrix operations on Tensix cores
   - Real-time visualization of compute patterns

2. **Mandelbrot Set** - Fractal generation with parallel compute
   - Shows how to map complex algorithms to TT architecture
   - Achieves significant speedup over CPU

3. **Audio Processing** - Real-time audio filtering and effects
   - Uses TT-Metalium for low-latency DSP operations
   - Examples: reverb, EQ, compression

4. **Image Filters** - Convolution and image transformations
   - Demonstrates TT tensor operations on image data
   - Supports various filter kernels (blur, sharpen, edge detect)

5. **Particle Life** - Physics simulation with emergent behavior
   - Many-body particle interactions
   - Beautiful emergent patterns from simple rules

All examples include source code, documentation, and performance
comparisons vs CPU/GPU.

Source: ../../code/tt-vscode-toolkit/content/lessons/cookbook-overview.md#L28-L89

tt launch openclaw?

Not in tt 1.2.1. tt launch points a client at the model you are serving and writes the smallest config it needs; OpenCode and pi get a tenstorrent provider in their JSON config, and that is exactly the shape OpenClaw's openclaw.json has. An openclaw adapter would:

  1. read openclaw.json (the path from OPENCLAW_CONFIG_PATH, else ~/.openclaw/openclaw.json),
  2. add a tenstorrent entry under models.providers with baseUrl from the running server and one models[] row for its reported id and context length,
  3. set agents.defaults.model.primary to tenstorrent/<id>, after showing the change and asking once,
  4. hand the terminal to openclaw tui, and on tt launch disconnect openclaw remove only the block it wrote.

Today each client is a Python module inside tt-cli, so this would be a tt-cli change. A lighter route, a client described by a small YAML or JSON definition that tt loads from a user directory, is proposed separately. Until one of those lands, the edit to openclaw.json above is the connection, and tt model ps plus curl /v1/models are the check.

Alternative: n150 with 8B Model

If you have an n150 (single chip) instead of TT-QuietBox 2, you can run OpenClaw with a smaller model.

n150 Specifications

Deployment for n150

tt serve Llama-3.1-8B-Instruct --port 8000

tt detects the n150 and serves the 8B model with the same tool-calling settings. On the cloud pod, which has no Docker, use the no-container route in Serve Without Containers and add --port 8000 to its vllm serve command.

Update OpenClaw Configuration

Edit ~/tt-claw/runtime/openclaw.json:

{
  "models": {
    "providers": {
      "vllm": {
        "models": [{
          "id": "meta-llama/Llama-3.1-8B-Instruct",
          "name": "Llama 3.1 8B Instruct",
          "contextWindow": 8192,
          "maxTokens": 2048
        }]
      }
    }
  },
  "agents": {
    "defaults": {
      "model": {
        "primary": "vllm/meta-llama/Llama-3.1-8B-Instruct"
      }
    }
  }
}

Performance Comparison: 70B vs community vs 8B

Feature TT-QuietBox 2 (70B) TT-QuietBox 2 (Qwen3-Coder-Next, community) n150 (8B)
Response time 14+ seconds 2-4 seconds (107 tok/s measured on two dies) 2-3 seconds
Context window 131K tokens 262K tokens 8K tokens
Reasoning depth Excellent Very good for code and tools; 3B active Good
Hardware required 2x p300c 2x p300c (or one, --profile p300) 1x n150
Power consumption ~300W ~300W ~75W
Model loading time 10-20 minutes 10 minutes first start, then minutes 2-5 minutes
Best for Complex reasoning Fast agents and coding Fast responses

When to use 8B:

When to use 70B:

Troubleshooting

Gateway Fails to Start

Symptom: Gateway exits with "Failed to connect to provider"

Solution:

# 1. Verify vLLM is running
curl http://localhost:8000/v1/models
# Should return model list

# 2. Check openclaw.json points to port 8000 (not 8001)
grep baseUrl ~/tt-claw/runtime/openclaw.json
# Should show: "baseUrl": "http://127.0.0.1:8000/v1"

# 3. Restart gateway
pkill -f 'openclaw.*gateway'
cd ~/openclaw && ./openclaw.sh gateway run

No Responses from Model

Symptom: Gateway starts but queries timeout or return empty

Solution:

# 1. Test vLLM directly
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.3-70B-Instruct",
    "messages": [{"role": "user", "content": "Hi"}],
    "max_tokens": 10
  }'
# Should return completion within 10-20 seconds

# 2. Check if model is still warming up
docker logs $(docker ps -q) | tail -50
# Look for "Readiness file created"

# 3. Check context window setting
grep contextWindow ~/tt-claw/runtime/openclaw.json
# Should be: 131072 (not 16000 or lower)

# 4. Verify gateway logs
tail -100 /tmp/openclaw-*/openclaw-*.log | jq 'select(.level=="error")'

Memory Search Not Working

Symptom: Agent responds but doesn't use indexed knowledge

Solution:

# 1. Check memory databases were created
ls -lh ~/tt-claw/runtime/memory/*.sqlite
# Should show 2-3 files, ~50 MB each after first run

# 2. Verify paths exist
ls ~/code/tt-vscode-toolkit/content/lessons/*.md | wc -l
# Should show 45+ files

# 3. Check indexing logs
grep -i "index" /tmp/openclaw-*/openclaw-*.log | head -20
# Should show "Indexed X documents"

# 4. Restart gateway to re-index
pkill -f 'openclaw.*gateway'
rm ~/tt-claw/runtime/memory/*.sqlite  # Force re-index
cd ~/openclaw && ./openclaw.sh gateway run
# Wait 1-2 minutes for indexing

TUI Shows "Connecting..."

Symptom: TUI stuck at "Connecting to gateway..."

Solution:

# 1. Verify gateway is running
ps aux | grep 'openclaw.*gateway'

# 2. Check gateway is listening
netstat -tlnp | grep 18789
# Should show: LISTEN on 0.0.0.0:18789

# 3. Check gateway logs for startup errors
tail -50 /tmp/openclaw-*/openclaw-*.log | grep -i error

# 4. Verify token matches (if auth is enabled)
grep '"token":' ~/tt-claw/runtime/openclaw.json
# Should match in both "auth" and "remote" sections

# 5. Try connecting with curl (WebSocket test)
curl -i -N \
  -H "Connection: Upgrade" \
  -H "Upgrade: websocket" \
  http://127.0.0.1:18789/
# Should return: "101 Switching Protocols"

Docker Container Keeps Restarting

Symptom: vLLM container crashes during warmup

Solution:

# 1. Check available RAM
free -h
# Need at least 175 GB free for 70B model

# 2. Check for OOM killer
dmesg | grep -i "out of memory"
# If present, system ran out of RAM

# 3. Reset hardware and retry
tt-smi -r
sleep 5
docker stop $(docker ps -q)
docker system prune -f
# Then re-run deployment from Step 1

# 4. Check Docker logs for specific error
docker logs $(docker ps -aq | head -1) | tail -100

Gateway Gets 400 Bad Request from vLLM

Symptom: Gateway connects but gets "400 Bad Request" from vLLM

All current TT vLLM images accept unknown fields silently (including strict, store, prompt_cache_key that OpenClaw sends). If you're seeing 400 errors, the issue is likely a model name mismatch or malformed payload, not the extra fields.

Solution:

# 1. Test vLLM directly with the exact model name OpenClaw uses
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.3-70B-Instruct",
    "messages": [{"role": "user", "content": "Hi"}],
    "strict": true,
    "store": false
  }'
# Both strict and store are accepted — should return 200 OK

# 2. Verify the model ID in openclaw.json matches what vLLM reports
curl http://localhost:8000/v1/models | python3 -m json.tool
grep '"id"' ~/tt-claw/runtime/openclaw.json
# IDs must match exactly

What You've Built

Complete AI Assistant Stack

User Interface Layer:
  ├─ Terminal UI (interactive chat)
  ├─ WebSocket API (port 18789)
  └─ Channel Integrations (optional: Slack, Discord, etc.)

Agent Framework Layer:
  ├─ OpenClaw Gateway (agent management, memory, tools)
  ├─ Memory Search (1,217 chunks from 45+ lessons)
  ├─ Tool Calling (file ops, system commands, memory ops)
  └─ Multi-Agent Support (main + 3 adventure game agents)

LLM Inference Layer:
  ├─ vLLM Server (inference engine, port 8000)
  └─ Model: Llama-3.3-70B-Instruct (128K context, tool calling)

Hardware Layer:
  ├─ TT-Metalium Framework (kernel compilation, device management)
  ├─ Firmware (19.4.2.0+ on each chip)
  └─ 4x Blackhole ASICs (480 Tensix cores, 2,654 TFLOPS)

Capabilities

Ask about Tenstorrent:

Technical assistance:

Creative applications:

Key Features

Next Steps

Explore Built-in Agents

OpenClaw includes 3 adventure game agents as examples:

# Launch adventure menu
cd ~/tt-claw
./bin/adventure-menu

# Or use TUI to switch agents:
# 1. In TUI, type: /agent chip-quest
# 2. Say: "start the adventure"
# 3. Enjoy educational adventure inside a TT chip!

Build Custom Agents

Create your own agents by:

  1. Adding agent definition to openclaw.json
  2. Creating agent directory in runtime/agents/
  3. Writing system prompt (SOUL.md or system.md)
  4. Testing with TUI

See runtime/agents/chip-quest/ for example structure.

Integrate with Channels

Connect OpenClaw to external services:

See OpenClaw docs: https://openclaw.io/docs/channels

Optimize Performance

For faster responses:

For better quality:

Resources

Documentation:

Community:

Support:

Summary

You've deployed a complete AI assistant on your TT-QuietBox 2:

Test it now:

cd ~/openclaw && ./openclaw.sh tui

Ask: "What is TT-QuietBox 2?" or "How do I deploy vLLM?" and watch it respond with citations!

Your TT-QuietBox 2 is now an expert on Tenstorrent hardware, deployment, and programming. 🎉