n150 n300 T3000 p100 p150 p300c Galaxy 30 min Validated

Exploring the TT-Metalium™ Playground

Welcome to the heart of Tenstorrent development! In this lesson you'll discover what's possible with TT-Metalium and TT-NN™, run real hardware code in minutes, and understand the architecture that makes it all tick.

What You'll Do

Before You Start: Run This Right Now

If you have TT-Metalium built and your venv activated, you can be running real TT-NN code in 60 seconds. No Jupyter, no setup — just Python:

# Activate the tt-metal Python environment
source ~/tt-metal/python_env/bin/activate
export TT_METAL_HOME=~/tt-metal
export PYTHONPATH=$TT_METAL_HOME:$PYTHONPATH

# Run the first tutorial — adds two tensors on TT hardware
python3 ~/tt-metal/ttnn/tutorials/basic_python/ttnn_add_tensors.py

⚡ Sim-ready: The ttnn/tutorials/basic_python/ scripts all use ttnn.open_device(device_id=0) and run on the ttsim simulator. Add export TT_METAL_SIMULATOR=~/sim/libttsim_wh.so before the commands above to run without hardware.

You'll see the device open, the computation run, and the device close. That's real silicon doing real work (or the simulator, if you set TT_METAL_SIMULATOR). The full tutorial collection lives at:

~/tt-metal/ttnn/tutorials/basic_python/

No Jupyter required — every notebook also has a .py companion you can run directly.

Don't have ~/tt-metal built yet? Start with Build TT-Metalium from Source first, then return here.

Why This Hardware is Different

Before diving in, here's what makes Tenstorrent hardware worth exploring:

Wormhole™ n150 (single chip, 8 TOPS):

Tenstorrent Galaxy (32 Wormhole chips, 256 TOPS):

The same TT-NN Python code runs on all of these. You write for n150, scale to Galaxy by changing a device count. That's the architecture advantage this lesson explores.

Part 1: Run the Tutorial Scripts

The Quickest Path: basic_python Scripts

Every TT-NN concept has a runnable Python script. These are the best starting point because they don't require Jupyter and have clear, commented code:

cd ~/tt-metal
source python_env/bin/activate

# Tensor basics: create, fill, add on device
python3 ttnn/tutorials/basic_python/ttnn_add_tensors.py

# Core operations: element-wise, reductions, broadcasting
python3 ttnn/tutorials/basic_python/ttnn_basic_operations.py

# Matrix multiplication: the workhorse of neural nets
python3 ttnn/tutorials/basic_python/ttnn_basic_matrix_multiplication.py

# 2D convolution on TT hardware
python3 ttnn/tutorials/basic_python/ttnn_basic_conv.py

# Full inference pipeline: MLP on MNIST
# ⚠️  Train weights first (CPU-only, ~1 min): saves mlp_mnist_weights.pt
python3 ttnn/tutorials/basic_python/train_and_export_mlp.py
python3 ttnn/tutorials/basic_python/ttnn_mlp_inference_mnist.py

# Transformer building block: multi-head attention
python3 ttnn/tutorials/basic_python/ttnn_multihead_attention.py

# CNN inference end-to-end
# ⚠️  Train weights first: saves simplecnn_mnist_weights.pt
python3 ttnn/tutorials/basic_python/train_and_export_cnn.py
python3 ttnn/tutorials/basic_python/ttnn_simplecnn_inference.py

Training step required: ttnn_mlp_inference_mnist.py and ttnn_simplecnn_inference.py load weights from .pt files. Without them the scripts use random weights and report ~20% accuracy. Run the corresponding train_and_export_*.py first — CPU-only, ~1 minute each.

Recommended order: ttnn_add_tensors → ttnn_basic_operations → ttnn_basic_matrix_multiplication → train_and_export_mlp → ttnn_mlp_inference_mnist.

Jupyter Notebooks

If you prefer interactive Jupyter notebooks, the same content is available as .ipynb files in the same directory:

~/tt-metal/ttnn/tutorials/

📓 Open TT-NN Tutorials

Available notebooks:

Part 2: The Model Zoo — What Runs Today

Tenstorrent's model repository is one of the most extensive collections of hardware-optimized AI models available. Here's what you can run right now:

🔍 Browse Model Zoo

Production-Ready (models/demos/)

Language Models:

Vision Models:

Audio:

Experimental (models/experimental/)

Hardware-Organized Demos

Models are organized by target hardware for easy discovery:

models/demos/wormhole/   — n150/n300 optimized
models/demos/t3000/      — T3000 (8-chip) configurations
models/demos/blackhole/  — p100/p300c (Blackhole<sup>®</sup>)
models/demos/tg/         — Galaxy (32-chip)

🎯 What's possible:

  1. Run a 685B parameter model — DeepSeek-V3 on Galaxy
  2. 128K context windows — Read entire books as context
  3. Real-time object detection — YOLO v12 on n150
  4. Train models on device — nanoGPT is buildable from scratch
  5. Native video generation — Mochi and Wan 2.2 (experimental)

Part 3: Understanding the Architecture

The Tensix Core

Each Tenstorrent chip contains a grid of Tensix cores. Understanding their architecture helps you write efficient code.

Inside a Tensix Core:

┌─────────────────────────────────────────────────┐
│                  Tensix Core                    │
├─────────────────────────────────────────────────┤
│                                                 │
│  ┌──────────┐    ┌────────────────┐           │
│  │  5 RISC-V│───▶│  1.5 MB SRAM   │           │
│  │  "Baby"  │    │   (L1 Memory)  │           │
│  │  CPUs    │    └────────────────┘           │
│  └──────────┘            │                     │
│                           │                     │
│       ┌───────────────────┴──────────┐         │
│       │                               │         │
│  ┌────▼─────┐                  ┌─────▼────┐   │
│  │  Matrix  │                  │  Vector  │   │
│  │  Engine  │                  │  Unit    │   │
│  │  (FPU)   │                  │  (SFPU)  │   │
│  │          │                  │          │   │
│  │ 32×32    │                  │ Element- │   │
│  │ Tiles    │                  │  wise    │   │
│  └──────────┘                  └──────────┘   │
│                                                 │
│  ┌──────────────────────────────────────────┐ │
│  │  Network-on-Chip (NoC) - 2 Paths        │ │
│  │  NoC 0: Reads    NoC 1: Writes          │ │
│  └──────────────────────────────────────────┘ │
└─────────────────────────────────────────────────┘
        │                           │
        ▼                           ▼
    DRAM Banks              Other Tensix Cores

Key components:

  1. 5 RISC-V "Baby" CPUs — Control and orchestration; run your kernel code
  2. 1.5 MB L1 SRAM — Fast local memory, explicitly managed (no cache)
  3. Matrix Engine (FPU) — Hardware accelerator for 32×32 tile matmul
  4. Vector Unit (SFPU) — Element-wise ops: ReLU, GELU, Softmax, custom math
  5. Network-on-Chip (NoC) — Two independent paths; connects DRAM and cores

Tile-Based Computing

Why 32×32 tiles?

Traditional GPUs process data in linear layouts. Tenstorrent uses 32×32 tiles as the native format because it matches the Matrix Engine hardware perfectly:

import ttnn
import torch

device = ttnn.open_device(device_id=0)

# ROW_MAJOR layout (like NumPy/PyTorch)
row_major = ttnn.from_torch(
    torch.rand((3, 4)),
    layout=ttnn.ROW_MAJOR_LAYOUT,
    device=device
)
print(f"Shape: {row_major.shape}, Padded: {row_major.padded_shape}")
# Output: Shape([3, 4]), Padded: Shape([3, 4])

# TILE_LAYOUT — native format, padded to 32×32 minimum
tile = ttnn.to_layout(row_major, ttnn.TILE_LAYOUT)
print(f"Shape: {tile.shape}, Padded: {tile.padded_shape}")
# Output: Shape([3, 4]), Padded: Shape([32, 32])
# Padding added automatically to fill 32×32 tile!

ttnn.close_device(device)

Performance tip: Operations on tile-aligned shapes (multiples of 32) are fastest! Non-aligned shapes work but waste some compute on the padding.

The Three-Kernel Programming Model

Most operations use three kernels working together in a pipeline:

     Reader Kernel              Compute Kernel             Writer Kernel
     (Data Movement)            (Math Operations)         (Data Movement)
            │                          │                         │
┌───────────▼──────────┐   ┌──────────▼─────────┐   ┌──────────▼─────────┐
│  Fetch from DRAM     │──▶│  Process in SRAM   │──▶│  Store to DRAM     │
│  via NoC 0           │   │  (Matrix/Vector)   │   │  via NoC 1         │
└──────────────────────┘   └────────────────────┘   └────────────────────┘

Circular Buffers in L1 SRAM enable pipelining:
- Reader fills buffer while Compute processes previous batch
- Compute fills output buffer while Writer stores previous batch

This architecture means there is no hidden cache thrashing — every data movement is explicit. That's why profiling Metalium programs is precise: you know exactly what's moving where.

Two Levels of Abstraction

TT-NN (Python) — High Level:

import ttnn

device = ttnn.open_device(device_id=0)

a = ttnn.rand((32, 32), device=device, layout=ttnn.TILE_LAYOUT)
b = ttnn.rand((32, 32), device=device, layout=ttnn.TILE_LAYOUT)

c = ttnn.matmul(a, b)   # Matrix multiply
d = ttnn.add(c, 1.0)    # Add scalar
e = ttnn.gelu(d)        # Activation

result = ttnn.to_torch(e)
ttnn.close_device(device)

Use TT-NN for: rapid prototyping, standard model inference, Python-first development.

TT-Metalium (C++) — Low Level:

#include "tt_metal/host_api.hpp"

using namespace tt::tt_metal;

int main() {
    Device* device = CreateDevice(0);
    Program program = CreateProgram();

    // Define reader, compute, and writer kernels
    auto reader = CreateKernel(program, "kernels/reader.cpp", core,
                               DataMovementConfig{...});
    auto compute = CreateKernel(program, "kernels/compute.cpp", core,
                                ComputeConfig{...});

    EnqueueProgram(command_queue, program, false);
    Finish(command_queue);
    CloseDevice(device);
}

Use TT-Metalium for: maximum performance, custom operations, novel algorithms, research.

Part 4: Programming Examples

Build and Run Examples

The programming examples demonstrate Metalium kernels from hello world through multi-core matrix multiply. Build them with:

cd ~/tt-metal
./build_metal.sh --build-programming-examples

This takes an additional 5–10 minutes but gives you standalone executables.

Beginner:

Example What It Teaches
Hello World Compute Your first compute kernel
Hello World Data Movement Your first reader/writer kernel
Add 2 Integers Basic arithmetic on device
DRAM Loopback Buffer creation, data movement 
# Run after building with --build-programming-examples
./build/programming_examples/hello_world_compute_kernel
./build/programming_examples/hello_world_datamovement_kernel
./build/programming_examples/add_2_integers_in_compute

Intermediate:

Example What It Teaches
Eltwise Binary Element-wise ops with circular buffers
Eltwise SFPU Vector operations (SFPU math)
Matmul Single Core Using the matrix engine
Matmul Multi Core Parallel execution across cores

Hands-On: Tile Padding Experiment

Run this short script to see how TT-NN handles the 32×32 tile requirement:

cat > /tmp/tile_experiment.py << 'EOF'
import ttnn
import torch

device = ttnn.open_device(device_id=0)

cases = [(5, 5), (100, 50), (128, 128), (1024, 1024)]

for shape in cases:
    t = ttnn.from_torch(
        torch.rand(shape),
        layout=ttnn.TILE_LAYOUT,
        device=device
    )
    pad_r = t.padded_shape[-2] - shape[0]
    pad_c = t.padded_shape[-1] - shape[1]
    print(f"{shape[0]:5}×{shape[1]:<5}  →  padded {t.padded_shape[-2]}×{t.padded_shape[-1]}  "
          f"(wasted: {pad_r * t.padded_shape[-1] + pad_c * shape[0]} elements)")

ttnn.close_device(device)
print("\nRule: dimensions always pad to next multiple of 32.")
print("For best performance, design your model shapes to be multiples of 32.")
EOF
cd ~/tt-metal && python3 /tmp/tile_experiment.py

Observe:

Key Takeaways

What's Next?

In the Metalium Cookbook, you'll apply these concepts building four creative projects:

  1. Conway's Game of Life — Cellular automata with parallel tile computing
  2. Audio Processor — Real-time mel-spectrogram and effects
  3. Mandelbrot Explorer — GPU-style fractal rendering
  4. Custom Image Filters — Creative visual effects

🚀 Continue to JAX Inference with TT-XLA

Resources

Troubleshooting

ttnn.open_device() fails:

tt-smi    # Check device status
tt-smi -r # Reset if showing errors

Jupyter notebooks won't open:

code --install-extension ms-toolsai.jupyter

Out of memory:

Slow performance: