TT-Buda Installation

Overview

The TT-Buda software stack can compile models from several different frameworks and execute them in many different ways on Tenstorrent hardware.

This user guide is intended to help you setup your system with the appropriate device drivers, firmware, system dependencies, and compiler / runtime software.

Note on terminology:

While TT-Buda is the official Tenstorrent AI/ML compiler stack, PyBuda is the Python interface for TT-Buda. TT-Buda is the core technology; however, PyBuda allows users to access and utilize TT-Buda’s features directly from Python. This includes directly importing model architectures and weights from PyTorch, TensorFlow, ONNX, and TFLite.

OS Compatibility

Currently, Tenstorrent software is supported on Ubuntu 20.04 LTS (Focal Fossa) and Ubuntu 22.04 LTS (Jammy Jellyfish) operating systems.

Release Versions

To access the PyBuda software and associated files, please navigate to the Releases section of this repository.

Once you have identified the release version you would like to install, you can download the individual files by clicking on their name.

Table Of Contents

Installation Instructions
PyBuda Installation
1. Python Environment Installation
2. Docker Container Installation
Tests
1. Smoke Test

Installation Instructions

PyBuda can be installed using two methods: Docker or Python virtualenv.

If you would like to run PyBuda in a Docker container, then follow the instructions for PCI Driver Installation and Device Firmware Update and followed by Docker Container Installation.

If you would like to run PyBuda in a Python virtualenv, then follow the instructions for the Setup HugePages, PCI Driver Installation, Device Firmware Update, and Backend Compiler Dependencies, followed by the Python Environment Installation.

Setup HugePages

Download latest setup_hugepages.py script.

wget https://raw.githubusercontent.com/tenstorrent/tt-metal/main/infra/machine_setup/scripts/setup_hugepages.py

Run first setup script.

sudo -E python3 setup_hugepages.py first_pass

Reboot
```
sudo reboot now
```

Run second setup script & check setup.

sudo -E python3 setup_hugepages.py enable && sudo -E python3 setup_hugepages.py check

PCI Driver Installation

Please navigate to tt-kmd homepage and follow instructions within the README.

Pro-Tip: ensure that you are within the home directory of the local clone version of tt-kmd when performing the installation steps

Device Firmware Update

The tt-firmware file needs to be installed using the tt-flash utility, for more details visit TT-Flash homepage and follow instructions within the README.

Backend Compiler Dependencies

Instructions to install the Tenstorrent backend compiler dependencies on a fresh install of Ubuntu Server 20.04 or Ubuntu Server 22.04.

You may need to append each apt command with sudo if you do not have root permissions.

For both operating systems run the following commands:

apt update -y
apt upgrade -y --no-install-recommends
apt install -y build-essential curl libboost-all-dev libgl1-mesa-glx libgoogle-glog-dev libhdf5-serial-dev ruby software-properties-common libzmq3-dev clang wget python3-pip python-is-python3 python3-venv

For Ubuntu 20.04, add:

apt install -y libyaml-cpp-dev

For Ubuntu 22.04, add:

wget http://mirrors.kernel.org/ubuntu/pool/main/y/yaml-cpp/libyaml-cpp-dev_0.6.2-4ubuntu1_amd64.deb
wget http://mirrors.kernel.org/ubuntu/pool/main/y/yaml-cpp/libyaml-cpp0.6_0.6.2-4ubuntu1_amd64.deb
dpkg -i libyaml-cpp-dev_0.6.2-4ubuntu1_amd64.deb libyaml-cpp0.6_0.6.2-4ubuntu1_amd64.deb
rm libyaml-cpp-dev_0.6.2-4ubuntu1_amd64.deb libyaml-cpp0.6_0.6.2-4ubuntu1_amd64.deb

TT-SMI

Please navigate to tt-smi homepage and follow instructions within the README.

PyBuda Installation

There are two ways to install PyBuda within the host environment: using Python virtual environment or Docker container.

Python Environment Installation

It is strongly recommended to use virtual environments for each project utilizing PyBuda and Python dependencies. Creating a new virtual environment with PyBuda and libraries is very easy.

Step 1. Navigate to Releases 

Step 2. Find the latest release package

Scroll to find the latest release package in .zip format under “Assets” that corresponds to your device and operating system

Step 3. Download the package and unzip

Download the .zip package and unzip to find the pybuda, tvm and torchvison wheel files

Step 4. Create your Python environment in desired directory

python3 -m venv env

Step 5. Activate environment

source env/bin/activate

Step 5. Pip install PyBuda, TVM and Torchvision whl files

pip install --upgrade pip==24.0
pip install pybuda-<version>.whl tvm-<version>.whl torchvision-<version>.whl

The pybuda-<version>.whl file contains the PyBuda library, the tvm-<version>.whl file contains the latest TVM downloaded release, and the torchvision-<version>.whl file bundles the torchvision library.

Step 6. Pip install Debuda (Optional Step)

For enhanced debugging capabilities, you may opt to install the Debuda library:

pip install debuda-<version>.whl

This wheel file installs the Debuda tool designed for debugging purposes.

Docker Container Installation

Alternatively, PyBuda and its dependencies are provided as Docker images which can run in separate containers. The Docker containers can be found under: https://github.com/orgs/tenstorrent/packages?repo_name=tt-buda

Step 1. Pull the docker image

To pull the Docker image, use the following command:

sudo docker pull ghcr.io/tenstorrent/tt-buda/<OS-VERSION>/<TT-DEVICE>:<TAG>

Supported OS <OS-VERSION> Versions:

ubuntu-20-04-amd64
ubuntu-22-04-amd64

Supported Tenstorrent <TT-DEVICE> Devices:

gs
wh_b0

For example, to run on an Ubuntu version 20.04 on a Grayskull device, use this command:

sudo docker pull ghcr.io/tenstorrent/tt-buda/ubuntu-20-04-amd64/gs:<TAG>

where <TAG> is the version number i.e. v0.12.3.

Step 2. Run the container

sudo docker run --rm -ti --cap-add=sys_nice --shm-size=4g --device /dev/tenstorrent -v /dev/hugepages-1G:/dev/hugepages-1G -v $(pwd)/:/home/ ghcr.io/tenstorrent/tt-buda/<OS-VERSION>/<TT-DEVICE>:<TAG> bash

Step 3. Change root directory

cd home/

Tests

Verify the correct installation of the PyBuda library and environment by conducting a smoke test.

Smoke Test

With your Python environment with PyBuda install activated, run the following Python script:

import pybuda
import torch


# Sample PyTorch module
class PyTorchTestModule(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.weights1 = torch.nn.Parameter(torch.rand(32, 32), requires_grad=True)
        self.weights2 = torch.nn.Parameter(torch.rand(32, 32), requires_grad=True)
    def forward(self, act1, act2):
        m1 = torch.matmul(act1, self.weights1)
        m2 = torch.matmul(act2, self.weights2)
        return m1 + m2, m1


def test_module_direct_pytorch():
    input1 = torch.rand(4, 32, 32)
    input2 = torch.rand(4, 32, 32)
    # Run single inference pass on a PyTorch module, using a wrapper to convert to PyBuda first
    output = pybuda.PyTorchModule("direct_pt", PyTorchTestModule()).run(input1, input2)
    print(output)
    print("PyBuda installation was a success!")


if __name__ == "__main__":
    test_module_direct_pytorch()