Getting Started
This document walks you through how to set up to run models using tt-torch. The following topics are covered:
Building tt-torch from Source
System Dependencies
tt-torch has the following system dependencies:
- Ubuntu 22.04
- Python 3.10
- python3.10-venv
- Clang 17
- GCC 11
- Ninja
- CMake 3.20 or higher
sudo apt install clang cmake ninja-build pip python3.10-venv
Configuring Hardware
This walkthrough assumes you are using Ubuntu 22.04.
Configure your hardware with tt-installer:
- Make sure your system is up-to-date:
sudo apt-get update
sudo apt-get upgrade -y
- Set up your hardware and dependencies using tt-installer:
/bin/bash -c "$(curl -fsSL https://github.com/tenstorrent/tt-installer/releases/latest/download/install.sh)"
Installing Dependencies
Install additional dependencies that were not installed by the tt-installer script:
sudo apt-get install -y \
libhwloc-dev \
libtbb-dev \
libcapstone-dev \
pkg-config \
linux-tools-generic \
ninja-build \
libgtest-dev \
ccache \
doxygen \
graphviz \
patchelf \
libyaml-cpp-dev \
libboost-all-dev \
lcov
Install OpenMPI:
sudo wget -q https://github.com/dmakoviichuk-tt/mpi-ulfm/releases/download/v5.0.7-ulfm/openmpi-ulfm_5.0.7-1_amd64.deb -O /tmp/openmpi-ulfm.deb && sudo apt install /tmp/openmpi-ulfm.deb
Installing CMake 4.0.2
Install CMake 4.0.2:
pip install cmake
Installing Clang 17
This section walks you through installing Clang 17.
- Install Clang 17:
wget https://apt.llvm.org/llvm.sh
chmod u+x llvm.sh
sudo ./llvm.sh 17
sudo apt install -y libc++-17-dev libc++abi-17-dev
sudo ln -s /usr/bin/clang-17 /usr/bin/clang
sudo ln -s /usr/bin/clang++-17 /usr/bin/clang++
- Check that the selected GCC candidate using Clang 17 is using 11:
clang -v
Look for the line that starts with: Selected GCC installation:
. If it is something other than GCC 11, please uninstall that and install GCC 11 using:
sudo apt-get install gcc-11 lib32stdc++-11-dev lib32gcc-11-dev
You do not need to uninstall other versions of GCC. Instead, you can use update-alternatives
to configure the system to prefer GCC 11:
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 100
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-11 100
This approach lets multiple GCC versions coexist on your system and you can switch between them as needed.
- Delete any non-11 paths:
sudo rm -rf /usr/bin/../lib/gcc/x86_64-linux-gnu/12
How to build from source
This section describes how to build tt-torch. You need to build tt-torch whether you plan to do development work, or run models.
- Clone the tt-torch repo:
git clone https://github.com/tenstorrent/tt-torch.git
cd tt-torch
- Create a toolchain directory and make the account you are using the owner:
sudo mkdir -p /opt/ttmlir-toolchain
sudo chown -R $USER /opt/ttmlir-toolchain
- Build the toolchain for tt-torch (this build step only needs to be done once):
cd third_party
cmake -B toolchain -DBUILD_TOOLCHAIN=ON
NOTE: This step takes a long time to complete.
-
Navigate back to the tt-torch home directory.
-
Build tt-torch:
source env/activate
cmake -G Ninja -B build
cmake --build build
cmake --install build
NOTE: It takes a while for everything to build.
Building tt-torch From Wheel
Wheel Pre-requisites
Please follow Configuring Hardware and install OpenMPI:
sudo wget -q https://github.com/dmakoviichuk-tt/mpi-ulfm/releases/download/v5.0.7-ulfm/openmpi-ulfm_5.0.7-1_amd64.deb -O /tmp/openmpi-ulfm.deb && sudo apt install /tmp/openmpi-ulfm.deb
How to Build from Wheel
- (Optional) Create a virtual environment
We recommend using a virtual environment.
VENV_DIR=./venv
python3 -m venv $VENV_DIR
source $VENV_DIR/bin/activate
- Install the wheel
You can find available wheel releases under https://pypi.eng.aws.tenstorrent.com/tt-torch/
pip install --pre --extra-index-url https://pypi.eng.aws.tenstorrent.com/ --upgrade tt_torch
Next Steps
Please follow Testing