Environment Setup

This document is currently under development.

OCAH workflows need a RISC-V cross-compiler, EDA tools (Verilator, Slang, Verible, Yosys), Python tooling and documentation tools. The recommended way to get all of them is the prebuilt OCAH container: you install a container engine, and the project’s scripts/docker-run.sh downloads the exact image that CI tests against. You do not install or configure any of the tools yourself.

Follow Quick start: the prebuilt container (recommended) from top to bottom. The other setups, described under Alternative setups, are for contributors who need to change the toolchain or cannot use a container.

1. Supported platforms

Host Prebuilt container (recommended) Native installation

Linux, x86_64 (for example Ubuntu 24.04 LTS or later)

Supported. This is what CI uses.

Documented for Ubuntu 26.04 LTS. Other distributions should work, but package names differ.

macOS

Supported with Podman. On Macs with Apple silicon (M-series) Podman pulls the native arm64 image.

Not documented.

Linux, arm64

Supported. CI builds and checks the arm64 image on arm64 runners.

Not documented.

Windows

Not tested.

Not documented.

The container needs about 10 GB of free disk space for the image, on top of the general CPU, memory and storage requirements listed in Dependencies.

2. Quick start: the prebuilt container (recommended)

Do these steps once per machine, in order.

2.1. Step 1: Install Git and a container engine

The container engine runs the OCAH image. Podman is recommended; Docker also works. If both are installed, docker-run.sh uses Podman.

2.1.1. On Linux (Ubuntu)

Install Git, Podman and fuse-overlayfs, then allow short image names to resolve to Docker Hub:

sudo apt update
sudo apt install -y git podman fuse-overlayfs

sudo mkdir -p /etc/containers/registries.conf.d
cat <<'EOF' | sudo tee /etc/containers/registries.conf.d/docker.conf
unqualified-search-registries = ["docker.io"]
EOF

On other distributions, follow the Podman installation guide.

To use Docker instead, follow the Docker Engine installation guide and its post-installation steps, which let you run docker without sudo.

2.1.2. On macOS

Install Git by running xcode-select --install, then install Podman. Podman runs containers in a Linux virtual machine. The default machine has 2 GB of memory, short of Dependencies, so set --memory (in MB) to match:

podman machine init --memory 16384
podman machine start

Note that the virtual machine has a significant memory overhead, so we recommend a Mac with 32GB RAM or more to be able to comfortably develop the OCA Harness.

After a reboot, run podman machine start again before using the container.

2.2. Step 2: Get the repository

Clone the repository and initialise its submodules:

git clone https://github.com/tenstorrent/tt-oca-harness.git
cd tt-oca-harness
git submodule update --init --recursive

The submodules are required: the Python environment uses them, and docker-run.sh reads them to identify the image.

2.3. Step 3: Select the prebuilt image

Tell docker-run.sh to download the image CI publishes instead of building it. Add these two lines to your shell startup file, ~/.bashrc on Linux or ~/.zshrc on macOS:

export OCAH_CONTAINER_REGISTRY_IMAGE=ghcr.io/tenstorrent/ocah-container
export OCAH_IMAGE_WITH_UV=true

Then open a new terminal, or run source ~/.bashrc (source ~/.zshrc on macOS), so they take effect. OCAH_IMAGE_WITH_UV=true selects the image variant CI publishes, which already contains the project’s Python packages.

2.4. Step 4: Download and check the image

From the repository root, run:

./scripts/docker-run.sh verify

The first run downloads the image, which takes a while. When it finishes, it prints the version of the RISC-V compiler inside the container, beginning riscv64-unknown-elf-gcc. Seeing that version means the setup works.

Two things are normal and do not mean anything is wrong:

  • Before every command, docker-run.sh computes which image matches your checkout. Without Nix installed on your machine this takes several minutes each time, even once the image is downloaded. Optional: make container commands start faster removes the wait.

  • Later commands reuse the downloaded image. It is downloaded again only when you update to a commit that changes the toolchain.

If the output contains absent locally and in cache; building, press Ctrl+C. There is no published image for your checkout, so docker-run.sh has started building one from source, which occupies the machine for a long time. This happens when CI has not yet published the image for the commit you are on, or when your checkout changes a file that defines the image, such as flake.lock, ocah_deps.nix, uv.lock or a file under nix/. Interrupting docker-run.sh does not always stop the build, so check podman ps (or docker ps) and stop any container still running. Then check out the latest main, or wait for CI to publish the image, or use the main image meanwhile.

Published images are listed on the ocah-container package page. No login is needed to download them.

2.5. Step 5: Use the container

All container use goes through scripts/docker-run.sh, run from the repository root. To work interactively, open a shell inside the container:

./scripts/docker-run.sh shell

The prompt changes to OCAH-Container > and every OCAH tool is available, so you can run any make target the rest of this guide describes. Run make help to list them, and exit to leave the container.

To run a single command without opening a shell:

./scripts/docker-run.sh run make dv-fw-tests TARGET=smc

Some workflows, such as synthesis, write absolute paths into their files. Run those with run-here, which mounts the repository at the same path it has on your machine (shell-here is the interactive equivalent):

./scripts/docker-run.sh run-here make ocah-synth-yosys-all

2.5.1. Which image runs

The image is tagged with a hash of your checkout’s toolchain definition. Before each command, docker-run.sh takes the first of these that works:

  1. The image with that tag, if it is already downloaded.

  2. The image with that tag from the registry.

  3. The image with that tag from the image cache named by OCAH_DOCKER_CACHE_DIR, or from an earlier docker-run.sh build.

  4. With OCAH_IMAGE_FALLBACK=main set, the main image: the last image CI verified on main. It downloads the current one, or uses one downloaded earlier if it cannot, and warns that it may not match your checkout, naming the image’s digest. It falls back only when the registry has no image for your checkout, or when OCAH_CONTAINER_REGISTRY_IMAGE is unset; a failed login or an unreachable registry skips it.

  5. A build from source, the default when nothing above applies.

For a few hours after a commit changes the toolchain, CI has not published its image, so without OCAH_IMAGE_FALLBACK=main that commit is built. The main image suits any work that does not depend on the toolchain change. Once CI publishes the image, the next command downloads it.

2.5.2. Managing downloaded images

Each toolchain change downloads a new image, and the old ones stay on disk:

./scripts/docker-run.sh images      # list downloaded images, their architecture, and which is yours
./scripts/docker-run.sh refresh     # download your checkout's image again and replace the local one
./scripts/docker-run.sh prune       # remove every OCAH image except your checkout's and main's
./scripts/docker-run.sh rmi <tag>   # remove the OCAH images with that tag

An image already downloaded is used whatever its architecture, so an x86_64 image downloaded before the arm64 one was published keeps running under emulation on an arm64 machine. refresh replaces it with the native one. It keeps the old image if a running container still uses it, so that container is not affected.

See scripts/docker.md for every container command and environment variable.

2.6. Optional: make container commands start faster

The image’s tag is a hash of the toolchain definition. Without Nix on the host, docker-run.sh computes it in a temporary container that downloads the definition again every time, which is why each command waits several minutes. Installing Nix on the host keeps that download cached, so commands start in seconds. Nothing else changes: you keep using the prebuilt image exactly as above.

2.7. Troubleshooting

Failed to fetch git repository

The submodules are missing. Run git submodule update --init --recursive.

Error: workdir "…​" does not exist on container

You reached the repository through a symbolic link. Change to its real path with cd "$(pwd -P)" and run the command again.

no space left on device while downloading the image

Rootless Podman stores images under ~/.local/share/containers, which may be on a small or quota-limited home filesystem. Move the store to a larger filesystem by setting graphroot in ~/.config/containers/storage.conf, then confirm the new location with podman info | grep -i graphroot.

podman: command not found or podman or docker is required

Step 1 is incomplete, or on macOS the Podman machine is not running. Run podman machine start.

Killed signal terminated program lto1

The Podman machine is short of memory. Stop it, run podman machine set --memory 16384 and start it again. For synthesis, this error may recur, in which case you may need to allocate 32GiB (32768), or 64GiB (65536) instead.

3. Alternative setups

The setups below are for contributors who change the toolchain itself or cannot use a container. They take considerably more effort than the prebuilt container and are less tested.

3.1. Building the container image yourself

Build the image locally only when you are changing its definition. The build runs on x86_64 or arm64 Linux, builds the image for the host’s architecture, uses many cores and takes a long time:

./scripts/docker-run.sh build

With OCAH_IMAGE_WITH_UV=true it builds the variant that bundles the Python packages, for fully offline use; otherwise it builds the default variant, which installs them on demand. docker-run.sh also builds the image automatically whenever OCAH_CONTAINER_REGISTRY_IMAGE is unset and no matching image is loaded or cached. See nix/nix-infrastructure.md for how the image is defined.

3.2. Native installation (Ubuntu)

A native installation puts every tool directly on the host. It gives the most control but needs the most setup, and you are responsible for matching the versions in Dependencies. These instructions are written for Ubuntu 26.04 LTS.

3.2.1. Python

We recommend the use of pyenv to ensure the correct version of python is in use. To install pyenv, visit their GitHub page, or run curl https://pyenv.run | bash.

The piping of a script from the internet as above, without checking it, is never safe. You should always check the contents of the script first to guarantee safety. This may be done using curl https://pyenv.run, which will print the script, instead of running it.

To install Python 3.11 with pyenv, first install the python build dependencies, and then Python 3.11 itself:

# Check Pyenv GitHub for up-to-date build environment
sudo apt update
sudo apt install make build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev curl git \
libncursesw5-dev xz-utils tk-dev libxml2-dev \
libxmlsec1-dev libffi-dev liblzma-dev libzstd-dev

pyenv install 3.11

From here, enter a pyenv shell, and install uv using the following commands:

pyenv shell 3.11

# In the shell
pip install uv

3.2.2. Package Dependencies

Once the repository is cloned, install the required open source tools:

sudo apt install asciidoctor asciidoctor-pdf verilator gcc \
ccache picolibc-riscv64-unknown-elf cmake default-jre-headless

default-jre-headless provides the Java runtime that renders PlantUML diagrams in the PDF builds. The asciidoctor-diagram extension those builds require is not packaged for Ubuntu; install it as a gem:

gem install asciidoctor-diagram

Follow the instructions on the following pages for tools not packaged within Ubuntu:

  • Antora

  • Bender

  • Verible

  • Slang

  • Yosys (Also packaged via apt, but newer version recommend)

    • SV-Elab (Formerly Yosys-Slang) Required to allow Yosys to synthesise SystemVerilog

  • Surfer (optional, alternatives available, used for wave trace debugging)

  • Drawio (optional, used for creating new diagrams for documentation)

  • Wavedrom (optional, used for creating new diagrams for documentation)

Enter the repository and use the following command to set up the Python environment:

uv sync # or `make uv-sync`

3.3. Nix development shell

If you have Nix installed, the dev shell provides the same tool set as the container directly on the host, with no container engine:

nix develop    # activates the dev shell; tools are then available on PATH

With direnv configured, the shell activates automatically on cd into the repo.

See the OCA Harness Nix Readme for more information on Nix integration.

The Nix dev shell is not first-class supported for the initial release. It has been deployed on NixOS and Linux hosts with Nix installed, but is not actively tested. MacOS builds have not been attempted. The container is the recommended path.