Getting started with compiler-backed emulation

The emule backend compiles TT-Lang programs and executes their generated kernels through tt-metal and tt-emule inside Docker. Install the environment once, then run programs with ./bin/tt-lang-sim --backend=emule.

Relative command paths in this guide assume the TT-Lang checkout root as the working directory, except where a container shell is specified.

The repository provides a pinned environment through config/tt-lang-emule-stack.json. This manifest records the required compiler baseline, tt-emule revision, tt-metal revision, base image, and hardware profiles. Installation builds the current TT-Lang checkout, which must contain that compiler baseline.

Host prerequisites

• Git and Python 3.10 or newer on PATH.

• A Docker-compatible daemon, running and accessible without sudo. The Docker installation guide covers Docker Desktop on macOS and Docker Engine on Linux.

• Support for linux/amd64 containers: tt-emule JITs x86-64 shared objects. On Apple Silicon, see Docker Desktop’s virtualization and Rosetta settings for x86-64 emulation support and acceleration.

• The approved tt-emule repository URL and Git access to that repository. The installer uses the host’s existing Git credentials.

Check the host tools before installation:

git --version
python3 --version
docker info

Obtain TT-Lang

Start from a TT-Lang source checkout containing the compiler-backed backend:

git clone https://github.com/tenstorrent/tt-lang.git
cd tt-lang

The installer uses the container’s prebuilt LLVM toolchain and pinned tt-metal source, so the checkout’s LLVM and tt-metal submodules can remain uninitialized. Keep the TT-Lang Git history available for the compiler baseline ancestry check.

The installer validates that the checkout contains the compiler baseline recorded in the stack manifest. It also fetches the exact tt-emule revision and verifies that tt-emule pins the same tt-metal revision as the manifest.

Install the environment

Set the emulator repository location supplied by the tt-emule owners, then run the installer from the checkout root:

export TTLANG_EMULE_RUNTIME_SOURCE_URL=REPOSITORY_URL
./scripts/install-tt-lang-emule.sh

The URL supplies the source for the manifest’s pinned emulator commit. The installer uses the recorded emulator, tt-metal, and base image together in a Linux/x86-64 environment and verifies their source revisions before building.

Maintainers can evaluate other combinations with the candidate manifests and runtime overrides described in Updating the supported stack.

Installation builds the pinned tt-emule/tt-metal Docker image and compiles this TT-Lang checkout into a persistent Docker volume. It can take substantial time, CPU, memory, and disk space on its first run. The installer prints the runtime image, compiler build volume, and runtime cache volume names. On success it also prints the compiler commit for which the environment was installed.

Installation prepares the compiler and runtime before the first program run. Selecting another target reuses the runtime image and compiler build. Default runtime cache volumes are separate for each target. Subsequent tt-lang-sim runs reuse the installed environment, which Docker retains in its image and volume caches. Run the installer again after changing compiler commits or build inputs, or to restore a missing image or incomplete compiler environment. The launcher checks the installation before executing a program and reports when reinstallation is needed. The compiler build and the tt-metal and tt-emule JIT caches live in named Docker volumes.

Image contents and disk usage

The image uses a separate build stage to prepare tt-metal and tt-emule. Its final stage contains one toolchain Python environment, the LLVM SDK, Clang, and the emulator runtime’s shared libraries, headers, JIT sources, and firmware. The LLVM development libraries support compiling the current TT-Lang checkout; pytest and lit support the existing compiler test commands. Documentation, lint, and hardware-management tools remain available in the development images.

Image builds install the compiler and test dependencies from requirements-test.txt, including its nested build/runtime requirements. The image retains those requirement files under /opt/tt-emule-runtime/requirements. Installation and program execution verify that they match the checkout, so the compiler uses the image’s prepared Python environment. Requirement changes select a new runtime image identity.

The toolchain’s src/python directory supplies installed MLIR Python resources referenced by its CMake package exports. These files support building TT-Lang’s Python bindings. The virtual environment links to the final image’s system Python interpreter. Both Ninja and make support the existing tests, including packaging probes that use CMake’s default Makefiles generator.

The Docker build context is limited to .github/containers; the pinned emulator sources, stack manifest, and Python requirement files use separate named contexts. Local compiler and dependency source checkouts remain outside the image build context.

Disk budgeting includes the final image, compiler build volume, and JIT cache volume. Building the image locally also requires space for the larger build stage and Docker’s build cache. CI workers can consume a prebuilt image to avoid that build-stage cost. Docker’s compressed download size and unpacked image size measure different parts of this budget.

Run a program

After installation:

./bin/tt-lang-sim --backend=emule examples/eltwise_add.py

The program imports the real ttl and ttnn packages from the installed environment. The compiler generates kernels and tt-emule executes them on the default emulated P150 target.

Select emulated hardware

The emule backend accepts --target=NAME or --target NAME before the program argument separator (--). Available profiles are recorded in the stack manifest:

Target	Emulated hardware	Devices
p150 (default)	Blackhole P150, unharvested (13 x 10 compute grid)	1
p100	Blackhole P100 (11 x 10 compute grid)	1
p150-harvested	Blackhole P150, harvested (11 x 10 compute grid)	1
p300	Blackhole P300, both chips accessible over PCIe	2
p150x4	Blackhole 4 x P150 mesh	4
p150x8	Blackhole LoudBox, harvested	8
p150x8-unharvested	Blackhole LoudBox, unharvested	8
galaxy	Blackhole Galaxy, unharvested single-tray torus	32
n150	Wormhole N150	1
n300	Wormhole N300	2
q1	Quasar Q1 (experimental; runtime and compiler support pending)	1

These entries cover all 11 YAML descriptors shipped in the pinned emulator’s cluster_descriptors directory. Harvesting disables a subset of chip resources; the harvested and unharvested entries select the corresponding resource layouts.

For example, run the same program on P100:

./bin/tt-lang-sim --backend=emule --target=p100 examples/eltwise_add.py

Target selection configures the cluster descriptor and, where defined, the mesh-device setting together. Q1 uses the descriptor without a mesh-device override. Compiler, tt-metal, and tt-emule revisions remain pinned. Programs must fit the selected devices’ resources and open the devices required by the workload; selecting a multi-device profile does not distribute a single-device program automatically. This option selects hardware for the emule backend; Python simulation uses its existing grid and memory options.

Profile selection and workload support are separate. The 2026-09-28 smoke checks at checkout 6bdcb630 passed device opening, examples/eltwise_add.py, and examples/compiler_only_external_call.py on all ten Blackhole/Wormhole profiles. The examples execute on device 0, including on multi-device profiles. Separate mesh-opening checks passed for P300, P150x4, both P150x8 profiles, and N300. Galaxy testing covered device 0 only; full 32-device opening was skipped because its memory requirements exceeded the local Docker VM’s memory. These checks do not qualify collective operations or distributed workloads.

The pinned runtime currently rejects Q1 device opening: its 4 MiB worker L1 exceeds the emulator memory pool’s 2 MiB slot size. Q1 also requires TT-Lang compiler support; the current compiler supports Blackhole and Wormhole targets.

Arguments belonging to the program follow --:

./bin/tt-lang-sim --backend=emule program.py -- --program-option value

The launcher mounts the checkout and the current working directory. Use relative paths for program inputs beneath the current working directory. Absolute paths are passed unchanged and must refer to locations visible inside the container.

The launcher can also run from another working directory through its absolute path. In this example, program.py is relative to the current directory:

/path/to/tt-lang/bin/tt-lang-sim --backend=emule program.py

To run programs directly on the host with the default Python backend, first install and activate the environment described in Python backend setup. The emule installer prepares the container environment; native Python simulation uses its own host dependencies. From the activated host environment:

./bin/tt-lang-sim program.py
./bin/tt-lang-sim --backend=python program.py

Run tests with the existing test framework

Select tests and generate reports with TT-Lang’s CMake, pytest, and lit commands from an activated compiler build environment.

Enter the installed Docker environment

Open a shell with the same installed compiler, runtime settings, mounts, and working directory used by the emulator launcher:

./scripts/shell-tt-lang-emule.sh

Pass --target=NAME to this helper to select any profile in the table above. For example, --target=n150 selects the Wormhole N150 environment. Omitting the option selects P150, matching the simulator launcher.

The helper verifies the installed compiler and activates its environment before starting Bash. The compiler build is available at /ttlang-build, and the source checkout at /workspace. Linked worktrees use the same Git metadata mount as program execution.

Select tests

Inside the activated container shell, use the commands in Testing or the detailed test/TESTING.md guide. Those references cover full suites, individual cases, pytest filtering, lit, and report locations. Apply the installed environment’s paths:

• Use /ttlang-build wherever the testing instructions use build.

• For direct pytest invocations, pass -c /ttlang-build/test/pytest.ini to load the installed build’s generated configuration.

• Select Python lit cases under /ttlang-build/test/python, which contains their configured test environment.

A separate native Linux build uses its own build directory. Compiler-only tests exercise compiler behavior; device execution tests exercise tt-emule in the installed Linux environment. Compiler suite on tt-emule records a historical full-suite result and failure triage at the recorded revisions. Validate later revisions with a fresh test run.

Validate and inspect the environment

The installer validates the current TT-Lang checkout against the manifest’s compiler baseline. It also verifies the emulator checkout commit, the default descriptor, and the emulator’s exact tt-metal pin before building. Run the same checks directly with:

python3 scripts/tt-lang-emule-stack.py \
  --manifest config/tt-lang-emule-stack.json \
  validate --compiler-source . --emulator-source /path/to/emulator

Place --target=p100 before validate to validate the P100 descriptor instead.

Every built image records its resolved inputs and default target as OCI labels and in /opt/tt-emule-runtime/stack.json. The original supported-stack manifest is stored beside it as source-manifest.json, and its SHA-256 is verified while the image is built. These records identify the supported manifest and exact runtime inputs used to build the image. Before installation, program execution, or shell access, the launcher checks the image’s provenance labels against the selected manifest and runtime inputs. This check also applies to images selected with TTLANG_EMULE_IMAGE. The container runs by its verified immutable image ID, so changing an image tag after validation does not select a different runtime. An image with missing or mismatched provenance requires a matching image or a rebuild with TTLANG_EMULE_REBUILD=1 ./scripts/install-tt-lang-emule.sh.

Inspect an artifact without running a workload with:

docker image inspect tt-lang-emule:TAG \
  --format '{{json .Config.Labels}}'
docker run --rm --entrypoint cat tt-lang-emule:TAG \
  /opt/tt-emule-runtime/stack.json

Known limitations

The launcher selects the profile’s descriptor and configures tt-metal’s hybrid allocator before opening devices. Selecting a profile establishes the requested hardware configuration, not workload compatibility. Complete models and multi-device workloads require further validation, and Quasar kernels require compiler support beyond this change.

Known compiler-suite failures with the pinned runtime include RISC-V inline assembly rejected by the x86 JIT, a missing RMSNorm SFPU header, incorrect results in some dynamic-buffer reuse and collective tests, and missing DPRINT output. The incorrect-output and DPRINT failures still require isolated reproducers; they are not all established tt-emule defects. Device execution tests provide the evidence for emulated execution; compiler-only tests cover the compiler itself.

Troubleshooting

Docker is unavailable

Check the daemon and selected context:

docker info
docker context show

The environment is not installed

Run the installer from the same checkout:

./scripts/install-tt-lang-emule.sh

The installer prepares the image and compiler build for subsequent program runs.

The compiler checkout changed

The installed compiler is tied to the exact TT-Lang source used during installation. Re-run the installer after switching branches, pulling new commits, or changing compiler/build inputs. The installed compiler can be reused while editing workload scripts and generating output files.

The image’s Python requirements differ from the checkout

Run the installer to build an image for the checkout’s current requirements. For an explicitly selected image tag, request a rebuild:

TTLANG_EMULE_REBUILD=1 ./scripts/install-tt-lang-emule.sh

The rebuild installs the matching compiler/test dependencies into the image and prepares the compiler volume. The configured emulator source location is used again for the rebuild.

Emulator source access fails

Confirm that TTLANG_EMULE_RUNTIME_SOURCE_URL names the approved repository and that the host can read it. The installer uses the host’s Git credentials to fetch the manifest’s exact emulator commit.