tt-inference-server

official

Python · Apache-2.0 · 75⭐ ·

Production-ready model serving for Tenstorrent hardware with OpenAI-compatible REST API. Supports continuous batching, multiple models, and all TT hardware configurations.

Links

📦 Repo 📖 Production Inference lesson (VSCode Toolkit)

Releases

LATEST v0.24.0 2026-10-06T19:13:47Z Release notes ↗

⬇ v0.24.0-release_artifacts.zip

4 previous releases

v0.23.0 2026-09-30T11:47:26Z

v0.22.0 2026-09-15T11:31:30Z

v0.21.0 2026-08-31T17:25:03Z

v0.20.0 2026-08-14T14:11:44Z

See all releases on GitHub ↗

serving openai-compatible production rest-api

Works on

wormhole blackhole quietbox galaxy