tt-inference-server

official

Python · Apache-2.0 · 69⭐ ·

Production-ready model serving for Tenstorrent hardware with OpenAI-compatible REST API. Supports continuous batching, multiple models, and all TT hardware configurations.

Links

📦 Repo 📖 Production Inference lesson (VSCode Toolkit)

Releases

LATEST v0.20.0 2026-08-14T14:11:44Z Release notes ↗

⬇ v0.20.0-release_artifacts.zip

4 previous releases

v0.19.0 2026-07-24T16:58:16Z

v0.18.0 2026-07-10T15:21:25Z

v0.17.0 2026-06-26T19:29:31Z

v0.16.0 2026-06-12T18:21:42Z

See all releases on GitHub ↗

serving openai-compatible production rest-api

Works on

wormhole blackhole quietbox galaxy