tt-inference-server

official

Python · Apache-2.0 · 73⭐ ·

Production-ready model serving for Tenstorrent hardware with OpenAI-compatible REST API. Supports continuous batching, multiple models, and all TT hardware configurations.

Links

📦 Repo 📖 Production Inference lesson (VSCode Toolkit)

Releases

LATEST v0.22.0 2026-09-15T11:31:30Z Release notes ↗

⬇ v0.22.0-release_artifacts.zip

4 previous releases

v0.21.0 2026-08-31T17:25:03Z

v0.20.0 2026-08-14T14:11:44Z

v0.19.0 2026-07-24T16:58:16Z

v0.18.0 2026-07-10T15:21:25Z

See all releases on GitHub ↗

serving openai-compatible production rest-api

Works on

wormhole blackhole quietbox galaxy