tt-inference-server

official

Python · Apache-2.0 · 66⭐ ·

Production-ready model serving for Tenstorrent hardware with OpenAI-compatible REST API. Supports continuous batching, multiple models, and all TT hardware configurations.

Links

📦 Repo 📖 Production Inference lesson (VSCode Toolkit)

Releases

LATEST v0.18.0 2026-07-10T15:21:25Z Release notes ↗

⬇ v0.18.0-release_artifacts.zip

4 previous releases

v0.17.0 2026-06-26T19:29:31Z

v0.16.0 2026-06-12T18:21:42Z

v0.15.0 2026-05-29T15:55:11Z

v0.14.0 2026-05-15T22:34:02Z

See all releases on GitHub ↗

serving openai-compatible production rest-api

Works on

wormhole blackhole quietbox galaxy