Open in tt-awesome →

vllm-tt-plugin

official
Python · Apache-2.0 · 4⭐ ·

Tenstorrent backend for vLLM, built on vLLM's standard plugin mechanism — install it alongside vLLM and TT hardware registers itself as a platform whenever `ttnn` is importable. Self-contained: model registration, platform detection, scheduling, worker execution, model loading, async decode, and data-parallel/multi-lane execution all live in the plugin, so nothing Tenstorrent-specific has to land in vLLM core.

📦 Repo
vllm serving inference llm openai-compatible plugin
wormhole blackhole