Models

VLM

Models of this kind. A tt-inference-server model's kind is its model type. A tt-model bundle's kind is the task tag on its card.

Community models are tt-model bundles published by their owners and are not in tt-inference-server's release spec. Tenstorrent has not reviewed them. Serve at your own risk.

7 models
VLMOfficial

olmOCR-2-7B-1025

allenai/olmOCR-2-7B-1025

Listed in tt-inference-server on 1 hardware type; best status Complete.

Runs on
WH LoudBox/QuietBox
Speed
≥ 22 tok/s≤ 212 ms WH LoudBox/QuietBox
Status
Complete
tt launch
2 clients
7B parameters
VLMOfficial

Qwen2.5-VL-7B-Instruct

Qwen/Qwen2.5-VL-7B-Instruct

Listed in tt-inference-server on 2 hardware types; best status Complete.

Runs on
WH Galaxy, WH LoudBox/QuietBox
Speed
≥ 22 tok/s≤ 100 ms WH Galaxy
Status
Complete
tt launch
7 clients
7B parameters
VLMOfficial

Qwen2.5-VL-72B-Instruct

Qwen/Qwen2.5-VL-72B-Instruct

Listed in tt-inference-server on 1 hardware type; best status Functional.

Runs on
WH LoudBox/QuietBox
Speed
≥ 2.0 tok/s≤ 3.1 s WH LoudBox/QuietBox
Status
Functional
tt launch
7 clients
72B parameters
VLMOfficial

Qwen3-VL-32B-Instruct

Qwen/Qwen3-VL-32B-Instruct

Listed in tt-inference-server on 1 hardware type; best status Functional.

Runs on
WH LoudBox/QuietBox
Speed
≥ 3.8 tok/s≤ 1.5 s WH LoudBox/QuietBox
Status
Functional
tt launch
2 clients
32B parameters
VLMOfficial

gemma-3-27b-it

google/gemma-3-27b-it

Listed in tt-inference-server on 1 hardware type; best status Experimental.

Runs on
WH Galaxy
Speed
≥ 4.3 tok/s≤ 760 ms WH Galaxy
Status
Experimental
tt launch
2 clients
27B parameters
VLMOfficial

Qwen2.5-VL-32B-Instruct

Qwen/Qwen2.5-VL-32B-Instruct

Listed in tt-inference-server on 1 hardware type; best status Experimental.

Runs on
WH LoudBox/QuietBox
Speed
≥ 4.2 tok/s≤ 630 ms WH LoudBox/QuietBox
Status
Experimental
tt launch
7 clients
32B parameters
VLMCommunity model

Qwen3.8-Flash-Next

Qwen/Qwen3.8-Flash-Next

Qwen3.8-Flash-Next, a 125B-parameter mixture-of-experts model with about 6B parameters active per token (48 layers mixing gated delta-net and sparse attention, 512 routed experts, top-10 plus one shared expert, a 51B-parameter n-gram…

Runs on
BH 4xP150, BH LoudBox, BH QuietBox 2
Speed
1.0 tok/s1 ms 4 chips
Status
Not in the release spec
tt launch
7 clients
5 tt-model bundles