Models
VLM OfficialQwen family

Qwen2.5-VL-7B-Instruct

Listed in tt-inference-server on 2 hardware types; best status Complete.

In tt-inference-server

HardwareEngine and implementationStatusContextUsersBuildSupport page
WH Galaxy
GALAXY
vLLM
tt-transformers
Complete 128K 128
tt-metal 09acccb
vLLM 6098002
Image ghcr.io/tenstorrent/tt-inference-server/vllm-tt-metal-src-release-ubuntu-22.04-amd64:0.26.0-09acccb-6098002
Open
WH LoudBox/QuietBox
T3K
vLLM
tt-transformers
Complete 128K 32
tt-metal 09acccb
vLLM 6098002
Image ghcr.io/tenstorrent/tt-inference-server/vllm-tt-metal-src-release-ubuntu-22.04-amd64:0.26.0-09acccb-6098002
Open

Status, context and users are copied from tt-inference-server's release spec (commit 0ad0f4be9, release 0.26.1). Context is the longest prompt plus reply the entry allows. Users is its highest concurrency. Each row carries its own code link, tt-metal and vLLM commits and image, because they differ between hardware. Nothing on this page is measured.

tt-model bundles

No tt-model bundle has been published for this model.

Details

KindVLM (from tt-inference-server model type)
Parameters7B
FamilyQwen (from the start of the name)
Implementationtt-transformers
Disk needed34 GB
RAM needed17.5 GB
Tool-call parserhermes
Listed bytt-inference-server