gemma-4-31B-it
Listed in tt-inference-server on 2 hardware types; best status Complete.
In tt-inference-server
| Hardware | Engine and implementation | Status | Context | Users | Build | Support page |
|---|---|---|---|---|---|---|
BH GalaxyBLACKHOLE_GALAXY |
vLLM tt-transformers |
Complete | 256K | 128 | tt-metal 978f93bvLLM a723030Image ghcr.io/tenstorrent/tt-inference-server/vllm-tt-metal-src-release-ubuntu-22.04-amd64:0.25.0-978f93b-a723030 |
Open |
BH QuietBox 2P300X2 |
vLLM gemma4-31b-qb2 |
Complete | 256K | 1 | tt-metal 2fd9445vLLM 7250ddfImage ghcr.io/tenstorrent/tt-inference-server/vllm-tt-metal-src-release-ubuntu-22.04-amd64:0.23.0-2fd9445-7250ddf |
Open |
BH QuietBox 2P300X2 |
vLLM tt-transformers |
Complete | 256K | 32 | tt-metal 978f93bvLLM a723030Image ghcr.io/tenstorrent/tt-inference-server/vllm-tt-metal-src-release-ubuntu-22.04-amd64:0.25.0-978f93b-a723030 |
Open |
Status, context and users are copied from tt-inference-server's release spec (commit 2db98e38c, release 0.25.0). Context is the longest prompt plus reply the entry allows. Users is its highest concurrency. Each row carries its own code link, tt-metal and vLLM commits and image, because they differ between hardware. Nothing on this page is measured.
tt-model bundles
No tt-model bundle has been published for this model.
Details
| Kind | LLM (from tt-inference-server model type) |
| Parameters | 31B |
| Family | gemma (from the start of the name) |
| Implementation | gemma4-31b-qb2, tt-transformers |
| Disk needed | 82 GB |
| RAM needed | 77.5 GB |
| Tool-call parser | gemma4 |
| Reasoning parser | gemma4 |
| Listed by | tt-inference-server |