Mistral-7B-Instruct-v0.3

Listed in tt-inference-server on 3 hardware types; best status Complete.

In tt-inference-server

Hardware	Engine and implementation	Status	Context	Users	Build	Support page
WH LoudBox/QuietBox `T3K`	vLLM tt-transformers	Complete	32K	32	Code: https://github.com/tenstorrent/tt-metal/tree/9b67e09/models/tt_transformers tt-metal `9b67e09` vLLM `a91b644` Image `ghcr.io/tenstorrent/tt-inference-server/vllm-tt-metal-src-release-ubuntu-22.04-amd64:0.2.0-9b67e09-a91b644`	Open
n150 `N150`	vLLM tt-transformers	Complete	32K	32	Code: https://github.com/tenstorrent/tt-metal/tree/9b67e09/models/tt_transformers tt-metal `9b67e09` vLLM `a91b644` Image `ghcr.io/tenstorrent/tt-inference-server/vllm-tt-metal-src-release-ubuntu-22.04-amd64:0.2.0-9b67e09-a91b644`	Open
n300 `N300`	vLLM tt-transformers	Complete	32K	32	Code: https://github.com/tenstorrent/tt-metal/tree/9b67e09/models/tt_transformers tt-metal `9b67e09` vLLM `a91b644` Image `ghcr.io/tenstorrent/tt-inference-server/vllm-tt-metal-src-release-ubuntu-22.04-amd64:0.2.0-9b67e09-a91b644`	Open

Status, context and users are copied from tt-inference-server's release spec (commit 2db98e38c, release 0.25.0). Context is the longest prompt plus reply the entry allows. Users is its highest concurrency. Each row carries its own code link, tt-metal and vLLM commits and image, because they differ between hardware. Nothing on this page is measured.

tt-model bundles

No tt-model bundle has been published for this model.

Details

Kind	LLM (from tt-inference-server model type)
Parameters	7B
Family	Mistral (from the start of the name)
Implementation	tt-transformers
Disk needed	34 GB
RAM needed	17.5 GB
Listed by	tt-inference-server