LLM Official

AFM-4.5B

Listed in tt-inference-server on 2 hardware types; best status Experimental.

In tt-inference-server

Hardware	Engine and implementation	Status	Context	Users	Build	Support page
WH LoudBox/QuietBox `T3K`	vLLM tt-transformers	Experimental	64K	32	Code: https://github.com/tenstorrent/tt-metal/tree/ae65ee5/models/tt_transformers tt-metal `ae65ee5` vLLM `35f023f` Image `ghcr.io/tenstorrent/tt-inference-server/vllm-tt-metal-src-release-ubuntu-22.04-amd64:0.3.0-ae65ee5-35f023f`	Open
n300 `N300`	vLLM tt-transformers	Experimental	64K	32	Code: https://github.com/tenstorrent/tt-metal/tree/ae65ee5/models/tt_transformers tt-metal `ae65ee5` vLLM `35f023f` Image `ghcr.io/tenstorrent/tt-inference-server/vllm-tt-metal-src-release-ubuntu-22.04-amd64:0.3.0-ae65ee5-35f023f`	Open

Status, context and users are copied from tt-inference-server's release spec (commit 2db98e38c, release 0.25.0). Context is the longest prompt plus reply the entry allows. Users is its highest concurrency. Each row carries its own code link, tt-metal and vLLM commits and image, because they differ between hardware. Nothing on this page is measured.

tt-model bundles

No tt-model bundle has been published for this model.

Details

Kind	LLM (from tt-inference-server model type)
Parameters	4B
Family	AFM (from the start of the name)
Implementation	tt-transformers
Disk needed	28 GB
RAM needed	10 GB
Listed by	tt-inference-server