Models

Llama

Models whose name starts with Llama.

9 models
LLMOfficial

Llama-3.1-70B

meta-llama/Llama-3.1-70B

Listed in tt-inference-server on 5 hardware types; best status Complete.

Runs on
BH 4xP150, BH LoudBox, BH QuietBox 2 +2
Speed
≥ 10 tok/s≤ 238 ms WH Galaxy
Status
Complete
tt launch
7 clients
70B parameters
LLMOfficial

Llama-3.1-70B-Instruct

meta-llama/Llama-3.1-70B-Instruct

Listed in tt-inference-server on 5 hardware types; best status Complete.

Runs on
BH 4xP150, BH LoudBox, BH QuietBox 2 +2
Speed
≥ 9.5 tok/s≤ 226 ms WH Galaxy
Status
Complete
tt launch
7 clients
70B parameters
LLMOfficial

Llama-3.1-8B

meta-llama/Llama-3.1-8B

Listed in tt-inference-server on 8 hardware types; best status Complete.

Runs on
BH LoudBox, BH P300, BH QuietBox 2 +5
Speed
≥ 85 tok/s≤ 46 ms WH LoudBox/QuietBox
Status
Complete
tt launch
7 clients
8B parameters
LLMOfficial

Llama-3.1-8B-Instruct

meta-llama/Llama-3.1-8B-Instruct

Listed in tt-inference-server on 9 hardware types; best status Complete.

Runs on
BH LoudBox, BH P300, BH QuietBox 2 +6
Speed
≥ 36 tok/s≤ 58 ms WH LoudBox/QuietBox
Status
Complete
tt launch
7 clients
8B parameters
LLMOfficial

Llama-3.3-70B-Instruct

meta-llama/Llama-3.3-70B-Instruct

Listed in tt-inference-server on 5 hardware types; best status Complete.

Runs on
BH 4xP150, BH LoudBox, BH QuietBox 2 +2
Speed
≥ 14 tok/s≤ 76 ms WH Galaxy
Status
Complete
tt launch
7 clients
70B parameters
LLMOfficial

Llama-3.2-1B

meta-llama/Llama-3.2-1B

Listed in tt-inference-server on 3 hardware types; best status Functional.

Runs on
WH LoudBox/QuietBox, n150, n300
Speed
≥ 18 tok/s≤ 60 ms WH LoudBox/QuietBox
Status
Functional
tt launch
7 clients
1B parameters
LLMOfficial

Llama-3.2-1B-Instruct

meta-llama/Llama-3.2-1B-Instruct

Listed in tt-inference-server on 3 hardware types; best status Functional.

Runs on
WH LoudBox/QuietBox, n150, n300
Speed
≥ 36 tok/s≤ 70 ms WH LoudBox/QuietBox
Status
Functional
tt launch
7 clients
1B parameters
LLMOfficial

Llama-3.2-3B

meta-llama/Llama-3.2-3B

Listed in tt-inference-server on 3 hardware types; best status Functional.

Runs on
WH LoudBox/QuietBox, n150, n300
Speed
≥ 6.1 tok/s≤ 170 ms WH LoudBox/QuietBox
Status
Functional
tt launch
7 clients
3B parameters
LLMOfficial

Llama-3.2-3B-Instruct

meta-llama/Llama-3.2-3B-Instruct

Listed in tt-inference-server on 3 hardware types; best status Functional. 1 tt-model bundle also published.

Runs on
WH LoudBox/QuietBox, n150, n300 +2
Speed
≥ 17 tok/s≤ 120 ms WH LoudBox/QuietBox
Status
Functional
tt launch
7 clients
3B parameters1 tt-model bundle