granite-4.1-tt-metal
communityA community port of IBM Granite 4.1-8B to Wormhole through tt-metal and the Tenstorrent vLLM plugin, with the porting code, a verification harness and measured performance results. The author states the project is not affiliated with IBM, Tenstorrent or Networld.
Links
📦
Repo
📋 Changelog
# Changelog
All notable changes to this repository are recorded here. This file follows the
spirit of Keep-a-Changelog and Semantic Versioning, adapted for a hardware-port
project (each entry names the compatibility surface it affects).
Sections used per release:
- **Compatibility** — supported stack changes (tt-metal commit, FW, KMD, container).
- **Serving** — runtime behavior of `scripts/serving/` and `overlays/`.
- **Validation** — new gates, new evidence, or shifted validation scope.
- **Performance** — measured numbers change or new configurations characterized.
- **Known limitations** — new or removed items from the open-gaps list.
## [Unreleased]
- **Backlog (from private-side operational baseline)**: the maintainer's private setup has since
upgraded to `MESH_DEVICE=N300, max_num_seqs=2, max_model_len=32768, chunk=16` on 2026-07-15,
with OpenClaw agent end-to-end measured. This upgrade is **not yet reflected in v0.1.x** — it
will land in a subsequent MINOR (0.2.0) release once the new configuration is characterized in
the public evidence + validation docs. The v0.1.0 baseline (msq=4, mml=8192, 2026-07-11
measurement) is preserved as the initial published snapshot.
## [0.1.1] — 2026-07-15 (docs)
- **Docs**: added Japanese (JA/EN bilingual) to the four highest-signal sections only,
keeping the English body as canonical:
- HF Model Card: Card-only / no-weights warning.
- Both READMEs: Non-affiliated disclaimer (IBM / Tenstorrent / Networld — no involvement,
no endorsement).
- GitHub README: `Status at a glance` table (single bilingual table so numbers are not
duplicated).
- GitHub README: Quickstart intro + preflight legend.
- **No changes**: to code, scripts, overlays, measured values, pinned stack, or validation
scope. The v0.1.0 evidence remains authoritative.
## [0.1.0] — 2026-07-15 (initial public release)
- **Compatibility**: `tt-metal 805f43d` / container image
`0.12.0-805f43d-a45c614` @ sha256:6cbb9532... / FW 19.6.0 / KMD 2.7.0 / Wormhole b0.
- **Serving**: N300 msq=4 mml=8192 with `--tool-call-parser hermes`. `msq` restricted
to `{1,2,4,8,16,32}` by upstream tt_transformers.
- **Validation**: N150 single-device, single-prompt PCC 0.999410 with top1 match
and top5 4/5. Byte-exact chat_template round-trip on N300. Structured tool-call
round-trip on N300 (turn 1 name/JSON/city + turn 2 tool-result reflection + streaming).
- **Performance**: N300 baseline (msq=2, mml=4096) 23.5 tok/s single, 41.3 tok/s
aggregate at c=2 (1.76×). Operational (msq=4, mml=8192) 24.8 tok/s single,
46.0 tok/s aggregate at c=3.
- **Known limitations**: no multi-device teacher-forced PCC, no multi-prompt
statistics, no T3K measurements, no p50/p95 latency. See [`docs/validation.md`](docs/validation.md).
Works on
wormhole