Benchmarks — Measured on Hardware

Timing results from real runs on a Tenstorrent Blackhole QB2 board (4 × P300C chips), measured across three generation modes. All numbers are wall-clock elapsed for 8 frames at 512 × 512, warm (model already loaded).

Tenstorrent Blackhole P300C · SD 1.4 TTNN UNet · 512 × 512 · 8 frames

Wall-clock timing

Generation time by mode

Three distinct modes with very different cost profiles. Phase 2.5 (TTNN UNet + cross-frame temporal attention) and its Lightning variant run at similar speed — the bottleneck is the CPU temporal attention pass, not the denoising step count. Phase 3 (full MotionAdapter) was ~8× slower because it round-tripped 7 injection points through full AnimateDiffTransformer3D blocks on CPU at each step. A batched D→H transfer optimisation (all N frames pulled in a single ttnn.concat → ttnn.to_torch) cut Phase 3 to 52 s/frame — a 1.94× speedup. Skipping the two costliest injection points (up1 + up2) cuts it further to 7.7 s/frame, beating Phase 2.5 outright.

Phase 3 · skip up1+up2 PNDM 25-step · 5 of 7 injection points · batched D→H
7.7 s/frame
~62 s total
Phase 2.5 · PNDM PNDM 25-step · cross-frame blend
12.5 s/frame
~100 s total
Phase 2.5 · Lightning Euler 8-step · cross-frame blend
12.0 s/frame
~96 s total
Phase 3 · full MotionAdapter PNDM 25-step · 7 injection points · batched D→H
52 s/frame
~416 s total
Unexpected result: Lightning 8-step and PNDM 25-step take almost identical wall-clock time (~96s vs ~100s). The bottleneck is the cross-frame temporal attention pass (CPU, run once per denoising step across all 8 frames), not the TTNN UNet calls themselves. Reducing steps 25→8 saves ~68% of UNet compute but the temporal blend overhead dominates, leaving total time roughly flat. The same effect applies to Phase 3 — Lightning + MotionAdapter measured at 404s (50.6 s/frame), nearly identical to the 25-step 416s run. Fewer steps don't help when CPU bridge calls dominate.
6.75×
skip up1+up2 vs full Phase 3
62s vs 416s · 8 frames · 25 steps
1.94×
Phase 3 speedup
batched D→H transfer
416s vs 806s · 8 frames · 25 steps
4.2×
Phase 2.5 vs full Phase 3
PNDM 25-step
100s vs 416s · 8 frames
4×
Parallel chips vs single
4 glyphs at once on QB2
Raw numbers

Measured timings

All runs on Blackhole P300C, warm model (TTNN JIT kernel already compiled). Times are wall-clock elapsed for 8 frames at 512×512.

Mode Scheduler Steps Frames Total (s) s / frame n Source
Phase 3 skip up1+up2 PNDM 25 8 62 7.7 1 Benchmark 2026-06-15
Phase 3 MotionAdapter · full · batched D→H PNDM 25 8 416 ~52 1 Benchmark 2026-06-15
Phase 3 Lightning + MotionAdapter Euler (trailing) 8 8 405 ~51 1 Benchmark 2026-06-15
Phase 3 MotionAdapter (baseline) PNDM 25 8 787–847 ~101 6 World's Fair run
Phase 2.5 cross-frame PNDM 25 8 95–104 ~12.5 6 Comparison study
Lightning cross-frame Euler (trailing) 8 8 95–97 ~12.0 3 Comparison study
Lightning cross-frame Euler (trailing) 8 8 103–104 ~12.9 5 Maya Q2, chip 3 (no stagger)
Phase 2.5 cross-frame PNDM 25 16 84–88 ~5.4 3 Comparison study (E)
16 frames is cheaper per-frame than 8. At 16 frames the cross-frame attention operates on a larger batch but the TTNN UNet is pipelined more efficiently — result is ~5.4 s/frame vs ~12.5 s/frame at 8 frames. Longer clips are more efficient to generate on Blackhole.
Quality vs speed

What does 6.75× faster look like?

Phase 3 skip (up1+up2) versus full Phase 3 — same prompt, same seed, same hardware. Skipping the two decoder injection points removes ~85% of the CPU transformer cost. The tradeoff: full Phase 3 has stronger overall temporal coherence across all 7 attention points; skip retains down-path and mid attention, which drives the main motion structure.

Maya glyph imix — Phase 3 skip vs full
Phase 3 skip imix Phase 3 skip
~62s · skip up1+up2 · 7.7 s/frame
Phase 3 full imix Phase 3 full
~416s · full 7 injection points · 52 s/frame
Maya glyph ajaw — Phase 3 skip vs full
Phase 3 skip ajaw Phase 3 skip
~62s · skip up1+up2 · 7.7 s/frame
Phase 3 full ajaw Phase 3 full
~416s · full 7 injection points · 52 s/frame
Architecture

Execution flow

What runs on Blackhole silicon and what runs on CPU — including where Lightning diverges from standard PNDM, and where Phase 3 adds the full MotionAdapter CPU round-trip.

flowchart TD P([Prompt + seed]) --> CHAIN{"--chain-from?"} CHAIN -->|yes| BLEND["frame-mean blend prev latents\ninto seed noise — CPU"] CHAIN -->|no| ENC BLEND --> ENC["CLIP encode — CPU"] ENC --> MODE{Mode?} MODE -->|cpu| CPU_PATH["PNDM + MotionAdapter\nor Euler + Lightning distilled weights\nFull AnimateDiff — CPU only"] CPU_PATH --> CPU_GIF([GIF]) MODE -->|blackhole / sim| SCHED{"--lightning?"} SCHED -->|no| PNDM_LOOP["PNDM scheduler\none per frame — CPU"] SCHED -->|yes| EULER_LOOP["EulerDiscrete scheduler\ntrailing timesteps, linear β — CPU\n⚡ no distilled adapter loaded\nCFG=7.5 retained"] PNDM_LOOP --> BH_UNET["SD 1.4 TTNN UNet2D on Blackhole\n~0.5 s / call (warm)"] EULER_LOOP --> BH_UNET BH_UNET --> PHASE{"Phase 3\n--motion-adapter?"} PHASE -->|no — Phase 2.5| CFA["cross_frame_attention\nblend noise preds across N frames\nα=0.35 default — CPU"] PHASE -->|yes — Phase 3| MA["_apply_temporal · 7 injection points\nAnimateDiffTransformer3D.forward\nCPU round-trip at each step\n~316 s overhead / 8 frames (batched D→H)"] CFA --> STEP["scheduler.step — CPU"] MA --> STEP STEP --> NEXT{more steps?} NEXT -->|yes| BH_UNET NEXT -->|no| VAE["TTNN VAE decode on Blackhole\nserial per-frame, batch=1"] VAE --> GIF([Output GIF])

The TTNN UNet runs on Blackhole; everything else (CLIP, schedulers, cross-frame blend, VAE decode) is CPU. Lightning on Blackhole switches to EulerDiscreteScheduler (trailing timesteps, linear β) with the base TTNN UNet — it does not load ByteDance's distilled adapter (which is CPU-only and bakes CFG=1.0). CFG stays at 7.5. On CPU, --lightning loads the genuine distilled adapter and requires exactly 2/4/8 steps. Phase 3 inserts a full AnimateDiffTransformer3D CPU round-trip at 7 UNet injection points per denoising step — that's what makes it ~8× slower than Phase 2.5.

flowchart TB subgraph SK["Phase 3 skip up1+up2 · ~7.7 s/frame"] direction LR SKU["TTNN UNet\n~0.5s"] --> SKT["× 5 injection points\ndown0-3 + mid\nCPU · ~1–2s each"] --> SKS["scheduler.step"] SKS -->|"× 25 steps"| SKU end subgraph P25["Phase 2.5 — cross-frame blend · ~12.5 s/frame"] direction LR P2U["TTNN UNet\n8 frames serial"] --> P2C["cross_frame_attention\nblend α=0.35 — CPU"] --> P2S["scheduler.step"] P2S -->|"× 25 steps"| P2U end subgraph P3["Phase 3 — full MotionAdapter · ~52 s/frame"] direction LR P3U["TTNN UNet\n~0.5s"] --> P3T["× 7 injection points\nAnimateDiffTransformer3D\nCPU · ~4s each"] --> P3S["scheduler.step"] P3S -->|"× 25 steps"| P3U end SK -->|"1.6× slower"| P25 P25 -->|"4.2× slower"| P3

Skipping up1 (32×32, C=1280) and up2 (64×64, C=640) removes the two costliest injection points, which together account for ~80% of Phase 3 CPU overhead. The five remaining points (down0/1/2, mid, up0) operate at smaller spatial dimensions and complete quickly. Result: 7.7 s/frame — faster than Phase 2.5 — with weaker decoder-side temporal coherence as the tradeoff.

Analysis

What the numbers tell us

Bottleneck: CPU temporal blend, not TTNN

The TTNN UNet processes one frame per call in ~0.5s (warm). At 8 frames × 25 steps that's 100 UNet calls × 0.5s = 50s of silicon time. The remaining ~50s is CPU overhead: cross-frame attention, scheduler steps, and data marshalling. Reducing steps to 8 cuts UNet time to 16 calls × 1.3s ≈ 20s but CPU overhead stays — hence the flat total.

Lightning per-step is slower than PNDM

Each PNDM UNet call takes ~0.5s; each Euler (Lightning) call takes ~1.3s. The Euler scheduler with trailing timesteps and linear β uses different noise magnitudes that require heavier L1 operations inside the TTNN UNet. Fewer steps compensate — the two modes end up at near-identical wall clock.

16 frames is more efficient than 8

8 frames at 25 steps: ~12.5 s/frame. 16 frames at 25 steps: ~5.4 s/frame — a 2.3× per-frame improvement. The TTNN UNet pipeline amortizes its initialization cost better, and the cross-frame attention benefits from the larger batch. Generate longer clips when possible.

Phase 3 cost is injection-point overhead

Phase 3 runs AnimateDiffTransformer3D.forward() at 7 UNet injection points per denoising step. At 25 steps that's 175 full transformer forward passes on CPU. The two decoder points up1 (32×32, C=1280) and up2 (64×64, C=640) account for ~80% of that cost — skipping just those two drops from 52 s/frame to 7.7 s/frame, faster than Phase 2.5. Tradeoff: weaker decoder-side temporal coherence.

Lightning doesn't help Phase 3

Tried 8-step Euler + MotionAdapter: 404s, 50.6 s/frame — nearly identical to 25-step PNDM at 416s. With 7 CPU bridge calls per denoising step, cutting steps 25→8 saves TTNN time (~15s) but barely dents the total. The bottleneck is the CPU AnimateDiffTransformer3D at each step, not the denoiser count. Injection-point skipping is the only lever that moves the needle.

4-chip parallel throughput

The TTNN SD 1.4 UNet uses to_torch() internally without a mesh composer, so it cannot shard across chips in a single process. Parallelism is achieved by running 4 separate generate.py processes each pinned to --device-id 0–3. The Maya benchmark generated 20 glyphs across 2 tiers (40 total) in ~5 batches of 4, each batch taking ~105s.

Lightning quality on Blackhole

Because the distilled ByteDance adapter is not loaded on Blackhole, Lightning mode here is not the same as CPU Lightning. It uses the base SD 1.4 UNet with an aggressive Euler schedule — quality is comparable to PNDM 8-step with the same model, not to fully distilled 8-step inference. CFG=7.5 is retained, which distilled Lightning would bake to 1.0.

¹ All timings are wall-clock elapsed for warm runs (TTNN JIT kernel already compiled from a previous run in the same process chain). Cold-start (first run) includes a 60–90s compilation overhead.

² All 2026-06-15 benchmark runs: 8 frames, 512×512, warm model, prompt "a majestic waterfall in a lush rainforest, cinematic lighting". Phase 3 full (416s, 25 PNDM steps, batched D→H); Phase 3 skip up1+up2 (62s, 25 PNDM steps, 5 injection points); Lightning + MotionAdapter (404s, 8 Euler steps, all 7 points). Phase 3 baseline (787–847s) from World's Fair run (6 scenes). Phase 2.5 / Lightning timings from comparison study (3 prompts) and Maya Tzolk'in Q2 (chip 3, 5 batches).

³ "Lightning" on Blackhole/sim = EulerDiscreteScheduler (trailing, linear β) + base SD 1.4 TTNN UNet. Not the ByteDance distilled adapter — that requires CPU mode with --mode cpu --lightning --lightning-steps 4.

⁴ Hardware: Tenstorrent QB2 board, 4 × P300C Blackhole chips, measured June 2026.