Paolo Rosson
@redp314
Update to this benchmark: I reran @ashxhart’s TensorFold 0.6.0 on the same DGX Spark, Qwen3.8-Flash-Next NVFP4 weights and MTP-6. Versus 0.3.6.3: +5.1% decode and 26% lower cold TTFT across 10 contexts. At 260k (one run): 430s → 208s cold; cached resend 0.52s. Chart ↓
Paolo Rosson@redp314 · Oct 1Tested @ashxhart's TensorFold v0.3.6.3 on one DGX Spark with Qwen3.8-Flash-Next (NVFP4, MTP-6).
Fixed 2,048-token code task:
74.8 tok/s single-stream decode,
313 tok/s across 8 streams.
Context sweep: 56–65 tok/s through 132k input;
50.8 at 260k (one run).
Cold TTFT at
Fixed 2,048-token code task:
74.8 tok/s single-stream decode,
313 tok/s across 8 streams.
Context sweep: 56–65 tok/s through 132k input;
50.8 at 260k (one run).
Cold TTFT at
Open quoted post →
0 4