Banghua Zhu
@BanghuaZ
SGLang delivers a good speed for K3 at day-0. Behind it were many sleepless nights across the SGLang ecosystem, and an incredible relay race across the ecosystem.
@nvidia and @AMD put serious engineering power into kernels and hardware enablement. @KVCache_AI pushed support
@nvidia and @AMD put serious engineering power into kernels and hardware enablement. @KVCache_AI pushed support
LMSYS Org@lmsysorg · Jul 27SGLang day-0 speed on Kimi K3: 423 tok/s (measured on gsm8k), plus RL support ready in Miles @radixark!
How the largest open-source model runs this fast: we natively implemented and deeply optimized K3’s new architecture with fused KDA decode kernels, DP attention, DSpark, PD
How the largest open-source model runs this fast: we natively implemented and deeply optimized K3’s new architecture with fused KDA decode kernels, DP attention, DSpark, PD
0:53 4 46