Banghua Zhu
@BanghuaZ
Since the release of GLM 5.2, SGLang team have been constantly working on pushing the performance to the limit, building in the open, and sharing it with everyone. Now you can achieve 535 token/s with latest version of SGLang!
Those won't happen without great efforts and support
Those won't happen without great efforts and support
LMSYS Org@lmsysorg · Jul 14Serving GLM5.2 NVFP4 Agentic Workload with SGLang: How We Reached 500 TPS on 8xB300 at bs=1
In this deep dive, we explain how SGLang reaches 500+ tok/s/user at bs=1 on 8xB300, with 18 to 34% higher single-user interactivity within two weeks since day-0, and 6 to 11% better peak
In this deep dive, we explain how SGLang reaches 500+ tok/s/user at bs=1 on 8xB300, with 18 to 34% higher single-user interactivity within two weeks since day-0, and 6 to 11% better peak
0:47 2 59