RadixArk

We brought SGLang to NVIDIA Vera Rubin and accelerated Kimi K3 inference.
Working closely with @NVIDIA, we optimized attention, MoE, and speculative verification kernels on early-access Rubin hardware.

Highlights:
• Up to 20% faster FP8 MLA at batch 1 / 128K context
• 20% faster KDA verification, with bitwise-identical output
• 5.9% end-to-end inference speedup from MoE tail fusion, removing 276 kernel launches per decode step

SGLang also powers rollouts for Miles' end-to-end RL training on Rubin, including agentic RL with 64 concurrent sandboxes on the Vera CPU.

Full results and engineering details 👉 lmsys.org/blog/2026-10-0…

Our training framework is built on top of miles. Training runs on Modal's on-demand GPU clusters and we use its serverless GPUs to flexibly scale our inference workloads

Our sandboxing infrastructure is built on Kubernetes and able to handle millions of concurrent rollouts

Announcing the speakers for our SF Tech Week panel (10/7), exploring the open-source technologies shaping how the next generation of AI products is built and scaled!

-Walter Korman (@shaper), Tech Lead at @vercel
-Sam Herring (@yaboilyrical), ML Engineering Lead at @NousResearch
-Mao Cheng, Member of Technical Staff at @radixark
-Akshay Deshpande, Member of Technical Staff at @ArtificialAnlys

Announcing the first SGLang Summit: November 12–13 at Fort Mason, San Francisco.

Join us for two days of talks, discussions, and deep dives into frontier AI infrastructure, silicon, models, and applications.

Register: sglang.io/summit

a lot of the features in open-source rl / inference frameworks will eventually show up in other implementations. the more interesting question is how quickly a framework gets there, and how much work it takes to actually use them.

two related things seem to matter:

1 - the primitives. can you change the part you care about without pulling apart everything around it? @radixark miles’ async rollout buffer is a good example: you can change which groups of samples reach training and which prompts get retried, while reusing the rollout worker and trainer. the granularity of that interface matters. extension points should let people work on different parts of the system without constantly stepping on each other, and the contracts should be stable enough that those extensions survive upgrades.

2 - contribution velocity. getting a new attention backend running is one step. making it work with the KV cache formats, prefix caching, and speculative decoding your workload needs is more work. what matters is how quickly contributors and maintainers can get an improvement through that process and into something people can depend on.

cheaper code generation makes this more interesting. it gets easier to add features and easier for everyone else to catch up. there’s still the work of integrating changes, checking correctness, and measuring performance on real workloads. good primitives help keep each contribution from turning into a project across the entire stack.

if your framework makes a useful technique practical a couple weeks/months earlier, that’s invaluable time for experiments and learning. the question is whether a framework can keep giving you that head start, without making you do all the integration work yourself.

We built high-throughput materials labs in Menlo Park to create a loop between experiments and models. The labs generate fresh data, the models learn from it, and then help us decide what to try next.

Using only 1,300 H200s, plus months of our experimental data, we mid-trained and RL’d an open-source model to surpass GPT-6 Astra on our analysis benchmark. We call it Neon.

This is real footage from our lab. We’re focusing first on hard problems in materials science, including superconductors, magnets, and semiconductor materials.

Read our blog posts below.

SGLang-Diffusion with VDN-H3 now generates 14.4s of 768p video in just 9.0s 🚀

On 8× B200, 8 step denoising takes just 6.9s, reaching over 2× real time. The 9.0s figure covers the full generation request after warmup.

No measured quality regression versus dense 50-step H3 across 103 test prompts. 🧵

DeepSeek V4.1 Flash weights are out! We are shipping day-0 inference and RL support in SGLang and Miles.

V4.1 extends the V4 stack with compressed KV shared across layers, a two-stage sparse indexer, and a 196B Engram lookup memory.

It is natively multimodal with 552B backbone parameters, 16B active decode and 8B prefill, and supports up to 1M context.

There will be some very exciting performance upgrades for this model in the next few days. Stay tuned!

Supported features, blog and cookbook in the comments ⬇️

Huge congrats to the @radixark team on Miles v0.1! 🚀

Fully async agentic RL on a 744B MoE across 64 GB300s, with @sgl_project powering rollouts. Love the focus on getting rollout/training fidelity right alongside throughput.

huggingface.co/papers/2609.08…

Today we're launching Miles v0.1, an open-source RL framework for LLMs and multimodal models.

RL training is easy to start and hard to debug. Miles helps you ensure your run is correct, use hardware efficiently, and keep RL running at scale.

Over the past 9 months, 72 contributors have landed 1,326 commits, 85 GPU E2E CI tests, battle-testing Miles on frontier open models like Kimi K3, DeepSeek V4, Qwen 3.8, GLM 5.2, Inkling, MiniMax H3, etc.

Miles powers frontier-model development and production RL workloads at @humansand, @periodiclabs, @modal, @DecagonAI, @Eigent_AI, @nebiusai, @IBM and more, on both @NVIDIAAI and @AIatAMD hardware.

Here is what we built, and why teams picked Miles🧵

open-source RL-as-a-service github.com/radixark/miles

GLM-5.3 weights from @Zai_org are live, with SGLang powering day-0 serving support!

GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months.

On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300.

It's fast, efficient, and production-ready today on @NVIDIAAI Blackwell and Hopper, and @AIatAMD MI300X/325X/355X.

We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability.

Can't wait to see what people build with GLM-5.3 and SGLang🚀

Cookbook👇

GLM-5.3-Flash (320B-A18B) from @Zai_org drops today with day-0 support in SGLang. You may know it as ox-alpha from the past few days!

👀 It's the first native multimodal model in the GLM-5 series, able to review its own output visually and fix what's wrong.
🚀 It outperforms GLM-5.2 at 1/10 the cost. Hybrid sparse plus linear attention enables stable 1M long-context performance at extreme cost efficiency.
💻 It goes beyond coding into professional work: slides, documents, spreadsheets, and finance research, all handled end to end.

Run GLM-5.3-Flash with SGLang, and welcome to a new era of efficient, production-ready open intelligence!

Congrats to @Alibaba_Qwen on launching Qwen3.8-Flash! SGLang is proud to be a day-0 partner supporting the new architecture preview for Qwen4.

It's a 125B main model with 51B of N-gram embeddings and 6B activated per token.

The 51B N-gram embeddings scale model capacity with almost no extra compute per token, and can sit in host memory with async prefetch instead of occupying GPU memory.

The GDN + QSA hybrid attention gives you efficient memory and precise retrieval at the same time on long-horizon tasks, while Gated Residual gives the model 4 lanes instead of 1 to pass information between layers. And it's trained with Muon!

We're excited for what's next with Qwen4, and we already have plenty of ideas for how to use the N-gram embeddings in new deployment setups. Stay tuned!

Blog and cookbook in the comments👇