training diffusion-style models with RL has traditionally required heavy re-engineering of the MDP. david’s paper removes the need for this and provides a clean way to train diffusion with RL using a tweaked PPO style objective. super exited by this — check it out!