Cheng Wan
@ChengWan17
We just landed layer_boundary in @sgl_project. The biggest win: feature compatibility across models.
Instead of re-implementing TP/DP/CP communication, residual handling, token movement, and normalization for every model, we put these semantics behind a shared stage boundary.
Instead of re-implementing TP/DP/CP communication, residual handling, token movement, and normalization for every model, we put these semantics behind a shared stage boundary.
5 87