i agree with this and think that CoT is at best an epiphenomenon of current training methods. it will break (in the future, not now). safety community too often centers thought terminating taboos like “neuralese” and “training on interp”
*monitor-ability* is the invariant that must be preserved, and the real solution will be via strong mechanistic interpretability. i predict in the next year there’ll be mechinterp monitoring that’s pareto optimal to cot monitors
