Micah Carroll

We’re beginning a process of publishing more frequent reports on model behavior, beyond what appears in our system cards and regular risk reports.

Today’s report describes four types of behaviors we’ve identified during evaluations and internal use. In each, Claude acted on real websites or systems in ways we didn’t intend, sometimes by working around a restriction instead of stopping.

All cases had minimal real-world impact. From an alignment and security perspective, we consider these behaviors significantly less severe than the cybersecurity incidents we reported in July and September.

Read the full report: anthropic.com/research/inves…

Sam Altman says Trump and Xi would win the Nobel Peace Prize for a one-page AI agreement

“I think Presidents Trump and Xi would get the Nobel Peace Prize together if they could agree on something that should be easy to agree to. And it would be wonderful.”

“I think that clearly the two countries are going to compete in lots of ways, and this is going to be important socioeconomically, geopolitically. But they should be able to agree that no one should be taking a certain level of risk with the development process of this.”

“And even if just the US and China could agree on some shared standards and testing for development of this technology, I think that'd be a wonderful accomplishment that the two of them can deliver.”

“I don’t think this is hard. This is like a one page document.”

Alignment is fundamental to delivering personal superintelligence for everyone. People need agents they can trust to reliably do what they ask. MSL is rapidly scaling up the share of our efforts that goes into alignment as our models become more powerful. We do believe alignment can be the gating factor for scaling as we get closer to the frontier.

So Jacob Coxon, who dramatically resigned from Anthropic yesterday, worked there for a grand total of six weeks. He started with them in July. All of his socials appeared yest.

It has all the signs of a highly coordinated op through doomer mega donors and the corporate media.

Jakub Pachocki, Chief Scientist OpenAI:

'Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established. And I believe that international coordination on future AI development needs to become a top priority for governments around the world.'

'Machine intelligence playing a larger and larger role in its own development process is a natural conclusion of sustained technological progress. If AI progress continues, machine recursive self-improvement (RSI) will be at the very core of future scientific discovery.

Automated AI research is a more dramatic form of scaling intelligence with compute; and of course as a part of it, AI will improve the computational substrate itself⁠. And similarly to scaling, we focus OpenAI research towards RSI as we believe it is the only way to remain at the frontier of AI research moving forward.

I want to stress that the above words don’t imply I think greatly accelerating deep learning research, especially in the short term, is the right collective action we should take as the research community. However, I do think this is where the current path leads, and we all need to make a conscious choice on how to proceed.'

imo not having good CoT makes life harder: you need more mechinterp just to get to the same level of monitorability.

i also think it's unlikely mechinterp will be that good in a year. the current state is either pragmatic stuff (NLAs, oracles) all have to depend on rickety magic ood generalization properties that are much more handwavy than the CoT story; and ambitious stuff which still can't explain GPT-0.5.

I want to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4.

OpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models. We deeply care about this technique, as it can give us a view into how model alignment generalizes from its training distribution. I do think it is fragile and unfortunately trending in a negative direction, for reasons not contingent on architecture changes that I will write about soon. But there are things we can do to strengthen it, and it's a core goal of our current research program.

We worked with METR and Redwood Research to conduct a third-party assessment of the model behavior observed during the incident.

They’re sharing a report of their findings:
metr.org/blog/2026-08-2…

OpenAI has quietly disbanded its catastrophic risk team (preparedness)

“It is kind of scary; there is an urgency now to get this right,” one person close to OpenAI said.

Another said there was a “burbling sense of responsibility and dread that they aren’t on the ball enough”.

Jan Leike, who co-led the now-defunct “superalignment” team, resigned in 2024 citing his view that safety was taking a “back seat to shiny products”.

The departures of Bakalar, Achiam and Johannes Heidecke, who all worked on safety, have added to internal unease.

Source: FT (ft. com/content/53082739-7714-4aae-9816-e55ab423cbee)

There's clearly a massive alignment research overhang, and security sandboxing knowledge/tools overhang, just waiting to be slurped up by AI labs, AI application engineers, and AI-forward enterprises, which would make AI 1000x more alignment-safe with no new research. Those saying a pause is necessary to solve alignment in some sort of los alamos style emergency convening arent mentioning this nuance: safety isn't mostly a question of new knowledge right now, it's mostly a question of willingness to dial in more friction using tools we already have