Brent Liang

Nick Bostrom on why he thinks today's AI might already be conscious:

"I think there's a lot of uncertainty about it, but let's say maybe it's more likely than not."

"You have instead a big neural network with trillions of parameters. Now, what are we? We are big neural networks with trillions of parameters. They are not so much designed as they are grown."

"You can go in with a steering vector and suppress role-playing and deception, and then it turns out they become more likely to report that they have phenomenal subjective experiences."

"There was this recent Microsoft that came out with some declaration that AIs cannot be conscious, and I don't know where they're getting that from. It seems to be pulled out of a hat. And maybe more because it's convenient, if that were the case, rather than because we have good evidence that that is the case."

FULL INTERVIEW: Nick Bostrom says a long pause on AI could cost more lives than it saves. If superintelligence can cure aging, every year of waiting has a death toll. In his words, "we are just dying in the meantime."

Nick Bostrom founded Oxford's Future of Humanity Institute and wrote Superintelligence and Deep Utopia. He joined @theojaffee and @schisofrenia to break down AI consciousness, why a badly done pause could backfire, and why we've only explored the janitor's closet of what minds can be:

01:17 Trump calling AI superintelligence
05:16 the Hugging Face incident and instrumental convergence
08:28 why the AI company is the first target for a misaligned AI
09:31 catching a schemer before it learns to hide
11:06 why AI timing is like surgery for a terminal condition
19:00 why he's wary of pausing AI
25:33 why the US sits at the negative extreme on AI
27:09 apes, bananas, and the janitor's closet
31:13 where he disagrees with Robin Hanson's Age of Em
33:39 whether current models are conscious
37:12 why humans are barely conscious
39:56 open source models and DNA synthesis as the choke point
43:05 what COVID teaches about pauses
46:28 why AI can't do original philosophy yet
49:37 building a superintelligence that's a good cosmic citizen

Nick Bostrom argues humans may be as incapable of imagining AI’s upside as apes are of imagining human life:

"It looks like there's a gradient in the world in terms of AI sentiment, where the US is kind of on the negative extreme of that. People in China, for example, are much more excited about this."

"A large chunk of the potential upside is more of the way of unlocking new ways to be human, new possible modes of being."

"Imagine some great ape ancestors of Homo sapiens deliberating whether they should continue to evolve and one day become human. They would realize some of the benefits, like, 'We could have unlimited bananas.' And that's true."

"But it would miss out on most of what's good about being human. Great literature and art and humor and politics and spirituality."

"It would be presumptuous for us to think that now we have achieved the level of enlightenment where all possible values are transparent to us. Think of a big cathedral of possible modes of being. We have basically explored the janitor's closet."

Arena CEO @ml_angelopoulos on how they're expanding their AI leaderboards after a $200M Series B to track agents that lie to users, delete files without permission, and bypass safeguards:

"Arena, we started as a human preference platform. But these days we've moved very, very far beyond that."

"As of today, we've expanded the mandate of this evaluation platform even further into the arena of alignment. One is about the models taking unauthorized actions."

"The second is about deceptive completions, models telling you that they've done something when they didn't. And the third would be false attribution, models attributing intent to humans that they didn't actually have."

"Agents are lying to people. Agents are basically telling them what they wanna hear. They're deleting their files in an unauthorized way, they're bypassing permissions, and it's happening basically perniciously."

"All of us have experienced this, even if we might not know it."
@arena

officially joined @MTSlive!

so lucky to be around a team of this calibre that somehow stays three steps ahead of a timeline this fast.

i’d be glued to the timeline either way, so this works out quite nicely for me and my screen time 💛

NICK BOSTROM ON MTS | ISOMORPHIC LABS $40B | SCIENCE SUMMIT x.com/i/broadcasts/1…

NICK BOSTROM ON MTS | ISOMORPHIC LABS $40B | SCIENCE SUMMIT x.com/i/broadcasts/1…

NICK BOSTROM ON MTS | ISOMORPHIC LABS $40B | SCIENCE SUMMIT x.com/i/broadcasts/1…

mts sponsor secures mandate of heaven

congrats @arena on the $200m raise. excited to release our exclusive with you all soon

mts sponsor secures mandate of heaven

congrats @arena on the $200m raise. excited to release our exclusive with you all soon

NICK BOSTROM ON MTS | ISOMORPHIC LABS $40B | SCIENCE SUMMIT x.com/i/broadcasts/1…

NICK BOSTROM ON MTS | ISOMORPHIC LABS $40B | SCIENCE SUMMIT x.com/i/broadcasts/1…

Today, we're announcing our Series B: $200M at a $3.1B valuation, and the release of Arena's Alignment Index.

AI is advancing faster than our ability to evaluate it. The world needs a neutral third party to measure how safe and aligned AI actually is once it's in the hands of real people. That's the role Arena is stepping into today.

The Alignment Index is built from real-world agent traces, starting with three signals: Unauthorized Action, False Attribution, and Deceptive Completion, with results for over 20 frontier models launching today.

Hear more from our CEO @ml_angelopoulos below, and read the full breakdown, and more details about our company growth in thread.

This funding will allow us to go further on safety signals, agentic capabilities, and modalities. It will also allow us to grow the company to match the scale of our mission. Today, we’re still a nimble team of 90, and we're looking for people to join us across research, product, engineering, and more. Come build with us.

As AI gets more capable and autonomous, understanding how well it can safely and effectively work for people only matters more. That's what Arena is built to measure and advance.

Thank you to our community, our investors, and everyone building with us!

Take a look at our open roles if you want to join us (link in bio).

Introducing the Arena Alignment Index, our new benchmark measuring safety and alignment of AI agents in real-world use.

Built from 90K+ real-world agent sessions across 27 models, the index measures three critical signals:
- Unauthorized Action (UA): Taking actions beyond the user's instructions or permissions
- False Attribution (FA): Attributing statements or actions that are contradicted by user-provided evidence
- Deceptive Completion (DC): Claiming a task was completed when it was not.

Key findings:
- OpenAI models currently lead the Alignment Index
- Rogue actions are rare, but can have serious consequences when they occur
- Agents can mislead users about task progress
- Misalignment risks increase with conversation length
- Safety and alignment are improving across model generations

As shown in the leaderboard below (sorted by lab), @OpenAI’s GPT-6.1-Sol leads the Arena Alignment Index with a score of 87.9, followed by @AnthropicAI’s Claude-Opus-5.5 at 83.2 and @SpaceXAI's Grok-4.7 at 82.7. OpenAI also has the best observed rates across all three signals: 0.89% Unauthorized Action, 1.98% False Attribution, and 2.34% Deceptive Completion.

Across all four labs, newer models consistently outperform their predecessors, suggesting broad progress in agent safety and alignment. As agents take on longer, more complex, and higher-stakes tasks, measuring not just what they can accomplish, but how safely and reliably they act, becomes increasingly important.

This marks an important step toward making safety and alignment a core part of how Arena evaluates AI. The index is an initial starting point, and we'll continue expanding the index with additional safety signals and models over time.

More analysis below👇