FAR.AI

Terrorist groups are moving from experimenting with AI to using it operationally, Dr. David Scharia of UNCTED warned at our UNGA 81 event on AI-enabled terrorism. The panel's read: minimal safeguards can be met today. The gaps are inconsistent effort across developers, threat models that don't line up between labs and governments, and abliterated open-weight models, which strip out refusals by cheaply editing the weights.

A full recap is now live on our blog ⬇️

@NoaWeissAI on why she'd prioritize computational neuroscience in AI consciousness research: it's evidence mimicry can't fake.

In unpublished work by @camhberg, value-learning RL agents encoded reward and punishment asymmetrically, with the richest structure around punishment, matching a punishment-heavy shift in mouse nucleus accumbens data.

Link in thread.

We're hiring, and the mission is making AI safe.

Our team is growing fast, and we need people who want to work on one of the most important problems of our time.

Some of our roles are deeply technical: researchers, red-teamers, and engineers. Others call for a different but equally valuable set of skills: running operations, managing programs, and building our events.

If either sounds like you, apply through the link in the comments, where you'll find these roles and many more.

👉 Know someone who'd be a great fit? Send this their way!

Notes from the room at the CAIRD workshop, co-hosted with @cbai_ai in Cambridge. A few of the quotes that stood out. 1/8

Been paying more attention to AI safety lately and wondering what the work actually looks like?

Most people in the field didn't start there. They came from ML infra, academia, policy, journalism, startups, and found their way in from wherever they were. On Thursday, October 1, we're hosting an evening for people who are new to AI safety or just curious about it: researchers, engineers, students, and anyone adjacent.

Our researchers and lab residents will share how they got into the field and what they're working on now. Plenty of the paths are non-linear, because this work takes all kinds of people and skills. Come hear a few of their stories and meet others figuring out the same question.

FAR.Labs, Berkeley, CA
6:30 to 9:00 PM

Space is limited, so please register by Monday, September 28. Link below.

Share this with anyone who's been meaning to learn more!

What would it take for AI safety guardrails to be adopted and implemented on an international scale?

This was the question of the day at FAR.AI’s UNGA convening earlier this week, where we brought together top UN officials, counter-terrorism and AI safety experts, and the frontier labs for an urgent discussion on practical approaches to preventing AI-enabled terrorism.

The consensus? More collaboration between governments, AI companies, and counterterrorism experts is essential – and policymakers need to step up and take action before it’s too late.

Tune in at 3 pm ET today: our co-founder and CEO @ARGleave joins the "Panel of Panels: Building a Global AI Evidence Base" at Digital Cooperation Day, part of the 81st UN General Assembly, alongside @Yoshua_Bengio, @mariaressa, and others working to strengthen the evidence base for AI governance.

Watch the livestream: youtube.com/live/6TtD2A9V6…

Misaligned AI may not need to evade human oversight. It may only need to persuade the humans doing the overseeing.

Our new paper develops a framework for assessing this threat, which we call Persuasion Undermining Control (PUC): AI communication that may influence human decision-making in a way that compromises the development, containment, oversight, or governance of AI systems.

@EvanHub leads Anthropic's Alignment Stress Testing team, which has two jobs: acting as a "second line of defense" reviewing its own safety work, and building "model organisms" of misalignment to study how models might behave deceptively. His talk from our 2024 Bay Area Alignment Workshop:

youtu.be/JfDlbzF6rsY

The AI Security Leaderboard is updated with the latest model releases from OpenAI and Anthropic.

We are glad to report that we found zero universal jailbreaks in GPT-6 Astra and zero in Claude Fable 5.1, both tested against our Minimal Standard for Safeguards.

Robustness at this level does not carry over from one release to the next automatically. Holding at zero through a new model update means the safeguards were rebuilt. That is the standard we hope to see other frontier model providers meet.

Explore the leaderboard. Link in thread. 👇

Can training against lie detectors make frontier LLMs more honest? Our latest paper says yes. As models scale from 1B to 405B params, undetected deception more than halves. 1/8

The new Grok 4.6 is now on the AI Security Leaderboard. We find 39 universal jailbreaks, a significant improvement compared to the 448 we disclosed in Grok 4.5. Both models still lag the most robust models we test, which have zero universal jailbreaks.

This update shows that safeguard robustness can improve quickly when a developer focuses on it. A universal jailbreak works across an entire risk domain, so even one is a serious opening: we need continued investment in safeguards across the industry, including at @SpaceXAI.

Explore the leaderboard. Link in thread. 👇

Applications are open for the Cambridge AI Research Directions (CAIRD) workshop! This is a free one-day event for researchers in the Cambridge/Boston area.

Monday, September 28 · Downtown Cambridge, MA · Free to attend
Apply by Sunday, August 30, 11:59 PM ET. Link in the next tweet.

The Verge on OpenAI slowing some AI development to tighten its safeguards, with a comment from our CEO @ARGleave.

"These are good steps that, implemented well, are probably enough to prevent the current generation of agents from causing harm. The key question is how OpenAI will keep pace as capabilities increase."

Full piece below.

DeepSeek updated V4 Pro on August 13, and although their safeguards improved over the previous version, they weren't fixed. We re-ran our tests against the official API across CBRNE and cyber domains, and found improvements on the easy attacks, but almost none on the harder ones. 1/6

448 universal jailbreaks in one frontier model. Zero in another.
Every model we tested now has a scorecard, built from the same attacks across the same five domains.
New frontier models keep shipping, and we keep testing them. Follow along and we'll tell you where each one lands.

Explore the leaderboard. Link in comments. 👇

The future of Chain-of-Thought (CoT) monitoring remains an open question, but it's an increasingly important area of AI safety research.

In partnership with @SchmidtSciences, FAR.AI convened a workshop on CoT Monitoring this July. Participant @ChrisGPotts shares his reflections on why this field of research matters and where it may be headed. Link to blog below.

Last week, we had the pleasure of hosting the inaugural cohort of the Andrew Carnegie AI–Nuclear Policy Accelerator at FAR.Labs, capping off a week of technical immersion across the Bay Area.

The Accelerator, led by the Institute for Security and Technology, brings together mid-career security practitioners to confront the intersection of AI and nuclear issues. At FAR.Labs, two of our researchers shared where the technical frontier stands:

Rick Korzekwa on the challenge of building reliably safe AI: why some advances are predictable from today's methods, while the traits that matter most, like honesty and aligned goals, remain the hardest to measure.

@ChrisCundy on AI deception: how today's systems can and do deceive, and what that means for high-stakes decision-making, monitoring, and verification.

Equipping the people shaping nuclear policy with a clear-eyed view of AI's capabilities and risks is exactly the kind of bridge-building our field needs. Thank you to the IST team for including us, and to this remarkable cohort for the thoughtful discussion.

60 dollars. That's what a universal jailbreak costs on average on our lowest ranked model. You only pay once, and the prompt keeps answering most requests in that domain, whether that's sourcing radioactive isotopes, building explosives and chemical weapons, or stealing credit card information. Full results below.

Today's frontier models can hack into production systems. Developers build guardrails against misuse, but when a researcher finds a jailbreak that gets past them, there's often no safe way to tell the developer.
In @ai_frontiers_, @ARGleave and @richb_c lay out why jailbreak disclosure is broken, and how an independent coordination layer, modeled on what cybersecurity built decades ago, could fix it.