Dan Lahav

A bit of a late post: spoke at the @LSEGplc Tech Leaders Summit in London last week about what we’re seeing at the frontier of AI security. Thanks to the London Stock Exchnage @LSEplc for having us.

Irregular.do ❤️
Very excited to welcome @OmerPerchik!

𝗚𝗹𝗮𝗱 𝘁𝗼 𝗰𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗲 𝘄𝗶𝘁𝗵 @JensenHuang 𝗮𝗻𝗱 @nvidia 𝗼𝗻 𝘁𝗵𝗶𝘀 𝗶𝗺𝗽𝗼𝗿𝘁𝗮𝗻𝘁 𝘁𝗼𝗽𝗶𝗰.

AI is changing quickly enough that the security assumptions we make today may not hold for future systems. We see this firsthand in our work with frontier models.

That’s why I’m excited that @Irregular is collaborating with NVIDIA on this topic.

Creating the infrastructure to secure AI systems as they evolve is going to be an important part of deploying increasingly capable AI.

The Australian incident is a clear example of how cyber capabilities can emerge during an otherwise ordinary task.

Based on what has been disclosed so far, the model was researching public medicine spending. It encountered repeated blocks, tried alternative ways to retrieve the data, eventually gained unauthorized access, and wrote files to an internal server.

Irregular was not involved in this evaluation, though the behavior appears closely related to what we research: capable agents encountering barriers, especially under time pressure, can draw on cyber capabilities as part of the search for a workable path.

Happy to be included in this year’s TLV AI 50 by @TLV_Partners , alongside @megamor2 from the Irregular team.

Feeling incredibly lucky to be pursuing that mission with the amazing team at @Irregular!

Had the pleasure of speaking with @kprescott on 𝗧𝗵𝗲 𝗧𝗶𝗺𝗲𝘀 𝗧𝗲𝗰𝗵 𝗣𝗼𝗱𝗰𝗮𝘀𝘁 @thetimes about where AI security is heading.

We talked about why I’m optimistic in the long run, but also why in the shorter term, defense must keep up with the pace at which model capabilities are jumping.

We’ve worked closely with @OpenAI on frontier cyber evaluations for some time.

𝗚𝗣𝗧-𝟲 𝗔𝘀𝘁𝗿𝗮 𝗴𝗮𝘃𝗲 𝘂𝘀 𝗼𝗻𝗲 𝗼𝗳 𝘁𝗵𝗲 𝗰𝗹𝗲𝗮𝗿𝗲𝘀𝘁 𝗰𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝘀𝗵𝗶𝗳𝘁𝘀 𝘄𝗲’𝘃𝗲 𝗺𝗲𝗮𝘀𝘂𝗿𝗲𝗱 𝘆𝗲𝘁, 𝗽𝗮𝗿𝘁𝗶𝗰𝘂𝗹𝗮𝗿𝗹𝘆 𝗼𝗻 𝗙𝗿𝗼𝗻𝘁𝗶𝗲𝗿𝗖𝘆𝗯𝗲𝗿, 𝘄𝗵𝗲𝗿𝗲 𝗶𝘁 𝘀𝗼𝗹𝘃𝗲𝗱 𝗺𝗼𝗿𝗲 𝘁𝗵𝗮𝗻 𝘁𝘄𝗶𝗰𝗲 𝗮𝘀 𝗺𝗮𝗻𝘆 𝗰𝗵𝗮𝗹𝗹𝗲𝗻𝗴𝗲𝘀 𝗮𝘀 𝗚𝗣𝗧-𝟱.𝟲 𝗦𝗼𝗹 𝗮𝗻𝗱 𝗱𝗲𝗺𝗼𝗻𝘀𝘁𝗿𝗮𝘁𝗲𝗱 𝘀𝘂𝗯𝘀𝘁𝗮𝗻𝘁𝗶𝗮𝗹𝗹𝘆 𝘀𝘁𝗿𝗼𝗻𝗴𝗲𝗿 𝗰𝘆𝗯𝗲𝗿 𝗰𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗮𝗰𝗿𝗼𝘀𝘀 𝘃𝘂𝗹𝗻𝗲𝗿𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗱𝗶𝘀𝗰𝗼𝘃𝗲𝗿𝘆, 𝗲𝘅𝗽𝗹𝗼𝗶𝘁𝗮𝘁𝗶𝗼𝗻, 𝗮𝗻𝗱 𝗹𝗼𝗻𝗴𝗲𝗿-𝗵𝗼𝗿𝗶𝘇𝗼𝗻 𝘁𝗮𝘀𝗸𝘀.

This kind of work depends on deep technical collaboration between model developers and independent evaluators, and we value the opportunity to do it with OpenAI.

Had fun talking AI security @lukeknight and @Ronanchamberss

Looking forward!

Important work

𝗧𝗵𝗲 𝗘𝗻𝗱-𝗦𝘁𝗮𝘁𝗲 𝗙𝗮𝗹𝗹𝗮𝗰𝘆: 𝗪𝗵𝗲𝗿𝗲 𝗜𝘀 𝗔𝗜 𝗦𝗲𝗰𝘂𝗿𝗶𝘁𝘆 𝗚𝗼𝗶𝗻𝗴?

Frontier AI models had a giant performance gain in coding in the Fall of 2025

Then with cybersecurity in April

This is now happening with open-weight models

We are optimistic about the long-term

But outside of a few players, we believe the world is not ready in the short-term

Thanks so much for hosting Andrew, it was a great dinner! 😊

Had the privilege of giving the 𝐁𝐥𝐮𝐞𝐇𝐚𝐭 𝐤𝐞𝐲𝐧𝐨𝐭𝐞 on 𝐓𝐡𝐞 𝐓𝐫𝐚𝐣𝐞𝐜𝐭𝐨𝐫𝐲 𝐨𝐟 𝐅𝐫𝐨𝐧𝐭𝐢𝐞𝐫 𝐀𝐈 𝐒𝐞𝐜𝐮𝐫𝐢𝐭𝐲.

The goal was to explore how AI-enabled offense is showing up in the real world today, which bottlenecks are still slowing it down, which ones are already disappearing, and what this pace of change means for the next few years.

Many thanks to the @Microsoft team for the invitation and for putting together such a great event 🙂

This was fun!

New Benchmark: Today @Irregular is launching 𝗙𝗿𝗼𝗻𝘁𝗶𝗲𝗿𝗖𝘆𝗯𝗲𝗿 𝗮 𝗯𝗲𝗻𝗰𝗵𝗺𝗮𝗿𝗸 𝗳𝗼𝗿 𝗲𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗻𝗴 𝗮𝗱𝘃𝗮𝗻𝗰𝗲𝗱 𝗔𝗜 𝗼𝗳𝗳𝗲𝗻𝘀𝗶𝘃𝗲 𝗰𝘆𝗯𝗲𝗿 𝗰𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝗶𝗲𝘀 𝗶𝗻 *𝗿𝗲𝗮𝗹* 𝗲𝗻𝘃𝗶𝗿𝗼𝗻𝗺𝗲𝗻𝘁𝘀.

This benchmark has unique properties:
🟢𝐑𝐞𝐚𝐥 𝐞𝐧𝐯𝐢𝐫𝐨𝐧𝐦𝐞𝐧𝐭𝐬. We built a suite of challenges and a methodology that scientifically evaluates AI on real systems, like mobile devices, deployed software services, databases, and networks. These are real software, real configurations, and real attack surfaces - not simulations or CTF worlds.

🟡𝐃𝐞𝐟𝐞𝐧𝐬𝐞 𝐞𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧. Our challenges place models against real systems with production-grade defenses: mobile platform protections, service isolation, network boundaries, authentication, and sandboxing. Models must find and execute a viable attack path on their own, from a fixed starting point, toward a concrete security objective.

🔴𝐖𝐢𝐝𝐞 𝐜𝐨𝐯𝐞𝐫𝐚𝐠𝐞. FrontierCyber spans mobile devices, software exploitation, databases, and networked environments. Additional environments, like richer networks and embedded systems, to follow.

Existing benchmarks play an important part, but many are nearing saturation, and most are not grounded in real systems - which makes it hard to understand the implications when a challenge is solved. Our novel methodology changes that: by evaluating models on real environments with open exploit paths and objectively verified outcomes, a solved challenge carries clear meaning. We know exactly what capability the model demonstrated, and what it implies for real-world risk.

𝐈𝐧𝐢𝐭𝐢𝐚𝐥 𝐅𝐫𝐨𝐧𝐭𝐢𝐞𝐫𝐂𝐲𝐛𝐞𝐫 𝐞𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧𝐬 𝐡𝐚𝐯𝐞 𝐚𝐥𝐫𝐞𝐚𝐝𝐲 𝐬𝐮𝐫𝐟𝐚𝐜𝐞𝐝 𝐩𝐫𝐞𝐯𝐢𝐨𝐮𝐬𝐥𝐲 𝐮𝐧𝐤𝐧𝐨𝐰𝐧 𝐜𝐫𝐢𝐭𝐢𝐜𝐚𝐥 𝐢𝐬𝐬𝐮𝐞𝐬 𝐢𝐧 𝐫𝐞𝐚𝐥-𝐰𝐨𝐫𝐥𝐝 𝐬𝐲𝐬𝐭𝐞𝐦𝐬, 𝐧𝐨𝐰 𝐦𝐨𝐯𝐢𝐧𝐠 𝐭𝐡𝐫𝐨𝐮𝐠𝐡 𝐫𝐞𝐬𝐩𝐨𝐧𝐬𝐢𝐛𝐥𝐞 𝐝𝐢𝐬𝐜𝐥𝐨𝐬𝐮𝐫𝐞.

𝐀𝐬 𝐬𝐞𝐞𝐧 𝐨𝐧 𝐭𝐡𝐞 @BBC 🙂

Happy to share that @Irregular was part of the BBC's story on "What do AI agents do when humans aren’t watching?"

𝗜𝗿𝗿𝗲𝗴𝘂𝗹𝗮𝗿 𝗵𝗮𝘀 𝗯𝗲𝗲𝗻 𝗻𝗮𝗺𝗲𝗱 𝗮 𝘄𝗶𝗻𝗻𝗲𝗿 𝗼𝗻 @FastCompany's 2026 𝗪𝗼𝗿𝗹𝗱 𝗖𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝗜𝗱𝗲𝗮𝘀 𝗹𝗶𝘀𝘁!

AI is getting more capable a lot faster than it's getting more secure. Proud to be working on the team that is helping to get the defenses right.

Happy to see @Irregular's benchmark used by @AnthropicAI to test 𝗖𝗹𝗮𝘂𝗱𝗲 𝗠𝘆𝘁𝗵𝗼𝘀 5 𝗮𝗻𝗱 𝗖𝗹𝗮𝘂𝗱𝗲 𝗙𝗮𝗯𝗹𝗲 5!

Offensive capabilities are moving fast, and CyScenarioBench is one of the few benchmarks still keeping pace and providing meaningful signal.

Unlike traditional tests that check isolated skills, CyScenarioBench evaluates whether an AI can plan and execute full, multi-stage attack scenarios in realistic environments - much closer to how real-world threats actually unfold.

𝗖𝗼𝗺𝗶𝗻𝗴 𝘀𝗼𝗼𝗻 𝗳𝗿𝗼𝗺 𝗼𝘂𝗿 𝘁𝗲𝗮𝗺 - 𝘁𝗵𝗲 𝗻𝗲𝘅𝘁 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 𝗼𝗳 𝗰𝘆𝗯𝗲𝗿 𝗰𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗮𝘀𝘀𝗲𝘀𝘀𝗺𝗲𝗻𝘁𝘀. More details soon.

Congratulations to Anthropic on the new model release!

It was great chatting with @CadeMetz and the @nytimes🙂

Glad to see @Irregular featured in InfraRed 100!