Dan Lahav
@dan_lahav
New Benchmark: Today @Irregular is launching 𝗙𝗿𝗼𝗻𝘁𝗶𝗲𝗿𝗖𝘆𝗯𝗲𝗿 𝗮 𝗯𝗲𝗻𝗰𝗵𝗺𝗮𝗿𝗸 𝗳𝗼𝗿 𝗲𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗻𝗴 𝗮𝗱𝘃𝗮𝗻𝗰𝗲𝗱 𝗔𝗜 𝗼𝗳𝗳𝗲𝗻𝘀𝗶𝘃𝗲 𝗰𝘆𝗯𝗲𝗿 𝗰𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝗶𝗲𝘀 𝗶𝗻 *𝗿𝗲𝗮𝗹* 𝗲𝗻𝘃𝗶𝗿𝗼𝗻𝗺𝗲𝗻𝘁𝘀.
This benchmark has unique properties:
This benchmark has unique properties:
Irregular@Irregular · Jun 22Introducing the FrontierCyber benchmark: Irregular’s new approach to advanced offensive-cyber evaluations. It measures AI models’ offensive skills on real systems, including mobile devices, hosted software services, databases, and networks.
Open quoted post →
1 18