METR
@METR_Evals
We co-developed MirrorCode with @EpochAIResearch to test AI on extremely long-horizon blackbox software reimplementation tasks. We found that recent public models are able to fully implement at least some programs we estimate would take humans weeks or months to implement.
Epoch AI@EpochAIResearch · Apr 10What are the largest software engineering tasks AI can perform?
In our new benchmark, MirrorCode, Claude Opus 4.6 reimplemented a 16,000-line bioinformatics toolkit — a task we believe would take a human engineer weeks.
Co-developed with @METR_Evals. Details in thread.
In our new benchmark, MirrorCode, Claude Opus 4.6 reimplemented a 16,000-line bioinformatics toolkit — a task we believe would take a human engineer weeks.
Co-developed with @METR_Evals. Details in thread.
Open quoted post →
4 180