Our algorithm trains LLM classification systems to block harmful inputs and outputs based on a “constitution” of harmful and harmless categories of information.
6 5 161
Anthropic
@AnthropicAI
Replying to @AnthropicAI
We challenged jailbreakers to try to break a prototype version of the system to test its robustness.
After thousands of hours of red teaming, not one participant found a reliable jailbreak that extracted detailed information across a set of 10 harmful questions.
After thousands of hours of red teaming, not one participant found a reliable jailbreak that extracted detailed information across a set of 10 harmful questions.
7 162