Building strong safety evaluations · Anthropic Engineering Blog
Science, Technology & Innovation · Nov 13, 2026
Single-turn safety evaluations are largely saturated regression checks, while multi-turn simulations reveal more realistic safety failures by modeling escalation, accumulated context, and adaptive user behavior. Anthropic reports 94%–98% harmless-response rates on recent models in single-turn tests, but much wider variation in multi-turn settings, showing why robust safety requires realistic multi-turn testing rather than relying on single-prompt scores.
Building strong safety evaluations · Anthropic Engineering Blog
Science, Technology & Innovation · Nov 13, 2026
Safety evaluation should prioritize context-appropriate behavior—not refusal frequency—by measuring supportive crisis responses, safe handling of dual-use requests, and domain-specific multi-turn outcomes alongside both harmful compliance and benign over-refusal.
Building strong safety evaluations · Anthropic Engineering Blog
Science, Technology & Innovation · Nov 13, 2026
Safety-score differences may be statistically meaningless, while deployment configuration can have a larger effect than model version; comparisons therefore require confidence intervals and fully matched, documented setups.
Building strong safety evaluations · Anthropic Engineering Blog
Science, Technology & Innovation · Nov 13, 2026
Anthropic improved Claude Fable 5.1’s self-harm safety behavior by adding a targeted system-prompt prohibition against validating self-harm, raising appropriate responses from 60% on the API to 94% on claude.ai and demonstrating the need for continuous, configuration-specific evaluation.