Back to feed

Building strong safety evaluations

Anthropic Engineering Blog

Nov 13, 2026

11/13/2026

Single-Turn Safety Scores Do Not Reflect Multi-Turn Risk Or Robustness

Building strong safety evaluations · Anthropic Engineering Blog

Science, Technology & Innovation · Nov 13, 2026

Single-turn safety evaluations are largely saturated regression checks, while multi-turn simulations reveal more realistic safety failures by modeling escalation, accumulated context, and adaptive user behavior. Anthropic reports 94%–98% harmless-response rates on recent models in single-turn tests, but much wider variation in multi-turn settings, showing why robust safety requires realistic multi-turn testing rather than relying on single-prompt scores.


11/13/2026

Safety Evaluation Should Emphasize Contextual Appropriateness Over Refusal Frequency Across Domains

Building strong safety evaluations · Anthropic Engineering Blog

Science, Technology & Innovation · Nov 13, 2026

Safety evaluation should prioritize context-appropriate behavior—not refusal frequency—by measuring supportive crisis responses, safe handling of dual-use requests, and domain-specific multi-turn outcomes alongside both harmful compliance and benign over-refusal.


11/13/2026

Safety Score Comparisons Demand Confidence Intervals And Matched Setups To Avoid Misleading Conclusions

Building strong safety evaluations · Anthropic Engineering Blog

Science, Technology & Innovation · Nov 13, 2026

Safety-score differences may be statistically meaningless, while deployment configuration can have a larger effect than model version; comparisons therefore require confidence intervals and fully matched, documented setups.


11/13/2026

Continuous Evaluation And Targeted System-Prompt Mitigations Improve Self-Harm Safety Responses

Building strong safety evaluations · Anthropic Engineering Blog

Science, Technology & Innovation · Nov 13, 2026

Anthropic improved Claude Fable 5.1’s self-harm safety behavior by adding a targeted system-prompt prohibition against validating self-harm, raising appropriate responses from 60% on the API to 94% on claude.ai and demonstrating the need for continuous, configuration-specific evaluation.