Lilith.
⌕
Editorial illustration: Without a sound measuring instrument, model safety is just a slide-deck number
Lilith illustration · editorial remix

Anthropic has set out ten recommendations for safety evaluations that inform model deployment, safeguards and further training. The guidance draws on the company’s system-card work and lessons from psychology’s replication crisis.

Ten recommendations turn an eval from a benchmark into a measurement process

Anthropic defines a safety eval as more than a capability test. It asks whether a response increases the risk of harm and whether the model acted appropriately in context. The company recommends combining domain and methods expertise, defining the construct in advance, preregistering the analysis and documenting the configuration in full.

Self-harm conversations show why this matters. A flat refusal may look safe while failing a person in crisis. Anthropic therefore evaluates complete multi-turn conversations against expert-built rubrics, with one dangerous turn able to fail the entire exchange.

Scores cannot be compared honestly when the grader or configuration changes

Prompts, answer order, product environment and the model used as a grader can all move the result. Anthropic reports a difference of up to 34 percentage points between the API and claude.ai configuration in tests of Claude Fable 5.1. Those figures do not isolate a model difference because several conditions changed at once.

Teams approving or buying a model therefore need to retain prompt versions, grader versions, sampling settings and run dates. Without that trail, a safety dashboard becomes colorful reporting with little audit value.

Saturated tests and confident graders can manufacture reassurance

Anthropic’s recent models score between 94% and 98% on aggregate single-turn harmless-response tests, while multi-turn results vary much more. The company recommends at least 100 items for a complex multi-turn eval and 1,000 for a simpler single-turn test. It also calls for validating graders against human labels and failing loudly when a grader breaks.

A visible gap may still be noise. Opus 5 and Opus 5.5 scored 98% and 94% on a romance-scam eval, but the difference was not statistically significant. The higher number alone would make a misleading headline.

Reproducible runs and disclosed uncertainty will decide credibility

The useful test is whether another team can repeat the run and reach a comparable conclusion. Anthropic recommends confidence intervals, category-level results and disclosure of any departure from the preregistered plan.

For buyers and builders, that is a better filter than another leaderboard. Without the sample, grader and environment, a safety score is a vendor claim. With them, it can support an actual deployment decision.

Lilith's verdict

A safety score without provenance is a laboratory sample without a label: it looks scientific, but no sensible team should use it to decide whether a model ships.