Anthropic Is Letting Evaluators Inside While Paying for Their Independence

Anthropic has announced a partnership that will give Accenture unusually deep access to frontier model development. Embedded evaluation can expose decisions and incidents that an external pre-release test cannot see. It also recreates an old audit problem in a new industry: the organization under review is paying the reviewer.

Accenture will receive access comparable to Anthropic employees

The program will be led by Faculty, Accenture's specialist AI business. Its remit includes evaluating and red-teaming models, conducting alignment assessments and testing safeguards. Embedded evaluators are meant to watch models take shape during training, follow development and deployment decisions and speak directly with employees.

Anthropic and Accenture each expect to invest at least $1 billion in building capacity in this area over the next 5 years, according to the announcement. The arrangement is non-exclusive. Anthropic is also discussing pilots with METR and other nonprofit evaluators.

Continuous access turns evals from product tests into organizational oversight

Conventional external evals see a model or API at a chosen moment. An evaluator inside the company can observe why a team selected a safeguard, what internal tests found and how management reacted to an incident. The subject of evaluation expands from a model's technical capability to the behavior of the lab itself.

That is a more valuable class of information for buyers, regulators and the public. A report on a finished model describes an outcome. Continuous oversight can reveal the process behind it, including blind spots and decisions that never fit into a model card.

Direct funding leaves independence without a settled structure

Anthropic acknowledges that there are no established standards for evaluator access, reporting or independent funding. It will therefore fund Accenture's work directly. METR and other nonprofits, by contrast, are expected to use their own funding for pilot work.

Multiple evaluators can add different expertise, but they can also let a company emphasize the friendliest conclusion. Without predefined publication rights, protection from retaliation and disclosure of conflicts, employee-level access remains a strong promise with an uncertain output.

The first public report will show whom the evaluator truly answers to

The decisive issue is not the number of partners, but whether they can publish an uncomfortable finding without Anthropic's permission. The signals to watch are the scope of access, editorial independence, funding terms and whether reports describe incidents and disagreements.

Embedded evaluation is useful when it reduces the lab's information advantage. If the public receives only an approved summary, it gets a prestigious seal instead of oversight. The first sharp disagreement will tell us more about this program than the joint announcement did.

Lilith's verdict

The evaluator has a badge for the laboratory door, but the person inside pays the bill. Independence begins when it can carry out a report Anthropic would rather not frame for display.