Anthropic opens its doors to METR: employee-level access to audit its risk work

Anthropic has started giving a team from METR, the nonprofit that evaluates frontier AI models, what it calls employee-like access to its internal systems. METR will publish the results of its risk assessments, and Anthropic says it cannot veto publication. The announcement follows two steps Anthropic took last month: a commitment to stronger independent, third-party assessment of its models and safety practices, and a signature on the White House Accord on Super Intelligence, in which companies pledged to partner with independent external auditors or evaluators.

What the access actually includes

Anthropic has onboarded six METR employees with company hardware, office space, and access to internal tools and systems that it describes as roughly on par with what its own risk assessment teams have. Outside a small set of excluded categories, customer data, attorney-client privileged information, and information about model architectures, Anthropic says its employees are encouraged to share information with the METR team the same way they would with a colleague working on alignment risk.

The engagement is deliberately narrow. It covers a small team operating under enforceable agreements, and other METR employees, including researchers who work with Anthropic on other projects, are siloed from that team's work. Anthropic frames this as a security boundary: the arrangement gives a few qualified people a close view of its operations under the company's security standards, not a channel for other organizations to obtain Anthropic's or its customers' information.

The veto question

The part that matters for independence is the publication clause. METR will publish its analysis of Anthropic's risk levels, incidents, and practices. Anthropic cannot block that publication, but it can require redaction of information that falls into the predefined categories above. Two caveats come with that: Anthropic ultimately decides what does and does not fall into those categories, and METR reserves the right to make certain statements about the level of access and information it actually received.

Anthropic's own analogy is bank supervision: banks know more about their own risk positions than outsiders do, so regulators place resident examiners inside the largest banks, and if a bank's risk-taking could threaten the public, third parties try to see it in advance. The company says the goal is to close the gap between what an external evaluator understands about a frontier AI company's systems and what the company itself understands. It also lists what it believes evaluators need to do that job: expertise in assessing frontier capabilities, including how evaluations can go wrong, such as alignment faking; independence from the company, with no equity and no financial or operational dependence; deep access to models, training, and safeguards; and freedom to publish.

The three tiers, and who is in them

Anthropic says it is developing three tiers of evaluator access. In the first tier, evaluators run narrowly scoped risk investigations; Anthropic decides what access they get, and they can publish findings only with the company's approval. In the second tier, evaluators can publish without approval and see more information; Anthropic can still require removal of specific categories, but evaluators can publish their views on whether the redactions were justified. The third tier is the METR arrangement. The stated end goal is to have several evaluators at that tier, each publishing its own analysis of Anthropic's risk posture and key decisions.

The other partnerships are earlier and more limited. SecureBio is reviewing the chemical and biological weapons chapter of Anthropic's August Risk Report, after completing and publishing a pilot review of the previous report; when that is done, Anthropic says it will consider moving the relationship to higher access. With Accenture and Faculty, the company is working toward a partnership that includes evaluating and red-teaming models, running alignment assessments, testing safeguards, and building capabilities across both organizations. Anthropic says it is in early discussions with several other organizations, including Hugging Face, and that these partners get more limited information and more limited publication rights for now.

The post closes with a claim that is worth watching: Anthropic says it is embedding evaluators voluntarily and believes every frontier AI company should do the same.