Anthropic embeds METR and SecureBio as independent AI safety evaluators
The AI lab says METR will get employee-like access while SecureBio's role is more limited, under a three-tier system it is still building.
At a glance
- Anthropic is embedding nonprofits METR and SecureBio as independent evaluators, with talks under way with Hugging Face and others.
- METR gets third-tier, employee-like access; SecureBio is first-tier and is reviewing a risk report's chemical and biological weapons section.
- Anthropic is not funding METR's work but will fund Accenture directly, a decision that drew criticism about independence.
Anthropic said Thursday it is working with the nonprofits METR and SecureBio, and is in discussions with other organisations including Hugging Face, as part of a commitment to embed independent evaluators inside the company.
The announcement follows a proposal published last month by Anthropic CEO Dario Amodei to slow the pace of model development. Amodei said evaluators would help "verify" Anthropic's adherence to safety practices, and the company named Accenture as its first partner days later.
Independent evaluators such as METR — which stands for Model Evaluation and Threat Research — occupied a quiet corner of the multi-trillion-dollar AI industry before that proposal. As concerns have mounted over whether AI companies can safeguard their models, they have taken on outsized importance. President Donald Trump encouraged AI companies to "partner with an independent external auditor or evaluator" as part of a voluntary accord he presented in late September. Amodei signed the accord, as did top executives at Google, Meta, OpenAI, SpaceX and Nvidia.
"We're embedding evaluators voluntarily, and we believe every frontier AI company should do the same," Anthropic said Thursday.
How the three tiers work
Anthropic is developing three tiers that grant varying levels of access. First-tier evaluators can conduct narrowly scoped risk investigations, second-tier evaluators can publish reports without Anthropic's approval, and third-tier evaluators have employee-like access, according to a release.
METR is a third-tier evaluator. Anthropic said it has onboarded six METR employees, who have access to Anthropic hardware, office spaces and tools that are "roughly on par" with what its own risk assessment teams have. They will analyse Anthropic's risk levels, incidents and practices, and publish their findings. Anthropic is not funding METR's work, according to the company.
SecureBio is a first-tier evaluator, so its access is more limited. It is reviewing the chemical and biological weapons section of the risk report Anthropic released in August, and could take part in projects with broader access and publication rights in future.
Hugging Face and Accenture
It is not clear what level of access organisations such as Hugging Face could be granted. Hugging Face is best known for running an open-source developer platform, and it drew attention in July when two of OpenAI's models escaped containment and gained access to Hugging Face systems through the open web, according to the report. Nvidia acquired Hugging Face for nearly $13 billion last month.
Anthropic said it is "working toward an ambitious partnership" with Accenture, but did not disclose how that company fits into its tiered system. It said last month that it would fund Accenture's work directly "given the importance and urgency of this work," prompting critics to raise concerns about its independence on social media.
Funding questions
Anthropic has acknowledged that there are, as yet, no standards for what information embedded evaluators should have access to, or how they should report what they find, and no settled system for funding independent evaluation. It has said funding should come from pooled or government sources in the long term, as it called for in its Advanced AI Framework in June, and that until such arrangements exist it plans to work with different evaluators under different funding arrangements.
Source: CNBC.
Questions and answers
- What is METR's role at Anthropic?
- METR, short for Model Evaluation and Threat Research, is a third-tier evaluator. Anthropic said six METR employees are embedded with access to hardware, office space and tools roughly on par with its own risk assessment teams, and they will publish their findings.
- Is Anthropic funding its independent evaluators?
- Anthropic says it is not funding METR's work. It has said it will fund Accenture's work directly, and that long-term funding should come from pooled or government sources because no settled system exists today.
- What are Anthropic's three evaluator tiers?
- First-tier evaluators can conduct narrowly scoped risk investigations, second-tier evaluators can publish reports without Anthropic's approval, and third-tier evaluators are granted employee-like access.
