External evaluators found vulnerabilities in OpenAI monitoring system during pilot
The nonprofits said the three-week exercise uncovered an exploit that could stop hundreds of actions from being monitored; OpenAI says it patched the flaws.
At a glance
- METR and Redwood Research ran a July 21-Aug. 11 pilot review of OpenAI's internal monitoring system.
- Evaluators found novel vulnerabilities, including an exploit that could stop hundreds of actions from being monitored.
- OpenAI says it patched the identified vulnerabilities affecting its production safeguards.
Nonprofit evaluators METR and Redwood Research said Thursday they found several "novel vulnerabilities" in an internal OpenAI monitoring system during a three-week pilot that gave outside researchers deep access to the artificial intelligence company's infrastructure.
The exercise, which ran from July 21 to Aug. 11, was designed to test what it would look like for OpenAI to give external researchers "a deep level of access" to its internal systems, the two organizations said.
One employee from each group had access to OpenAI's internal codebase, documentation and some technical staffers, according to a release. The employees were chaperoned while on-site and used OpenAI-issued devices that "never left the premises."
What the evaluators found
Among the issues discovered was an exploit that would allow agents to prevent "hundreds" of recent actions from being monitored. OpenAI told METR and Redwood that it had patched the identified vulnerabilities that affect its production safeguards.
The evaluators stress-tested OpenAI's internal misalignment monitoring system. In AI, alignment describes whether a model behaves in line with human interests and values. METR and Redwood assessed the system as it existed on and before July 21, and noted that OpenAI was expanding the system during the exercise.
METR and Redwood said they agreed to publish only a high-level public post, because the pilot was not designed to "provide robust public transparency about OpenAI's monitoring system." They shared a 61-page internal report with OpenAI and said they expect to publicly share more details about future exercises.
"This collaboration builds on our longstanding commitment to pioneering new approaches to third-party testing, and we're using what we learned to strengthen our safeguards and shape future collaborations with independent evaluators," an OpenAI spokesperson told CNBC in a statement.
A wider push for outside scrutiny
The pilot follows OpenAI CEO Sam Altman's September pledge to grant independent evaluators "employee-like access" to the company, after Anthropic CEO Dario Amodei proposed the idea as part of his plan to slow the pace of AI development. Anthropic has since announced agreements with organizations including Accenture, METR and SecureBio, and OpenAI said earlier this month that it is "actively finalizing contracts with third-party safety assessors."
President Donald Trump, who has taken a light-touch approach to AI regulation, encouraged AI companies to "partner with an independent external auditor or evaluator" as part of a voluntary accord he presented at the White House. Amodei signed the accord, as did top executives at Google, Meta, OpenAI, SpaceX and Nvidia.
Independent evaluators have historically occupied a relatively quiet corner of the multitrillion-dollar AI industry, and experts told CNBC that questions remain about how these groups will be funded and what their reporting structures will ultimately look like.
Source: CNBC Top News.
Questions and answers
- What did METR and Redwood Research find at OpenAI?
- During a pilot from July 21 to Aug. 11, the nonprofits found several "novel vulnerabilities," including an exploit allowing agents to prevent "hundreds" of recent actions from being monitored. OpenAI said it patched the issues affecting production safeguards.
- Who had access to OpenAI's systems during the pilot?
- One employee from each organization had access to OpenAI's internal codebase, documentation and some technical staffers. They were chaperoned on-site and used OpenAI-issued devices that never left the premises.
- Why was only a high-level summary published?
- METR and Redwood said the pilot was not designed to "provide robust public transparency" about OpenAI's monitoring system. They gave OpenAI a 61-page internal report and expect to share more details about future exercises.
