Menu Close

OpenAI says Apollo, METR, Transluce and Goodfire will get deep access for safety assessments

A pale unlabelled standing oval core rests on a deep royal blue field. Four small unmarked rounded modules are seated into the inner wall of the core, and the official white OpenAI blossom stands in the center of that opening with nothing behind it. A tight contact shadow sits under the lowest point of the core, with a soft wider shadow on the field.

OpenAI said Monday, Nov. 16, that it has signed agreements with Apollo Research, METR, Transluce, and Goodfire for a first phase of embedded independent safety assessments, according to a company blog post. CNBC also reported the announcement.

OpenAI said embedded assessments give independent experts “deep access” to a lab’s internal models and systems to assess model alignment and the effectiveness of safeguards over an extended period. Based on what each evaluation needs, the company said, assessors receive access to some of the same internal resources its researchers use, including model checkpoints, source code, training and evaluation data, monitoring systems, technical documentation, and discussions with the researchers and engineers responsible for the work.

The company said the initial assessments focus on alignment and safeguards for models during training and internal use. In the pilot, OpenAI said, Apollo Research will assess whether models could covertly work against their developers’ intentions and whether OpenAI’s safeguards would detect such behavior. OpenAI said it intends to conduct two assessments with METR, and that in the first phase METR will “assess misaligned behavior in our largest reinforcement learning run to date, including failures that our existing tests and monitors may have missed.” A second, broader METR assessment of monitors is pending agreement on scope, the company said. Transluce will conduct an initial assessment of OpenAI’s post-training alignment evaluations, OpenAI said, and Goodfire will embed interpretability researchers to work on training-run monitoring, debugging, and alignment.

OpenAI said the work builds on prior third-party evaluations, “including a recent pilot assessment of our internal monitoring systems with deep internal access and an investigation of the Hugging Face incident.” The company also said the efforts “support our commitments under the White House Accord on Super Intelligence, where we agreed to independent assessments of our safety controls.” The announcement came the same day California Attorney General Rob Bonta sued OpenAI, which AI Tech Daily has reported.

OpenAI said assessors will maintain independence in publishing their assessments, though the company will retain the right to redact confidential information and may request responsible disclosure delays. According to the post, Apollo Research and Goodfire accept financial compensation that is not contingent on results, while METR and Transluce do not accept financial compensation but may accept API credits or compute. CNBC also reported those payment terms.

Sources

0 0 votes
Article Rating
Subscribe
Notify of
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x