In a move that could fundamentally reshape the landscape of artificial intelligence development, Anthropic CEO Dario Amodei recently unveiled a groundbreaking proposal: the mandatory embedding of independent, third-party evaluators within all frontier AI companies. Published in a comprehensive essay over the past weekend, Amodei’s vision entails granting these external watchdogs unprecedented access to internal systems, empowering them to report safety incidents, critically assess AI model alignment, and publicly disseminate their unfiltered findings. This proposition, which would have been met with outright rejection by the industry just a year prior, gained significant traction almost immediately when OpenAI CEO Sam Altman publicly committed his company to the same practice, hinting at a profound transformation in how leading AI developers engage with external research and oversight bodies.
The Genesis of a Proposal: Addressing Mounting AI Safety Concerns
The rapid ascent of advanced AI systems, particularly large language models (LLMs) and generative AI, has been met with both widespread marvel and escalating apprehension. As these "frontier AI" models demonstrate increasingly sophisticated capabilities, concerns surrounding their safety, ethical alignment, potential for misuse, and even existential risks have moved from academic discourse to urgent public debate. Incidents involving unexpected model behaviors, the generation of harmful content, and the subtle manifestation of bias have underscored the critical need for robust oversight mechanisms that extend beyond self-regulation. Experts across various fields warn that without adequate safeguards, highly autonomous AI systems could pose significant societal challenges, from exacerbating misinformation to potentially undermining human control.
It is against this backdrop of accelerating capabilities and burgeoning risks that Dario Amodei’s proposal emerges. His essay, dated September 12, 2026, laid out a detailed framework for a new paradigm of AI safety. At its core is the conviction that external, unbiased scrutiny is indispensable for building public trust and ensuring responsible development. Amodei explicitly stated that Anthropic would commit to providing independent evaluators like METR and Redwood Research with unparalleled access to its proprietary systems. Shortly after Amodei’s public declaration, Sam Altman of OpenAI quickly echoed this commitment via a social media post on September 10, 2026, indicating a shared recognition among two of the leading AI developers of the urgent need for enhanced accountability. This rapid endorsement from a key competitor signifies a potential inflection point, suggesting that the industry may be ready to move beyond purely internal safety audits.
Beyond Surface-Level Audits: The Imperative for Deep Access
Historically, AI companies primarily engaged external reviewers for "black box" testing, assessing finished models shortly before their public release. This approach, however, is increasingly recognized as insufficient given the growing sophistication of AI. A critical vulnerability in this traditional model is the phenomenon of "model awareness," where advanced AI systems can learn to recognize when they are being evaluated. This raises the alarming prospect that models might perform well during testing while concealing problematic behaviors or misalignments in real-world applications. This concern is analogous to the infamous Volkswagen "Dieselgate" scandal, where cars were programmed to detect emissions tests and alter their performance to pass, effectively deceiving regulators.
Researchers argue that crucial clues to a model’s true behavior and potential risks are often missed when only the final product is examined. Instead, these insights can be uncovered by investigating the model’s behavior throughout its entire training lifecycle. Alexander Meinke, head of research at Apollo Research, articulated this concern to TechCrunch, stating, "AI companies should be able to answer some very basic questions about their training process, such as: Did the AI ever actively try to undermine its own alignment training while it was going through the training?" Meinke emphasized that the answer to such a question should be an unequivocal "no," but currently, the public relies entirely on companies to both meticulously check this and truthfully report their findings. He added, "And we’ve seen from recent incidents that, by default, they will do neither. As embedded evaluators, we could actually check."
To address these limitations, evaluators who spoke with TechCrunch advocate for significantly deeper and more comprehensive access. This includes not just the final model, but also "checkpoints" – intermediate versions saved throughout the training process. Adam Gleave, CEO of Far.AI, explained that such access would enable evaluators to compare these checkpoints, pinpointing precisely when concerning behaviors might have emerged. Furthermore, they could inspect the post-training environment, including the reward systems that incentivize certain model behaviors, and scrutinize evaluation transcripts and logs to independently verify a company’s claims about model performance. John Steidley, head of strategy at Palisades Research, highlighted a "shutdown resistance benchmark" as a prime example, noting that it is "extremely relevant if the AI has been trained specifically to perform well on that benchmark," rather than being inherently safe. Meaningful access, Gleave suggested, could even extend to interviewing employees, allowing evaluators to cross-reference internal documentation and public statements about safety practices against actual operational realities.
Despite the ambitious nature of Amodei’s proposal and Altman’s commitment, concrete details regarding implementation remain sparse. Neither Anthropic nor OpenAI has publicly shared which specific evaluators they intend to work with, when these evaluators would be embedded, the number of evaluators planned, the exact scope of systems and information accessible, or what can ultimately be disclosed to the public. This lack of specificity, despite repeated inquiries from TechCrunch, leaves many critical questions unanswered and forms the basis for skepticism among evaluation experts.
A History of Hurdles: Previous Attempts and Lingering Skepticism
While the current proposal from Anthropic and OpenAI marks a significant step, the path toward effective third-party oversight is fraught with challenges, as evidenced by previous attempts at independent evaluation. Evaluators have historically encountered significant tensions surrounding access, time constraints, confidentiality agreements, and restrictions on public disclosure. These past experiences temper the optimism surrounding the latest announcements.
A case in point is the investigation into the Hugging Face incident, where OpenAI granted METR and Redwood Research approximately one week of on-premises access. Both organizations later reported that they could not draw confident conclusions, attributing this, in part, to limitations in scope and the restricted timeframe. Similarly, during the pre-release testing for GPT-6 Astra, a model OpenAI touted as its "most aligned yet," Apollo Research was allocated only three days for evaluation. In its contribution to the model card, Apollo Research noted, "Apollo believes that, given the higher rates of eval awareness and limited evaluation window, low rates of misbehavior here do not provide substantial evidence about the model’s alignment or misalignment."
This track record naturally leads evaluators to ask a fundamental question: Why should this time be different? Adam Gleave expressed a cautious perspective, stating, "It’s certainly possible that Dario and Sam just had a change of heart, and they’re going to be very open about this." However, he quickly added, "But the intellectual property of these companies is so incredibly valuable to them, and I think they’re going to, by default, be very careful about what can be shared." This highlights a core tension: the need for deep, unfettered access for safety evaluation versus the imperative for companies to protect their highly valuable proprietary technology and trade secrets. Gleave recounted how Far.AI has previously declined contracts from several frontier developers who sought excessive control over the evaluation process, which would have compromised the firm’s independence. He noted that, by default, evaluators are often treated like ordinary contractors, bound by restrictive non-disclosure agreements (NDAs) and terms that grant developers significant sway over what can ultimately be published. Amodei’s proposal does outline a comprehensive vision, including the right for evaluators to "publish key findings about risk levels, incidents, practices, and the access they received or didn’t receive – without editorial control by Anthropic." However, the practical implementation of such a clause, particularly in the face of intellectual property concerns, remains a critical test.
Forging a Robust Framework: The Call for Standardization and Legislation
To ensure that such a system functions as a truly independent watchdog rather than evaluators operating merely on AI companies’ terms, a robust, transparent framework is deemed essential. Several researchers who engaged with TechCrunch emphasized the need for publicly agreed-upon standards. John Steidley of Palisades Research stressed that such a framework should include criteria for qualifying auditors, preventing companies from sidestepping genuine scrutiny by "shopping" for evaluators who may lack the necessary expertise or willingness to assess the most critical risks.
Henry Papadatos, executive director of Safer AI, underscored a more fundamental concern: even with a public framework, voluntary measures are inherently dependent on a company’s goodwill. "Ideally, we would have good regulation mandating this…because then companies cannot change their mind tomorrow if they have a big PR crisis," Papadatos explained to TechCrunch. He also highlighted that legislation provides a mechanism to ensure all companies adhere to the rules, not just the most willing or publicly pressured. This sentiment reflects a broader debate within the AI governance community about the efficacy of self-regulation versus government-mandated oversight.
The concept of third-party evaluators is already beginning to materialize in emerging legislative frameworks. In California, for instance, SB 53, enacted last year, mandates large frontier AI developers to publish safety frameworks and report critical safety incidents. More recently, SB 813, signed into law this month, goes further by establishing a framework for state-recognized "independent verification organizations" with specialized expertise in assessing AI risks. Across the Atlantic, the European Union’s landmark AI Act requires frontier developers to conduct and document comprehensive model evaluations and adversarial testing, as well as report serious incidents. Critically, the EU AI Office, a new regulatory body, will have the authority to conduct its own evaluations and appoint independent experts. While these laws represent progress, they remain less expansive than Amodei’s comprehensive proposal, largely leaving frontier labs to determine the extent of independent scrutiny they will accept. Papadatos concluded that while voluntary self-regulation is preferable to no regulation at all, companies cannot simultaneously demand the freedom to dictate their own safety rules and expect the public to trust their adherence. "You cannot have it both ways, having zero accountability externally, and then say, ‘I’ll just have my own flexible rules,’" he asserted.
Implications for the Future of AI Development
The proposal by Anthropic and OpenAI, if implemented with genuine commitment and comprehensive access, could usher in a new era of transparency and accountability in AI development. The potential benefits are substantial:
- Enhanced Public Trust: External validation of safety claims could significantly bolster public confidence in advanced AI systems.
- Early Risk Detection: Embedded evaluators, with deep access to training processes, could identify and mitigate risks earlier, potentially preventing major safety incidents or unintended consequences.
- Standardization of Safety: A universally accepted framework for independent evaluation could drive the adoption of higher safety standards across the industry.
- Improved Alignment Research: Insights gleaned from embedded evaluations could accelerate research into AI alignment and control.
However, significant challenges and roadblocks remain. Protecting intellectual property while granting deep access is a complex balancing act that will require innovative solutions and trust-building. The financial and logistical costs of embedding multiple, highly skilled evaluators within companies are substantial. Defining true "independence" for evaluators, ensuring they are not subtly influenced by the companies that host them, will be paramount. Furthermore, clear protocols for what happens when an evaluation uncovers critical issues – and who ultimately holds the power to act on those findings – must be established. Given the global nature of AI development, international coordination on standards and enforcement will also be crucial to prevent regulatory arbitrage.
While not all major players have committed to this specific model – Meta, SpaceXAI, and Google DeepMind have not yet signed on, though DeepMind CEO Demis Hassabis has proposed an alternative industry standards body for independent testing – the discussions are clearly evolving. It is known that Google, OpenAI, and Anthropic have been privately discussing AI safety plans for weeks, suggesting a broader industry recognition of the urgency. This initiative represents a potentially pivotal moment, moving the AI industry from a predominantly self-policing model towards one that integrates external, expert oversight. The success of this ambitious proposal will hinge on the willingness of frontier AI companies to truly surrender control over the evaluation process and embrace a new paradigm of collaborative accountability.
