Are AI companies committed to a safe future?

We rate frontier AI companies' risk management practices. Our objective is to make frontier AI safer through improved risk management practices and to enhance the accountability of private actors shaping the development of AI.

Advanced AI systems are arguably one of the most transformative technologies ever built.

Frontier AI companies are advancing technology at an astonishing rate, leading the three most cited AI researchers—Geoffrey Hinton, Ilya Sutskever, and Yoshua Bengio—to believe it will have a profound impact on societies and could even potentially contribute to their downfall. 

Yet, we don’t currently know how to properly manage AI risks. Frontier models are quickly becoming more capable, and they already pose potential large-scale risks. For example, they can now significantly help people with basic technical backgrounds create biological weapons. As AI capabilities continue to advance rapidly, the gap between these growing risks and our inadequate risk management frameworks widens. This situation underscores the urgent need for robust risk management practices in the AI industry.

At the AI Seoul Summit, multiple companies pledged to publish safety frameworks by February 2025, detailing how they manage risks when developing and deploying frontier AI models and systems. We have assessed the 12 companies that fulfilled this commitment: Amazon, Anthropic, Cohere, G42, Google DeepMind, Magic, Meta, Microsoft, Naver, NVIDIA, OpenAI, and xAI.

Six companies: 01.AI, Inflection AI, Minimax, Mistral AI, Technology Innovation Institute, and Zhipu AI, have not published their frameworks despite their commitments. Additionally, IBM summarized its existing AI governance practices without addressing severe AI risks, while Samsung published a framework limited to on-device AI safety, neither of which met our criteria for frontier AI risk management evaluation. These approaches fall outside the scope of frontier AI risk management. 

Without published frameworks, these companies cannot demonstrate any risk management practices, resulting in the lowest possible assessment, a score of 0%.

• Risk identification is partially addressed, with companies typically covering 2-4 risks, though many improperly exclude the risks that they do not fully understand. Companies generally address core risks from domains such as CBRN (Chemical, Biological, Nuclear, Radiological), cyber offense, and in a few cases, automated AI R&D. Some companies, like GoogleDeepMind and OpenAI, show sophistication by breaking down complex loss of control risks into granular categories such as deceptive alignment and autonomous replication.
• Some companies recognize the importance of risk modeling and define measurable Key Risk Indicators (KRIs) relatively well. Companies have established KRI thresholds such as capability thresholds, which indicate types of evaluations that will be needed to determine if the capability threshold has been crossed. However, KRIs should in general map to the actual evaluations performed, which is not often described currently in companies’ frameworks. 
• Some companies have established clear risk governance structures. For instance, companies like G42, OpenAI, Microsoft, and Anthropic have created specialized governance bodies including dedicated risk officers, safety advisory groups, and risk councils that represent meaningful institutionalization of risk management. Nonetheless, these remain fairly high-level and crucial details about final decision authority, oversight mechanisms, and handling of edge cases, are lacking. 
• Additional areas where companies show promise include early steps towards transparency and inclusion of third parties. Companies like Anthropic, Microsoft, and Cohere have made commitments to share evaluation results with stakeholders, engage third parties in assessments, and report externally on governance structures, representing positive movement toward industry accountability. 

• Risk tolerance (the maximum level of risk companies are willing to impose on society) is almost universally undefined, creating a fundamental gap where companies set capability thresholds and select mitigations without explicit bounds to ensure risks remain acceptable. This fundamental weakness undermines the entire risk management process, as companies cannot make principled decisions about acceptable risk levels without explicit tolerance thresholds. Instead, the criteria for these decisions remain discretionary and vague. 
• Open-ended red teaming, which is necessary to discover unknown, emergent risks, remains largely absent from frameworks, leaving companies unprepared for unforeseen dangers that arise as AI capabilities advance beyond the existing risk categories. Most companies focus narrowly on known risk domains without having systematic processes for identifying novel threats that may emerge as capabilities evolve. Given that AI companies are explicitly aiming to develop novel capabilities in their frontier models, this is crucial. 
• Most companies do not have a policy to put development on hold if they cannot sufficiently mitigate risk, or their threshold for doing so is vaguely defined, discretionary and non pre-emptive. While companies provide thresholds for when deployment would be paused, the same does not apply for development. This is problematic since many risks arise already during development. This represents a critical failure to prioritize safety over business objectives. 
• Separation of roles and responsibilities remains limited. A sound risk governance framework calls for distinct and separate entities and roles that can act as checks and balances on each other. The well-known Three Lines model, for example, calls for independent internal audit, clear responsibilities for the Board of Directors and an appropriately resourced central risk function that can challenge management on its risk decisions. None of the AI companies currently have all necessary components in place and only a few have distinct audit or risk teams with any independence.
• Most frameworks are reactive rather than proactive with mitigations, failing to develop comprehensive, measurable  ex ante risk management plans specifying precisely how they will mitigate risk, and why they believe these measures will be sufficient. To improve, frameworks should include more details on how they will evaluate the quality of their risk mitigation measures and provide proof that such measures will be sufficient before risks materialize. This is a critical gap given that effective mitigation measures require substantial development time and extensive advance planning.
• There is a concerning development of ‘marginal risk clauses’, which make deployment decisions contingent on other companies’ risk tolerances. One company to include such a clause is OpenAI. Risk management is predicated on having an ‘unacceptable risk level’ that companies commit not to cross. Importantly then, this level should be independent of other companies’ risk levels. We strongly caution against these clauses, as they go against the spirit of risk management.