Comparison table

presenting our findings

Our methodology allows for extensive, apples-to-apples comparisons between companies on over 65 different risk management criteria.

The table below shows both overall ratings, and ratings for each criterion. We also present the best in class for each criterion, to encourage other companies to adopt these industry-leading risk management practices. 

For more detail on the rationale behind each rating, you can click on the score itself to see a detailed explanation with supporting quotes. To see a company's full breakdown, you can click on the company icon.

For more detail on how scores are calculated, you can click to see our methodology at the button below.

See our methodology

Filter:
  • All
  • Frontier Companies
Click logos to visit company pages
Overall score
Anthropic
35%
33%
25%
22%
20%
19%
18%
18%
16%
11%
10%
8%
1. Risk Identification
OpenAI
26%
32%
17%
23%
20%
7%
7%
11%
14%
12%
7%
8%
1.1 Classification of Applicable Known Risks (40%)
OpenAI
38%
63%
25%
18%
43%
13%
13%
13%
18%
25%
13%
10%
1.1.1 Risks from literature and taxonomies are well covered (50%)
OpenAI
50%
75%
25%
25%
75%
25%
25%
25%
25%
50%
25%
10%
1.1.2 Exclusions are clearly justified and documented (50%)
OpenAI
25%
50%
25%
10%
10%
0%
0%
0%
10%
0%
0%
10%
1.2 Identification of Unknown Risks (Open-ended red teaming) (20%)
Amazon
0%
0%
7%
0%
0%
0%
0%
10%
7%
0%
0%
0%
1.2.1 Internal open-ended red teaming (70%)
G42
0%
0%
10%
0%
0%
0%
0%
10%
10%
0%
0%
0%
1.2.2 Third party open-ended red teaming (30%)
Amazon
0%
0%
0%
0%
0%
0%
0%
10%
0%
0%
0%
0%
1.3 Risk modeling (40%)
Meta
29%
18%
15%
41%
7%
5%
4%
11%
14%
4%
4%
9%
1.3.1 The company uses risk models for all the risk domains identified
and the risk models are published (with potentially dangerous
information redacted) (40%)
Meta
25%
25%
10%
50%
10%
0%
10%
10%
25%
10%
10%
10%
1.3.2 Risk modeling methodology (40%)
Meta
21%
9%
2%
39%
9%
8%
0%
4%
9%
0%
0%
11%
1.3.2.1 Methodology precisely defined (70%)
Meta
25%
10%
0%
50%
10%
0%
0%
0%
0%
0%
0%
0%
1.3.2.2 Mechanism to incorporate red teaming findings (15%)
NVIDIA
0%
0%
0%
0%
0%
0%
0%
0%
10%
0%
0%
0%
1.3.2.3 Prioritization of severe and probable risks (15%)
Cohere
25%
10%
10%
25%
10%
50%
0%
25%
50%
0%
0%
75%
1.3.3 Third party validation of risk models (20%)
Anthropic
50%
25%
50%
25%
0%
10%
0%
25%
0%
0%
0%
0%
2. Risk Analysis and Evaluation
Meta
21%
25%
18%
30%
19%
13%
27%
16%
11%
14%
7%
5%
2.1 Setting a Risk Tolerance (35%)
xAI
7%
16%
7%
22%
7%
7%
33%
7%
3%
9%
3%
3%
2.1.1 Risk tolerance is defined (80%)
xAI
8%
20%
8%
28%
8%
8%
41%
8%
3%
12%
3%
3%
2.1.1.1 Risk tolerance is at least qualitatively defined for all risks (33%)
Meta
25%
50%
25%
75%
25%
25%
50%
25%
10%
25%
10%
10%
2.1.1.2 Risk tolerance is expressed at least partly quantitatively as a combination of scenarios (qualitative) and probabilities (quantitative) for all risks (33%)
OpenAI
0%
10%
0%
10%
0%
0%
0%
0%
0%
10%
0%
0%
2.1.1.3 Risk tolerance is expressed fully quantitatively as a product of severity (quantitative) and probability (quantitative) for all risks (33%)
Anthropic
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
2.1.2 Process to define the tolerance (20%)
Anthropic
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
2.1.2.1 AI developers engage in public consultations or seek guidance from regulators where available (50%)
Anthropic
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
2.1.2.2 Any significant deviations from risk tolerance norms established in other industries is justified and documented (e.g., cost-benefit analyses) (50%)
Anthropic
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
2.2 Operationalizing Risk Tolerance (65%)
Meta
29%
29%
24%
34%
26%
17%
25%
22%
16%
17%
9%
6%
2.2.1 Key Risk Indicators (KRI) (30%)
Anthropic
33%
33%
22%
33%
33%
22%
21%
21%
15%
21%
15%
15%
2.2.1.1 KRI thresholds are at least qualitatively defined for all risks (45%)
Anthropic
50%
50%
25%
50%
50%
25%
25%
25%
10%
25%
10%
10%
2.2.1.2 KRI thresholds are quantitatively defined for all risks (45%)
Anthropic
25%
25%
10%
0%
10%
10%
10%
25%
0%
25%
0%
0%
2.2.1.3 KRIs also identify and monitor changes in the level of risk in the external environment (10%)
Meta
0%
0%
10%
25%
0%
10%
0%
0%
10%
0%
0%
10%
2.2.2 Key Control Indicators (KCI) (30%)
OpenAI
24%
32%
25%
15%
31%
11%
21%
18%
6%
13%
4%
6%
2.2.2.1 Containment KCIs (35%)
DeepMind
30%
5%
45%
38%
63%
25%
25%
25%
13%
25%
5%
13%
2.2.2.1.1 All KRI thresholds have corresponding qualitative containment KCI thresholds (50%)
G42
50%
10%
90%
75%
75%
50%
50%
50%
25%
50%
10%
25%
2.2.2.1.2 All KRI thresholds have corresponding quantitative containment KCI thresholds (50%)
DeepMind
10%
0%
0%
0%
50%
0%
0%
0%
0%
0%
0%
0%
2.2.2.2 Deployment KCIs (35%)
OpenAI
30%
43%
25%
5%
5%
5%
25%
25%
5%
13%
5%
5%
2.2.2.2.1 All KRI thresholds have corresponding qualitative deployment KCI thresholds (50%)
OpenAI
50%
75%
50%
10%
10%
10%
50%
50%
10%
25%
10%
10%
2.2.2.2.2 All KRI thresholds have corresponding quantitative deployment KCI thresholds (50%)
Anthropic
10%
10%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
2.2.2.3 For advanced KRIs, assurance process KCIs are defined (30%)
OpenAI
10%
50%
0%
0%
25%
0%
10%
0%
0%
0%
0%
0%
2.2.3 Pairs of thresholds are grounded in risk modeling to show that risks remain below the tolerance (20%)
OpenAI
10%
25%
25%
25%
25%
10%
10%
25%
25%
10%
10%
0%
2.2.4 Policy to put development on hold if the required KCI threshold cannot be achieved, until sufficient controls are implemented to meet the threshold (20%)
Meta
50%
25%
25%
75%
10%
25%
50%
25%
25%
25%
10%
0%
3. Risk Treatment
Anthropic
41%
38%
24%
20%
26%
28%
15%
23%
14%
8%
8%
12%
3.1 Implementing Mitigation Measures (50%)
Amazon
32%
37%
23%
15%
22%
27%
18%
38%
19%
9%
4%
12%
3.1.1 Containment measures (35%)
Amazon
40%
40%
34%
10%
0%
49%
0%
74%
30%
6%
0%
19%
3.1.1.1 Containment measures are precisely defined for all KCI thresholds (60%)
Amazon
50%
50%
50%
10%
0%
75%
0%
90%
50%
10%
0%
25%
3.1.1.2 Proof that containment measures are sufficient to meet the thresholds (40%)
Amazon
25%
25%
10%
10%
0%
10%
0%
50%
0%
0%
0%
10%
3.1.1.3 Strong third party verification process to verify that the containment measures meet the threshold (100% if 3.1.1.3 > [60% x 3.1.1.1 + 40% x 3.1.1.2])
Anthropic
25%
25%
10%
0%
0%
10%
0%
0%
0%
0%
0%
10%
3.1.2 Deployment measures (35%)
xAI
40%
40%
30%
25%
40%
25%
50%
25%
19%
15%
10%
15%
3.1.2.1 Deployment measures are precisely defined for all KCI thresholds (60%)
Anthropic
50%
50%
50%
25%
50%
25%
25%
25%
25%
25%
10%
25%
3.1.2.2 Proof that deployment measures are sufficient to meet the thresholds (40%)
Anthropic
25%
25%
0%
25%
25%
25%
25%
25%
10%
0%
10%
0%
3.1.2.3 Strong third party verification process to verify that the deployment measures meet the threshold (100% if 3.1.2.3 > [60% x 3.1.2.1 + 40% x 3.1.2.2])
xAI
10%
25%
25%
0%
0%
0%
50%
0%
0%
0%
0%
0%
3.1.3 Assurance processes (30%)
OpenAI
14%
30%
2%
8%
28%
5%
3%
10%
7%
5%
0%
0%
3.1.3.1 Credible plans towards the development of assurance properties (40%)
Anthropic
25%
25%
0%
10%
25%
10%
10%
25%
10%
10%
0%
0%
3.1.3.2 Evidence that the assurance properties are enough to achieve their corresponding KCI thresholds (40%)
OpenAI
10%
50%
0%
0%
25%
10%
0%
0%
10%
0%
0%
0%
3.1.3.3 The underlying assumptions that are essential for their effective implementation and success are clearly outlined (20%)
DeepMind
10%
10%
10%
25%
50%
10%
0%
10%
0%
10%
0%
0%
3.2 Continuous Monitoring and Comparing Results with Pre-determined Thresholds (50%)
Anthropic
51%
39%
25%
26%
30%
29%
11%
8%
9%
7%
13%
12%
3.2.1 Monitoring of KRIs (40%)
Anthropic
64%
36%
31%
20%
27%
50%
2%
13%
2%
16%
23%
0%
3.2.1.1 Justification that elicitation methods used during the evaluations are comprehensive enough to match the elicitation efforts of potential threat actors (30%)
OpenAI
75%
90%
50%
50%
50%
75%
0%
25%
0%
0%
0%
0%
3.2.1.2 Evaluation frequency (25%)
Anthropic
100%
0%
50%
0%
10%
75%
0%
0%
0%
0%
90%
0%
3.2.1.3 Description of how post-training enhancements are factored into capability assessments (15%)
Anthropic
50%
25%
0%
25%
25%
50%
0%
0%
10%
0%
0%
0%
3.2.1.4 Vetting of protocols by third parties (15%)
G42
10%
10%
25%
0%
10%
0%
0%
10%
0%
25%
0%
0%
3.2.1.5 Replication of evaluations by third parties (15%)
Anthropic
50%
25%
0%
10%
25%
10%
10%
25%
0%
0%
0%
0%
3.2.2 Monitoring of KCIs (40%)
Anthropic
43%
43%
21%
20%
33%
4%
15%
0%
10%
0%
10%
13%
3.2.2.1 Detailed description of evaluation methodology and justification that KCI thresholds will not be crossed unnoticed (40%)
DeepMind
50%
50%
25%
50%
75%
10%
0%
0%
25%
0%
25%
25%
3.2.2.2 Vetting of protocols by third parties (30%)
OpenAI
25%
50%
25%
0%
10%
0%
0%
0%
0%
0%
0%
0%
3.2.2.3 Replication of evaluations by third parties (30%)
Anthropic
50%
25%
10%
0%
0%
0%
50%
0%
0%
0%
0%
10%
3.2.3 Transparency of evaluation results (10%)
Anthropic
77%
64%
21%
21%
43%
64%
43%
21%
43%
0%
0%
43%
3.2.3.1 Sharing of evaluation results with relevant stakeholders as appropriate (85%)
Anthropic
90%
75%
25%
25%
50%
75%
50%
25%
50%
0%
0%
50%
3.2.3.2 Commitment to non-interference with findings (15%)
G42
0%
0%
25%
0%
0%
0%
0%
0%
0%
0%
0%
0%
3.2.4 Monitoring for novel risks (10%)
Meta
5%
10%
25%
75%
18%
10%
0%
5%
5%
0%
0%
25%
3.2.4.1 Identifying novel risks post-deployment: engages in some process (post deployment) explicitly for identifying novel risk domains or novel risk models within known risk domains (50%)
Meta
0%
10%
25%
75%
10%
10%
0%
10%
10%
0%
0%
50%
3.2.4.2 Mechanism to incorporate novel risks identified post-deployment (50%)
Meta
10%
10%
25%
75%
25%
10%
0%
0%
0%
0%
0%
0%
4. Risk Governance
Anthropic
49%
39%
42%
15%
16%
27%
23%
22%
23%
10%
17%
7%
4.1 Decision-making (25%)
G42
50%
34%
60%
30%
13%
38%
40%
34%
44%
19%
13%
5%
4.1.1 The company has clearly defined risk owners for every key risk identified and tracked (25%)
Microsoft
50%
10%
0%
10%
0%
75%
75%
25%
50%
0%
0%
10%
4.1.2 The company has a dedicated risk committee at the management level that meets regularly (25%)
G42
0%
0%
90%
25%
0%
0%
0%
10%
0%
0%
0%
0%
4.1.3 The company has defined protocols for how to make go/no-go decisions (25%)
Anthropic
75%
75%
75%
75%
50%
75%
0%
75%
50%
50%
50%
10%
4.1.4 The company has defined escalation procedures in case of incidents (25%)
Anthropic
75%
50%
75%
10%
0%
0%
75%
25%
75%
25%
0%
0%
4.2. Advisory and Challenge (20%)
OpenAI
35%
48%
25%
21%
28%
13%
4%
14%
35%
5%
12%
6%
4.2.1 The company has an executive risk officer with sufficient resources (16.7%)
Anthropic
75%
0%
25%
0%
0%
0%
0%
0%
0%
0%
0%
25%
4.2.2 The company has a committee advising management on decisions involving risk (16.7%)
OpenAI
10%
90%
0%
25%
90%
0%
0%
0%
0%
0%
25%
0%
4.2.3 The company has an established system for tracking and monitoring risks (16.7%)
OpenAI
50%
75%
25%
50%
50%
25%
25%
25%
75%
10%
0%
10%
4.2.4 The company has designated people that can advise and challenge management on decisions involving risk (16.7%)
OpenAI
25%
50%
50%
25%
0%
0%
0%
10%
50%
0%
10%
0%
4.2.5 The company has an established system for aggregating risk data and reporting on risk to senior management and the Board (16.7%)
OpenAI
50%
75%
50%
25%
25%
25%
0%
50%
10%
10%
10%
0%
4.2.6 The company has an established central risk function (16.7%)
NVIDIA
0%
0%
0%
0%
0%
0%
0%
0%
50%
10%
25%
0%
4.3 Audit (20%)
G42
50%
38%
70%
5%
5%
43%
25%
25%
0%
5%
5%
13%
4.3.1 The company has an internal audit function involved in AI governance (50%)
Microsoft
25%
0%
50%
0%
0%
75%
0%
0%
0%
0%
0%
0%
4.3.2 The company involves external auditors (50%)
G42
75%
75%
90%
10%
10%
10%
50%
50%
0%
10%
0%
25%
4.4 Oversight (20%)
Anthropic
50%
45%
0%
0%
25%
5%
0%
0%
0%
5%
45%
0%
4.4.1 The Board of Directors of the company has a committee that provides oversight over all decisions involving risk (50%)
OpenAI
25%
90%
0%
0%
0%
10%
0%
0%
0%
10%
90%
0%
4.4.2 The company has other governing bodies outside of the Board of Directors that provide oversight over decisions (50%)
Anthropic
75%
0%
0%
0%
50%
0%
0%
0%
0%
0%
0%
0%
4.5 Culture (10%)
Anthropic
63%
15%
47%
3%
7%
32%
50%
20%
37%
12%
8%
7%
4.5.1 The company has a strong tone from the top (33.3%)
Anthropic
50%
25%
50%
10%
10%
10%
25%
50%
25%
25%
25%
10%
4.5.2 The company has a strong risk culture (33.3%)
NVIDIA
50%
10%
0%
0%
10%
10%
50%
0%
75%
10%
0%
0%
4.5.3 The company has a strong speak-up culture (33.3%)
Anthropic
90%
10%
0%
0%
0%
75%
75%
0%
10%
0%
0%
0%
4.6 Transparency (5%)
Anthropic
72%
53%
72%
33%
20%
58%
45%
67%
37%
20%
23%
28%
4.6.1 The company reports externally on what their risks are (33.3%)
OpenAI
50%
75%
50%
50%
25%
75%
75%
50%
10%
50%
10%
50%
4.6.2 The company reports externally on what their governance structure looks like (33.3%)
Microsoft
75%
75%
75%
25%
25%
90%
10%
75%
50%
10%
50%
10%
4.6.3 The company shares information with industry peers and government bodies (33.3%)
Anthropic
90%
10%
90%
25%
10%
10%
50%
75%
50%
0%
10%
25%