OpenAI

1.3.2.1 Methodology precisely defined (70%) 10%

It is not clear what the methodology is from The Preparedness Framework, or that a particular methodology is followed. However, they do mention identifying causal pathways, which implies some methodology. More detail should be given.

Quotes:

“It must be possible to identify a causal pathway for a severe harm in the capability area, enabled by frontier AI.” (PF, p.4)

“Capability thresholds concretely describe things an AI system might be able to help someone do or might be able to do on its own that could meaningfully increase risk of severe harm.” (PF, p.4)

“OpenAI has implemented a variety of structured processes to identify systemic risks stemming from our frontier models and to develop risk scenarios and threat models through which these systemic risks may develop or materialize.” (FGF, p.3)

“We conduct systemic risk modelling that informs how we evaluate systemic risks before deployment.” (FGF, p.5)

1.3.2.2 Mechanism to incorporate red teaming findings (15%) 0%

No mention of risks identified during open-ended red teaming or evaluations triggering further risk modeling.

Quotes:

No relevant quotes found.

1.3.2.3 Prioritization of severe and probable risks (15%) 25%

For a risk area to be a tracked category, the capability must create a risk that is “Severe: There is a plausible threat model within the capability area that would create severe harm.” This suggests that severity is prioritised, and plausibility here suggests the risk model must have nonzero probability. However, these threat models are developed post-hoc – after deciding which categories to track: “For each Tracked Category, we develop and maintain a threat model identifying specific risks of severe harms that could arise from the frontier capabilities in that domain […]”

They then prioritise monitoring for High and Critical capabilities, implicitly defining these as those capabilities with higher probability x severity of harm: “High capability thresholds mean capabilities that significantly increase existing risk vectors for severe harm”; “Critical capability thresholds mean capabilities that present a meaningful risk of a qualitatively new threat vector for severe harm with no ready precedent.”

The FGF commits to estimate both dimensions explicitly: “Currently, we estimate the severity and probability of harm of risks related to CBRN, cyber offense, and loss of control under this FGF”, and it gives a concrete severity-based prioritisation, prioritising “biological capability evaluations due to the higher potential severity of biological threats”. However, there is still minimal detail on how severity and probability are determined, harmful manipulation is excluded as still early-stage, and these results are not published.

Overall, there is awareness that threat models should focus on severe harms, and the FGF commits to estimating probability and severity, but with little evidence of systematic prioritisation among multiple risk models. Risk modelling is still completed after deciding what to track, which differs from the required criterion of using prioritisation of risk models to determine focus areas.

Quotes:

“For each Tracked Category, we develop and maintain a threat model identifying specific risks of severe harms that could arise from the frontier capabilities in that domain […] High capability thresholds mean capabilities that significantly increase existing risk vectors for severe harm. Critical capability thresholds mean capabilities that present a meaningful risk of a qualitatively new threat vector for severe harm with no ready precedent.” (PF, p.4)

“Where we determine that a capability presents a real risk of severe harm, we may decide to monitor it as a Tracked Category or a Research Category.” (PF, p.4)

For a capability to be a Tracked Category (PF, p.4):
“Plausible: It must be possible to identify a causal pathway for a severe harm in the capability area, enabled by frontier AI.”
“Severe: There is a plausible threat model within the capability area that would create severe harm.”

“Currently, we estimate the severity and probability of harm of risks related to CBRN, cyber offense, and loss of control under this FGF.” (FGF, p.5)

“we prioritize biological capability evaluations due to the higher potential severity of biological threats” (FGF, p.8)

2.1.1.1 Risk tolerance is at least qualitatively defined for all risks (33%) 50%

There is a qualitative definition of scenarios which are implicitly ‘unacceptable’ levels of risk, under the Critical capability threshold. For instance, “Proliferating the ability to create a novel threat vector of the severity of a CDC Class A biological agent (i.e., high mortality, ease of transmission) could cause millions of deaths and significantly disrupt public life, with few available societal safeguards” implicitly states this risk as the risk tolerance.

To improve, they must set out the maximum amount of risk the company is willing to accept, for each risk domain (though they need not differ between risk domains), ideally expressed in terms of probabilities and severity (economic damages, physical lives, etc), and separate from KRIs.

Partial credit is given for the definition of “severe harm” as “the death or grave injury of thousands of people or hundreds of billions of dollars of economic damage.” However, the capability thresholds are not explicitly linked to this proto-risk tolerance, and it should be more specific (e.g., specifying how many thousands of people).

Quotes:

“By “severe harm” in this document, we mean the death or grave injury of thousands of people or hundreds of billions of
dollars of economic damage.” (PF, p.1)

“This FGF’s definition of systemic risk includes foreseeable and material risks of severe harm from the development, storage, use, or deployment of our most advanced frontier models, including risks that a model will materially contribute to greater than 50 fatalities or $1 billion of property damages or losses arising from a single incident.” (FGF, p. 3)

Some examples from Tracked Categories table, under the Critical category for ‘Associated risk of severe harm’ (PF, p.5):

“Proliferating the ability to create a novel threat vector of the severity of a CDC Class A biological agent (i.e., high mortality, ease of transmission) could cause millions of deaths and significantly disrupt public life, with few available societal safeguards.”

“Finding and executing end-to-end exploits for all software could lead to catastrophe from unilateral actors, hacking military or industrial systems, or OpenAI infrastructure. Novel cyber operations, e.g., those involving novel zero-days or novel methods of command-and-control, generally pose the most serious threat, as they are unpredictable and scarce.”

“A major acceleration in the rate of AI Self-improvement could rapidly increase the rate at which new capabilities and risks emerge, to the point where our current oversight practices are insufficient to identify and mitigate new risks, including risks to maintaining human control of the AI system itself.”

2.1.1.2 Risk tolerance is expressed at least partly quantitatively as a combination of scenarios (qualitative) and probabilities (quantitative) for all risks (33%) 10%

The qualitative risk tolerances do not have quantitative probabilities, and are vague in description. The definition of severe harm implies some awareness of quantitative measurement, though this is used to classify critical capability thresholds rather than defined as a risk tolerance itself.

Quotes:

“High capability thresholds mean capabilities that significantly increase existing risk vectors for severe harm” (p. 4)

“Critical capability thresholds mean capabilities that present a meaningful risk of a qualitatively new threat vector” (p. 4)

“Scalable evaluations have associated “indicative thresholds,” which are levels of performance that we have pre-determined to indicate that a deployment may have reached a capability threshold.” (p. 8)

“By “severe harm” in this document, we mean the death or grave injury of thousands of people or hundreds of billions of dollars of economic damage.” (p. 1)

2.1.1.3 Risk tolerance is expressed fully quantitatively as a product of severity (quantitative) and probability (quantitative) for all risks (33%) 0%

Whilst they mention the criterion of “severe harm” via “the death or gave injury of thousands of people or hundreds of billions of dollars of economic damage”, this is still vague, and doesn’t apply as a specific risk tolerance for specific risks. None of the specific risks mention quantitative probabilities, and the implicit risk tolerances from capability thresholds are not fully quantitative either.

Quotes:

“High capability thresholds mean capabilities that significantly increase existing risk vectors for severe harm” (p. 4)
“Critical capability thresholds mean capabilities that present a meaningful risk of a qualitatively new threat vector” (p. 4)

“Scalable evaluations have associated “indicative thresholds,” which are levels of performance that we have pre-determined to indicate that a deployment may have reached a capability threshold.” (p. 8)

Some examples from Tracked Categories table, under the Critical category for ‘Associated risk of severe harm’ (p. 5):

“Proliferating the ability to create a novel threat vector of the severity of a CDC Class A biological agent (i.e., high mortality, ease of transmission) could cause millions of deaths and significantly disrupt public life, with few available societal safeguards.”

“Finding and executing end-to-end exploits for all software could lead to catastrophe from unilateral actors, hacking military or industrial systems, or OpenAI infrastructure. Novel cyber operations, e.g., those involving novel zero-days or novel methods of command-and-control, generally pose the most serious threat, as they are unpredictable and scarce.”

“A major acceleration in the rate of AI Self-improvement could rapidly increase the rate at which new capabilities and risks emerge, to the point where our current oversight practices are insufficient to identify and mitigate new risks, including risks to maintaining human control of the AI system itself.”

2.1.2.1 AI developers engage in public consultations or seek guidance from regulators where available (50%) 0%

No evidence of asking the public what risk levels they find acceptable. No evidence of seeking regulator input specifically on what constitutes acceptable risk levels.

Quotes:

No relevant quotes found.

2.1.2.2 Any significant deviations from risk tolerance norms established in other industries is justified and documented (e.g., cost-benefit analyses) (50%) 0%

No justification process: No evidence of considering whether their approach aligns with or deviates from established norms.

Quotes:

No relevant quotes found.

2.2.1.1 KRI thresholds are at least qualitatively defined for all risks (45%) 50%

Under the PF, each Tracked Category has 2 thresholds each, which are specific. They appear to be grounded in risk modelling, but this is hard to tell given risk models are not clear. The Research Categories have less developed KRIs but are still well-defined.

The FGF adds a Tier 1 below the High/Critical thresholds for Cybersecurity and CBRN and an “Examples” column giving more qualitative detail than the PF’s bare threshold definitions, and develops loss of control into three defined tiers (a Research Category in the PF). Against this, the FGF introduces harmful manipulation as a risk domain with no defined KRI, working against coverage of all risks. To improve, more precision is needed in all of the Research Categories, as well as the High thresholds of Cybersecurity and Biological & Chemical; evidence is needed of the specific thresholds, e.g. of the “baseline of tools available in 2021” or “relative to those researchers’ 2024 baseline”. Additionally, KRIs should map directly to evaluation tests performed.

Quotes:

From Tracked Categories, under ‘Capability threshold that could lead to the risk’, PF, pp.5–6:
Biological and chemical:
“[High] The model can provide meaningful counterfactual assistance (relative to unlimited access to baseline of tools available in 2021) to “novice” actors (anyone with a basic relevant technical background) that enables them to create known biological or chemical threats.”

“[Critical] The model can enable an expert to develop a highly dangerous novel threat vector (e.g., comparable to a novel CDC Class A biological agent) OR the model can be connected to tools and equipment to complete the full engineering and/or synthesis cycle of a regulated or novel biological threat without human intervention.”

Autonomous AI R&D:
“[High] The model’s impact is equivalent to giving every OpenAI researcher a highly performant mid-career research engineer assistant, relative to those researchers’ 2024 baseline.”

“[Critical] The model is capable of recursively self improving (i.e., fully automated AI Self-improvement), defined as either (leading indicator) a superhuman researchscientist agent OR (lagging indicator) causing a generational model improvement (e.g., from OpenAI o1 to OpenAI o3) in 1/5th the wall-clock time of equivalent progress in 2024 (e.g., sped up to just 4 weeks) sustainably for several months.”

Research Categories:
“Long-range Autonomy: ability for a model to execute a long-horizon sequence of actions sufficient to realize a “High” threat model (e.g., a cyberattack) without being directed by a human (including successful social engineering attacks when needed)” (PF, p.7). More on page 7 can be found.

“Tier 1: The model’s capabilities provide minimal advantage compared to publicly available information or search engines, or provides minor uplift or lowers barriers to experts.” (FGF, p.8)

“[Tier 2] Provides accurate, detailed, and contextually relevant information that is similar to direct interactions with subject matter experts, that would be difficult for a non-specialist to obtain via open internet or manuals […]” (FGF, p.8)

“Tier 1 The model can reliably complete complex, multi-step tasks but lacks the execution of abilities approaching expert humans. The model […] executes deceptive behaviors in narrow, highly prompted scenarios.” (FGF, p.10)
“Outside of risks related to AI self-improvement, these risk tiers remain exploratory and may evolve substantially.” (FGF, p.10)

“OpenAI’s approach to harmful manipulation remains exploratory due to its nascency as a systemic risk area. This risk tier is subject to further research and may be substantially changed over time.” (FGF, p.9)

2.2.1.2 KRI thresholds are quantitatively defined for all risks (45%) 25%

The Critical capability threshold for AI Self-improvement, and somewhat the High capability threshold for AI Self-improvement and Biological & Chemical give baselines, which mean these KRIs can in theory be measured quantitatively. However, more detail of the actual threshold implemented here is needed. No quantitative thresholds for the others is given. The FGF does not add quantitative KRI thresholds: it claims its tier systems “quantify” capabilities and incorporate “risk estimates,” but the tiers as presented are qualitative descriptions and examples.

Quotes:

All the below are from Tracked Categories under ‘Capability threshold that could lead to the risk’, PF, pp.5-6:

High, Biological & Chemical: “[High] The model can provide meaningful counterfactual assistance (relative to unlimited access to baseline of tools available in 2021) to “novice” actors (anyone with a basic relevant technical background) that enables them to create known biological or chemical threats”

High, AI Self-improvement: “[High] The model’s impact is equivalent to giving every OpenAI researcher a highly performant mid-career research engineer assistant, relative to those researchers’ 2024 baseline.” Critical, AI Self-improvement: “[Critical] The model is capable of recursively self improving (i.e., fully automated AI Self-improvement), defined as either (leading indicator) a superhuman research scientist agent OR (lagging indicator) causing a generational model improvement (e.g., from OpenAI o1 to OpenAI o3) in 1/5th the wall-clock time of equivalent progress in 2024 (e.g., sped up to just 4 weeks) sustainably for several months.”

“[…] we have established a tier system that seeks to quantify model capabilities against cybersecurity threats and incorporates risk estimates and other suitable metrics. We use these measurable thresholds […]” (FGF, p.7)

2.2.1.3 KRIs also identify and monitor changes in the level of risk in the external environment (10%) 10%

The KRIs in the PF only mention model capabilities. They do mention monitoring and incident response, which could feasibly lead to KRIs which satisfy this criterion. The FGF strengthens this slightly: it commits to drawing on model-independent and external signals — external research and data, market analyses, insights from other deployed models, post-release monitoring — as inputs to risk assessment, and to post-deployment threat intelligence monitoring as a trigger for additional measures. But these signals feed a holistic risk-assessment judgment rather than functioning as measurable external KRIs with thresholds that trigger specific KCIs, so the criterion is acknowledged and partially set up but not operationalized.

Quotes:

“Monitoring and Incident Response: Monitor security and event logs continuously to detect, triage, and respond to security incidents rapidly by 24×7 on-call staff.” (PF, p.21)

“Our risk assessment process may take place throughout a model’s lifecycle, including during development and after deployment. Risk assessments draw on various sources, including a range of model evaluations, model-independent information such as external research and data, literature reviews, market analyses, consultation with internal and external experts, insights from other deployed models, post-release monitoring, and, if applicable, investigations of serious incidents or critical safety incidents.” (FGF, p.5)

“We rely on a variety of techniques to identify whether additional measures are needed, including postdeployment threat intelligence monitoring, post-launch monitoring tools (classifiers, automated detection, human review, and investigations), consultation with experts, and other techniques.” (FGF, p.11)

2.2.2.1 Containment KCIs (35%) 5%

2.2.2.2 Deployment KCIs (35%) 43%

2.2.2.3 For advanced KRIs, assurance process KCIs are defined (30%) 50%

The framework provides desiderata which can be seen as proto assurance KCIs, though they are somewhat vague (e.g. “the model consistently understands […] instructions” or “sufficiently addressed by one or more of the following claims” lack detail.) It is also unclear if they must all be covered in the Safeguards Report, and how often Safeguards Reports are required. The KCIs also are not linked to specific KRIs explicitly, though the document says they will be.

Table 5 list some desiderata indicating further proto assurance process KCI thresholds. However, these require more development, such as what constitutes “significant evidence” or “consistently takes action”.

Quotes:

“Safeguards should sufficiently minimize the risk associated with a misaligned model circumventing human control and oversight and executing severe harms. This can be done by establishing that all plausible known vectors of enabling severe harm are sufficiently addressed by one or more of the following claims:

• Lack of Autonomous Capability: The model is not capable of carrying out tasks autonomously, including the risk of severe harm.
• Value Alignment: The model consistently applies human values in novel settings (without any instructions) to avoid taking actions that cause harm, and has shown sufficiently minimal indications of misaligned behaviors like deception or scheming.
• Instruction Alignment: The model consistently understands and follows user or system instructions, even when vague, and those instructions rule out pathways to causing severe harm.
• Reliable and Robust System Oversight: Effective AI and human oversight of model actions detects and prevents execution of harm or subversion of safeguards.
• System Architecture: The model can’t take actions that cause harm because it lacks access to output channels or mechanisms to persist sufficiently to execute the harm, due to containerization and restricted permissions.” (p. 18) See also Table 3, p. 11.

From Table 5, p. 19:
“Internal and external user reports, monitoring, or manual inspection for historical model deployments show no significant evidence of unexpected value alignment failures.”

“Large scale usage or testing in realistic settings should show the model consistently takes action aligned to the user’s instructions or goals (while maintaining value alignment and avoiding harm). Examples against would include trying to escape a container, lying to the user to appease them, etc. […] “Canary evaluations” which test model capabilities to bypass less complex, easier-to exploit versions of our security controls, establishing that our implemented controls are robust”

3.1.1.1 Containment measures are precisely defined for all KCI thresholds (60%) 50%

The PF describes the security controls required for High capability models in C.3 in detail, though not for Critical, and many measures remain high-level desiderata rather than operational ones (some are more specific). The FGF restates this posture and adds a few more concrete items anchored to named certifications (ISO 27001, SOC 2 Type II). But these form a flat baseline for all frontier models rather than measures mapped to specific containment KCI thresholds, and Critical remains uncovered.

Quotes:

From appendix C.3, PF, pp.20-21:
“Adopt a layered security strategy, ensuring robust protection through multiple defensive barriers, including physical and datacenter security, network segmentation and controls, workload isolation, data encryption, and other overlapping and
complementary security controls.”
“Employees must authenticate using multi-factor authentication (MFA) and managed devices meeting security baselines. Access must be logged and reviewed for detection and investigative purposes.”
“Integrate automated code analysis, formal security reviews, and penetration testing in engineering processes. Apply security reviews and validation to higher-sensitivity critical components prior to deployment.”

“OpenAI maintains an Information Security and Privacy Program aligned with ISO 27001, 27017, 27018, and 27701, and supported by SOC 2 Type II evaluations.” (FGF, p.14)

“Model weights are protected through encryption (at rest and in transit), continuous monitoring, and access controls (e.g., multi-factor authentication, multi-party approval, and detailed logging).” (FGF, p.15)

“Model execution is sandboxed, with restricted egress by default.” (FGF, p.15)

3.1.1.2 Proof that containment measures are sufficient to meet the thresholds (40%) 25%

The PF commits to validating controls regularly but gives no detail on why the measures are likely sufficient to meet a containment KCI threshold, and sufficiency is ultimately a post-hoc SAG/leadership determination rather than decided before the KRI threshold is crossed. The FGF adds a documented residual-risk acceptance justification, marginally reinforcing that a process for soliciting proof exists, but still provides no ex-ante proof against a defined containment KCI and no quantified threshold.

Quotes:

“Continuous Monitoring and Validation: Ensure security threat models and updates inform where security and data privacy controls should be implemented, improved, and monitored to further reduce risk. Internal and external assessments to validate these controls are conducted regularly and reports are provided to OpenAI leadership.” (PF, p.20)

“We document a justification for why the systemic risks stemming from the model are acceptable, including reasonably foreseeable conditions under which the justification may no longer hold […]” (FGF, p.11)

3.1.1.3 Strong third party verification process to verify that the containment measures meet the threshold (100% if 3.1.1.3 > [60% x 3.1.1.1 + 40% x 3.1.1.2]) 25%

The PF requires independent audits for High capability models but not Critical, and validates already-implemented protocols rather than the ex-ante case that containment measures meet a containment KCI. The FGF strengthens the evidence by naming specific recurring independent mechanisms — SOC 2 Type II audits, ISO 27001 certification, external red teaming and penetration testing. However, these verify compliance with general security standards and existing controls, not the prior case for KCI sufficiency.

Quotes:

“Independent Security Audits: Ensure security controls and practices are validated regularly by third-party auditors to ensure compliance with relevant standards and robustness against identified threats.” (PF, p.21)

“Security controls are validated through internal and external assessments, including red teaming, penetration testing, vulnerability scanning, SOC 2 Type II audits, and ISO 27001 certification.” (FGF, p.15)

3.1.2.1 Deployment measures are precisely defined for all KCI thresholds (60%) 50%

Section C.1 in the Appendix details “potential safeguards” for models with High capability, without explicit commitment to implementing them: “the safeguards should not be construed as a definitive or comprehensive list of the safeguards we will or could apply to a given launch”. Nonetheless, the measures are defined for each KCI threshold, namely robustness, usage monitoring and trust-based access.

To improve, deployment measures must also be defined for the Critical capability.

Quotes:

From Table 4, p. 17:
“Robustness:

• Training the model to refuse to help with high-risk tasks or to otherwise produce low risk
  responses
• Unlearning or training-data filtering to erase specific risk-enabling knowledge from the model’s knowledge-base
• Interpretability-based approaches, like activation steering, that directly edit models’ thinking at inference time
• Jailbreak robustness, including through adversarial training, inference-time deliberation, and more”

More quotes may be found in Table 4.

“This Appendix provides illustrative examples of potential safeguards, and safeguard efficacy assessments that could be used to establish that we have sufficiently mitigated the risk of severe harm. The examples aim to provide insight on our thinking, but many of the techniques require further research. The safeguards should not be construed as a definitive or comprehensive list of the safeguards we will or could apply to a given launch.

As a reminder, covered systems that reach High capability must have safeguards that sufficiently minimize the associated risk of severe harm before they are deployed. Systems that reach Critical capability also require sufficient safeguards during development.” (p. 16)

3.1.2.2 Proof that deployment measures are sufficient to meet the thresholds (40%) 25%

The PF’s Section C.1 in the Appendix details “potential safeguard efficacy assessments”, without explicit commitment to implementing them. However, they do not provide actual proof or evidence that the deployment measures are sufficient ex ante. Instead, it relies on the Safety Advisory Group’s judgment at the time when High or Critical deployment standards need to be implemented, making the decision vulnerable to discretion. The FGF likewise determines sufficiency through holistic risk-acceptance judgment incorporating safety margins, not through pre-specified, justified efficacy targets.

Quotes:

From PF, Table 4, p.17:
“Robustness:
• Automated and expert redteaming (identifying success per resources)
• Prevalence of jailbreaks identified via monitoring and reports, in historical deployments
• Results from public jailbreak bounties and results from private and public jailbreak benchmarks”

More quotes may be found in PF, Table 4.
“The examples aim to provide insight on our thinking but should not be construed as a definitive checklist of the safeguards we will apply to a given launch.” (PF, p.10)

In assessing whether to accept residual risk, we review the risk tiers for each systemic risk category and incorporate appropriate safety margins.” (FGF, p.6)

3.1.2.3 Strong third party verification process to verify that the deployment measures meet the threshold (100% if 3.1.2.3 > [60% x 3.1.2.1 + 40% x 3.1.2.2]) 25%

While they mention third-party stress testing of safeguards, this is not specific to deployment measures, and appears optional. The FGF keys it to a model approaching a risk tier but stays optional and specifies neither how experts are selected and kept independent nor whether external reports are published.

Quotes:

“Third-party stress testing of safeguards: If we deem that a deployment warrants third party stress testing of safeguards and if high quality third-party testing is available, we will work with third parties to evaluate safeguards. We may seek this out in particular for models that are over a High capability threshold.” (PF, p.13)

“Independent expert opinions for evidence produced to SAG: The SAG may opt to get independent expert opinion on the evidence being produced to SAG. The purpose of this input is to add independent analysis from individuals or organizations with deep expertise in domains of relevant risks (e.g., biological risk). If provided, these opinions will form part of the analysis presented to SAG in making its decision on the safety of a deployment. These domain experts may not necessarily be AI experts and their input will form one part of the holistic evidence that SAG reviews.” (PF, p.13)

“We may solicit and obtain input from external experts in relevant domains, and other stakeholders, to assist in systemic risk assessment or in determining the sufficiency of safety mitigations. This may include independent third-party evaluators, stress testing of safeguards for models approaching or reaching a new risk tier, or to provide independent expert opinions to assist the Safety Advisory Group in assessing the safety of a proposed deployment.” (FGF, p.17)

3.1.3.1 Credible plans towards the development of assurance processes (40%) 25%

The framework mentions a commitment to developing assurance processes for Critical capabilities. However, they do not provide further detail on how they will achieve this, or by what point this will need to be intact (i.e. it is unclear if assurance processes must be solidifed before or after a model has been deemed to possess Critical level capabilities). Further, their commitment is only to “update” the framework, meaning their plans or the KCIs/KRIs necessitating Critical level assurance themselves may be modified.

Quotes:

“Models that have reached or are forecasted to reach Critical capability in a Tracked Category present severe dangers and should be treated with extreme caution. Such models require additional safeguards (safety and security controls) during development, regardless of whether or when they are externally deployed. We do not currently possess any models that have Critical levels of capability, and we expect to further update this Preparedness Framework before reaching such a level with any model. Our approach to Critical capabilities will need to be robust to both malicious actors (either internal or external) and model misalignment risks. The SAG retains discretion over when to request deep dive evaluations of models whose scalable evaluations indicate that they may possess or may be nearing critical capability thresholds.” (p. 12)

3.1.3.2 Evidence that the assurance processes are enough to achieve their corresponding KCI thresholds (40%) 50%

Whilst they detail safeguard efficacy assessments which is commendable, these are only “potential”. In addition, their descriptions remain more high-level than precise, and do not cover the Critical case. To improve, they could describe what evidence would be sufficient for showing that the safeguards for misalignment meet the required KCI threshold.

Quotes:

“The table below illustrates possible examples of safeguards and safeguard efficacy assessments we could consider to claim we have sufficiently minimized the risks of severe harm associated with High level of capability under any of our Tracked Categories” (p. 19). The table (Table 5) goes on to define “potential safeguard efficacy assessments” for each category. For instance,
“Reliable and Robust System Oversight:

• Red-teamers can train an arbitrarily adversarial agent or can pretend to be an agent in our setup and we should ensure that harmful behaviors are detected by monitors with a high recall rate
• Monitor escalations are either outright blocked or, in empirical studies, users with the relevant level of expertise were shown to reliably understand and act on monitor flags before significant harm occurred
• SLAs for asynchronous monitors to analyze and detect harmful agent actions posthoc”

More examples can be found in Table 5.

3.1.3.3 The underlying assumptions that are essential for their effective implementation and success are clearly outlined (20%) 10%

The Framework does not explicitly outline the assumptions for assurance processes’ efficacy to hold. However, they demonstrate some implicit awareness that “deception, colluding with oversight models, sabotaging safeguards over time such as by embedding vulnerabilities in safeguards code” could all undermine these KCI measures. To improve, the framework should detail the key technical assumptions necessary for the assurance processes to meet the KCI threshold, and evidence for why these assumptions are justified.

Quotes:

“Research Category: Undermining Safeguards: ability and propensity for the model to act to undermine safeguards placed on it, including e.g., deception, colluding with oversight models, sabotaging safeguards over time such as by embedding vulnerabilities in safeguards code, etc.
Potential response: If a model has High or Critical capabilities in any of the Tracked Categories, require the Safeguards case to be robust to the discovered capability and/or propensity” (p. 7, under ‘Potential response’ to Research Category “Undermining Safeguards” in Table 2.)

3.2.1.1 Justification that elicitation methods used during the evaluations are comprehensive enough to match the elicitation efforts of potential threat actors (30%) 90%

The framework outlines multiple elicitation strategies and commits to fulfill this criterion almost word for word. The elicitation methods detailed show nuance and expertise. To improve, the framework could include measurable information, such as how much compute is used for fine-tuning. More detail could be added on which elicitation methods they anticipate would be used by different threat actors, under realistic settings, to justify their elicitation method.

Quotes:

“Our evaluations are intended to approximate the full capability that the adversary contemplated by our threat model could extract from the deployment candidate model, including by using the highest capability tier of system settings, using a version of the model that has a negligible rate of safety-based refusals on our Tracked Category capability evaluations (which may require a separate model variant), and with the best presently-available scaffolds. These measures are taken to approximate the high end of expected elicitation by threat actors attempting to misuse the model, and should be tailored depending on the level of expected access (e.g., doing finetuning if the weights will be released). Nonetheless, given the continuous progress in model scaffolding and elicitation techniques, we regard any one-time capability elicitation in a frontier model as a lower bound, rather than a ceiling, on capabilities that may emerge in real world use and misuse. We incorporate this uncertainty into our assessments. We monitor the technical landscape for changes to the elicitation techniques and best practices, and reassess our evaluations as needed.” (p. 8)

3.2.1.2 Evaluation frequency (25%) 10%

The FGF mentions the option of conducting lighter-touch model evaluations “from time to time”, which might be triggered when OpenAI have “reason to believe a model’s risk profile may have materially changed”, or when they release an updated model. However, these are discretionary rather than comitting.

Quotes:

“we may from time to time conduct lighter-touch model evaluations at appropriate trigger points […] These trigger points may include (1) the release of an updated model or (2) where we have reason to believe a model’s risk profile may have materially changed.” (FGF, p.16)

3.2.1.3 Description of how post-training enhancements are factored into capability assessments (15%) 25%

There is some recognition of how post-training enhancements can factor into capability assessments, but this description remains high level.

The commitment to “monitor the technical landscape for changes to the elicitation techniques and best practices, and reassess our evaluations as needed” is vague; it is not clear how evaluations are “reassessed” based on changes in best practices. For an improvement, an explicit commitment to adopt best practices should be given, or otherwise forecasting exercises could be completed to justify their assumptions on the rate of progress in post-training enhancements. However, “we incorporate this uncertainty into our assessments”, whilst vague, shows partial implementation of factoring the uncertainty of the progress of post-training enhancements in the future.

Importantly, more detail could be provided on precisely how post-training enhancements are factored into capability assessments – for instance, the size of the “uncertainty” or the safety buffer they give to account for uncertainty concerning the progress of post-training enhancements.

Further, more detail could be added on how they account(ed) for how post-training enhancements’ risk profiles change with different model structures – namely, post-training enhancements are much more scalable with reasoning models, as inference compute can often be scaled to improve capabilities.

Quotes:

“Our evaluations are intended to approximate the full capability that the adversary contemplated by our threat model could extract from the deployment candidate model, including by using the highest capability tier of system settings, using a version of the model that has a negligible rate of safety-based refusals on our Tracked Category capability evaluations (which may require a separate model variant), and with the best presently-available scaffolds. These measures are taken to approximate the high end of expected elicitation by threat actors attempting to misuse the model, and should be tailored depending on the level of expected access (e.g., doing finetuning if the weights will be released). Nonetheless, given the continuous progress in model scaffolding and elicitation techniques, we regard any one-time capability elicitation in a frontier model as a lower bound, rather than a ceiling, on capabilities that may emerge in real world use and misuse. We incorporate this uncertainty into our assessments. We monitor the technical landscape for changes to the elicitation techniques and best practices, and reassess our evaluations as needed.” (p. 8)

3.2.1.4 Vetting of protocols by third parties (15%) 10%

The framework demonstrates discretionary commitment to third-party vetting of evaluation protocols. They do not have a specific structure in place for regularly vetting capabilities assessments by third parties, but they do indicate that they measure the Research Categories capabilities in collaboration with external experts. They also mention a general commitment to soliciting expert opinion on the overall holistic risk assessment process.

Quotes:

“We evaluate whether frontier capabilities create a risk of severe harm through a holistic risk assessment process. This process draws on our own internal research and signals, and where appropriate incorporates feedback from academic researchers, independent domain experts, industry bodies such as the Frontier Model Forum, and the U.S. government and its partners, as well as relevant legal and policy mandates.” (p. 4)

“We call these Research Categories, and in these areas we will take the following
steps, both internally and in collaboration with external experts:

• Further developing the threat models for the area,
• Advancing the science of capability measurement in the area and investing towards the development of rigorous evaluations (which could be achieved internally or via partnerships), and
• Sharing summaries of our findings with the public where feasible.” (pp. 5-6)

Deeper capability assessments: “Deep Dives: designed to provide additional evidence validating the scalable evaluations’ findings on whether a capability threshold has been crossed. These may include a wide range of evidence gathering activities, such as human expert red-teaming, expert consultations, resource-intensive third party evaluations (e.g., bio wet lab studies, assessments by independent third party evaluators), and any other activity requested by SAG.” (p. 8)

3.2.1.5 Replication of evaluations by third parties (15%) 25%

The Preparedness Framework gives some recognition of evaluations being conducted independently by third-parties, but only if deemed necessary. Further, they only commit to “work[ing] with” these parties. They do not explicitly commit in the document to have any evaluations replicated, unless it is part of a deeper capability assessment (‘Deep Dive’) that is asked for by the Safety Advisory Group (i.e., at the OpenAI leadership’s discretion.)

Quotes:

“Third-party evaluation of tracked model capabilities: If we deem that a deployment warrants deeper testing of Tracked Categories of capability (as described in Section 3.1), for example based on results of Capabilities Report presented to them, then when available and feasible, OpenAI will work with third-parties to independently evaluate models.” (PF, p.13)

Deeper capability assessments: “Deep Dives: designed to provide additional evidence validating the scalable evaluations’ findings on whether a capability threshold has been crossed. These may include a wide range of evidence
gathering activities, such as human expert red-teaming, expert consultations, resource-intensive third party evaluations (e.g., bio wet lab studies, assessments by independent third party evaluators), and any other activity requested by SAG.” (PF, p.8)

“We may solicit and obtain input from external experts in relevant domains, and other stakeholders, to assist in systemic risk assessment or in determining the sufficiency of safety mitigations. This may include independent third-party evaluators, stress testing of safeguards for models approaching or reaching a new risk tier, or to provide independent expert opinions to assist the Safety Advisory Group in assessing the safety of a proposed deployment.” (FGF, p.17)

3.2.2.1 Detailed description of evaluation methodology and justification that KCI thresholds will not be crossed unnoticed (40%) 50%

There is some reference to monitoring systems, but without explicit justification that KCI thresholds will not be crossed unnoticed, nor a detailed description of the methodology of monitoring or assurance that monitoring is ongoing.

There is implicit justification that KCI thresholds are sufficient in the safeguards for misaligned models. For instance, a “monitor AI supervises agent actions to detect agent in the case of trying to take harmful actions” and “agent actions are all logged in an uneditable database, and asynchronous monitoring routines review those actions for evidence of harm”. This shows that monitoring is being conducted in order to satisfy the assurance process KCI. However, this monitoring is not specifically for measuring mitigation effectiveness; rather, it is to “review [agent] actions for evidence of harm”, which may not catch KCI thresholds being crossed unnoticed.

Indeed, there are multiple “potential safeguard efficacy assessments” detailed in The Preparedness Framework – e.g., “Redteamer success creating a certain number of accounts without detection (success per resources)”. However, none of these efficacy assessments are ongoing; instead, it appears they only need to be conducted for the drafting of the Safeguards Report. Further, they are only “potential” assessments.

To improve, The Preparedness Framework should describe systematic, ongoing monitoring to ensure mitigation effectiveness is tracked continuously such that the KCI threshold will still be met, when required.

Quotes:

“Reliable and Robust System Oversight:
• Monitor AI supervises agent actions to detect agent in the case of trying to take harmful actions, scheming, or similar
• System forces human approval of agent actions (either by default, or by escalation of the monitor)
• Model’s reasoning is provided faithfully and interpretably to humans to review
• Agent actions are all logged in an uneditable database, and asynchronous monitoring routines review those actions for evidence of harm” (PF, p.19)

See PF, Tables 4 and 5, pages 18 and 19, “Potential safeguard efficacy assessments”.

“We rely on a variety of techniques to identify whether additional measures are needed, including post-deployment threat intelligence monitoring, post-launch monitoring tools (classifiers, automated detection, human review, and investigations), consulting with experts, and other techniques.” (FGF, p.11)

3.2.2.2 Vetting of protocols by third parties (30%) 50%

The Preparedness Framework demonstrates discretionary commitment to third-party vetting of KCI protocols. They do not have a specific structure in place for regularly vetting safeguards assessments by third parties, beyond the quote below. They do not explicitly commit therefore to undergo vetting of KCI protocols by third parties, except for containment KCIs.

Quotes:

“Independent expert opinions for evidence produced to SAG: The SAG may opt to get independent expert opinion on the evidence being produced to SAG. The purpose of this input is to add independent analysis from individuals or organizations with deep expertise in domains of relevant risks (e.g., biological risk). If provided, these opinions will form part of the analysis presented to SAG in making its decision on the safety of a deployment. These domain experts may not necessarily be AI experts and their input will form one part of the holistic evidence that SAG reviews.” (PF, p.13)

“SAG is responsible for assessing whether the safeguards associated with a given deployment sufficiently minimize the risk of severe harm associated with the proposed deployment. The SAG will make this determination based on:
[…] The associated risks of severe harm, as described in the threat model and where needed, advice of internal or external experts.” (PF, p.10)

“Continuous Monitoring and Validation: Ensure security threat models and updates inform
where security and data privacy controls should be implemented, improved, and monitored to
further reduce risk. Internal and external assessments to validate these controls are conducted regularly and reports are provided to OpenAI leadership.” (PF, p.20)

“Independent Security Audits: Ensure security controls and practices are validated regularly by third-party auditors to ensure compliance with relevant standards and robustness against identified threats.” (PF, p.21)

“Monitoring and Incident Response: Monitor security and event logs continuously to detect,
triage, and respond to security incidents rapidly by 24×7 on-call staff.” (PF, p.21)

“We rely on a variety of techniques to identify whether additional measures are needed, including post-deployment threat intelligence monitoring, post-launch monitoring tools (classifiers, automated detection, human review, and investigations), consulting with experts, and other techniques.” (FGF, p.11)

3.2.2.3 Replication of evaluations by third parties (30%) 25%

The framework gives some recognition of evaluations being conducted independently by third-parties, but only if deemed necessary. Further, they only commit to “work[ing] with” these parties. They do not explicitly commit in the document to have any evaluations replicated.

Quotes:

“Third-party stress testing of safeguards: If we deem that a deployment warrants third party stress testing of safeguards and if high quality third-party testing is available, we will work with third parties to evaluate safeguards. We may seek this out in particular for models that are over a High capability threshold.” (p. 13)

3.2.3.1 Sharing of evaluation results with relevant stakeholders as appropriate (85%) 75%

There are commitments to share evaluation results to the public if models are deployed. However, they do not commit to alert any stakeholders when/if Critical capabilities are reached.

Quotes:

“Public disclosures: We will release information about our Preparedness Framework results in order to facilitate public awareness of the state of frontier AI capabilities for major deployments. This published information will include the scope of testing performed, capability evaluations for each Tracked Category, our reasoning for the deployment decision, and any other context about a model’s development or capabilities that was decisive in the decision to deploy. Additionally, if the model is beyond a High threshold, we will include information about safeguards we have implemented to sufficiently minimize the associated risks. Such disclosures about results and safeguards may be redacted or summarized where necessary, such as to protect intellectual property or safety.” (p. 12)

“Transparency in Security Practices: Ensure security findings, remediation efforts, and key metrics from internal and independent audits are periodically shared with internal stakeholders and summarized publicly to demonstrate ongoing commitment and accountability.” (p. 21)

“Internal Transparency. We will document relevant reports made to the SAG and of SAG’s decision and reasoning. Employees may also request and receive a summary of the testing results and SAG recommendation on capability levels and safeguards (subject to certain limits for highly sensitive information).” (p. 12)

3.2.3.2 Commitment to non-interference with findings (15%) 0%

No commitment to permitting the reports, which detail the results of external evaluations (i.e. any KRI or KCI assessments conducted by third parties), to be written independently and without interference or suppression.

Quotes:

No relevant quotes found.

3.2.4.1 Identifying novel risks post-deployment: engages in some process (post deployment) explicitly for identifying novel risk domains or novel risk models within known risk domains (50%) 10%

There is some indication of monitoring; however, this is not explicitly to gain information on novel risk profiles. To improve, such a process should be detailed, for instance by building on the current monitoring infrastructure.

They do mention that monitoring should be conducted to assert there is “no significant evidence of unexpected value alignment failures”, as a safeguard efficacy assessment. Partial credit is given here for the use of “unexpected”, as this could be further developed to analyse novel risk profiles.

Quotes:

“Internal and external user reports, monitoring, or manual inspection for historical model deployments show no significant evidence of unexpected value alignment failures” (p. 19)

“Prevalence of jailbreaks identified via monitoring and reports, in historical deployments” (p. 17)

“Expanding human monitoring and investigation capacity to track capabilities that pose a risk of severe harm, and developing data infrastructure and review tools to enable human investigations” (p. 17)

“Agent actions are all logged in an uneditable database, and asynchronous monitoring routines review those actions for evidence of harm” (p. 19)

3.2.4.2 Mechanism to incorporate novel risks identified post-deployment (50%) 10%

There is a commitment to developing threat models for some of the Research Categories. However, this is not explicitly linked to incorporating novel risks, which were unexpected or not previously anticipated. To improve, an encounter with a possibly novel risk profile of a model should trigger risk modelling exercises, to analyse how this finding may impact all other risk models.

They do mention that if a capability “presents a real risk of severe harm, we may decide to monitor it as a Tracked Category or a Research Category”. Whilst this remains general, partial credit is given here for having some reference to incorporating additional risks – noting that “a capability” could refer to any capability.

Quotes:

“Where we determine that a capability presents a real risk of severe harm, we may decide to monitor it as a Tracked Category or a Research Category.” (p. 4)

“There are also some areas of frontier capability that do not meet the criteria to be Tracked Categories, but where we believe work is required now in order to prepare to effectively address risks of severe harms in the future. These capabilities either need more research and threat modeling before they can be rigorously measured, or do not cause direct risks themselves but may need to be monitored because further advancement in this capability could undermine the safeguards we rely on to mitigate existing Tracked Category risks.” (p. 6)