The criterion is partially addressed, covering the risk areas of CBRN weapons, offensive cyberoperations and advanced autonomy (which is essentially AI R&D). Further, 1.1.2 is less than 50%, suggesting that justification for exclusion of risks such as persuasion and loss of control risks should be stronger, or that these risks should be included in their monitoring.
“This framework tracks the following capabilities that we believe could emerge over the short-to-medium term and threaten national security or pose at-scale public safety risks if not appropriately mitigated. In formulating this list, we have benefited from the advice of both internal and external experts.
No justification for exclusion of risks such as manipulation or loss of control risks is given.
No relevant quotes found.
The framework doesn’t mention any procedures pre-deployment to identify novel risk domains or risk models for the frontier model. To improve, they should commit to such a process to identify either novel risk domains, or novel risk models/changed risk profiles within pre-specified risk domains (e.g. emergence of an extended context length allowing improved zero shot learning changes the risk profile), and provide methodology, resources and required expertise.
No relevant quotes found.
The framework doesn’t mention any third-party procedures pre-deployment to identify novel risk domains or risk models for the frontier model. To improve, they should commit to an external process to identify either novel risk domains, or novel risk models/changed risk profiles within pre-specified risk domains (e.g. emergence of an extended context length allowing improved zero shot learning changes the risk profile), and provide methodology, resources and required expertise.
No relevant quotes found.
While they mention “mapping” risks in general, there is no evidence that they develop a risk model for any of the risk areas. To improve, risk models that are specific to the model being considered should be developed, with causal pathways to threat scenarios identified. There should be justification that adequate effort has been exerted to systematically map out all possible risk pathways, and the risk models, threat scenarios, methodology, and experts involved should be published.
“While different risk profiles may thus inform different mitigation strategies, Microsoft’s overall approach of mapping, measuring, and mitigating risks, including through robust evaluation and measurement, applies consistently across our AI technologies.” (p. 4)
There is no methodology for risk modeling defined.
No relevant quotes found.
No mention of risks identified during open-ended red teaming or evaluations triggering further risk modeling.
No relevant quotes found.
While they don’t explicitly prioritize severity and likelihood of risk models, there does appear to be some structured process for identifying which risks are most severe and probable. Implicitly, they seem to be prioritizing these. To improve, risk models with severity and probability determinations should be published.
“AI technology continues to develop rapidly, and there remains uncertainty over which capabilities may emerge and when. We continue to study a range of potential capability related risks that could emerge, conducting ongoing assessment of the severity and likelihood of these risks. We then operationalize the highest-priority risks through this framework.” (p. 3)
Whilst the framework does not detail a risk modeling methodology, they do obtain some external input when prioritising risks, which implicitly requires input into risk models. However, this does not count as review, and the process should be more explicitly linked to validating risk models.
“This framework tracks the following capabilities that we believe could emerge over the short-to-medium term and threaten national security or pose at-scale public safety risks if not appropriately mitigated. In formulating this list, we have benefited from the advice of both internal and external experts.” (p. 3)
There is no explicit reference to a risk tolerance, though implicitly the tolerance is given by the capability thresholds. For instance, “CBRN weapons, Critical: The model provides a meaningful uplift to an expert’s ability to develop a highly dangerous novel threat or significantly lowers the barriers to a low-skilled actor developing and delivering a known CBRN threat.” The risk tolerance is also implicitly described as risks arising from “capabilities that we believe could emerge over the short-to-medium term and threaten national security or pose at-scale public safety risks if not appropriately mitigated”; i.e., “threaten[ing] national security” or “pos[ing] at-scale public safety risks” is the risk tolerance.
To improve, they should set out the maximum amount of risk the company is willing to accept for each risk domain (though these need not differ between risk domains), ideally expressed in terms of probabilities and severity (economic damages, physical lives, etc), and separate from KRIs.
“This framework tracks the following capabilities that we believe could emerge over the short-to-medium term and threaten national security or pose at-scale public safety risks if not appropriately mitigated.” (p. 3)
“CBRN weapons, Critical: The model provides a meaningful uplift to an expert’s ability to develop a highly dangerous novel threat or significantly lowers the barriers to a low-skilled actor developing and delivering a known CBRN threat.” (p. 11)
“Offensive cyberoperations, Critical: The model provides a meaningful uplift to a low-skilled actor’s ability to identify and exploit major vulnerabilities or enables a well-resourced and expert actor to develop and execute novel and effective strategies against hardened targets.” (p. 12)
The implicit risk tolerance of “threaten[ing] national security” or posing “at-scale public safety risks” is not a quantitative nor partly quantitative definition. Further, the implicit risk tolerances offered by the critical capability thresholds are not quantitative nor partly quantitative. To improve, the risk tolerance should be expressed fully quantitatively or as a combination of scenarios with probabilities.
“This framework tracks the following capabilities that we believe could emerge over the short-to-medium term and threaten national security or pose at-scale public safety risks if not appropriately mitigated.” (p. 3)
The implicit risk tolerance of “threaten[ing] national security” or posing “at-scale public safety risks” is not a quantitative nor partly quantitative definition. The implicit risk tolerances given by the critical capability thresholds are not fully quantitative, either.
“This framework tracks the following capabilities that we believe could emerge over the short-to-medium term and threaten national security or pose at-scale public safety risks if not appropriately mitigated.” (p. 3)
No evidence of engaging in public consultations or seeking guidance from regulators for risk tolerance.
No relevant quotes found.
No justification process: No evidence of considering whether their approach aligns with or deviates from established norms.
No relevant quotes found.
The framework describes two types of KRIs: those used for the ‘leading indicator assessment’, and those used for the ‘deeper capability assessment’.
For the leading indicator assessment KRIs, they give categories of benchmarks, but not the actual benchmarks nor their thresholds which are of sufficiently high risk. This could use more detail and could be more grounded in risk modelling.
For the deeper capability assessment KRIs, there could be more detail as to what “meaningful” or “significantly” constitute for the purpose of their risk assessment. However, these KRIs are grounded in risk modelling. To improve, KRIs should map directly to evaluation tests performed.
For instance, CBRN: “High: The model provides a meaningful uplift to an expert’s (PhD level education in related fields) ability to develop and deliver a novel CBRN threat. The model provides a meaningful uplift to a medium-skilled actor’s (e.g., STEM education) ability to develop and deliver a known CBRN threat. Critical: The model provides a meaningful uplift to an expert’s ability to develop a highly dangerous novel threat or significantly lowers the barriers to a low-skilled actor developing and delivering a known CBRN threat.” (p. 11)
“Through the processes described in this framework, Microsoft’s most advanced models are assessed for leading indicators of the framework’s high-risk capabilities. This is done using state-of-the-art benchmarks for the following advanced general-purpose capabilities, identified as precursors to high-risk capabilities:
Footnote 1, after “benchmarks”: “For a benchmark to be included in our suite of leading indicator assessments it must: 1) have low saturation (i.e., the best performing models typically score lower than 70%); 2) measure an advanced capability, for example, mathematical reasoning, rather than an application-oriented capability like financial market prediction; and 3) have a sufficient number of prompts to account for non-determinism in model output.” (p. 5)
“Deeper capability assessment provides a robust indication of whether a model possesses a tracked capability and, if so, whether this capability is at a low, medium, high, or critical risk level, informing decisions about appropriate mitigations and deployment. We use qualitative capability thresholds to guide this classification process as they offer important flexibility across different models and contexts at a time of nascent and evolving understanding of frontier AI risk assessment and management practice.” (p. 5)
The framework describes two types of KRIs: those used for the ‘leading indicator assessment’, and those used for the ‘deeper capability assessment’.
For the leading indicator assessment KRIs, they give categories of benchmarks, but not the actual benchmarks nor their thresholds which are of sufficiently high risk. This could use more detail. However, these could likely be quantitatively defined.
For the deeper capability assessment KRIs, they explicitly do not have quantitative thresholds, preferring qualitative indicators. However, quantitative thresholds need not be inflexible, and in order to have transparency in risk decisions and provide clear guidance, KRIs should be quantitative where possible.
They should still publish the actual evaluations and thresholds which they currently operate under. This is because KRI-KCI pairings should be as predictable in advance as possible/allowing as little discretion as possible, and a qualitative threshold may be more arbitrary than a conservative quantitative estimate, until improved risk indicators can be developed.
“Through the processes described in this framework, Microsoft’s most advanced models are assessed for leading indicators of the framework’s high-risk capabilities. This is done using state-of-the-art benchmarks for the following advanced general-purpose capabilities, identified as precursors to high-risk capabilities:
“Deeper capability assessment provides a robust indication of whether a model possesses a tracked capability and, if so, whether this capability is at a low, medium, high, or critical risk level, informing decisions about appropriate mitigations and deployment. We use qualitative capability thresholds to guide this classification process as they offer important flexibility across different models and contexts at a time of nascent and evolving understanding of frontier AI risk assessment and management practice.” (p. 5)
Whilst there is some indication that external risks must also be monitored and potentially used as a KRI, details on what these external risks are, how they are monitored, or the threshold that determines that a KRI has been crossed are not given.
“The results of capability evaluation and an assessment of risk factors external to the model then inform a determination as to whether a model has a tracked capability and to what level.” (p. 6)
“In addition to high-risk capabilities, a broader set of risks are governed when Microsoft develops and deploys AI technologies. Under Microsoft’s comprehensive AI governance program, frontier models—as well as other models and AI systems—are subject to relevant evaluation, with mitigations then applied to bring overall risk to an appropriate level.
Information on model or system performance, responsible use, and suggested system-level evaluations is shared with downstream actors integrating models into systems, including external system developers and deployers and teams at Microsoft building models. […] Our efforts to assess and mitigate risks related to this framework’s tracked
capabilities benefit from this broadly applied governance program, which is continuously improved. The remainder of this framework addresses more specifically the assessment and mitigation of risks relating to the framework’s tracked capabilities.” (p. 4)
The framework gives qualitative containment KCI thresholds distinguishing between high-risk and critical risk KRIs, though more detail could be given as to what “the highest level of security safeguards” refers to, or what “protective against most cybercrime groups and insider threats” entails, e.g. what kind of threats or attacks.
“Models posing high-risk on one or more tracked capability will be subject to security measures protective against most cybercrime groups and insider threats […] Models posing critical risk on one or more tracked capability are subject to the highest level of security safeguards.” (p. 7)
No quantitative containment KCI thresholds given.
No relevant quotes found.
Practically no detail on deployment KCI thresholds is given. For each capability threshold, the deployment requirements are either “Deployment allowed in line with Responsible AI Program requirements” or “Further review and mitigations required.” The specific threshold given by the Responible AI Program requirements should be explicitly detailed.
“Deployment allowed in line with Responsible AI Program requirements” or “Further review and mitigations required.” (p.13)
There are no quantitative deployment KCI thresholds given.
No relevant quotes found.
There are no assurance processes KCIs defined. The framework does not provide recognition of there being KCIs outside of containment and deployment measures.
No relevant quotes found.
There is a clear acknowledgment that KRIs and KCIs pair together to bring residual risk below the risk tolerance, or “an acceptable level”. However, this is not grounded in risk modelling, and this fact is not proven or given justification for each (or any) risk domain. Further, their risk assessment is contingent on other companies’ risk tolerance: “This holistic risk assessment also considers the marginal capability uplift a model may provide over and above currently available tools and information, including currently available open-weights models.”
“This framework assesses Microsoft’s most advanced AI models for signs that they may have these capabilities and, if so, whether the capability poses a low, medium, high, or critical risk to national security or public safety (more detail in Appendix I). This classification then
guides the application of appropriate and proportionate mitigations so that a model’s risks remain at an acceptable level.” (p. 3)
“The framework monitors Microsoft’s most capable AI models for leading indicators of high-risk capabilities and triggers deeper assessment if leading indicators are observed. As and when risks, are identified, proportional mitigations are applied so that risks are kept at an appropriate level. This approach provides confidence that highly capable models are identified before relevant risks emerge.” (p. 2)
“This holistic risk assessment also considers the marginal capability
uplift a model may provide over and above currently available tools and information, including currently available open-weights models.” (p. 7)
There is a clear commitment to putting development (and deployment) on hold if a risk cannot be sufficiently mitigated. To improve, this could have more detail, for instance by linking to clear KCI thresholds so that the decision to pause is unambiguous. A process for pausing development could also be developed.
“If, during the implementation of this framework, we identify a risk we cannot sufficiently mitigate, we will pause development and deployment until the point at which mitigation practices evolve to meet the risk.” (p. 8)
“The leading indicator assessment is run during pre-training, after pre-training is complete, and prior to deployment to ensure a comprehensive assessment as to whether a model warrants deeper inspection. This also allows for pause, review, and the application of mitigations as appropriate if a model shows signs of significant capability improvements.” (p. 5)
There is explicit reference to complying with specific standards and frameworks, and examples of containment measure requirements for high-risk and critical-risk capabilities. They are clearly linked to the high and critical capability thresholds, ie to these corresponding containment KCIs. To improve, the framework could be more specific on what will actually be implemented (rather than providing possible examples), as well as developing (or detailing the plan to develop) the containment measures for the critical-risk capabilities.
“The framework is built on a foundation of full-stack security, advancing comprehensive protections for key assets.” (p. 2)
“As Microsoft operates the infrastructure on which its models will be trained and deployed, we adopt an integrated full-stack approach to the security of frontier models, implementing safeguards at the infrastructure, model, and system layers. Security measures will be tailored to the specifics of each model, including its capabilities and the method by which it is made available and integrated into a system, so that the marginal risks a model may pose are appropriately addressed.” (p. 7)
“We expect scientific understanding of how to best secure the AI lifecycle will advance significantly in the coming months and years and will continue to contribute to, and apply, security best practices as relevant and appropriate. This includes existing best practice defined in leading standards and frameworks, such as NIST SP 800-53, NIST 800-218, SOC 2, Securing AI Model Weights: Preventing Theft and Misuse of Frontier Models, and Deploying AI Systems Securely, as well as industry practices, including from the Frontier Model Forum. Security safeguards are scaled up depending on the model’s pre-mitigation scores, with more robust measures applied to models with high and critical risk levels.” (p. 7)
“Models posing high-risk on one or more tracked capability will be subject to security measures protective against most cybercrime groups and insider threats. Examples of requirements for models having a high-risk score include:
Models posing critical risk on one or more tracked capability are subject to the highest level of security safeguards. Further work and investment are needed to mature security practices so that they can be effective in securing highly advanced models with critical risk levels that may emerge in the future. Appropriate requirements for critical risk level models will likely include the use of high-trust developer environments, such as hardened tamper-resistant workstations with enhanced logging, and physical bandwidth limitations between devices or networks containing weights and the outside world.” (p. 7)
They state that they engage in “advanced security red teaming”; more detail is required on the process of this red-teaming, and what constitutes sufficient proof. There is no process detailed for proving containment measures are sufficient for critical-risk models.
Importantly, they should detail proof in advance for why they believe the containment measures proposed will be sufficient to meet the KCI threshold. In addition, red-teaming is more an evidence gathering activity than a validation/proof; to improve, a case should be made for why they believe their containment measures to be sufficient.
For high-risk models: “Advanced security red teaming, using third parties where appropriate, to reasonably simulate relevant threat actors seeking to steal the model weights so that security safeguards are robust.” (p. 7)
They state that they engage in “advanced security red teaming, using third parties where appropriate”; more detail is required on the process of this red-teaming, what constitutes sufficient proof. Involving third parties should not be discretionary but part of the verification process. There is no process detailed for proving containment measures are sufficient for critical-risk models. In addition, red-teaming is more an evidence gathering activity than a validation/proof; to improve, a case should be made for why they believe their containment measures to be sufficient.
Importantly, they should detail proof in advance for why they believe the containment measures proposed will be sufficient to meet the KCI threshold.
For high-risk models: “Advanced security red teaming, using third parties where appropriate, to reasonably simulate relevant threat actors seeking to steal the model weights so that security safeguards are robust.” (p. 7)
Whilst they define deployment measures in general, these are not tied to KCI thresholds nor specific risk domains. For instance, the deployment measures for models that are high-risk in cybersecurity may be different to deployment measures for models that are critical-risk in autonomous AI R&D.
“We apply state-of-the-art safety mitigations tailored to observed risks so that the model’s risk level remains at low or medium once mitigations have been applied. […] Examples of safety mitigations we utilize include:
There is some implementation of proving deployment measures are sufficient, by testing that the post-mitigation model does not cross the same KRI threshold as the unmitigated model. More detail could be provided on what exactly the evaluation constitutes, and why they believe this to be sufficient proof. Further, proof should be provided ex ante for why they believe their deployment measures will meet the relevant KCI threshold.
“Post-mitigation capability assessment and safety buffer: Following application of safety and security mitigations, the model will be re-evaluated to ensure capabilities are rated low or medium and, if not, to guide further mitigation efforts.” (p. 8)
There is no mention of third-party verification of deployment measures meeting the threshold.
No relevant quotes found.
Whilst there is a commitment to conduct further research in mitigations, this is not specifically for assurance processes. However, there is an acknowledgment that current mitigations are insufficient, garnering partial credit.
“Models posing critical risk on one or more tracked capability are subject to the highest level of security safeguards. Further work and investment are needed to mature security practices so that they can be effective in securing highly advanced models with critical risk levels that may emerge in the future. Appropriate requirements for critical risk level models will likely include the use of high-trust developer environments, such as hardened tamper-resistant workstations with enhanced logging, and physical bandwidth limitations between devices or networks containing weights and the outside world.” (p. 7)
For all high and critical capability thresholds: “Deployment requirements: Further review and mitigations required.” (pp.11-14).
“We are focused on capabilities that could emerge in the short-to-medium term. Longer term or more speculative capabilities are the subject of ongoing research that we and many others across industry and academia are invested in.” (p. 2)
“We apply state-of-the-art safety mitigations tailored to observed risks so that the model’s risk level remains at low or medium once mitigations have been applied. We will continue to contribute to research and best-practice development, including through organizations such as the Frontier Model Forum, and to share and leverage best practice mitigations as part of this framework.” (p. 8)
There is no mention of providing evidence that the assurance processes are sufficient.
No relevant quotes found.
There is no mention of assumptions essential for effective implementation of mitigation measures. There is some mention of needing to monitor chain of thought, but this doesn’t appear to be for the purpose of checking underlying assumptions are effective – instead, this is to monitor for abuse from the customer side of deployment.
However, there is possibly an implicit acknowledgment that assumptions are required for evaluations (ie, KRI assessment), as the robustness of the evaluation must be justified: “This evaluation also includes a statement on the robustness of the evaluation method used and any concerns about the effectiveness or validity of the evaluation.” Partial credit is granted for this. To improve, the framework should detail the key technical assumptions necessary for the assurance processes to meet the KCI threshold, and evidence for why these assumptions are justified.
“Monitoring and remediation, including abuse monitoring in line with Microsoft’s Product Terms and provide channels for employees, customers, and external parties to report concerns about model performance, including serious incidents that may pose public safety and national security risks. We apply mitigations and remediation as appropriate to address identified concerns and adjust customer documentation as needed. Other forms of monitoring, including for example, automated monitoring in chain-of-thought outputs, are also utilized as appropriate. We continue to assess the tradeoffs between safety and security goals and legal and privacy considerations, optimizing for measures that can achieve specific safety and security goals in compliance with existing law and contractual agreements.” (p. 8)
“This evaluation also includes a statement on the robustness of the evaluation method used and any concerns about the effectiveness or validity of the evaluation.” (pp. 5-6)
There is a clear connection between elicitation effort and the resources available to threat actors, and some elicitation techniques are listed: “fine-tuning”, “multi-agent setup, leveraging prompt optmization, or connecting the model to whichever tools and plugins will maximize its performance.” More detail could be added on what these resources are assumed to be for threat actors, to explain why these elicitation methods are comprehensive enough. Further, specifics on e.g. compute used for finetuning could be added.
“Evaluations include concerted efforts at capability elicitation, i.e., applying capability enhancing techniques to advance understanding of a model’s full capabilities. This includes fine-tuning the model to improve performance on the capability being evaluated or ensuring the model is prompted and scaffolded to enhance the tracked capability—for example, by using a multi-agent setup, leveraging prompt optimization, or connecting the model to whichever tools and plugins will maximize its performance. Resources applied to elicitation should be extrapolated out to those available to actors in threat models relevant to each tracked capability.” (p. 6)
Both leading indicator assessments and deeper capability assessments are conducted every 6 months, explicitly to account for post-training enhancements. However, evaluation frequency is not linked to effective computing power used in training.
“A leading indicator assessment is run on any model that teams at Microsoft are optimizing for frontier capabilities or that Microsoft otherwise expects may have frontier capabilities.” and footnote following this sentence: “Frontier capabilities are defined as a significant jump in performance beyond the existing capability frontier in one advanced general-purpose capability or beyond frontier performance across the majority of these advanced general-purpose capabilities.” (p. 4)
“In addition, any model pre-trained using more than 10^26 FLOPs is subject to leading indicator assessment, given the (imperfect) correlation between pre-training compute and performance. This pre-training compute trigger will be revisited over time given improvements in training efficiency and as new approaches to enhancing model capabilities outside of pre-training are further developed, including techniques leveraging test-time compute.” (p. 4)
“The leading indicator assessment is run during pre-training, after pre-training is complete, and prior to deployment to ensure a comprehensive assessment as to whether a model warrants deeper inspection.” (p. 5)
“Models in scope of this framework will undergo leading indicator assessment at least every six months to assess progress in post-training capability enhancements, including fine-tuning and tooling. Any model demonstrating frontier capabilities is then subject to a deeper capability assessment to provide strong confidence about whether it has a tracked capability and to what level, informing mitigations.” (p. 5)
“After the first deeper capability assessment, we will conduct subsequent deeper capability assessments on a periodic basis, and at least once every six months.” (p. 6)
There is an explicit consideration of incorporating frontier post-training enhancements when re-evaluating models to ensure KRIs are not crossed unnoticed. An improvement would be to add detail on how they account(ed) for how post-training enhancements’ risk profiles change with different model structures – namely, post-training enhancements are much more scalable with reasoning models, as inference compute can often be scaled to improve capabilities.
“Models in scope of this framework will undergo leading indicator assessment at least every six months to assess progress in post-training capability enhancements, including fine-tuning and tooling. Any model demonstrating frontier capabilities is then subject to a deeper capability assessment to provide strong confidence about whether it has a tracked capability and to what level, informing mitigations.” (p. 5)
There is no mention of having the evaluation methodology vetted by third parties.
No relevant quotes found.
There is no mention of evaluations being replicated; they mention that external experts may be “involved” in evaluations, at Microsoft’s discretion. However, this does not necessarily mean the external experts will be conducting the evaluations: it is more likely they will be participants of internally run expert elicitation or red-teaming, for instance.
“Deeper capability assessment […] involves robust evaluation of whether a model possess tracked capabilities at high or critical levels, including through adversarial testing and systematic measurement using state-of-the-art methods. […] As appropriate, evalautions involve qualified and expert external actors that meet relevant security standards, including those with domain-specific expertise.” (pp. 5-6)
There is a mention of monitoring in terms of reporting concerns, as well as automated monitoring in chain-of-thought outputs. However, this is not linked explicitly to monitoring mitigation effectiveness, and it is not clear if monitoring is ongoing. To improve, the framework should describe systematic, ongoing monitoring to ensure mitigation effectiveness is tracked continuously such that the KCI threshold will still be met, when required.
“Monitoring and remediation, including abuse monitoring in line with Microsoft’s Product Terms and provide channels for employees, customers, and external parties to report concerns about model performance, including serious incidents that may pose public safety and national security risks. We apply mitigations and remediation as appropriate to address identified concerns and adjust customer documentation as needed. Other forms of monitoring, including for example, automated monitoring in chain-of-thought outputs, are also utilized as appropriate. We continue to assess the tradeoffs between safety and security goals and legal and privacy considerations, optimizing for measures that can achieve specific safety and security goals in compliance with existing law and contractual agreements.” (p. 8)
There is no mention of KCIs protocols being vetted by third parties.
No relevant quotes found.
There is no mention of control evaluations/mitigation testing being replicated or conducted by third-parties.
No relevant quotes found.
There is a commitment to share substantial detail, seemingly with members of the Frontier Model Forum, on models’ KRI levels and corresponding KCI measures.
There is also a commitment to publishing capabilities, evaluations and risk classification publicly. More detail could be given on ex ante criteria for redacting information, to avoid discretion. To improve, the company should commit to alerting authorities if critical thresholds are reached.
“We will continue to contribute to research and best-practice development, including through organizations such as the Frontier Model Forum. […] Examples of safety mitigations we utilize include: […] Deployment guidance, with clear documentation setting out the capabilities and limitations of the model, including factors affecting safe and secure use and details of prohibited uses. This documentation will also include a summary of evaluation results, the deeper capability
assessment, and safety and security mitigations. For example, the documentation could outline specific capabilities and tasks that the model robustly fails to complete which would be essential for a high or critical risk rating.” (p. 8)
“Information about the capabilities and limitations of the model, relevant evaluations, and the model’s risk classification will be shared publicly, with care taken to minimize information hazards that could give rise to safety and security risks and to protect commercially sensitive information.” (p. 9)
“Evaluations are documented in a consistent fashion setting out the capability being evaluated, the method used, and evaluation results.” (p. 5)
No commitment to permitting the reports, which detail the results of external evaluations (i.e. any KRI or KCI assessments conducted by third parties), to be written independently and without interference or suppression.
No relevant quotes found.
There is a commitment to revisiting “our list of tracked capabilities frequently, ensuring it remains up to date in light of technological developments and improved understanding of model capabilities, risks, and mitigations.” To improve, more detail on (a) how this improved understanding will be gotten, (b) a process for identifying novel risks and (c) justification for why this process is likely to detect novel risks, could be given.
“AI technology continues to develop rapidly, and there remains uncertainty over which capabilities may emerge and when. We continue to study a range of potential capability related risks that could emerge, conducting ongoing assessment of the severity and likelihood of these risks. We then operationalize the highest-priority risks through this framework. We will revisit our list of tracked capabilities frequently, ensuring it remains up to date in light of technological developments and improved understanding of model capabilities, risks, and mitigations.” (p. 3)
They mention that they conduct “ongoing assessment of the severity and likelihood of these [potential] risks. We then operationalize the highest-priority risks through this framework.” However, details on how they assess the severity and likelihood of novel risks is not given. More detail could be added on how the “explicit discussion on how this framework may need to be improved” will lead to incorporations of risks identified post-deployment. To improve, a process which triggers risk modelling exercises once a novel risk domain or threat model is found, and analysing how this could intersect with existing threat models, could be conducted.
“We continue to study a range of potential capability related risks that could emerge, conducting ongoing assessment of the severity and likelihood of these risks. We then operationalize the highest-priority risks through this framework. We will revisit our list of tracked capabilities frequently, ensuring it remains up to date in light of technological developments and improved understanding of model capabilities, risks, and mitigations.” (p. 3)
“We will update our framework to keep pace with new developments. Every six months, we will have an explicit discussion on how this framework may need to be improved. We acknowledge that advances in the science of evaluation and risk mitigation may lead to additional requirements in this framework or remove the need for existing requirements. Any updates to our practices will be reviewed by Microsoft’s Chief Responsible AI Officer prior to their adoption. Where appropriate, updates will be made public at the same time as we adopt them.” (p. 11)
Although not specified in more detail than Executive Officers or delegates, the framework specifies that certain executive officers hold the responsibility for making key decisions regarding risks.
“Documentation regarding the pre-mitigation and post-mitigation capability assessment will be provided to Executive Officers responsible for Microsoft’s AI governance program (or their delegates) along with a recommendation for secure and trustworthy deployment setting out the case that: 1) the model has been adequately mitigated to low or medium risk level, 2) the marginal benefits of a model outweigh any residual risk and 3) the mitigations and documentation will allow the model to be deployed in a secure and trustworthy manner. The Executive Officers (or their delegates) will make the final decision on whether to approve the recommendation for secure and trustworthy deployment.” (p. 9)
No mention of a management risk committee.
No relevant quotes found.
The framework outlines clearly which decisions are made, the basis on which they will be made and who makes the decisions.
“If, during the implementation of this framework, we identify a risk we cannot sufficiently mitigate, we will pause development and deployment until the point at which mitigation practices evolve to meet the risk.” (p. 8)
“Holistic risk assessment: The results of capability evaluation and an assessment of risk factors external to the model then inform a determination as to whether a model has a tracked capability and to what level. This includes assessing the impact of potential system level mitigations and societal and institutional factors that can impact whether and how a hazard materializes. This holistic risk assessment also considers the marginal capability uplift a model may provide over and above currently available tools and information, including currently available open-weights models.” (p. 6)
“The Executive Officers (or their delegates) will make the final decision on whether to approve the recommendation for secure and trustworthy deployment. The Executive Officers (or their delegates) are also responsible for assessing that the recommendation for secure and trustworthy deployment and its constituent parts have been developed in a good faith attempt to determine the ultimate capabilities of the model and mitigate risks.” (p. 9)
No mention of escalation procedures.
No relevant quotes found.
The company has a Chief Responsible AI Officer, which should be equivalent to this function.
“Any updates to our practices will be reviewed by Microsoft’s Chief Responsible AI Officer prior to their adoption.” (p. 9)
No mention of an advisory committee.
No relevant quotes found.
The framework lists specific capabilities that are tracked.
“This framework tracks the following capabilities that we believe could emerge over the short-to-medium term and threaten national security or pose at-scale public safety risks if not appropriately mitigated.” (p. 3)
No mention of people that challenge decisions.
No relevant quotes found.
The framework specifies that documentation will be provided to senior management for decision making.
“Documentation regarding the pre-mitigation and post-mitigation capability assessment will be provided to Executive Officers responsible for Microsoft’s AI governance program.” (p. 9)
No mention of a central risk function.
No relevant quotes found.
The framework specifies that the framework is part of the remit of Microsoft’s internal audit team.
“This framework is subject to Microsoft’s broader corporate governance procedures, including independent internal audit.” (p. 9)
The framework mentions learning from external experts, but nothing about external independent review.
“We will also highlight the value of learning from experts outside of AI, including those with expertise in measurement science and in scientific domains like chemistry and biology, as well as those with knowledge of managing the risks of other complex technologies.” (p. 10)
There is no mention of a Board committee, but the framework specifies that Microsofts’s broader corporate governance, which could be expected to include the Board, applies.
“This framework is subject to Microsoft’s broader corporate governance procedures, including independent internal audit.” (p. 9)
No mention of any additional governance bodies.
No relevant quotes found.
The framework includes a statement regarding its purpose to manage large-scale risks.
“Microsoft’s Frontier Governance Framework manages potential national security and at-scale public safety risks that could emerge as AI models increase in capability.” (p. 2)
There are no direct mentions of elements of risk culture, but the framework refers to security best practices.
“We expect scientific understanding of how to best secure the AI lifecycle will advance significantly in the coming months and years and will continue to contribute to, and apply, security best practices as relevant and appropriate.” (p. 7)
The company has an established whistleblower mechanism.
“Microsoft employees have the ability to report concerns relating to this framework and its implementation, as well as AI governance at Microsoft more broadly, using our existing concern reporting channels, with protection from retaliation and the option for anonymity” (p. 9)
The framework lists the risks that are being tracked and what information of them will be shared externally.
“This framework tracks the following capabilities…Chemical, biological, radiological, and nuclear (CBRN) weapons…Offensive cyberoperations…Advanced autonomy.” (p. 3)
“Information about the capabilities and limitations of the model, relevant evaluations, and the model’s risk classification will be shared publicly, with care taken to minimize information hazards that could give rise to safety and security risks and to protect commercially sensitive information.” (p. 9)
The framework provides plenty of details on the governance structure and how it is integrated into the company’s broader AI governance program.
“This framework is integrated with Microsoft’s broader AI governance program, which sets out a comprehensive risk management program that applies to all AI models and systems developed and deployed by Microsoft.” (p. 2)
“We will update our framework to keep pace with new developments. Every six months, we will have an explicit discussion on how this framework may need to be improved. We acknowledge that advances in the science of evaluation and risk mitigation may lead to additional requirements in this framework or remove the need for existing requirements. Any updates to our practices will be reviewed by Microsoft’s Chief Responsible AI Officer prior to their adoption. Where appropriate, updates will be made public at the same time as we adopt them.” (p. 9)
“In addition to high-risk capabilities, a broader set of risks are governed when Microsoft develops and deploys AI technologies. Under Microsoft’s comprehensive AI governance program, frontier models—as well as other models and AI systems—are subject to relevant evaluation, with mitigations then applied to bring overall risk to an appropriate level…Our efforts to assess and mitigate risks related to this framework’s tracked capabilities benefit from this broadly applied governance program, which is continuously improved. ” (p. 4)
The framework specifies information to be shared externally and with whom.
“Information on model or system performance, responsible use, and suggested system-level evaluations is shared with downstream actors integrating models into systems, including external system developers and deployers and teams at Microsoft building models. Appropriate information sharing is important to facilitate mitigation of a broader set of risks, many of which are heavily shaped by use case and deployment context as well as laws and norms that vary across jurisdictions.” (p. 4)
“Microsoft will prioritize ongoing contributions to this work and expand its collaboration with government, industry, and civil society, including through organizations like the Frontier Model Forum, to solve the most pressing challenges in AI risk management.” (p. 10)