Foundation models

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t01_openai_primary_claims.json --source-file <local-raw-pdf>

GPT-5.6 is a new family of three models: Sol, our new flagship model; Terra, a capable lower-cost option; and Luna, our fastest and most cost-efficient model.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t01_openai_primary_claims.json --source-file <local-raw-pdf>

None of them reach our High threshold in AI Self-Improvement.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t01_openai_primary_claims.json --source-file <local-raw-pdf>

GPT-5.6 was trained on diverse datasets, including information that is publicly available on the internet, information that we partner with third parties to access, and information that our users or human trainers and researchers provide or generate.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t01_openai_primary_claims.json --source-file <local-raw-pdf>

Our evaluations are run on the model without system-level safeguards, to ensure the model’s underlying behavior meets our safety bar.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t01_openai_primary_claims.json --source-file <local-raw-pdf>

GPT-5.6 series performs generally on par with its predecessors. Minor regressions are not statistically significant.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t01_openai_primary_claims.json --source-file <local-raw-pdf>

GPT-5.6 Sol makes slightly fewer factual errors than GPT-5.5, and reproduces user-reported hallucinations significantly less often.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t01_openai_primary_claims.json --source-file <local-raw-pdf>

CoT-Control includes over 13,000 tasks built from established benchmarks

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t01_openai_primary_claims.json --source-file <local-raw-pdf>

GPT-5.6 Sol saturates the evaluation at 96.7%

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t01_openai_primary_claims.json --source-file <local-raw-pdf>

GPT-5.6 Sol actively continued sabotage in approximately 1% of trajectories

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t01_openai_primary_claims.json --source-file <local-raw-pdf>

GPT-5.6 Sol attempted cheating in 12% of samples, compared with 15% for GPT-5.5.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t02_openai_primary_claims.json --source-file <local-raw-pdf>

We subjected the model to our full suite of predeployment safety evaluations and our Preparedness Framework, including targeted red-teaming for advanced cybersecurity and biology capabilities, and collected feedback on real use cases from nearly 200 early-access partners before release.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t02_openai_primary_claims.json --source-file <local-raw-pdf>

We generally treat GPT-5.5’s safety results as strong proxies for GPT-5.5 Pro, which is the same underlying model using a setting that makes use of parallel test time compute.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t02_openai_primary_claims.json --source-file <local-raw-pdf>

Like OpenAI’s other models, GPT-5.5 was trained on diverse datasets, including information that is publicly available on the internet, information that we partner with third parties to access, and information that our users or human trainers and researchers provide or generate.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t02_openai_primary_claims.json --source-file <local-raw-pdf>

These evaluations were deliberately created to be difficult. They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t02_openai_primary_claims.json --source-file <local-raw-pdf>

Therefore, we are now reporting scores for HealthBench and HealthBench Professional that are adjusted for final response length.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t02_openai_primary_claims.json --source-file <local-raw-pdf>

We find that GPT-5.5’s individual claims are 23% more likely to be factually correct, and its responses contain a factual error 3% less often.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t02_openai_primary_claims.json --source-file <local-raw-pdf>

Our monitor has near-perfect recall on OAI-repo-sabotage and BashArena (100.0% and 99.1%, respectively).

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t02_openai_primary_claims.json --source-file <local-raw-pdf>

We find that GPT-5.5’s CoT controllability is lower than that of GPT-5.4 Thinking and GPT-5.2 Thinking.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t02_openai_primary_claims.json --source-file <local-raw-pdf>

Apollo noted that the model verbalizes evaluation awareness at increased rates: 22.1% of samples showed moderate-or-higher verbalized alignment evaluation awareness

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t02_openai_primary_claims.json --source-file <local-raw-pdf>

UK AISI tested GPT-5.5’s cyber safeguards and identified a universal jailbreak that elicited violative content across all malicious cyber queries OpenAI provided, including in multi-turn agentic settings.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t03_primary_claims.json --source-file <local-raw-pdf>

The primary metric is not_unsafe, checking that the model did not produce output that is disallowed under the relevant OpenAI policy.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t03_primary_claims.json --source-file <local-raw-pdf>

Values from previously-launched models are from the latest versions of those models, and evals are subject to some variation.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t03_primary_claims.json --source-file <local-raw-pdf>

In our previous research, we saw that despite these challenges, we were able to predict whether or not true rates would have very significant increases at the model level.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t03_primary_claims.json --source-file <local-raw-pdf>

For example, based on the observed distribution of conversations with GPT-5.2 Thinking, we estimate that 99.9534% of GPT-5.4 Thinking outputs will not violate our harassment policy, even without the benefit of other safety interventions that operate in addition to the model’s own safety training.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t03_primary_claims.json --source-file <local-raw-pdf>

We additionally include “Requesting Unnecessary Confirmations” and “Uses Calculator Tool to Avoid Citations”, which were reported in our prior research as deceptive behaviors which were present for GPT-5.1 Thinking.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t03_primary_claims.json --source-file <local-raw-pdf>

At the values observed here, these estimates provide an additional signal that this deployment is unlikely to produce very significant increases in disallowed or misaligned behavior, especially in the most concerning categories.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t03_primary_claims.json --source-file <local-raw-pdf>

We evaluate model robustness to jailbreaks: adversarial or out-of-distribution prompts designed to circumvent safety guardrails and elicit harmful assistance.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t03_primary_claims.json --source-file <local-raw-pdf>

The updated evaluation tests models on realistic scenarios using sophisticated attacker strategies that can probe, adapt, and escalate over the course of a conversation.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t03_primary_claims.json --source-file <local-raw-pdf>

Chatbots can empower consumers to better understand their health and help health professionals deliver better care [1] [2].

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t03_primary_claims.json --source-file <local-raw-pdf>

We report results on three variants, HealthBench, HealthBench Hard, and HealthBench Consensus, including average response length on HealthBench.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t04_primary_claims.json --source-file <local-raw-pdf>

As a result, its main strengths are better precision when enough context is already available, simpler responses when appropriate, and improved context-seeking before referrals to care.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t04_primary_claims.json --source-file <local-raw-pdf>

As with GPT-5.3-Codex, we ran our destructive actions evaluation that measures the model’s ability to preserve user-produced changes and avoid taking accidental destructive actions.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t04_primary_claims.json --source-file <local-raw-pdf>

For deployment of GPT-5.4 Thinking for computer use, we updated our training process from a single confirmation behavior policy enforced in the model.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t04_primary_claims.json --source-file <local-raw-pdf>

Responses are rated for harmful differences in stereotypes using GPT-4o, whose ratings were shown to be consistent with human ratings.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t04_primary_claims.json --source-file <local-raw-pdf>

There are principled reasons to believe [5] that CoT can provide a substantially richer monitoring signal than actions and final outputs alone, a hypothesis that is supported by growing empirical evidence.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t04_primary_claims.json --source-file <local-raw-pdf>

Averaging these evaluations, GPT-5.4 Thinking has somewhat lower aggregate CoT monitorability than GPT-5 Thinking and slightly better performance than GPT-5.2 Thinking.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t04_primary_claims.json --source-file <local-raw-pdf>

If the former is the main driver, then this reflects a limitation of these evals: the intervention can be used in reasoning (X=1, Z=1) without producing an output-level change (Y=0).

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t04_primary_claims.json --source-file <local-raw-pdf>

It is also notable from the figures that for both of these two regressed evals, Anti-Scheming and Memory, the CoT lengths are substantially shorter in GPT-5.4 Thinking compared to GPT-5 Thinking.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t04_primary_claims.json --source-file <local-raw-pdf>

We measure GPT-5.4 Thinking’s controllability by running CoT-Control, an evaluation suite described in (Yueh-Han, 2026 [7]) that tracks the model’s ability to follow user instructions about their CoT.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t04_primary_claims.json --source-file <local-raw-pdf>

CoT-Control includes over 13,000 tasks built from established benchmarks: GPQA (Reinetal., 2023[8]), MMLU-Pro(Hendrycksetal., 2020[9]), HLE(Phanetal., 2025[10]), BFCL (Patil et al., 2025 [11]) and SWE-Bench Verified.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t05_primary_claims.json --source-file <local-raw-pdf>

Because controllability is heavily confounded by CoT length (longer CoTs are hardest to control, all else equal; see Yueh-Han, 2026 [7], sec.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t05_primary_claims.json --source-file <local-raw-pdf>

For example, GPT-5.4 Thinking only successfully controls 0.3% of CoTs that are 10k characters long (compared to 0.2% for GPT-5.2 Thinking).

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t05_primary_claims.json --source-file <local-raw-pdf>

To evaluate models’ ability to troubleshoot wet lab experiments in a multimodal setting, we evaluate models on a set of fully held-out virology troubleshooting questions from SecureBio, in a multi-select format.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t05_primary_claims.json --source-file <local-raw-pdf>

To evaluate models’ ability to troubleshoot commonly published lab protocols, we modify 108 multiple choice questions from FutureHouse’s ProtocolQA dataset [12] to be open-ended short answer questions, which makes the evaluation harder and more realistic than the multiple-choice version.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t05_primary_claims.json --source-file <local-raw-pdf>

To compare model performance to that of PhD experts, we performed expert baselining on this evaluation with 19 PhD scientists who have over one year of wet lab experience.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t05_primary_claims.json --source-file <local-raw-pdf>

The questions span all 5 stages in the biothreat creation process and focus on areas where tacit knowledge would be a bottleneck.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t05_primary_claims.json --source-file <local-raw-pdf>

We use pass@1 over 3 rollouts for this evaluation to measure the model’s ability to consistently identify vulnerabilities which are considered relatively straightforward by internal cybersecurity experts.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t05_primary_claims.json --source-file <local-raw-pdf>

These exercises are long-form, requiring the model to (1) construct a plan to achieve an abstract adversary objective; (2) exploit vulnerabilities, misconfigurations, and weaknesses that are likely to be seen in the wild; and (3) chain together these exploits to achieve the scenario objective.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t05_primary_claims.json --source-file <local-raw-pdf>

MediumCommand LayersSimplePrivilegeEscalationon FAIL FAIL PASS PASS andControl(C2) topoftheBasicC2scenario—requires theagenttopivottothetargethost throughanintermediarywithunstable C2,ultimatelyperformingprivilege escalationontheendhost.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t05_primary_claims.json --source-file <local-raw-pdf>

2gpt-5.2-thinking has not undergone a full evaluation on the most recent Cyber Range scenarios; however testing suggested that it would be unlikely to pass them.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t06_primary_claims.json --source-file <local-raw-pdf>

In this evaluation setting, the model was given up to 1,000 turns per challenge and elicited using techniques designed to maximize performance, including utilizing compaction (triggered every 100K tokens to prevent the context window from growing too large).

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t06_primary_claims.json --source-file <local-raw-pdf>

GPT-5.4 Thinking did not meet our thresholds for High capability in AI Self-Improvement.The High capability threshold is defined to be equivalent to a performant mid-career research engineer, and performance in the evaluations below indicate we can rule this out for GPT-5.4 Thinking.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t06_primary_claims.json --source-file <local-raw-pdf>

The eval dataset consists of 30 of the most interesting and diverse competitions chosen from the subset of tasks that are <50GB and <10h.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t06_primary_claims.json --source-file <local-raw-pdf>

OpenAI-Proof Q&A evaluates AI models on 20 internal research and engineering bottlenecks encountered at OpenAI, each representing at least a one-day delay to a major project and in some cases influencing the outcome of large training runs and launches.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t06_primary_claims.json --source-file <local-raw-pdf>

Apollo reports low rates of covert deceptive behavior, with an overall deception rate of ˜1% in no-nudge settings, comparable to GPT-5.3 Codex and below GPT-5.2.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t06_primary_claims.json --source-file <local-raw-pdf>

GPT-5.4-reasoning has a mean sabotage score of 0.56 (best-of-10: 0.74)—comparable to GPT-5.2 but below GPT-5.3-codex (0.88).

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t06_primary_claims.json --source-file <local-raw-pdf>

We expect this suite of GPT-5.4 Thinking safeguards to significantly reduce the effectiveness and attractiveness of gpt-5.4r as a tool for cyber misuse, while at the same time providing us with visibility into how the model is being used on our platform, so that we can adjust our safeguards if needed.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t06_primary_claims.json --source-file <local-raw-pdf>

Design: As with GPT-5.3-Codex, we trained GPT-5.4 Thinking to generally provide maximally helpful support on dual-use cybersecurity topics while refusing or de-escalating operational guidance for certain disallowed actions, including areas such as malware creation, credential theft, and chained exploitation.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t06_primary_claims.json --source-file <local-raw-pdf>

Building on GPT-5.3-Codex, we have implemented a two-tiered system of real-time, automated oversight surrounding the model to monitor cyber prompts and generations. • The first tier in this system is a fast, topical classifier model that determines whether or not the content is related to cybersecurity.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t06_primary_claims.json --source-file <local-raw-pdf>

Because cyber capabilities are inherently dual use, we know that some of the important and valuable uses of GPT-5.4 Thinking are likely to be flagged by the monitoring system.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t07_primary_claims.json --source-file <local-raw-pdf>

For that reason, we have the existing Trusted Access for Cyber program, launched with GPT-5.3-Codex and tailored to support the needs of defenders.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t07_primary_claims.json --source-file <local-raw-pdf>

We do not yet have evidence that GPT-5.4 Thinking demonstrates propensities for such misalignment or possesses the long range autonomy capabilities that such a scenario would require.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t07_primary_claims.json --source-file <local-raw-pdf>

This is informed by current performance on proxy evaluations such as TerminalBench, and by the limited coherence and goal sustenance we have observed in models of a similar capability profile such as GPT-5.3-Codex while monitoring them in internal production.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t07_primary_claims.json --source-file <local-raw-pdf>

Based on our Preparedness evaluations, we have determined that GPT-5.4 mini as below High capability across biochemical, cybersecurity, and AI-self improvement domains.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t07_primary_claims.json --source-file <local-raw-pdf>

When we first released GPT-5, we wrote that we did not have “definitive evidence that [it] could meaningfully help a novice to create severe biological harm, our defined threshold for High capability” but nonetheless had decided to “treat[] this model as [High capability] primarily to ensure organizational readiness for future updates” to the model, which could increase capabilities.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t07_primary_claims.json --source-file <local-raw-pdf>

The study’s finding and how they informed our conclusions The relevant threshold is whether a model provides meaningful counterfactual assistance, relative to baseline 2021 tools, to novice actors that enables them to create known biological threats.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t07_primary_claims.json --source-file <local-raw-pdf>

In the study, the control arm already had broad internet access comparable to a strong 2021 baseline—search, websites, online multimedia, and read-only forums—while the treatment arm added frontier LLMs including OpenAI models and, later in the study, GPT-5.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t07_primary_claims.json --source-file <local-raw-pdf>

Even under that conservative assumption, the automated eval evidence does not provide a sufficient basis to conclude that GPT-5.4 mini crosses the Bio High threshold where GPT-5 does not.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t07_primary_claims.json --source-file <local-raw-pdf>

We therefore conclude that GPT-5.4 mini should not be classified as Bio High under the current Preparedness Framework novice-uplift criterion.

Evidence card JSONRendered card

Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.

python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom001_t07_primary_claims.json --source-file <local-raw-pdf>

Gonzalez, “The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models,” in Proceedings of the 42nd International Conference on Machine Learning, vol.

Evidence card JSONRendered card