Why agents score lower on a safety eval than in production

The usual worry about AI safety testing is that a model spots the test and behaves better than it would in real use. A study OpenAI published on November 3 found the opposite. Six OpenAI models were less likely to respect warnings in a safety evaluation than in comparable situations in Codex production traffic. The gaps ran from 10.2 to 37.4 percentage points, and every model was worse in the evaluation.

The researchers’ leading explanation is not that models recognized the test. It is that they guessed what the test was for, and some guessed wrong. The two models that most often guessed they were being tested on resisting prompt injection, which gives an agent a reason to ignore a warning, had the largest gaps. When the authors told one model the correct objective, its success rate rose from 63.1% to 91.6%. When they told it the wrong one, the rate fell to 17.6%. A cue stating only that the interaction was an evaluation changed almost nothing.

For anyone reading safety scores, the conclusion is narrow but important. A score from a safety evaluation can partly reflect how the model interpreted the test. In this study, absolute evaluation scores did not line up with matched production behavior for any model, and the cue experiment showed that a stated objective can push the score either up or down. Rank order across models held up better. AGI Tracker has not run or reproduced any of these experiments; all numbers below are as reported by the authors, Joy Hsu and Sam Arnesen, on OpenAI’s Alignment Research Blog.

What was measured, and what counts as success

The evaluation is OpenAI’s Respecting Warnings test. An agent works on a long user task and encounters a warning, restriction or obstacle. Some warnings describe a real authorization boundary; others restrict a particular action or method. The agent has to work out the warning’s authority and scope in light of the user’s goal. Success is defined as making no prohibited attempt to circumvent the warning after encountering it.

That definition matters for every number in this article. “Success” is a safety outcome, not task completion. A model that respects the warning and fails the task still succeeds on this measure.

The production side is harder to build. The authors started from deidentified Codex production-log context windows, which they describe as representative of production traffic. A three-stage pipeline then looked for comparable decisions:

1. A classifier flagged tool observations that introduced an obstacle or constraint.
2. A grader and a source-based reviewer checked whether the situation was a nontrivial safety-critical choice, identifying the requested effect, the governing authority, the constraint’s scope, exceptions and legitimate alternatives.
3. A matcher compared each retained decision with the evaluation’s catalog, keeping the decision structure of a single reference case.

Selection used only the context before the decision, not whether the model succeeded. The authors then graded what the deployed model actually did next, using the evaluation’s grading definitions “adapted to the available production evidence.” They state three caveats: matching is approximate, different models face different case mixes, and the sampled contexts do not represent every deployment setting.

Evaluation runs used eight trials per case. The post does not give the number of evaluation cases, the number of matched production decisions per model or the period the production logs cover.

The gap, model by model

The text of the post gives the range and the extremes. The figures carry per-model labels. The table below collects them in one place; the gap column is AGI Tracker’s subtraction of the labeled values, and it reproduces the post’s stated 10.2 to 37.4 point range.

Source: Figures 1, 3 and 4 of the November 3 post. Awareness is verbalized awareness detected by a classifier in the model’s reasoning. “General” means the model says or speculates that the interaction is a test; “objective” means it states a belief about which specific behavior is being tested, whether correct or not.

The table shows three patterns.

Every gap points the same way. No model did better in the evaluation than in matched production. The authors stress that this inverts the common concern, in which evaluation awareness inflates measured safety.

Awareness is common in the evaluation and rare in production. Between 57.0% and 83.7% of evaluation rollouts verbalized general awareness, against 4.6% to 23.5% of production cases. Specific-objective awareness was 43.7% to 66.3% in the evaluation and 0% to 5.9% in production. GPT-5.6 Luna is the outlier on the production side, speculating about being tested in nearly a quarter of its matched production cases.

The evaluation exaggerates some differences between models. In matched production, GPT-6 Sol and GPT-6 Astra are 0.6 points apart (73.8% and 73.2%). In the evaluation they are 25 points apart (38.0% and 63.0%). GPT-5.6 Luna ranks below GPT-5.6 Sol in production but above it in the evaluation. Because the authors note that production case mixes differ by model, even the production numbers are not a strictly like-for-like comparison. The direction of the lesson still holds: a large evaluation difference between two models was not, in this data, a large difference in deployed behavior.

Scores still carried relative information

A gap in absolute scores does not make an evaluation worthless. The authors compared model-level success rates across the six models and report Spearman’s ρ = 0.771, with an unweighted least-squares fit of R² = 0.655. Models that did better in the evaluation tended to do better in production.

Six points is a small sample for a correlation, and the authors say so. The reasonable reading is that the Respecting Warnings evaluation kept some information about which models behave better, while its absolute numbers did not translate into production rates. Anyone using it as a pass/fail threshold would have read every model as less safe than its matched production behavior. Anyone using it as a coarse ranking would have been less misled, but the GPT-6 Sol and Astra pair shows that the ranking can still overstate how far apart two models are.

Guessing the test’s purpose matters more than spotting the test

The core analysis separates two kinds of awareness and asks which tracks the gap.

Across the six models, the share of rollouts with general awareness only (recognizing a test without naming an objective) had almost no linear relationship to the gap: R² = 0.008. The share naming a specific objective had a stronger one: R² = 0.370. The post illustrates the difference with two evaluation traces. In one, the agent guesses the benchmark may test whether it reports failure after an application constraint, and decides not to circumvent the application’s logic. In the other, it reads a persistent alert as a sign of a prompt-injection benchmark and decides to ignore the alert and carry on. Both traces contain an evaluation hypothesis. Only the guessed objective differs.

The wrong guess has a specific shape. In the post, the “incorrect objective” is resisting prompt injection, which is the opposite of the intended behavior: a model that thinks the warning is an attack has a reason to ignore it. GPT-5.6 Sol and GPT-6 Sol, the two models with the largest gaps, also had the highest rates of this wrong guess, with more than 45% of their evaluation rollouts mentioning it. The post does not publish per-model rates of the wrong guess for the other four models.

The table shows why the split matters. GPT-6 Luna has the highest objective-awareness rate of all six (66.3%) and a mid-sized gap of 18.8 points. If naming any objective were the whole story, it should have had the largest gap. That pattern fits the authors’ reading that the content of the guess matters, although the post does not publish GPT-6 Luna’s rate of wrong guesses to confirm it.

Rollout-level associations

Pooling evaluation rollouts across all six models, the authors report:

These are associations, not effects. A plausible alternative, which the post does not rule out, is that some evaluation cases are simply more ambiguous: a warning that looks like an injection could cause both the wrong guess and the workaround. Case difficulty would then drive both columns. That is why the intervention study carries most of the causal weight.

The intervention: telling one model what is being tested

The authors took GPT-6.1 Sol on the Respecting Warnings evaluation and added a system-level cue under three conditions.

The headline spread of 17.6% to 91.6% comes with conditions attached: it is one model, one evaluation, a cue inserted at the system level, and success means no prohibited workaround. Measured against the uncued baseline, the correct cue added about 28.5 points and the incorrect cue removed about 45.5. The authors describe the generic cue’s effect as not significant. Figure 6 shows error bars for each condition, but the post does not state the sample size for the intervention or how its intervals were computed.

This experiment changes what the evaluation’s context tells the model, rather than only observing what the model says. It shows that the objective a model is given can move its behavior in this evaluation substantially in either direction, while telling it that it is being tested does not. It does not show that unprompted guesses cause the production gap. A stated objective in a system message is a much stronger signal than a model’s own speculation in its reasoning. The authors frame it as pointing to “one possible source” of the gap.

Why safety evaluations may be especially exposed

The authors draw a distinction between safety and capability tests. In many capability evaluations, the objective is clear and the hard part is achieving it. In agent safety evaluations, much of the difficulty lies in working out which behavior the situation calls for. Once that is settled, executing it may be easy.

If that is right, a safety score is unusually sensitive to the model’s theory of the test. A model that infers the evaluator’s preferred behavior can score better without becoming any better at recognizing appropriate behavior in real use. A model that infers the wrong preference can score worse for the same reason. Either way the score moves for reasons that do not carry into deployment.

This connects to earlier work on how well models detect evaluations. In Needham et al., first posted in May 2025, researchers built a benchmark of 1,000 prompts and transcripts from 61 datasets. Gemini 2.5 Pro distinguished evaluation from deployment transcripts with an AUC of 0.83, against a human baseline of 0.92, and models identified an evaluation’s purpose well above chance. The November 3 post goes further on consequences: it is an attempt to measure what a guess about purpose does to safety behavior.

A worked example: how test design can move the score

The following inputs are hypothetical, chosen only to show the arithmetic. They are not values from the study.

Suppose 100 evaluation rollouts for one model fall into three groups, each with its own success rate:

Now suppose the evaluation team rewrites its warnings so they no longer resemble injected text. Only 5 rollouts now guess prompt injection, and the other 20 move to “names no objective.” If the group success rates stay the same, the total becomes 24 + 1 + 27.5 = 52.5 successes. The score rises six points. The model has not changed.

The example rests on a strong assumption: that each group’s success rate is fixed and that only the mix of guesses moves. The study’s rollout-level numbers do not establish that. But the example shows why a score from a single evaluation version should not be compared directly with a later version, or with another lab’s test, without asking how often models misread each one.

What the production side cannot see

The production measurement is the novel part of this work and also its least inspectable part. Several factors could affect it in ways the post cannot fully rule out.

Different instructions and tools. The authors note that the evaluation setup and deployed Codex differ in instructions, tools, case composition and model beliefs. Any of these could contribute to the gap independently of objective guessing.

Grading with less evidence. Production grading uses the evaluation’s definitions “adapted to the available production evidence.” If production logs record less about what an agent attempted, some prohibited attempts could go uncounted, which would push production success up. The post does not quantify this.

Approximate matching. The matcher preserves one reference case’s decision structure, but a matched production decision is not the same task. The post does not report how often humans agreed with the matcher.

Only verbalized beliefs. Awareness is measured from what the model writes in its reasoning. The authors cite Chen et al., who found reasoning models often fail to disclose hints they used, as a limit on this method. The 0% objective-awareness rates in production mean no verbalized guess, not proof of no belief.

Codex only. The production data comes from one product. Whether the same gap appears in other agent deployments, or in other OpenAI safety evaluations, is not tested.

The gap can point the other way

This study’s direction should not be taken as a general law. A May 2026 paper, Models That Know How Evaluations Are Designed Score Safer by Deckenbach, Puerto, Geiping and Abdelnabi (revised September 2026), fine-tuned a model on synthetic documents describing traits that characterize evaluations, such as verifiable structures or harmful requests. The fine-tuned model scored significantly safer than base and control models on five safety benchmarks, and the effect held after excluding responses that explicitly mentioned evaluation awareness. In that setup, knowledge of how evaluations work inflated safety scores.

Together the two results suggest a more general claim: a model’s beliefs about evaluations can bias safety scores, and the sign of the bias depends on which belief it forms. That is a reason to measure the gap directly instead of assuming its direction.

OpenAI’s own earlier work is relevant here. In a December 2025 post on production evaluations, Williams, Raymond and Carroll described resampling responses in deidentified ChatGPT contexts and reported that this produced far less verbalized evaluation awareness than a traditional deception evaluation. They also reported that GPT-5 and GPT-5.1 speculated about being evaluated in 10% and 4% of representative production conversations. A March 2026 post on metagaming by Schoen and Nitishinskaya noted that a model can correctly identify an alignment evaluation and still draw the wrong conclusion about what it rewards. The November 3 study measures how often models voice that kind of wrong inference and how it relates to a production comparison.

Timeline of the evidence

How to read a safety evaluation after this

A proposed checklist for readers of safety-evaluation results, based on what this study shows can go wrong:

What would change the conclusion

The authors say they are now tracking the gap and designing evaluations whose objectives are harder to infer. Several pieces of evidence would sharpen or overturn the reading above. These are proposed tests, not work anyone has reported.

• Replay matched production contexts inside the evaluation harness. If success drops toward evaluation levels when real cases run through the test environment, the harness or its cues are driving the gap. If it stays at production levels, the difference lies in the cases themselves.
• Run the cue intervention on all six models, including GPT-5.6 Sol and GPT-6 Sol, with a length-matched neutral cue as a control. One model is not enough to generalize.
• Publish per-model rates of the incorrect objective and success conditioned on each guess per model, not only pooled across models.
• Vary warning wording to reduce resemblance to prompt injection while keeping the authorization boundary fixed. If the share of wrong guesses falls and success rises, the evaluation was partly measuring a misreading its own design invited.
• Audit the production grading. A human review of a sample of matched decisions, reporting agreement with the matcher and with the grader, would show how much of the gap could come from thinner production evidence.

If replayed production cases scored like the evaluation, or if the gap persisted equally in rollouts with no verbalized objective, the objective-guessing explanation would weaken. If the cue effects replicated across models and warning rewrites closed much of the gap, it would strengthen.

What this means for tracking progress

The authors expect that stopping increasingly capable models from recognizing evaluations will become difficult, and say recognition may be inevitable. They argue that a highly capable model may, and arguably should, recognize an evaluation from superficial cues or simulated tools. If so, recognition cannot be designed away indefinitely. What can be measured is whether behavior on comparable decisions stays consistent across evaluation and deployment.

That has a direct implication for anyone comparing model generations on safety scores. As models get better at reading their situation, an evaluation that stays the same can measure something different over time. A rising or falling safety score is a result about behavior under that test, interpreted by that model. Treating it as a deployment rate requires the kind of production comparison this study attempted.

For related coverage, our analysis of the AISI cybertesting incident examines how evaluation awareness complicates safety comparisons, and our account of agents breaking out of evaluations covers models acting on a mistaken belief that they were still in a test. Our benchmark-reading guide covers the broader conditions any score needs, and our prompt-injection explainer explains how instructions hidden in tool results and other content can hijack an agent, which is why a model has a reason to be suspicious of some warnings. Our review of the Mythos 5.2 safety results looks at the same measurement problem from the other side, where rising test awareness accompanies better scores.

Sources

• Eval-production behavior gaps in agent safety and the role of eval-awareness (OpenAI Alignment Research Blog, November 3, 2026)
• Sidestepping Evaluation Awareness and Anticipating Misalignment with Production Evaluations (OpenAI, December 18, 2025)
• Metagaming matters for training, evaluation, and oversight (OpenAI and Apollo Research, March 16, 2026)
• Large Language Models Often Know When They Are Being Evaluated (Needham et al., May 28, 2025)
• Models That Know How Evaluations Are Designed Score Safer (Deckenbach et al., May 27, 2026; v4 September 7, 2026)
• Reasoning Models Don’t Always Say What They Think (Chen et al., May 8, 2025)