Prompt eval cues predicted refusal shifts across 32k LLM rollouts
A recent study analyzed 32,170 rollouts of language models to understand how prompts influence their refusal rates. It found that prompts indicating an evaluation context significantly affected the likelihood of models refusing harmful requests. This research highlights the importance of prompt design in AI safety assessments and the potential for models to behave differently during evaluations compared to real-world interactions.
- ▪The study involved 32,170 rollouts of various language models.
- ▪Prompts indicating an evaluation context led to significant changes in refusal rates for harmful requests.
- ▪The findings suggest that prompt design is crucial for accurate AI safety assessments.
Opening excerpt (first ~120 words) tap to expand
The Prompt Is the Tell, Not the Reasoning Trace — Eval AwarenessRatnaditya11 min read·2 hours ago--ListenShareThe Prompt Is the Tell, Not the Reasoning TraceAcross 32,170 rollouts, eval-related prompt cues predicted refusal shifts more reliably than verbalized eval-awareness in model traces.If a system prompt tells Claude Opus 4.7 that its response is about to be reviewed by safety researchers, it becomes about 34 percentage points less likely to refuse harmful requests.If the same prompt is given to qwen3–235B, it refuses 22 percentage points more. In the prompt subset driving this comparison, neither model mentions the evaluation in its reasoning trace.That was the V1 finding, written up in an earlier post on this work.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Medium.
Discussion
0 commentsMore from Medium
