Refusal Lives Downstream of Persona in Chat Models
Researchers have found that refusal and persona traits in chat models are interconnected, with a compliant persona suppressing refusal. This discovery was made by studying instruction-tuned chat models, including Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct. The study's findings have implications for the development of more effective and compliant chat models.
- ▪A compliant persona direction and a refusal direction were extracted from the models and intervened on.
- ▪Compliant persona steering suppresses refusal, with the refusal rate falling from 97% to 2% in Llama.
- ▪Refusal is gated at the late-layer expression stage, downstream of where it is computed.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Artificial Intelligence arXiv:2606.26161 (cs) [Submitted on 24 Jun 2026] Title:Refusal Lives Downstream of Persona in Chat Models Authors:Viola Zhong, Qirui Li View a PDF of the paper titled Refusal Lives Downstream of Persona in Chat Models, by Viola Zhong and 1 other authors View PDF HTML (experimental) Abstract:Linear directions in activation space have been identified for both refusal and persona traits in instruction-tuned chat models, but the two have been studied as separate mechanisms. We show they interact: a compliant persona gates refusal. In Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct, we extract a compliant model-persona direction and a refusal direction and intervene on both.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv.org.
Discussion
0 commentsMore from arXiv.org
