As X, Do Y: How Persona and Task Combine in Instruction-Tuned LLMs
The paper discusses how persona and task can be combined in instruction-tuned language models. It highlights the limitations of compressing role prompts into a single cached residual vector. The findings suggest that local additivity in the residual stream does not imply prompt compressibility.
- ▪Role prompts of the form As X, do Y allow for a linear decomposition at the prompt-to-answer transition.
- ▪The study shows that persona-conditioned behavior relies on a distributed prompt/KV mechanism.
- ▪Injecting cached predictions into a baseline prompt does not yield the desired clean long-persona target.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Computation and Language arXiv:2605.23147 (cs) [Submitted on 22 May 2026] Title:As X, Do Y: How Persona and Task Combine in Instruction-Tuned LLMs Authors:Eric Xu View a PDF of the paper titled As X, Do Y: How Persona and Task Combine in Instruction-Tuned LLMs, by Eric Xu View PDF HTML (experimental) Abstract:Role prompts of the form As X, do Y admit a clean linear decomposition at one specific site in the residual stream: the prompt-to-answer transition -- the last prompt token together with the first two generated tokens -- in an early/mid layer band. There, persona and task contribute through partially orthogonal additive directions.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
