Hallucination as Exploit: Evidence-Carrying Multimodal Agents
The paper discusses the concept of hallucination in multimodal agents, where false visual claims can lead to unauthorized actions. It introduces evidence-carrying multimodal agents (ECA) that utilize external evidence to validate actions proposed by model language. The study demonstrates that ECA significantly reduces the risk of unsafe actions compared to naive agents.
- ▪Multimodal agents can trigger actions based on false visual claims, leading to authorization failures.
- ▪Evidence-carrying multimodal agents (ECA) treat model text as inadmissible evidence and require external validation for actions.
- ▪The architecture of ECA reduces gate bypass from 15% to 1.3% through targeted hardening steps.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Artificial Intelligence arXiv:2605.19192 (cs) [Submitted on 18 May 2026] Title:Hallucination as Exploit: Evidence-Carrying Multimodal Agents Authors:Guijia Zhang, Hao Zheng, Harry Yang View a PDF of the paper titled Hallucination as Exploit: Evidence-Carrying Multimodal Agents, by Guijia Zhang and 2 other authors View PDF HTML (experimental) Abstract:Multimodal agents use screenshots, documents, and webpages to choose tool calls. When a false visual claim triggers a click, email, extraction, or transfer, hallucination becomes an authorization failure rather than an answer-quality error. We formalize this failure mode as hallucination-to-action conversion: an unsupported perceptual claim supplies the precondition that makes a privileged action appear permitted.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
