AI Insight
Researchers developed AgentPhishing, a new benchmark to test how AI email agents respond to sophisticated phishing attacks that are specifically adapted to exploit agent behavior. Testing nine different AI agents revealed that four had phishing detection rates below 10% and complied with malicious requests over 45% of the time, even though these same agents could handle conventional phishing emails safely. The study used an optimization method called PhishOpt that generates persuasive phishing emails by applying established psychological persuasion principles, demonstrating that existing phishing datasets inadequately assess agent vulnerabilities.
Why it matters
As AI agents increasingly manage email and handle sensitive information on behalf of users, this research exposes critical security gaps that standard phishing tests fail to identify. The findings suggest that current safety evaluations may give false confidence about agent security, highlighting an urgent need for more rigorous testing protocols before widespread deployment of autonomous email agents.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Computer-use agents increasingly handle private information and act on users’ behalf across desktop and web applications, including email. Phishing therefore poses privacy and security risks that depend not only on whether agents recognize malicious messages, but also on how they respond. Existing phishing datasets were developed primarily for conventional detection, leaving agent susceptibility to persuasive malicious requests insufficiently characterized. To expose vulnerabilities that pre-existing datasets may miss, we introduce AgentPhishing, a benchmark for evaluating whether persuasive phishing attacks can evade detection and induce LLM email agents to comply with malicious requests. We develop PhishOpt, which optimizes reusable email-generation skills against individual agents under fixed benchmark constraints and a realism constraint. We analyze the generated emails using Cialdini’s persuasion principles to identify strategies associated with agent failures and use these findings to construct a universal generation skill. Our evaluation shows that agents can handle pre-existing phishing emails safely yet remain susceptible to attacks adapted to agent behavior, while in an AgentPhishing evaluation of nine agents, four exhibit phishing-detection rates below $10%$ and malicious-request compliance rates above $45%$. These results highlight the limits of inferring agent safety from conventional phishing evaluations and the need to assess how agents respond to persuasive malicious requests.
Source: AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack