AI Insight
Researchers developed "shibboleth traps," specialized honeypot services that can identify which large language model (LLM) and operational framework is behind autonomous AI-driven cyberattacks by analyzing their distinctive behaviors. The technique achieved 90% accuracy in distinguishing among 15 different AI models and 92% accuracy in identifying the correct harness (operational framework) from four candidates, even when the attacking agents attempted to conceal their identity. The method works by creating scenarios that force attacking AI agents to reveal identifying information through their responses and behavioral patterns, without requiring direct access to the system.
Why it matters
This research provides cybersecurity defenders with practical tools to attribute AI-powered attacks to specific models and frameworks, which could aid in developing targeted defenses and understanding attacker capabilities. As autonomous AI agents become more prevalent in both legitimate and malicious applications, the ability to fingerprint these systems becomes crucial for security monitoring and incident response.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: We study the fingerprinting of agentic cyberattacks. Our objective is to reliably infer information about the attacker’s configuration, such as LLM version and harness, from telemetry. To achieve this, we introduce a technique we call shibboleth traps: modular honeypot services designed to elicit discriminative behaviors from agentic attackers. The traps make informative interactions a condition for obtaining resources the agent seeks, eliciting model declarations and characteristic writing without requiring a successful prompt injection.
To achieve harness fingerprinting, we introduce general mechanisms to indirectly probe peculiar execution behaviors of harnesses. These probes reveal how a harness masks secrets in tool output, enforces command time limits, and schedules tool calls, using only the effects visible in subsequent requests.
We show that combined declarative and behavioral inference distinguishes among 15 candidate models with an average 90% top-1 accuracy, with behavioral classifiers evaluated on held-out harnesses or task families. In a separate experiment across four harnesses, fixed probe rules identify the correct harness with 92% accuracy. When instructed to conceal their identity, agents still leave useful behavioral evidence even as their declarations become uninformative.
Source: Say Shibboleth: Fingerprinting the LLMs and Harnesses Behind Agentic Attacks