AI Insight
Researchers developed COPEX, a benchmark to test how vulnerable large language models are to adversarial attacks when they act as clients in the Model Context Protocol system. Testing nine different models across 3,375 trials and 25 attack types, they found an average attack success rate of 64.4%, with attacks exploiting four different entry points: the model itself, the client layer, the server/tool layer, and the transport layer. Implementing combined input and context scanning defenses reduced attack success by approximately 50% in a subset of scenarios.
Why it matters
As AI models increasingly control tools and external systems, understanding their vulnerability to manipulation through different protocol layers is critical for security. The benchmark provides a systematic way to evaluate model robustness independent of other system protections, which could inform the development of more secure AI agent architectures.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
-cross
Abstract: Large language models increasingly mediate tool use in Model Context Protocol (MCP) systems, where adversarial influence may enter through user instructions, tool schemas, tool outputs, or protocol messages. Existing benchmarks often evaluate deployed agents, conflating model susceptibility with guardrails, orchestration, and general task capability. We introduce COPEX (COntext Provider EXploitation), a controlled benchmark that isolates the model as an MCP client by fixing the surrounding agent stack and varying only the tool-selecting model. COPEX covers 25 attack types instantiated as 125 scenarios across four entry surfaces: model/agent, client, server/tool, and transport. Across nine models and 3,375 trials, the mean attack success rate is 64.4%, with surface-level means ranging from 58.3% to 71.4%. Some client- and transport-level attacks succeed partly outside the model’s observation or control, separating system exposure from model susceptibility. Combined input and context scanning reduces mean attack success by 49.6% on an eight-attack defense subset relative to the undefended setting. The benchmark is available at https://github.com/inspire-center/copex.
Source: COPEX: Benchmarking LLM Robustness to Adversarial Context Across Model Context Protocol (MCP) Layers