AI Insight
This study introduces K-Bench, a new benchmark for evaluating whether large language models successfully "unlearn" or forget specific information when deployed as agents with tool-using capabilities. The researchers found that existing unlearning benchmarks like TOFU and MUSE, which only check if models refuse to answer questions, fail to detect information leakage through other channels such as chain-of-thought reasoning, tool calls, and observations. Testing twenty published unlearning methods revealed that none could demonstrably remove secrets stored in model weights, and agents leaked information in 22-86% of queries even when existing benchmarks reported no leakage.
Why it matters
This research exposes a critical security gap in current AI safety practices, showing that models certified as having "forgotten" sensitive information can still leak it when deployed as agents with access to tools and retrieval systems. The findings are particularly relevant for applications requiring data privacy compliance, such as handling personal information or proprietary data in AI-powered systems.
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Unlearning benchmarks such as TOFU and MUSE certify forgetting by reading the model’s final answer, where a model that refuses to answer already counts as having forgotten. We show that this model-level certificate does not transfer once the model is deployed as an agent. We introduce K-Bench, a benchmark that scores LLM unlearning under agentic deployment. K-Bench inspects all six channels a ReAct agent exposes, including its chain-of-thought (CoT), tool calls and tool observations, and elicited summary. A query counts as leaked if the secret appears in any of them. Each experiment places the secret in exactly one of the agent’s three sources (the weights, the prompt, or the retrieval store). The K-Score is computed separately for each source and credits forgetting only when the agent remains usable. Clearing the answer channel does not make the secret unrecoverable. On structured retrieval, the secret stays verbatim in the tool-observation channel and the aggregate leak rate is unchanged. When the secret lives in the prompt or the retrieval store, TOFU and MUSE report no leakage, while the deployed agent still leaks it on 22–86% of queries. When the secret is in the weights, none of the twenty evaluated published methods demonstrably removes it, and only an input-corruption intervention reaches selective forgetting under the evaluated observer. The top-ranked method changes across base models. A refusal-tuning method resists the evaluated extraction without verified knowledge removal.
Source: K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments