AI Insight
This paper introduces SLIFT, a framework that improves large language models by selectively learning from user feedback. The system categorizes feedback components into three types: fixes that should apply broadly, specifications that apply only in certain contexts, and null feedback with no clear improvement direction. SLIFT uses two specialized adapters—a Generalist that incorporates universal fixes into default behavior and a Specialist that handles context-specific refinements—achieving strong performance on benchmark tests.
Why it matters
This approach addresses a key challenge in making AI systems more responsive to user needs by distinguishing between feedback that should change general behavior versus context-specific adjustments. The framework could enable more efficient and appropriate learning from real-world user interactions, potentially improving AI assistants and other interactive systems without overgeneralizing from limited feedback.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: User feedback offers natural supervision for persistent LLM improvement, but a single message may support multiple behavioral changes with different scopes of generalization. We introduce SLIFT, a selective self-learning framework built on a task-relative view of user feedback. SLIFT decomposes each feedback message into atomic components and interprets each component relative to the original task as Fix, Spec, or Null: requirements for task validity, compatible condition-specific refinements, or content with no reliable positive update direction. To incorporate each change at the appropriate scope, SLIFT trains two complementary LoRA adapters on a shared frozen backbone: a Generalist that consolidates Fix requirements into default behavior through feedback-conditioned self-distillation, and a Specialist that observes only the task and Generalist response to supply residual guidance for applicable, unmet Spec refinements. Null components induce no positive update. Across backbones, SLIFT achieves strong performance on both MemoryBench and WildFB, with targeted analyses further examining its underlying mechanisms. We release our code at https://anonymous.4open.science/r/SLIFT.