AI Insight
This paper introduces DualSFT, a unified algorithm that simultaneously selects both which parameters to update and which training data to use when fine-tuning large language models. The researchers show that parameter importance and data utility can be derived from the same mathematical framework as row and column aggregations of a gradient interaction matrix, eliminating the need for separate selection mechanisms. Testing on 3-9 billion parameter models demonstrates that this joint approach achieves better performance-efficiency trade-offs compared to applying parameter and data selection sequentially.
Why it matters
Fine-tuning large language models is computationally expensive, and this method offers a more efficient way to reduce costs by coordinating parameter and data selection through a single scoring system. The approach could make customizing LLMs more accessible and resource-efficient for organizations with limited computational budgets.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: In Large Language Model (LLM) fine-tuning, parameter and data selection are common strategies for reducing fine-tuning cost, yet they are typically driven by separate scoring mechanisms. When a parameter mask and data subset jointly determine restricted fine-tuning, this separation incurs redundant overhead and makes coordinated selection difficult. We cast parameter and data selection as two bilevel selection problems under a common validation objective and derive a shared local response-surrogate scoring rule. Under first- and second-order validation-improvement approximations, parameter importance and data utility emerge as column-wise and row-wise aggregations of a single gradient interaction matrix, yielding a closed-form row-column correspondence for co-extracting both signals. Building on this structure, we propose DualSFT (Dual-Selection Fine-Tuning), a one-shot dual-scoring algorithm that produces a parameter mask and data subset from shared gradient statistics. On 3B-9B LLMs, single-axis DualSFT variants strengthen target-task performance and stability-plasticity trade-offs within their comparison groups, while full DualSFT yields a more favorable joint-constrained trade-off than sequential hybrid baselines under matched budgets.
Source: One Algorithm, Two Goals: Dual Scoring for Parameter and Data Selection in LLM Fine-Tuning