AI Insight
This paper introduces Variational Model Merging, a new Bayesian method for efficiently estimating Pareto fronts in multitask machine learning model finetuning. The researchers demonstrate that using more flexible probability distributions (posteriors) when merging models mathematically guarantees better estimates of optimal task-mixing strategies compared to simpler approaches. The theoretical framework shows that existing model-merging techniques are special cases of their approach, and they validate their findings through experiments on vision and language transformer models.
Why it matters
This work reduces the computational cost of finding optimal strategies for training AI models on multiple tasks simultaneously, which is increasingly important as models become larger and more expensive to train. The method provides a principled mathematical framework for improving model merging techniques, potentially making multitask learning more accessible and efficient across various AI applications.
Understand the Science
arXiv:2412.08147v2 Announce Type: replace-cross
Abstract: Pareto fronts are useful to find good task-mixing strategies for multitask finetuning, but they are also costly to compute. To reduce costs, recent works have used existing model merging methods to help train cheap surrogate models to estimate the Pareto fronts. However, no work has yet considered designing new model-merging methods to directly, and provably, improve the quality of Pareto fronts. Here, we fill this gap by proposing a new Bayesian approach called Variational Model Merging. In this approach, existing model-merging methods are obtained as special cases of “posterior-merging” when Gaussian posteriors are used and new model-merging strategies can be derived by using non-Gaussian posteriors. Our main theoretical result is to show that more flexible posteriors necessarily yield better estimates of Pareto fronts. For instance, a Pareto front estimate obtained by merging full-Gaussian posteriors is expected to be better than that obtained by using isotropic Gaussian posteriors. We validate the theory through extensive empirical results on vision and language transformers where better Gaussian families consistently yields better or comparable Pareto fronts. Our work is a rare instance where Bayesian ideas are used to improve Pareto analysis.
Source: Variational Model Merging for Pareto Front Estimation in Multitask Finetuning