AI Insight
SuperAI is a new framework that combines capabilities from multiple AI foundation models with different architectures into a single host model. The method extracts task-relevant representations from different models, aligns them using orthogonal mathematical transformations, and trains a lightweight network to integrate these capabilities while preserving the host model's general abilities. Testing across three host models and 15 capability domains showed SuperAI outperforms existing cross-architecture fusion methods and maintains better general performance as more source models are added.
Why it matters
This approach could enable more efficient AI development by allowing researchers to combine specialized capabilities from different models without rebuilding from scratch. The framework addresses a key technical challenge in creating more versatile AI systems that can leverage the strengths of multiple existing models simultaneously.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
-cross
Abstract: Enabling the reuse and composition of knowledge and capabilities distributed across foundation models is an important step toward building scalable artificial intelligence (AI) systems. However, architectural differences and incompatible representations make it difficult to transfer and integrate these capabilities through a unified cross-model interface. To address this challenge, we propose SuperAI, a cross-architecture multi-model fusion framework that integrates complementary capabilities from multiple source models into a host model through orthogonal alignment of low-dimensional task-relevant representations. Specifically, we first extract task-relevant representations from the source models and the host model using shared anchor samples, and apply principal component analysis (PCA) to construct low-dimensional functional subspaces with the same dimensionality. Next, we use orthogonal Procrustes mappings to align the source and host coordinate systems, establishing cross-model correspondences while preserving the internal geometry of each source subspace. Finally, we train a lightweight residual multilayer perceptron (MLP) with joint supervision on task outputs and source-model increments, learning input-dependent residual increments that are injected into the host model for multi-model capability fusion. Experiments on three host models across 15 capability domains show that SuperAI achieves higher average domain scores than the evaluated cross-architecture fusion methods and better preserves general capabilities as the number of source models increases. Ablation studies further support the effectiveness of orthogonal alignment, nonlinear residual mapping, and joint supervision.
Source: SuperAI: Merging Capabilities Across Heterogeneous Foundation Models