AI Insight
Researchers have developed TaLK, a new method for compressing text-attributed graph datasets that preserves both textual meaning and structural relationships. The approach combines language models with graph-aware neural tangent kernels to create synthetic datasets that are 99% smaller than originals while maintaining 97% of training performance. This method avoids the computational expense of repeatedly training combined language model and graph neural network systems during the compression process.
Why it matters
This technique could significantly reduce the computational costs and time required to train AI systems on complex graph data that includes text, such as social networks, citation networks, or knowledge graphs. The ability to distill large datasets into much smaller synthetic versions while preserving performance could make advanced graph learning more accessible and environmentally sustainable.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Text-attributed graphs (TAGs) are widely used in many real-world domains, and learning on TAGs requires jointly modeling text semantics and graph structure. A standard approach for modeling TAGs is to combine a language model (LM) and a graph neural network (GNN), but joint training is computationally expensive and difficult to scale. Dataset distillation is a promising way to reduce training costs, but existing methods are not well suited to TAGs because they are typically designed for a single modality or still require repeatedly training expensive LM-GNN models on the full dataset during distillation. To address this, we propose TaLK, an effective dataset distillation method for TAGs that couples an LM with a graph-aware neural tangent kernel. This design enables efficient dataset distillation, avoiding repeated joint training on the full dataset while reflecting both textual and structural information for effective TAG learning. Experiments on multiple TAG benchmarks show that TaLK consistently outperforms existing baselines and achieves up to 97% of full-dataset performance with only 1% synthetic data.
Source: TaLK: Text-attributed Graph Dataset Distillation via Coupling Language Model with Graph-Aware Kernel