AI Insight
Researchers have provided the first mathematical proof explaining why transformers learn in two distinct stages during training, such as when GPT-2 first learns syntactic correctness before achieving semantic accuracy. The analysis focuses on how transformers process disentangled feature types (like syntax and semantics in language, or primary and secondary structures in proteins) through normalized ReLU self-attention mechanisms. The study demonstrates that this two-stage learning phenomenon is connected to the spectral properties of attention weights in the model.
Why it matters
This theoretical framework helps explain empirically observed training behaviors in large language models, potentially enabling more efficient training strategies and better model architectures. Understanding these dynamics could lead to improved approaches for teaching AI systems complex hierarchical concepts that build upon foundational features.
Understand the Science
-cross
Abstract: Transformers may exhibit two-stage training dynamics during the real-world training process. For instance, when training GPT-2 on the Counterfact dataset, the answers progress from syntactically incorrect to syntactically correct to semantically correct. However, existing theoretical analyses hardly account for this feature-level two-stage phenomenon, which could be conceptually attributed to disentangled two-type features like syntax and semantics. In this paper, we theoretically demonstrate how the two-stage training dynamics potentially occur in transformers. Specifically, we analyze the feature learning dynamics induced by the aforementioned disentangled two-type feature structure, grounding our analysis in a simplified yet illustrative setting that comprises normalized ReLU self-attention and structured data. Such disentanglement of feature structure is general in practice, e.g., natural languages contain syntax and semantics, and proteins contain primary and secondary structures. To our best knowledge, this is the first rigorous result regarding a feature-level two-stage optimization process in transformers within this theoretical framework. A corollary further indicates that such a two-stage process is closely related to the spectral properties of attention weights.