AI Insight
The authors introduce PAR (Protein Autoregressive modeling), a multi-scale autoregressive framework that generates protein backbone structures through a coarse-to-fine prediction process, progressing from global topology to fine structural details across hierarchical scales. The system combines multi-scale downsampling, an autoregressive transformer for conditional embedding generation, and a flow-based decoder for backbone atom prediction. To address exposure bias, a known limitation of autoregressive models arising from discrepancies between training and inference, the authors implement noisy context learning and scheduled sampling, improving generation robustness and quality.
Why it matters
Accurate and controllable protein backbone generation has direct applications in drug discovery, enzyme engineering, and the design of novel proteins with desired structural properties. PAR's zero-shot generalization to conditional generation tasks and motif scaffolding without fine-tuning reduces computational costs and broadens practical accessibility.
arXiv:2602.04883v2 Announce Type: replace-cross
Abstract: We present protein autoregressive modeling (PAR), the first multi-scale autoregressive framework for protein backbone generation via coarse-to-fine next-scale prediction. Using the hierarchical nature of proteins, PAR generates structures that mimic sculpting a statue, forming a coarse topology and refining structural details over scales. To achieve this, PAR consists of three key components: (i) multi-scale downsampling operations that represent protein structures across multiple scales during training; (ii) an autoregressive transformer that encodes multi-scale information and produces conditional embeddings to guide structure generation; (iii) a flow-based backbone decoder that generates backbone atoms conditioned on these embeddings. Moreover, autoregressive models suffer from exposure bias, caused by the training and the generation procedure mismatch, and substantially degrades structure generation quality. We effectively alleviate this issue by adopting noisy context learning and scheduled sampling, enabling robust backbone generation. Notably, PAR exhibits strong zero-shot generalization, supporting flexible human-prompted conditional generation and motif scaffolding without requiring fine-tuning. On the unconditional generation benchmark, PAR effectively learns protein distributions and produces backbones of high design quality, and exhibits favorable scaling behavior. Together, these properties establish PAR as a promising framework for protein structure generation.
Source: Protein Autoregressive Modeling via Multiscale Structure Generation