AI Insight
SAE-RNA introduces a sparse autoencoder (SAE) framework designed to interpret the internal representations of RNA language models, specifically applied to RiNALMo. The study investigates whether SAEs can decompose these representations into biologically meaningful features, mapping them to known human-level biological concepts such as RNA family identity and structural context. Rather than asserting definitive biological discoveries, the authors present this approach as a representation-level probing tool that characterizes how RNA language models internally organize biological information.
Why it matters
Understanding how RNA language models encode biological information is a critical step toward building more transparent and trustworthy AI tools for RNA biology, with potential implications for RNA structure prediction, functional annotation, and therapeutic RNA design.
arXiv:2510.02734v2 Announce Type: replace
Abstract: Deep learning, particularly with the advancement of Large Language Models, has transformed biomolecular modeling, with protein language models such as ESM inspiring emerging RNA language models such as RiNALMo. Recent work has begun applying sparse autoencoders (SAEs) to protein language model representations, exploring representation-level interpretability in biomolecular models. Here, we explore whether SAEs can provide interpretable feature decompositions of RNA language model representations, while also examining their limitations in this setting. We present SAE-RNA, interpretability model that analyzes RiNALMo representations and maps them to known human-level biological features. Rather than claiming definitive biological concept discovery, our study frames SAE-based analysis as a representation-level probe for characterizing how RNA language models organize biological information internally. More broadly, SAE-RNA provides a feature-level framework for comparing RNA groups and identifying sparse representation components associated with RNA family identity or structural context.
Source: SAE-RNA: A Sparse Autoencoder Model for Interpreting RNA Language Model Representations