AI Insight
This study reveals that neural networks applied to texture-driven visual domains rely on "low-frequency shortcuts," making predictions based primarily on a few low-frequency image components rather than more informative high-frequency details. The researchers demonstrate that removing these low-frequency components from training and test data can improve in-distribution accuracy by up to 10% and out-of-distribution accuracy by up to 40%. This shortcut learning phenomenon persists even in foundation models and is amplified by reduced image resolutions.
Why it matters
The findings have important implications for machine learning applications in texture-heavy domains like medical imaging, materials science, and remote sensing, where reliance on low-frequency shortcuts could lead to poor generalization. The work provides actionable strategies for improving model robustness without requiring computationally expensive large models.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
-cross
Abstract: Neural networks suffer from shortcut learning, where learned features generalize well to the training set but not to in-distribution (ID) or out-of-distribution (OOD) test sets. Existing studies are all based on a few standard benchmarks, which are shape-driven. Numerous application domains, however, are texture-driven. In this work, we present shortcut learning analysis for texture-driven domains and compare it with that of a standard benchmark. We show that texture-driven domains suffer from low-frequency shortcuts. They make the majority of their decisions based on a few low-frequency components (LFCs) with a skewed spectral behavior, despite that higher-frequency components (HFCs) have higher predictive power. Pruning LFCs from training and test sets mitigates the shortcut and provides a more balanced spectral behavior, improving the ID accuracy by up to 10% and OOD accuracy by up to 40% under algorithmic and real-world domain shifts. We show that general-purpose and domain-specific foundation models can also suffer from low-frequency shortcuts. While large models can mitigate the shortcuts, they incur a high computational cost and may result in a significantly lower accuracy than shortcut-pruned from-scratch trained small models. We show that reduced image resolutions amplify the degree of shortcuts; large frequency-transformation block sizes capture low-frequency shortcuts better than small block sizes; and, low-frequency shortcuts persist across different color spaces. Our findings provide valuable insights, which we hope will be useful for practitioners working on new, understudied domains.
Source: Low-Frequency Shortcuts in Texture-Driven Visual Learning