AI Insight
BridgeAfford is a new framework for teaching AI systems to identify functional regions on 3D objects based on language descriptions of intended interactions. The method addresses a key limitation in existing approaches by reducing the semantic gap between query features (representing interaction intents) and target-region features through two mechanisms: Interaction Region Alignment, which aligns geometrically similar interaction regions across different objects, and Visual Evidence Aggregation, which uses 2D affordance models to refine attention toward correct regions. Testing on LASO and PIAD datasets shows improved performance, particularly for objects not seen during training.
Why it matters
This research could enhance robotic manipulation systems and assistive technologies by enabling better generalization to new objects in real-world environments. The ability to accurately identify where and how to interact with unfamiliar objects based on natural language instructions has applications in household robotics, automated warehouses, and human-robot collaboration scenarios.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Language-guided 3D affordance grounding aims to localize functional regions on 3D objects according to interaction intents. However, existing methods exhibit poor generalization to unseen objects. One challenge is the query-target gap: a semantic discrepancy between query features encoding interaction intents and target-region features, leading to inaccurate grounding. End-to-end training favors query representations tailored to seen objects, limiting their compatibility with shifted target-region features on unseen objects. To address this issue, we propose BridgeAfford, a framework that reduces the query-target semantic gap from two complementary perspectives. First, Interaction Region Alignment (IRA) promotes cross-object alignment among geometrically similar interaction regions during training. This enables structures on unseen objects that resemble previously observed ones to produce representations similar to those learned during training, facilitating target-region recognition. Second, Visual Evidence Aggregation (VEA) leverages a 2D affordance model to refine the attention distribution and guide the query toward the correct region, yielding more target-aware queries. Extensive experiments on the LASO and PIAD datasets demonstrate that BridgeAfford outperforms existing methods, particularly on unseen objects, highlighting its robust generalization ability.
Source: BridgeAfford: Bridging Query-Target Gap for Generalizable 3D Affordance Grounding