AI Insight
This paper introduces H+ Embedding, a retrieval system that uses context-dependent phrases as an intermediate unit between single-vector and token-level approaches. The method dynamically partitions text into meaningful phrases while preserving individual tokens when needed, achieving retrieval quality that exceeds global vector methods by 6.91 nDCG@10 points across 16 tasks while using 86.3% fewer document vectors than full token-level systems. This approach is particularly beneficial for terminology-intensive domains like medical and scientific literature where multi-word entities and abbreviations are critical.
Why it matters
This work addresses a key efficiency-quality tradeoff in information retrieval systems, particularly for specialized domains requiring precise terminology matching. By reducing storage and computational costs while maintaining high accuracy, it could make advanced retrieval systems more practical for medical databases, scientific literature search, and other applications where both precision and scalability are essential.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Terminology-intensive retrieval, especially in medical settings, depends on preserving multi-word entities, abbreviations, numerical constraints, and compositional concepts. However, existing representations lie at two extremes: single-vector retrievers often over-compress local relevance signals, while token-level late interaction retains every tokenizer subword at substantial indexing, storage, and scoring cost. This mismatch raises a natural question: can context-dependent phrases provide a useful retrieval unit between global vectors and tokens? We introduce H+ Embedding, a unified multi-granularity retriever that predicts variable-length phrase partitions, preserves uncovered tokens as singletons, and applies importance-guided unit selection with weighted MaxSim interaction. Across 16 scientific, medical, and bilingual tasks, its phrase retrieval branch exceeds the global retrieval branch by 6.91 macro nDCG@10. It also nearly matches Token while using 13.7% fewer document vectors and outperforms content-independent grouping rules under moderate vector budgets. Context-dependent phrase interaction therefore provides an intermediate quality-cost point between global compression and token-level interaction for practical retrieval systems.
Source: H+ Embedding: Harmonizing Global and Token-Level Retrieval with Context-Dependent Phrases