AI Insight
TreeHop is a new framework for multi-hop question answering in retrieval-augmented generation (RAG) systems that operates directly in embedding space without requiring large language models for query rewriting. The method updates query embeddings by fusing semantic information from previous queries and retrieved documents, replacing the traditional multi-stage retrieval cycle with a streamlined embedding-based loop. Experimental results demonstrate that TreeHop achieves comparable performance to existing advanced RAG methods while using only 2.2-29.4% of the parameters and reducing query latency by 92.8-97.8%.
Why it matters
This approach significantly reduces computational costs and latency for complex question-answering systems, making RAG technology more accessible for deployment in resource-constrained environments or applications requiring real-time responses. The efficiency gains could enable broader adoption of sophisticated information retrieval systems in practical applications.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
-cross
Abstract: Retrieval-augmented generation (RAG) systems face significant challenges in multi-hop question answering (MHQA), where complex queries require synthesizing information across multiple document chunks. Existing approaches typically rely on iterative LLM-based query rewriting and routing, resulting in high computational costs due to repeated LLM invocations and multi-stage processes. To address these limitations, we propose TreeHop, an embedding-level framework without the need for LLMs in query refinement. TreeHop dynamically updates query embeddings by fusing semantic information from prior queries and retrieved documents, enabling iterative retrieval through embedding-space operations alone. This method replaces the traditional “Retrieve-Rewrite-Vectorize-Retrieve” cycle with a streamlined “Retrieve-Embed-Retrieve” loop, significantly reducing computational overhead. Moreover, a rule-based stopping criterion is introduced to further prune redundant retrievals, balancing efficiency and recall rate. Experimental results show that TreeHop rivals advanced RAG methods across four open-domain MHQA datasets, achieving comparable performance with only 2.2%-29.4% of the parameter size of concurrent solutions and reducing the query latency by 92.8%-97.8%. This makes TreeHop a faster and more cost-effective solution for low-resource or latency-sensitive deployment. For reproducibility purposes, codes and data are available here: https://github.com/allen-li1231/TreeHop-RAG