AI Insight
This paper introduces Agent-as-a-Router, a framework that dynamically routes coding tasks to the most suitable Large Language Model from multiple available options. Unlike existing static routers, this approach learns from execution feedback through a Context-Action-Feedback loop, improving routing decisions over time. Testing on CodeRouterBench with approximately 10,000 tasks across 8 frontier LLMs showed that the proposed ACRouter system achieves lower cumulative regret compared to static routing methods and generalizes well to new task types.
Why it matters
As organizations increasingly use multiple AI models from different providers, efficient task routing becomes essential for optimizing both performance and cost. This framework demonstrates that adaptive routing systems can outperform static classification approaches by learning from real execution outcomes, potentially reducing computational waste and improving user experience in production environments.
Understand the Science
arXiv:2606.22902v2 Announce Type: replace
Abstract: Real-world users typically have access to multiple Large Language Models (LLMs) from different providers, and these LLMs often excel at distinct domains, yet none dominate all. Consequently, routing each task to the most suitable model becomes critical for both performance and cost. Existing routers treat this as a static, one-off classification problem. However, we identify the performance bottleneck for these routers as information deficit: simply augmenting a vanilla LLM router with performance statistics at the task-dimension level yields a 15.3% relative gain, surpassing a heuristic router built on the same dimension-level priors. Motivated by this finding, we propose Agent-as-a-Router, a framework that formalizes routing as a C-A-F loop (Context->Action->Feedback->Context). It closes the information gap by accumulating execution-grounded experience during deployment. We instantiate this framework as ACRouter, composed of an Orchestrator, a Verifier, a Memory module, and introduce CodeRouterBench, an evaluation environment comprising ~10K task instances with verified scores from 8 frontier LLMs, enabling regret-based router comparison on streaming tasks. Experiments show that ACRouter achieves the lowest cumulative regret on in-distribution tasks and generalizes to out-of-distribution agentic-programming tasks, demonstrating that our routing framework actively closes the information gap. Codes and benchmarks are released at https://github.com/LanceZPF/agent-as-a-router.
Source: Agent-as-a-Router: Agentic Model Routing for Coding Tasks