AI Insight
This study introduces "AI-assistability" as a new metric to measure how well AI coding assistants can generate correct code for different multi-agent frameworks. Testing ten frameworks on a database-to-semantic mapping task, researchers found that Agno achieved the highest AI-assistability score (0.55) while DSPy scored lowest (0.07), despite DSPy being the most declarative framework. The results demonstrate that alignment with existing programming conventions, rather than declarative design principles alone, is the primary factor determining whether AI assistants can effectively work with a framework.
Why it matters
As AI coding assistants become increasingly prevalent in software development, this research provides concrete guidance for framework designers on how to make their tools more compatible with AI-assisted development. The findings challenge assumptions about what makes code AI-friendly and could influence how future programming frameworks are designed.
Understand the Science
arXiv:2602.11198v2 Announce Type: replace-cross
Abstract: Multi-agent frameworks (MAFs) promise to simplify LLM-driven software development, yet no principled metric captures how well AI coding assistants can generate correct, framework-specific code. We introduce textit{AI-assistability} ($mathcal{AI}$), a composite metric that quantifies a framework’s amenability to AI-assisted development by combining structural alignment ($bar{sigma}$) with functional correctness (pass@1). To evaluate this metric in a controlled setting, we design DDL2PropBank, a novel benchmark task that maps relational database schemas to PropBank semantic rolesets, and implement identical agent logic across ten frameworks using the Agent-as-a-Tool pattern. Our results challenge the intuition that declarative framework design guarantees AI-assistability: Agno, with a single canonical pattern and convention-aligned API, achieves the highest $mathcal{AI}$ score (0.55), while DSPy — the most declarative framework by design — scores lowest (0.07), as its novel abstractions are insufficiently represented in AI training data. We find that convention alignment, not declarative design alone, is the primary driver of AI-assistability ($r = 0.576$ between $bar{sigma}$ and pass@1). All artifacts — DDL2PropBank, PropBank MCP server, and all implementations — are available at https://github.com/ahmeshaf/ddl2propbank