Small coding agents improve with code graphs and embedding search
Structured tools can double accuracy for certain small model sizes.
Small coding agents perform better when given embedding search and code-graph tools. These tools help models understand a repository structure.
Researchers tested ten models on SWE-bench Verified. These included six Qwen3 sizes from 0.6B to 32B and three Gemma 4 sizes. They used a depth-balanced 160-task split and all 500 tasks.
Embedding search and code-graph tools were compared against grep only. Structured tools double the accuracy for Qwen3-8B and Gemma 4 E4B. The gain peaks at 17-21 points for models sized 1.7-4B.
The gain falls to 4-5 points for the 32B model. Every extra hop multiplies the odds of correct localization by 0.50-0.67. Agents chain expansions in only 12-14% of episodes.
Linear probes show agents carry a one-hop, largely lexical picture of their repository. This picture exists in weights and in use. The AUC for direct call edges is 0.74.
Key facts
- Structured tools double accuracy for Qwen3-8B and Gemma 4 E4B.
- The gain for 32B models falls to 4-5 points.
- Agents chain expansions in only 12-14% of episodes.
- Every extra hop multiplies the odds of correct localization by 0.50-0.67.