Scaling Laws for Tool-Use Agents via Controllable Environment Synthesis
The paper maps how synthetic data composition shapes tool-use agent generalization. Scaling tool and database diversity beats adding tasks as budgets grow.
The claim
The paper derives a scaling law for synthetic tool-use data, and diversity beats volume.
The problem
Tool-use agents learn from synthetically generated environments rather than live traffic.
How the composition of those environments shapes generalization was not measured.
Every team must still spend one fixed data budget across tasks, tools, and databases.
How it works
SynthAPI generates tool-use environments along three controllable axes.
The axes are tools, tasks, and the databases on which the tools operate.
The authors vary each axis across 458,752 generated tasks in total.
They train 849 separate models spanning four different parameter sizes.
A fitted scaling law predicts downstream generalization from scale and composition.
Figures
What changes
At larger data budgets, tool and database diversity beats adding tasks.
Increasing task complexity yields larger gains than repeating easier tasks.
Agents trained this way beat prior synthetic data work on four benchmarks.
Limits
The law covers environments produced by this specific pipeline.
Real production tool traffic may shift the trade-offs.
The paper reports no cost figures for generating the data.
Source · arXiv