From Interaction to Capability: Task Advancement as State and Supervision for GUI Agents
GUI agents forget which past actions actually worked. Task Advancement Memory keeps verified progress and lifts MobileWorld success from 12.0% to 28.2%.
The claim
Keeping verified task progress as state raises GUI agent success from 12.0% to 28.2%.
The problem
GUI memory should preserve the results of past actions and reusable operations.
Long-horizon workflows spread useful operations between many failed attempts across several applications.
Simple compression of history mixes reusable knowledge with dead ends from failed attempts.
How it works
Task Advancement State records how each verified result changes the remaining goals.
After each task, advancement supervision selects interactions that produced verified results.
Learned operations keep inputs, expected outcomes, and verification conditions together.
Locally successful operations survive from failed tasks as well as successful ones.
The agent then reuses those learned operations across other applications later.
Figures
What changes
Advancement state alone raises task success from 12.0% to 28.2%.
Full TAM gains 17.6% over the strongest evaluated baseline.
Lifecycle tokens per solved task fall by 38.7%.
Limits
Reported results cover the MobileWorld GUI-only tasks and the SPA-Bench benchmark suite.
The gain depends on tasks whose progress can be verified.
The abstract reports no cross-device or cross-platform performance figures.
Source · arXiv