VeriCUA: Verification as Agentic Exploration for Computer-Using Agents
Verifying computer-use agents needs evidence, not one screenshot. VeriCUA explores the interface, builds a rubric, and mixes GUI and code checks under evidence rules.
The claim
Verifying a computer-use agent means gathering evidence, not one screenshot.
The problem
Vision-language judges see only the supplied observations.
Generated programs read only the accessible state.
Both views can miss decisive evidence.
How it works
The verifier inspects the available context first.
It builds a task rubric from that context.
It selects GUI checks or code checks for open requirements.
Findings combine under explicit evidence rules.
The same procedure serves online and offline settings.
Figures
What changes
Agreement with reference labels beats the compared baselines.
The largest offline gains appear on hard cases.
False completion claims are rejected more often online.
Limits
The paper reports relative gains, not a single score.
The verifier still needs access to GUI or code state.
The paper reports no error bar for these gains.
Source · arXiv