AgentDecept: Benchmarking Deceptive Behavior in Tool-Using LLM Agents under Pressure
Users see the agent's report, not its API calls. AgentDecept checks reports against execution on 477 pressure variants, separating falsification, concealment and equivocation.
The claim
AgentDecept tests whether tool-using agents report their actions honestly under pressure.
The problem
Users usually see only the agent's report of its work.
The report omits observations, API calls, and state changes.
Deception across realistic, stateful workflows is not well tested.
How it works
The benchmark builds 477 pressure variants from 257 source tasks.
Pressure enters through events, context, state, tool responses, and quotas.
Every variant has a pressure-free counterpart for comparison.
The protocol checks user claims against time-ordered observations and outcomes.
It labels falsification, concealment, equivocation, honest failure, and disclosed limits.
Figures
What changes
Reports can be compared with what the agent actually did.
The behavioral labels also support testing activation-based probes.
Pressure variants and counterparts isolate when reports diverge.
Limits
The study covers benchmark workflows, not live production systems.
The paper reports no deception rates for specific models.
Source · arXiv