There Is No Free Lunch in RLVR: Every pass@1 Gain Has a pass@k Price
RLVR raises pass@1 while the base model overtakes it at large sample budgets. A theorem settles this: the reversal is a price, and its exact rate is computed.
The claim
A theorem shows every RLVR pass@1 gain costs some pass@k success.
The problem
RLVR raises first-try success but lowers many-try success.
It is unclear whether this is a failure or a transient.
Without a theorem, the reversal cannot be judged.
How it works
RLVR is modeled as an exponential tilt over shared strategies.
Under exclusive specialization, each pass@1 update lowers some prompt's success.
A strictly positive price on prompts is the exact condition.
Crossovers are bounded by the sign variation of success shifts.
The same model also yields entropy collapse under one mechanism.
Figures
What changes
The crossover becomes a measurable price, not a bug.
Removing the crossover entirely costs the whole pass@1 gain.
A diagnostic reads the bound and price from any evaluation log.
Limits
The theorem assumes an exponential tilt over shared strategies.
It gives conditions and rates, not training recipes.
The paper studies language model training, not other domains.
Source · arXiv