Reward-Adaptive Co-Evolution of Policy and Rubric
Static rubrics lose power as the policy improves. RACE routes rollout groups and switches phases by reward informativeness, beating fixed alternation on all eight tasks.
The claim
Rubrics and policies should co-evolve on their own schedule, not a fixed one.
The problem
Static rubrics lose discriminative power as the policy improves.
Fixed schedules ignore which rollout groups still teach.
How it works
Group routing labels rollouts as informative, collapsed, or failed.
Only informative groups update the policy.
Phase switching starts rubric training when rewards saturate.
Rubric phases train on cached pairs with a frozen judge.
Proportional control adapts both thresholds online.
Figures
What changes
RACE beats the strongest fixed-alternation baseline on all eight tasks.
Average performance is the best under the unified protocol.
Rubric discrimination improves alongside task scores.
Limits
The evaluation covers eight tasks under one protocol.
The rubric phase still relies on a frozen judge.
The abstract gives no single overall score for the rubric gain.
Source · arXiv