Uncovering Shortcut Amplification in Video RLVR through the Lens of Perceptual Uncertainty Collapse
Video RLVR gains partly ride on shortcuts: perceptual uncertainty collapses and amplifies them. Matters for anyone shipping RL-trained VLMs.
The explainer · 3-minute read · 263 words
The claim
Video RLVR can sharpen answers while the model leans less on the actual video.
The problem
RLVR boosts video reasoning in large multimodal models.
Outcome-level optimization can also amplify shortcuts.
Models increasingly favor text-prior answers that the video does not support.
How it works
The team tracks predictive uncertainty with and without the video.
RL training sharpens the answer distribution over time.
But uncertainty reduction from visual evidence keeps shrinking.
Gradient analysis shows updates dominated by text-prior compression.
BSPO then reweights the RL advantage with two distributional signals.
Figures
What changes
One signal finds reasoning-sensitive tokens from correct and incorrect rollouts.
The other penalizes predictions that stay unchanged when relevant visual evidence weakens.
The result keeps RLVR's task gains while grounding updates in vision.
Limits
The analysis covers video RLVR training runs, not other modalities.
BSPO adds perturbation steps, so training may cost more.
Visual faithfulness improves on the paper's benchmarks; independent replications are pending.
Source · arXiv wire