Incentivizing Self-Learning in Large Language Models via Reinforcement Learning
RL that rewards textbook study raised proof accuracy from 51.5% to 79.5%. Gains also hold on out-of-domain subjects.
The claim
RL rewards let a model study a textbook on its own and solve harder proofs.
The problem
Knowledge-intensive tasks need new knowledge, not just retrieval.
Fixed retrieval pipelines hand the model a preassembled context.
Problem solving in unfamiliar domains needs study and reasoning together.
How it works
The task is textbook-based mathematical proof writing with graded exercises.
Reference solutions are withheld, so the model must study first.
A BookNavigator interface lets the model browse the long-form textbook.
Reinforcement learning rewards each fine-grained step of the study process.
The model learns to retrieve, read and apply the right principles.
Figures
What changes
Proof accuracy rises from 51.5% to 79.5% on Intern-S2-Preview-35B.
The method beats textbook-free and fixed-retrieval RL baselines on the same task.
Long-context benchmarks improve by 1.7 and 2.3 points over the base model.
Limits
Results come from a single 35B model working on textbook subjects.
The task family is mathematical proofs drawn from textbooks, not open problems.
The paper reports no results outside textbook study and long-context settings.
Source · arXiv