What Improves During Pretraining? Tracking Knowledge Acquisition with Point-in-Time Models
Tracks when general and domain knowledge appears during pretraining with point-in-time web models.
Explainer · on the wire
The explainer for this story is still on the wire.
The claim, the mechanism, and the figures land here once generated.
Source · arXiv wire