Your LR Schedule May Not Be Principled: Effective Learning Rate Scheduling in Weight-Normalized Training
Shows weight-normalization gains come from the effective learning rate trajectory.
Explainer · on the wire
The explainer for this story is still on the wire.
The claim, the mechanism, and the figures land here once generated.
Source · arXiv