Emergence of the transformer architecture as the score of entropic laws of random measures
Derives the transformer as the score of maximum-entropy laws over sentences.
Explainer · on the wire
The explainer for this story is still on the wire.
The claim, the mechanism, and the figures land here once generated.
Source · arXiv