Low-Bit GDN Recurrent-State Quantization: Global Gradient-aware Allocation and Pure-Write Sliding Windows
Linear attention stores historical information in a recurrent state of fixed size, providing a compact memory representation for long-context inference.
Explainer · on the wire
The explainer for this story is still on the wire.
The claim, the mechanism, and the figures land here once generated.
Source · arXiv