Accuracy is not compatibility: two Evo 2 checkpoints hinge on one FP8 matrix multiplication
One FP8 matrix multiplication decides whether Evo 2 checkpoints work on new GPUs. Fixing it lifts BRCA1 AUROC from 0.434 to 0.869.
The claim
One FP8 matrix multiplication decides whether Evo 2 checkpoints work on new GPUs.
The problem
FP8 formats fix how values are stored, not how products are summed.
Evo 2 documents a Hopper FP8 requirement for its checkpoints.
Both big checkpoints fail the developers' forward test on Blackwell.
How it works
The authors build an FP8 GEMM emulator called mmaemu.
It is bit-exact on captured projections from the 1B model.
The failure sits in the first projection of 21.
Exact sums there fail unless shrunk by 1.4%.
One projection left native makes the 40B fail.
Figures
What changes
Emulating that one projection lifts BRCA1 AUROC from 0.434 to 0.869 on that GPU.
Weights stay unchanged; only the arithmetic emulation path itself changes.
On H100 the 40B already reaches 0.868 AUROC on the same task.
Limits
Only one model family is tested, not every FP8 workload in practice.
Training origin of the checkpoints was not checked by the authors.
Releases should specify reference arithmetic, the authors conclude firmly.
Source · arXiv