The Real Cost of Context: Predicting Compute Time and Energy Consumption in LLM Prefill and Decode Phases
Profiles prefill and decode time and energy; prefill scales with input length.
Explainer · on the wire
The explainer for this story is still on the wire.
The claim, the mechanism, and the figures land here once generated.
Source · arXiv