Genko 原稿Back to edition

Softmax Routing Dynamics in Mixture-of-Experts Models: Collapse and Recovery

Softmax routing in mixture-of-experts models can concentrate probability on a subset of experts, suppressing access to the others.

Explainer · on the wire

The explainer for this story is still on the wire.

The claim, the mechanism, and the figures land here once generated.

Source · arXiv

More from this edition