En vedetteIAApprentissage profondEN
Mixture of Experts and top-k routing
A mixture of experts hands each input to a few sub-networks chosen by a router. The parameter count grows with the number of experts, but the computation per token depends only on .
Avancé · 11 min de lecture ·