Aller au contenu

Recherche

1 résultat(s) · #sparsityEffacer les filtres

1 résultat(s)

En vedetteIAApprentissage profondEN

Mixture of Experts and top-k routing

A mixture of experts hands each input to a few sub-networks chosen by a router. The parameter count grows with the number of experts, but the computation per token depends only on kk.

Avancé · 11 min de lecture ·