Machine learning

[Google Scholar]

Notes:
Papers:

Mixture of experts

  • Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity [fedus:arxiv:2021]
  • Outrageously large neural networks: The sparsely-gated mixture-of-experts layer [shazeer:arxiv:2017]