 |
||
|
||
Accueil > Forum TERATEC > Ateliers > Atelier 8
Atelier 08 - de 14h00 à 17h30
IA et HPC dans l'industrie
Présidé par Cristel Saudemont, France Directrice, Supercomputing & AI , Enseignement supérieur et Recherche et Frédéric Parienté, senior manager in the solutions architecture and engineering group, Nvidia
Practical lessons from training extreme-scale language models for the multilingual Muse API
Par Julien Launay, PhD Student at ENS/Machine Learning Research Scientist, LightOn
Large language models have gained increased traction in the research community, but they remain only scarcely deployed at scale in the industry. In this talk, we go over how we are bringing large models with our Muse API to French, Arabic, Spanish, German, and English. We first discuss the importance of data collection and curation, and how it can be tackled at scale. We then highlight how an efficient training codebase based on Megatron+DeepSpeed is key to efficiently bringing these models to life. Finally, we review some use cases for these models, and how we can allow for efficient task specialisation at scale.
 |
Biographie : Julien Launay is the lead of the extreme-scale project at LightOn, working to extend the capabilities and the availability in various languages of large language models. |
|---|
Inscrivez-vous dès à présent et obtenez votre badge en cliquant ici
- Le Forum TERATEC est strictement réservé aux professionnels.
- Participation gratuite aux conférences et aux ateliers (sous réserve des places disponibles).
- L'enregistrement en ligne est obligatoire.
- Le badge délivré vous donnera accès gracieusement à l'ensemble des évènements du Forum TERATEC