 |
||
|
||
Home > TERATEC FORUM > Workshops > Workshop 8
Workshop 08 - 2:00 to 5:30 pm
High performance AI in the industry
Chaired by Cristel Saudemont, France Director, Supercomputing & AI , Higher Education and Research and Frédéric Parienté, senior manager in the solutions architecture and engineering group, Nvidia
Practical lessons from training extreme-scale language models for the multilingual Muse API
By Julien Launay, PhD Student at ENS/Machine Learning Research Scientist, LightOn
Large language models have gained increased traction in the research community, but they remain only scarcely deployed at scale in the industry. In this talk, we go over how we are bringing large models with our Muse API to French, Arabic, Spanish, German, and English. We first discuss the importance of data collection and curation, and how it can be tackled at scale. We then highlight how an efficient training codebase based on Megatron+DeepSpeed is key to efficiently bringing these models to life. Finally, we review some use cases for these models, and how we can allow for efficient task specialisation at scale.
 |
Biography: Julien Launay is the lead of the extreme-scale project at LightOn, working to extend the capabilities and the availability in various languages of large language models. |
|---|
Register now and get your badge here
- TERATEC Forum is strictly reserved for professionals.
- Participation to exhibition, conferences and workshops is free (subject to seats available)
- On line registration is obligatory to attend exhibition, conferences or the workshops.
- The badge is free of charge and give you access to all events TERATEC Forum.