 |
||
|
||
Home > TERATEC FORUM > Workshops > Workshop 6
Workshops
Thursday June 24
Workshop 06 - 16:00 to 18:00
New Storage Paradigms to Tackle Exascale Challenges
Chaired by Philippe Deniel, chef du Laboratoire architecture du stockage et des systèmes, CEA DAM
Storage Aware Scheduling with Slurm
By Iakovos Panourgia, HPC Applications Consultant, EPCC,
Adrian Jackson, Senior Researc Fellow, EPCC,
and Darren J. White, Applications Consultant, UEDIN
Handling enormous volumes of data is becoming a hot topic in High Performance Computing. The need to store, parse, analyse and safe-keep the data pushes the limits of current technologies and storage systems.
In this talk we will present the state-of-the-art for the Sage2 platform and our work to integrate this with job scheduling systems to enable users to easily and efficiently use the platform from their applications.
The Sage2 platform is a unified data storage system platform for AI, Deep learning, Big Data Analysis and High Performance Computing workloads. The architecture reflects the need for reducing data movement in order to improve energy efficiency, as well as the technology trend towards new non-volatile memory technologies.
Our scheduler developments have focussed on enabled Slurm to work in tandem with the Sage2 platform. This has included implementing a burst-buffer style plugin that allows the Sage2 platform to copy objects across data tiers whilst jobs are waiting, in order to improve throughput.
 |
Biography: |
|---|---|
 |
Biography: |
Register now and get your badge here
- TERATEC Forum is strictly reserved for professionals.
- Participation to exhibition, conferences and workshops is free (subject to seats available)
- On line registration is obligatory.