Big Data in High Performance Computing

Coordinator :BULL
Status : Ongoing
Start date : June 2013
Finish date : June 2015
Duration : 24 months
Members involved in the project :ACTIVEEON - ARMADILLO - BULL - CEA/DAM - CEA/LIST - INRIA - INSTITUT DE PHYSIQUE DU GLOBE DE PARIS - SENSEETIVE

October 2013 - The DataScale project gathers a large set of partners, from large research laboratories to SMEs, including also big companies. Their common objective is to design efficient Big Data solutions, suited to real use cases in High Performance Computing (HPC)

DataScale is a two-year project, created in June 2013 and funded by the “Investissements d’Avenir” (Investments for the Future) program supported by the French government. Its main mission is to develop synergies between Big Data and HPC, and more specifically to develop Big Data technological building blocks that will enrich the HPC ecosystem.

These technological building blocks will be focused on three topics that are at the heart of the Big Data challenge:

Efficient data management, a key issue to be able to accommodate and process ever increasing volumes of data (particularly by using hierarchical storage),

The opening of data management to third party environments, and specifically to cloud environments,

A database architecture with data mining technologies suited for the efficient handling of very large data volumes and of different types of data.

The project will also evaluate the relevance of the technological building blocks by implementing demonstrators based on real world applications, at real scale, in the areas of seismic event detection, HPC cluster management, and multimedia product analysis.

The project partners have very different profiles, and can thus bring to the project their respective expertise and know-how in areas as diverse as infrastructures, HPC, databases, cloud computing, system management, multimedia, data-mining and seismology. The DataScale partners will leverage their thorough knowledge of market needs to propose relevant methods and algorithms, and develop solutions that will give rise to actual products. The diversity of the partners’ profiles and approaches guarantees an excellent coverage of the Big Data issues and solutions, and is also the source of the synergies that the project can develop.

"The DataScale project is characterized by an approach that is both technological and guided by real use. It will bring a very practical solution to the challenges raised by the exponential growth of the volume of data to be processed. Bull and all the other DataScale partners are confident that the horizontal tools created or extended within the project will accelerate their innovation and enrich their respective commercial offers. » says Etienne Walter, DataScale project manager at Bull.

Denis Caromel (founder and CEO of ActiveEon) states "Big Data is too often reduced to the MapReduce issue, and particularly to Hadoop, which means that many companies make inadequate technological choices. In fact, the management of large amounts of data often raises different issues, and thus requires a combination of solutions, such as orchestration and scheduling, deployment, resource optimization. DataScale will reinforce the existing offers of the French partners of the project, and will make it possible in the short term to propose a complete and sovereign solution.“