Session

DescriptionIn the past few years, applications in scientific domains such as weather and forecast have evolved into complex data-centric workflows. This evolution has resulted in a data deluge which has been observed in large computing centers such as the National Energy Research Scientific Computing Center (NERSC), where the volume of data stored increased by a factor of 40 in the last ten years. However, the gap between computing and I/O capabilities continues to grow: the ratio of I/O bandwidth to computing power has decreased by an order of magnitude over the last ten years for the top 3 supercomputers listed in the top500. Faced with this critical situation, storage systems must adapt. In this minisymposium, we will examine the needs of scientific workflows in terms of massive data management through a concrete example. Then, we will explore the solutions proposed by the computer science research community, both from an infrastructure and a execution software stack perspective. We will see how these lines of research can improve data orchestration in scientific workflows where storage is central, and identify which research topics remain to be further developed.