Minisymposium: MS3A - HPC Workflow Automation and Management
Session Chair
Event TypeMinisymposium
Chemistry and Materials
Climate, Weather and Earth Sciences
Computer Science and Applied Mathematics
Life Sciences
TimeTuesday, June 2811:00 - 13:00 CEST
LocationOsaka Room
DescriptionComputational science projects today consist of numerous steps and methods which involve multiple software projects and may run on various devices, from classical HPC systems to cloud-based services. This makes the workflow itself an integral part of the result, and Workflow Managers one of the main tools computational scientists will have to use in their work. By using proper workflow tools, scientific results can more easily be shared and reproduced, while keeping the process to obtain them flexible and transferable. Together with our speakers we look at the current state of software packages for workflow orchestration and their building blocks to make such an automation possible across supercomputers. We give concrete examples demonstrating how to integrate Machine Learning into scientific workflows in a domain-agnostic way, and show how smart data management in a workflow allows for asynchronous data analysis to improve time to solution.