Session

DescriptionComputational science projects today consist of numerous steps and methods which involve multiple software projects and may run on various devices, from classical HPC systems to cloud-based services. This makes the workflow itself an integral part of the result, and Workflow Managers one of the main tools computational scientists will have to use in their work. By using proper workflow tools, scientific results can more easily be shared and reproduced, while keeping the process to obtain them flexible and transferable. Together with our speakers we look at the current state of software packages for workflow orchestration and their building blocks to make such an automation possible across supercomputers. We give concrete examples demonstrating how to integrate Machine Learning into scientific workflows in a domain-agnostic way, and show how smart data management in a workflow allows for asynchronous data analysis to improve time to solution.