Collaboratively developing workflows at the BSC-ES

At the Earth Sciences department of the Barcelona Supercomputing Center (BSC-ES) a variety of workflows are run for many different purposes like executing climate and atmospheric simulations, data downloading or performance evaluation. This implies having to deal with many different processes and time scales in different environments and machines.

To help conduct all these complex tasks, the Autosubmit workflow manager is used in the whole department as a unique framework. The fact that Autosubmit has been fully developed at the BSC-ES has led to the adoption of a co-design procedure between users, workflow developers and Autosubmit developers to fulfill the day-to-day department needs. The synergy and close collaboration among them allows the workflow engineers to gather the specific user requirements that later become new Autosubmit features available to everyone. Thanks to this continuous interaction at all levels, an efficient and very adaptable system could have been achieved, perfectly aligned with the constantly evolving user needs.

Here this collaborative strategy is presented from the workflow development point of view. Some real use cases and practical examples are used to show the positive impact it had in different operational and research projects, demonstrating how it can help in the achievement of high scientific productivity.