Developing scalable and portable computational workflows with Nextflow

Dr Rad Suchecki1

1CSIRO, Urrbrae, Australia

 

Large analysis workflows are fragile ecosystems of software tools, scripts and dependencies. This complexity commonly makes them hard to maintain, extend, and all but impossible to use outside their original development environment. Nextflow is a workflow framework and a domain specific programming language which follows the dataflow paradigm and offers an alternative, and arguably superior, approach to developing, executing and sharing pipelines. Nextflow offers seamless integration with code and container image hosting services such as GitHub and Docker Hub, and out of the box support for various HPC cluster schedulers and cloud compute systems.

In this workshop you will learn

  • about processes, channels and operators – the building blocks of Nextflow
  • how to run, port and customise existing Nextflow workflows
  • how to develop a simple Nextflow workflow from scratch
  • how to separate the pipeline logic from compute and software environment configuration

By the end of this workshop you will be ready to start developing shareable, version controlled, container-backed workflows, which can be seamlessly executed across different environments from a laptop to cluster to cloud.


Biography:

Rad Suchecki obtained his BSc and PhD from the School of Computing Sciences, University of East Anglia, Norwich, UK. During his postdoc at The University of Adelaide, he developed high-performance computational pipelines and web applications for integration and visualisation of biological data. He continues this work in CSIRO’s Aginformatics group where he applies and develops frameworks and software to drive reproducibility in crop informatics and data science.

ABOUT AeRO

AeRO is the industry association focused on eResearch in Australasia. We play a critical coordination role for our members, who are actively transforming research via Information Technology. Organisations join AeRO to advance their own capabilities and services, to collaborate and to network with peers. AeRO believes researchers and the sector significantly benefit from greater communication, coordination and sharing among the increasingly different and evolving service providers.

Conference Managers

Please contact the team at Conference Design with any questions regarding the conference.
© 2019 Conference Design Pty Ltd