Mr Daniel Collins1
Learn to tame your unruly data processing workflow with Snakemake, a modern workflow system to create reproducible and scalable data analyses. Workflows are described via a human readable, Python-based language. They can be seamlessly scaled to server, cluster, grid, and cloud environments, without the need to modify the workflow definition.
This workshop has two components: a hands-on introduction to Snakemake, and case studies on the impact of adopting workflow engines on selected science projects.
In the hands-on component, you will explore the limitations of manual processing and shell scripting before building up a reproducible, automated, and efficient workflow step by step with Snakemake. Along the way, you will learn the benefits of modern workflow engines and how to apply them to your own work.
At the end of this workshop, you will:
* Understand the benefits of workflow engines.
* Be able to create reproducible analysis pipelines with Snakemake.
All code and data are provided.
Users are required to bring their own laptop with appropriate software pre-installed.
The workshop will be delivered using a Software Carpentry approach.
* Some basic Python programming experience, ideally in Python 3.
* Familiarity with running programs on a command line.
A software engineer by training, Daniel has worked in CSIRO since 2010. His current role with Scientific Computing sees him working in the intersection between software engineering and research, bringing software engineering best practice to bear on research challenges.