ASKAPsoft/Yandasoft Deployment using Docker/Singularity
Dr Paulus Lahur1, Dr Stephen Ord2
1CSIRO, Clayton, Australia, 2CSIRO, Marsfield, Australia
Yandasoft is a suite of radio astronomy software developed by CSIRO Astronomy and Space Science (CASS) for the calibration and imaging of Interferometric Radio Telescope data. It was originally developed for the Australian Square Kilometre Array Pathfinder (ASKAP) project. To serve a wider community of radio astronomy, it has been released as open source code. However, due to the complexity of the code, and the large number of third-party dependencies, it takes a lot of time and effort to build correctly. If users must build it from the source code, they will require a significant amount of support. Shipping the executable binaries is not an option either, because the software must match the underlying computing environment.
This is where container technology called Docker and Singularity comes to the rescue. Docker was used to create the image of the software, complete with underlying dependencies. Users can then use Docker or Singularity to execute the image on their machines as if it’s just a normal software. For large scale HPC deployments Singularity is ideal, because it allows parallel computation and is now commonly installed on HPC facilities. The best part is that we pay almost no penalty for this convenience.
The container technology has significantly improved user experience and at the same time reduced the effort required to support them in terms of deployment. We will present the techniques that we use to build the Docker images for various computing environment and provide an example of HPC deployments using this technology.
Currently the presenter works as research software engineer in Scientific Computing in Information Management and Technology (IMT) in CSIRO, where he helps various research projects in DevOps for scientific software. Previously he worked in Japan Aerospace Exploration Agency (JAXA) as researcher in Computational Fluid Dynamics (CFD).