Prof. Daniel S. Katz1, Yadu Babuji2, Kyle Chard2,3, Ben Clifford2, Ian Foster2,3, Zhuozhao Li2, Michael Wilde2,4, Anna Woodard2, Justin Wozniak2
1University of Illinois at Urbana-Champaign , , , 2University of Chicago & Argonne National Laboratory, , , 3Globus, , , 4Parallel Works, ,
Python is increasingly the lingua franca of scientific computing. It is used as a higher level language to wrap lower-level libraries and to compose scripts from various independent components. However, scaling and moving Python programs from laptops to supercomputers remains a challenge. Here we present Parsl (http://parsl-project.org/), a parallel scripting library for Python. Parsl makes it straightforward for developers to implement parallelism in Python by annotating functions that can be executed asynchronously and in parallel, and to scale analyses from a laptop to thousands of nodes on a supercomputer or distributed system. We explain the features of Parsl, and show an example of how resources are specified, how parallel functions are decorated, and how a Parsl program takes advantage of these features.
Daniel S. Katz is Assistant Director for Scientific Software and Applications at the National Center for Supercomputing Applications (NCSA), Research Associate Professor in Computer Science (CS), Research Associate Professor in Electrical and Computer Engineering (ECE), and Research Associate Professor in the School of Information Sciences (iSchool) at the University of Illinois Urbana-Champaign. He is also Guest Faculty at Argonne National Laboratory, and Adjunct Faculty at the Center for Computation & Technology (CCT), Louisiana State University (LSU). His interest is in the development and use of advanced cyberinfrastructure to solve challenging problems at multiple scales. His technical research interests are in applications, algorithms, fault tolerance, and programming in parallel and distributed computing, including HPC, Grid, Cloud, etc. He is also interested in policy issues, including citation and credit mechanisms and practices associated with software and data, organization and community practices for collaboration, and career paths for computing researchers.