Published in

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '13

DOI: 10.1145/2503210.2503222

Links

Tools

Export citation

Search in Google Scholar

Parallelizing the Execution of Sequential Scripts

Proceedings article published in 2013 by Zhao Zhang, Daniel S. Katz ORCID, Timothy G. Armstrong, Justin M. Wozniak, Ian Foster
This paper is available in a repository.
This paper is available in a repository.

Full text: Download

Question mark in circle
Preprint: policy unknown
Question mark in circle
Postprint: policy unknown
Question mark in circle
Published version: policy unknown

Abstract

Scripting is often used in science to create applications via the composition of existing programs. Parallel scripting sys-tems allow the creation of such applications, but each sys-tem introduces the need to adopt a somewhat specialized programming model. We present an alternative scripting approach, AMFS Shell, that lets programmers express par-allel scripting applications via minor extensions to existing sequential scripting languages, such as Bash, and then exe-cute them in-memory on large-scale computers. We define a small set of commands between the scripts and a parallel scripting runtime system, so that programmers can compose their scripts in a familiar scripting language. The underly-ing AMFS implements both collective (fast file movement) and functional (transformation based on content) file man-agement. Tasks are handled by AMFS's built-in execution engine. AMFS Shell is expressive enough for a wide range of applications, and the framework can run such applications efficiently on large-scale computers.