Parsl: Parallel Scripting in Python
Parsl extends parallelism in Python beyond a single computer.
You can use Parsl just like Python’s parallel executors but across multiple cores and nodes . However, the real power of Parsl is in expressing multi-step workflows of functions. Parsl lets you chain functions together and will launch each function as inputs and computing resources are available.
import parsl from parsl import python_app # Start Parsl on a single computer parsl . load () # Make functions parallel by decorating them @python_app def f ( x ): return x + 1 @python_app def g ( x ): return x * 2 # These functions now return Futures, and can be chained future = f ( 1 ) assert future . result () == 2 future = g ( f ( 1 )) assert future . result () == 4 Start with the configuration quickstart to learn how to tell Parsl how to use your computing resource, then explore the parallel computing p
Parsl: Parallel Scripting in Python