Launch parallel Python programs
This topic describes how to launch parallel Python programs that use mpi4py with a supported MPI implementation. Use an MPI launcher to start these programs under TotalView. See Python debugging supported platforms and requirements for supported implementations.
On this page:
Validate your MPI Python runtime before debugging
Before launching your program in TotalView, check that it runs directly with MPI. This check helps distinguish runtime problems from problems with the debugger launch configuration.
From the command line, run the following command. Replace <N> with the number of processes:
mpirun --mca op ^avx -np <N> <path-to-python> <path-to-script.py>
If the command succeeds but the TotalView launch fails, the problem may be in the debugger launch configuration.
Launch a parallel Python program
-
From the command line, run one of the following commands:
-
Single-script parallel Python program: Replace <N> with the number of processes to launch and <script.py> with the path to your Python script. The command uses
pythonas the interpreter.totalview -args mpirun -np <N> %tv_python_wrapper% python <script.py> -
Multiple-program, multiple-data (MPMD) mpi4py session: Place
%tv_python_wrapper%immediately before eachpythoncommand. For example, to launch two different Python scripts, run:totalview -args mpirun -n <N> %tv_python_wrapper% python producer.py : -n <M> %tv_python_wrapper% python consumer.pyReplace <N> and <M> with the number of processes to launch for
producer.pyandconsumer.py, respectively. Replace the script names with the paths to your Python scripts. The colon (:) separates the two program specifications.
During launch, take the appropriate action for each condition that occurs:
-
If you encounter "mpirun" in the Source view, select Go (
) to continue the launch. -
If a dialog asks whether to stop the parallel job, select No.
-
If execution stops and displays the Python wrapper program (tv_python_wrapper.c) in the Source view, select Go (
).
TotalView loads your Python scripts and displays the MPI launcher and process ranks in the Processes & Threads view.
-
-
(Optional) Create a grouping of the Processes & Threads view that is tailored to multiprocessing Python programs, see Group Python processes and threads.
-
(Optional) Set any breakpoints you want in your Python code. See Set breakpoints in Python code.
-
Select Go (
) to run your Python program.
View and switch process context
Use the Processes & Threads view to examine a specific process or thread in your parallel Python program. You can also navigate between processes and threads. The Source, Call Stack, and Local Variables views update to show the selected process or thread's context.
To examine a specific process or thread, use one of the following methods:
-
In the Processes & Threads view, select the process or thread you want to examine.
-
Use the Processes & Threads toolbar to navigate to another process or thread. Options include the following:
-
P+ / P-: Move to the next or previous process. You can also use the menu options Process > Next Process and Process > Previous Process.
-
T+ / T-: Move to the next or previous thread in the current process. You can also use the menu options Thread > Next Thread and Thread > Previous Thread.
-
Px: Opens the Jump To dialog, which you can use to jump to a specific process or thread. In the Jump To dialog, specify a debugger ID or, for an MPI process, a rank, then select OK. You can also use the menu option Process > Jump to Process/Thread (Px).
-
To learn more, see Change the process or thread of focus.
Skip over the Python wrapper during MPI debugging
To avoid stopping at the Python wrapper each time you launch an MPI program, configure the parallel attach behavior:
-
Select File > Preferences to open the Preferences dialog.
-
On the PARALLEL tab, under After Attach Behavior, select Continue the processes.
With this setting enabled, TotalView continues MPI processes after attaching instead of stopping at the Python wrapper. This setting applies to MPI programs only. For single-process Python programs, select Go (
) if execution stops in the wrapper.
Resolve launch errors
If you encounter the following error when launching a parallel Python program, use the recommended action to resolve the issue.
Error: mpirun was unable to find the specified executable file
This error in the Input/Output view indicates a TotalView version that does not support native Python debugging. To resolve this error, use a TotalView version that supports native Python debugging.