IBM PE Debugging Tips

The following debugging tips apply only to IBM MPI (PE):

Avoid unwanted timeouts

Timeouts can occur if you place breakpoints that stop other processes too soon after calling MPI_Init() or MPL_Init(). If you create “stop all” breakpoints, the first process that gets to the breakpoint stops all the other parallel processes that have not yet arrived at the breakpoint. This can cause a timeout.

To turn the option off, select the Process Window Action Point > Properties command while the line with the stop symbol is selected. ­After the Properties Dialog Box appears, select the Process button in the When Hit, Stop area, and also select the Plant in share group button.

CLI: dbarrier location –stop_when_hit process

Control the poe process

Even though the poe process continues under debugger control, do not attempt to start, stop, or otherwise interact with it. Your parallel tasks require that poe continues to run. For this reason, if poe is stopped, TotalView automatically continues it when you continue any parallel task.

Avoid slow processes due to node saturation

If you try to debug a PE program in which more than three parallel tasks run on a single node, the parallel tasks on each node can run noticeably slower than they would run if you were not debugging them.

In general, the number of processes running on a node should be the same as the number of processors in the node.

This becomes more noticeable as the number of tasks increases, and, in some cases, the parallel tasks does not progress. This is because PE uses the SIGALRM signal to implement communications operations, and AIX requires that debuggers must intercept all signals. As the number of parallel tasks on a node increases, TotalView becomes saturated and can’t keep up with the SIGALRM signals being sent, thus slowing the tasks.