Displaying the Message Queue Graph Window

User_Guides : Classic TotalView User Guide : PART II Debugging Tools and Tasks : Debugging Strategies for Parallel Applications : MPI Debugging Tips and Tools : MPI Display Tools : Displaying the Message Queue Graph Window

TotalView can graphically display your MPI program’s message queue state. Select the Process Window Tools > Message Queue Graph command to display a graph of the current message queue state.

Figure 229, Tools > Message Queue Graph Window

If you want to restrict the display, select the Options button, Figure 230.

Figure 230, Tools > Message Queue Graph Options Window

Here, you can alter the way in which TotalView displays ranks within this window—for example, as a grid or in a circle.

Use the commands within the Cycle Detection tab to receive reports about cycles in your messages. This is a quick and efficient way to detect when messages are blocking one another and causing deadlocks.

Perhaps the most used of these tabs is Filter.

Figure 231, Tools > Message Queue Graph Options. Filter Tab

The button colors used for selecting messages are the same as those used to draw the lines and arrows in the Message Queue Graph Window, as follows:

Green: Pending Sends

Blue: Pending Receives

Red: Unexpected Messages

You can directly select which ranks you want displayed in the lower part of the window. The Filter on specified message tags area lets you name the tags to be used as filters. Finally, you can select a group or a communicator in the group pulldown. If you have created your own communicators and groups, they appear here.

Changes made within the Options dialog box do not occur until you click Apply. The graph window then updates to reflect your changes.

The message queue graph shows your program’s state at a particular instant. Select Update to fetch new information and redraw the graph.

The numbers in the boxes within the Message Queue Graph Window indicate the MPI message source or destination process rank. Diving on a box opens a Process Window for that process.

The numbers next to the arrows indicate the MPI message tags that existed when TotalView created the graph. Diving on an arrow displays the Tools > Message Queue Window, with detailed information about the messages. If TotalView has not attached to a process, it displays this information in a grey box.

You can use the Message Queue Graph Window in many ways, including the following:

Pending messages often indicate that a process can’t keep up with the amount of work it is expected to perform. These messages indicate places where you may be able to improve your program’s efficiency.

Unexpected messages can indicate that something is wrong with your program because the receiving process doesn’t know how to process the message. The red lines indicate unexpected messages.

After a while, the shape of the graph tends to tell you something about how your program is executing. If something doesn’t look right, you will want to determine why.

You can change the shape of the graph by dragging nodes or arrows. This is often useful when you’re comparing sets of nodes and their messages with one another. By default, TotalView does not persist changes to the graph shape. This means that if you select Update after you arrange the graph, your changes are lost. To retain your changes, select Keep nodes as positioned from with the Options dialog box.