dcalltree
Displays parallel backtrace data
Format: 
dcalltree [-data pbv_data_array] [-show_details] [-sort columns] [-hide_backtrace]
[-save_as_csv filename] [-save_as_dot filename]
Arguments: 
-data pbv_data_array
Captures the data from calling dcalltree in an associative Tcl array rather than writing the data to the console.
-show_details
Displays the data with all processes and threads displayed.
-hide_backtrace
Displays the data with only root and leaf nodes displayed.
-sort column
Sorts the data display based on the data in a particular column. The possible arguments are Processes, Location, PC, Host, Rank, ID, and Status.
-save_as_csv filename
Saves the backtrace data as a file of comma-separated values under the name filename.
-save_as_dot filename
Saves the backtrace data as a dot file under the name filename. Dot is a plain text graph description language.
Description: 
The TotalView GUI has a Parallel Backtrace View window that displays the state of every process and thread in a parallel job. The dcalltree command makes this same data available either in the console window, or, with the -data switch, as a Tcl associative array.
The associative array has the following format:
{
{
Key <value>
Level <value>
Processes <value>
Location <value>
PC <value>
Host <value>
Rank <value>
ID <value>
Status <value>
}
{
...
}
}
The -show_details and -hide_backtrace switches pull in opposite directions. The -show_details switch shows the maximum data, including all processes and threads. The -hide_backtrace command hides any intermediate nodes, displaying only the root and leaf nodes. If used together, this results in a display of root and leaf nodes and all threads. This reduction can help to de-clutter the data display if the number of processes and threads is large.
Examples: 
dfocus group dcalltree
This example first changes the focus to the group using dfocus, then calls dcalltree with no switches. Note that the ID column is a compressed ptlist describing process and thread count, range, and IDs. See Compressed List Syntax (ptlist) for more information.
Processes Location PC Host Rank ID Status
--------- -------- -- ---- ---- -- ------
12 / ... <local> -1 4:12[p1-4.1-3] ...
4 _start 0x004011b9 <local> -1 4:4[p1-4.1] ...
4 __libc_start_main 0x2b3425358184 <local> -1 4:4[p1-4.1] ...
4 main 0x004035bf <local> -1 4:4[p1-4.1] ...
4 fork_wrapper 0x00402790 <local> -1 4:4[p1-4.1] ...
4 forker 0x0040274b <local> -1 4:4[p1-4.1] ...
4 snore 0x00401c11 <local> -1 4:4[p1-4.1] ...
1 snore#681 0x00401c05 <local> -1 2.1 - 47502964801120 Stopped
1 snore#705 0x00401c9b <local> -1 4.1 - 47502964801120 Breakpoint
2 wait_a_while 0x00401a09 <local> -1 2:2[p1.1, p3.1] Stopped
2 __select_nocancel 0x2b34253f56e2 <local> -1 2:2[p1.1, p3.1] Stopped
8 start_thread 0x2b3424db1143 <local> -1 4:12[p1-4.1-3] ...
8 snore_or_leave 0x004021cb <local> -1 4:8[p1-4.2-3] ...
8 snore ... <local> -1 4:8[p1-4.2-3] ...
1 snore#681 0x00401c05 <local> -1 1.2 - 1082132800 Breakpoint
1 snore#681 0x00401c05 <local> -1 1.3 - 1090525504 Stopped
1 snore#705 0x00401c9b <local> -1 2.2 - 1082132800 Breakpoint
1 snore#681 0x00401c05 <local> -1 2.3 - 1090525504 Stopped
1 snore#681 0x00401c05 <local> -1 4.2 - 1082132800 Stopped
1 snore#681 0x00401c05 <local> -1 4.3 - 1090525504 Stopped
2 wait_a_while ... <local> -1 1:2[p3.2-3] ...
 
 
dcalltree -show_details
 
 
By adding the -show_details, switch, you get more complete output:
Processes Location PC Host Rank ID Status
--------- -------- -- ---- ---- -- ------
12 / ... <local> -1 4:12[p1-4.1-3] ...
4 _start 0x004011b9 <local> -1 4:4[p1-4.1] ...
4 __libc_start_main 0x2b3425358184 <local> -1 4:4[p1-4.1] ...
4 main 0x004035bf <local> -1 4:4[p1-4.1] ...
4 fork_wrapper 0x00402790 <local> -1 4:4[p1-4.1] ...
4 forker 0x0040274b <local> -1 4:4[p1-4.1] ...
4 snore 0x00401c11 <local> -1 4:4[p1-4.1] ...
1 snore#681 0x00401c05 <local> -1 2.1 - 47502964801120 Stopped
1 snore#705 0x00401c9b <local> -1 4.1 - 47502964801120 Breakpoint
2 wait_a_while 0x00401a09 <local> -1 2:2[p1.1, p3.1] Stopped
2 __select_nocancel 0x2b34253f56e2 <local> -1 2:2[p1.1, p3.1] Stopped
1 __select_nocancel 0x2b34253f56e2 <local> -1 1.1 - 47502964801120 Stopped
1 __select_nocancel 0x2b34253f56e2 <local> -1 3.1 - 47502964801120 Stopped
8 start_thread 0x2b3424db1143 <local> -1 4:12[p1-4.1-3] ...
8 snore_or_leave 0x004021cb <local> -1 4:8[p1-4.2-3] ...
8 snore ... <local> -1 4:8[p1-4.2-3] ...
1 snore#681 0x00401c05 <local> -1 1.2 - 1082132800 Breakpoint
1 snore#681 0x00401c05 <local> -1 1.3 - 1090525504 Stopped
1 snore#705 0x00401c9b <local> -1 2.2 - 1082132800 Breakpoint
1 snore#681 0x00401c05 <local> -1 2.3 - 1090525504 Stopped
1 snore#681 0x00401c05 <local> -1 4.2 - 1082132800 Stopped
1 snore#681 0x00401c05 <local> -1 4.3 - 1090525504 Stopped
2 wait_a_while ... <local> -1 1:2[p3.2-3] ...
1 __select_nocancel 0x2b34253f56e2 <local> -1 3.3 - 1090525504 Stopped
1 wait_a_while#580 0x004019e9 <local> -1 3.2 - 1082132800 Breakpoint
 
 
dcalltree -show_details -hide_backtrace
 
Adding the -hide_backtrace switch reduces the clutter somewhat:
Processes Location PC Host Rank ID Status
--------- -------- -- ---- ---- -- ------
12 / ... <local> -1 4:12[p1-4.1-3] ...
1 __select_nocancel 0x2b34253f56e2 <local> -1 3.3 - 1090525504 Stopped
1 __select_nocancel 0x2b34253f56e2 <local> -1 1.1 - 47502964801120 Stopped
1 __select_nocancel 0x2b34253f56e2 <local> -1 3.1 - 47502964801120 Stopped
1 snore#681 0x00401c05 <local> -1 2.1 - 47502964801120 Stopped
1 snore#705 0x00401c9b <local> -1 4.1 - 47502964801120 Breakpoint
1 snore#681 0x00401c05 <local> -1 1.2 - 1082132800 Breakpoint
1 snore#681 0x00401c05 <local> -1 1.3 - 1090525504 Stopped
1 snore#705 0x00401c9b <local> -1 2.2 - 1082132800 Breakpoint
1 snore#681 0x00401c05 <local> -1 2.3 - 1090525504 Stopped
1 snore#681 0x00401c05 <local> -1 4.2 - 1082132800 Stopped
1 snore#681 0x00401c05 <local> -1 4.3 - 1090525504 Stopped
1 wait_a_while#580 0x004019e9 <local> -1 3.2 - 1082132800 Breakpoint
 
 
Here is code to get the location of all threads that are at a breakpoint:
dcalltree -data pbv_data_array -show_details
foreach { data_record } [array get pbv_data_array] {
set print_location 0
set break_location
foreach {title value} $data_record {
if {$title == "Location"} {
set break_location $value
}
if {$value == "Breakpoint"} {
set print_location 1
}
if {1 == $print_location} {
puts stdout "Breakpoint found at $break_location"
set print_location 0
}
}
}