Configuring Cray XT for TotalView
When configuring your Cray XT system for use with the TotalView, you must:
Mount user home directories on all service nodes that you will use while debugging.
(Optional) Enable passwordless
ssh on all service nodes that you will use while debugging. In most cases, your system administrator will have enabled your system so that it uses
ssh instead or
rsh. If passwordless ssh is not enabled, you will be asked to enter a password each time a process is launched on a node.
(Optional) Automatically set
RS_DBG_CLIENTS_PER_SERVER and
-nodes_allowed.
On Cray XT systems, setting a -nodes_allowed command-line option to 64 will not work. Instead, configure TotalView to use RS_DBG_CLIENTS_PER_SERVER and -nodes_allowed to make best use of the cluster and TotalView licenses.
TotalView administrators can set installation preferences by editing (or creating) the $TVROOT/linux-x86-64/lib/.tvdrc file. Here are two simple scenarios:
If you have only one TotalView license, and that license is for less than 64 processors, then a server launch string like this would be best:
dset -set_as_default TV::server_launch_string \
{%C %R -n "%B/tvdsvr%K -working_directory %D \
-callback %L \
-nodes_allowed maximum_processor_license \
-set_pw %P -verbosity %V %F"}
where maximum_processor_license is the processor count for your TotalView license.
If you are running TotalView on a cluster where the ratio of service nodes to compute nodes is less than 1:64, use a server launch string. For example:
dset -set_as_default TV::server_launch_string \
{%C %R -n "%B/tvdsvr%K -working_directory %D \
-callback %L \
-nodes_allowed ${RS_DBG_CLIENTS_PER_SERVER-64} \
-set_pw %P -verbosity %V %F"}
You need to set the RS_DBG_CLIENTS_PER_SERVER environment variable before submitting all jobs where a service node-to-compute node ration of 1:64 is not possible. You should also set RS_DBG_CLIENTS_PER_SERVER to the number of compute nodes served by each service node. For example, if you have a service-to-node ratio of 1:128, set this variable to 128.
The
TV::server_launch_string variable is used for both Cray XT3 and Linux x86-64. This means that if you will also be using this TotalView installation on other linux-x86-64 machines, do not set the TV::server_launch_string variable in your global .tvdrc.