Using MRNet on Cray Computers
The following sections describe the options and state variables that were added to TotalView to control the configuration and use of MRNet on Cray. Please refer to the TotalView documentation for a general description of how options and state variables can be used with TotalView.
Is Cray XT Flag
State variable: TV::is_cray_xt boolean
Default value: Set to true if TotalView is running on Linux-x86_64 and /proc/cray_xt/nid exists; otherwise set to false.
Note that some Cray front-end (login) nodes do not have a /proc/cray_xt/nid file, in which case a job must be submitted to start TotalView on a Cray XT, XE, or XK node.
Cray XT MRNet Server Launch String
Option: –cray_xt_mrnet_server_launch_string string
State variable: TV::cray_xt_mrnet_server_launch_string string
Default value: /var/spool/alps/%A/toolhelper%A/tvdsvr%K \
-working_directory %D -set_pw %P -verbosity %V %F
Analogous to the standard MRNet server launch string, the Cray XT MRNet server launch string is used when MRNet launches the TotalView debugger servers on Cray XT, XE, or XK. TotalView expands the launch string using the normal launch string expansion rules.
Cray XT MRNet Transfer File List
Option: –cray_xt_mrnet_xfer_file_list stringlist
State variable: TV::cray_xt_mrnet_xfer_file_list stringlist
Default value:
The default value is calculated. at TotalView startup time, as follows. The following is used as a "base" list of files needed by TotalView on the Cray compute nodes when MRNet is in use.
TVROOT/bin/mrnet_commnode_main_cray_xt
TVROOT/bin/tvdsvr_mrnet
TVROOT/bin/tvdsvrmain_mrnet
TVROOT/shlib/mpa/obj_cray_xt/libmpattr.so.1
TVROOT/shlib/unwind/obj/libunwind-x86_64.so.8
TVROOT/shlib/unwind/obj/libunwind.so.8
TVROOT/shlib/mrnet/obj_cray_xt/libmrnet.so
TVROOT/shlib/mrnet/obj_cray_xt/libxplat.so
TVROOT/shlib/mrnet/obj_cray_xt/libservertree_filters.so.1
TVROOT/shlib/mrnet/obj_cray_xt/libtvwrapalps.so.1
/lib64/libthread_db.so.1
/usr/bin/ld
/usr/bin/objcopy
TVROOT/lib/libundodb_debugger_x64.so
TVROOT/lib/undodb_a_x64.o
TVROOT/lib/undodb_a_x32.o
TVROOT/lib/undodb_x64.so
TVROOT/lib/undodb_x32.so
TVROOT/lib/undodb_infiniband_preload_x64.so
TVROOT/lib/undodb_infiniband_preload_x32.so
TVROOT/lib/libtvheap_mic.so
TVROOT/lib/mic/libtvheap_64.so
The above list is then passed to the shell script named "cray_sysdso_deps.sh" to calculate the system shared libraries needed by the executables and shared libraries on the base list. The actual list of system libraries can vary from system to system, but typically consists of the following files:
/lib64/libdl.so.2
/lib64/libgcc_s.so.1
/lib64/libm.so.6
/lib64/libpthread.so.0
/usr/lib64/libbfd-<version>.so
/usr/lib64/libstdc++.so.5
The version of libbfd, which is needed by ld and objcopy, varies from system to system.
The default value is a space-separated string-list of file names that are transferred (staged) to the compute nodes. These files are the shell script, executable and shared library files required to run the MRNet commnode and TotalView debugger server processes on the compute nodes. When instantiating the MRNet tree on Cray XT, XE, or XK, the ALPS Tool Helper library is used to broadcast these files into the compute nodes' ramdisk under the /var/spool/alps/apid directory. TVROOT is the path to the platform-specific files in the TotalView installation.