Linking Your Application With the Agent
MemoryScape puts its heap agent between your program and its heap library which allows the agent to intercept the calls that your program makes to this library. After it intercepts the call, it checks the call for errors and then sends it on to the library so that it can be processed. The MemoryScape agent does not replace standard memory functions; it just monitors what they do. For more information, see “Behind the Scenes” in the introduction to Debugging Memory Problems with MemoryScape.
In most cases, MemoryScape arranges for the heap agent to be loaded automatically when it starts your program. In some cases, however, special steps must be taken to ensure the agent loads. One example is when you are starting an MPI program using a launcher that does not propagate environment variables. (If you start your MPI program in MemoryScape using the Add Parallel Program page, MemoryScape propagates the information for you.) Another is when you want to start your program outside, or independently of, TotalView, and want to attach to the program later after it has started.
There are two ways you can arrange for the heap agent to be loaded:
• Link the application with the agent, as described in this section.
Here is some important platform-specific information:
• On AIX, the malloc replacement code and heap agent application must be in directories searched by the dynamic loader. If they are not in any of the standard directories (you can check with your system administrator), you can set LIBPATH to search these directories when you run the program. Another option is to add the directories to the program's list of search directories when you link the program. To do this, use the -L option as described in the table below. If you are in doubt about the directories being searched, you can obtain a list of the searched directories with dump -Hv <program-name>.
• On Cray, TotalView supports both static and dynamic linking. See the table below for the link lines you need to use.
• On Blue Gene/L, a special heap agent must be used. This needs to be statically linked with your program, as shown in the table below.
• Blue Gene/P supports static and dynamic linking. If your program is dynamically linked and you start your job under TotalView using a launching program such as mpirun or runjob that honors the MPIRUN_ENV environment variable, you don't need to take any special steps to prepare your application for memory debugging. In this case, TotalView arranges for the heap agent to be loaded with your program.
If, however, you are starting the job outside TotalView, such as when you intend to attach to the job later, or you are using a launching program that does not honor MPIRUN_ENV, you must arrange for the heap agent to be loaded when the program runs. This can be done in one of two ways:
— Link your application with the heap agent. The link options for doing so are shown in the table below.
— Set the LD_PRELOAD environment variable in your program's environment to path/libtvheap.so. Note that path refers to the file system in which the program executes, not the one in which the launching program runs. If you are using mpirun you can use the -env option, or set MPIRUN_ENV to pass the environment variable setting. See the manual page for mpirun on your system.
There is also a version of the heap agent available for statically linked programs. The link options you should use when linking your program are also shown in the table below. Both statically and dynamically linked versions of the heap agent are thread-safe and can be used with multi-threaded programs.
• Blue Gene/Q supports static or dynamic linking. In either case, you must link your application with the heap agent as described in the table below, or, if your application is dynamically linked, arrange for it to be preloaded. Note also that memory debugging and event notification should be enabled on the starter program.
As with Blue Gene/P, if your program is dynamically linked you can avoid having to link the heap agent with your program if you set the LD_PRELOAD variable in your program's environment. If you are using runjob to launch your program, you can use its --envs option to pass this environment variable setting. If you are using srun, you will need to add the runjob --envs option to srun's --runjob-opts option; for example, --runjob-opts="--envs LD_PRELOAD=...". However you arrange for the environment variable to be passed, LD_PRELOAD should be set to path/libtvheap_64.so. Both statically and dynamically linked versions of the heap agent are thread-safe and can be used with multi-threaded programs.
• On Apple Mac OSX, you cannot link the agent into your program.
The following table lists additional command-line linker options that you must use when you link your program:
Platform | Compiler | Binary Interface | Additional linker options |
Cray XT, XE, XK CLE (dynamic) | - | 64 | -L<path> -ltvheap_cnl -Wl,-rpath,<path> |
Cray XT, XE, XK CLE (static) | - | 64 | -L<path> -ltvheap_cnl |
Cray XT3 Catamount | - | 64 | -L<path> -lgmalloc -ltvheap_xt3 |
IBM Blue Gene/L (static) | - | 32 | -L<path> -ltvheap_bluegene |
IBM Blue Gene/P (dynamic) | - | 32 | -L<path> -ltvheap -Wl,-rpath,<path> |
IBM Blue Gene/P (static) | - | 32 | -L<path> -ltvheap_bluegene_p |
IBM Blue Gene/Q (dynamic) | IBM/GCC | 64 | -L<path> -ltvheap_64 -Wl,-rpath,<path> |
IBM Blue Gene/Q (static) | IBM/GCC | 64 | -L<path> -Wl,@<path>/tvheap_bgqs.ld |
IBM RS/6000 (all) | IBM/GCC | 32/64 | -L<path_mr> -L<path> |
AIX 5 | IBM/GCC | 32 | -L<path_mr> -L<path> <path>/aix_malloctype.o |
| 64 | -L<path_mr> -L<path> <path>/aix_malloctype64_5.o |
Linux x86 | GCC/Intel/PGI | 32 | -L<path> -ltvheap -Wl,-rpath,<path> |
Linux x86-64 | GCC/Intel/PGI | 32 | -L<path> -ltvheap -Wl,-rpath,<path> |
| 64 | -L<path> -ltvheap_64 -Wl,-rpath,<path> |
Linux IA64 | GCC/Intel | 64 | -L<path> -ltvheap -Wl,-rpath,<path> |
Linux Power | GCC/Intel/PGI | 32 | -L<path> -ltvheap -Wl,-rpath,<path> |
| 64 | -L<path> -ltvheap_64 -Wl,-rpath,<path> |
Sun | Sun/Apogee | 32 | -L<path> -ltvheap -R <path> |
Sun | 64 | -L<path> -ltvheap_64 -R <path> |
GCC | 32 | -L<path> -ltvheap -Wl,-R,<path> |
| 64 | -L<path> -ltvheap_64 -Wl,-R,<path> |
The following list describes the options in this table:
<path>
The absolute path to the agent in the MemoryScape installation hierarchy. More precisely, this directory is:
<installdir>/toolworks/memoryscape.<version>/<platform>/lib
<installdir>
The installation base directory name.
<version>
The MemoryScape version number.
<platform>
The platform tag.
<path_mr>
The absolute path of the malloc replacement library. This value is determined by the person who installs the MemoryScape malloc replacement library.
NOTE >> The heap agent library path can be hard to determine. If you have access to the command line interface (CLI), you can use the following command to print out its path:
puts $TV::hia_local_dir
Other related topics are: