NextGen TotalView® for HPC 2019.3 Supported Platforms
Versions
Support Notes
Platform Support
NextGen TotalView for HPC User Guide
PART I An Introduction to NextGen TotalView for HPC
Chapter 1 Getting Started
Introducing NextGen TotalView for HPC
An Initial Look at the Interface
Customizing the Interface
Preferences
Resizing
Drawers
Undocking and Docking
A Tour of the Interface
Central Window
Toolbars
Processes and Threads View
Call Stack View and Local Variables View
Data View
Lookup View
Action Points, CLI, and Logger Views
Help
Starting TotalView and Creating a Debugging Session
Debugging Commands
Chapter 2 Creating and Managing Sessions
Overview
Setting up Debugging Sessions
Loading Programs from the Session Editor
Starting a Debugging Session
Debug a Program
Debug a Parallel Program
Attach to Process
Debug a Core or Replay Recording File
Load a Recent Session
Editing a Previous Session
Loading Programs Using the CLI
Options and Program Arguments
Options: Reverse Debugging
Program Environment: Environment Variables
Standard Input and Output
Modifying Arguments in an Open Session
Managing Sessions
Starting a Session from your Shell
Starting TotalView on a Script
Chapter 3 Basic Debugging
Overview
Program Load and Navigation
Load the Program to Debug
Initial Display
Program Navigation
Navigating from within the Source Pane
Searching with Highlighting and the Find Function
Using the Lookup File or Function View
Stepping and Executing
Simple Stepping
Setting and Running to a Breakpoint (Action Point)
Set and Control Breakpoints
Run Your Program and Observe the Call Stack
Examining Data
Viewing Variables in the Local Variables View
Viewing Variables in the Data View
Watching Data Values Update
Moving On
PART II Debugging Tools and Tasks
Chapter 4 Working with Processes and Threads
The Processes and Threads View
How the Processes and Threads View Works
Debugging Command Width
Understanding Thread Width
Understanding Process Width
Understanding Group Width
Controlling Processes and Threads in a Running Session
Controlling Processes by Detaching from the Debugger
Process > Detach
Group > Detach
Program Execution and Share Groups
Synchronizing Processes and Threads Using Run To
Breakpoint Width Impact on Execution
Setting the Program Counter
Summary of Process and Thread Attributes
Chapter 5 Setting and Managing Action Points (Breakpoints)
Introducing Action Points
Breakpoints
Setting Source-Level Breakpoints
Sliding Breakpoints
Breakpoints at a Specific Location
Pending Breakpoints
Pending Breakpoints on a Function
Pending Breakpoints on a Line Number
Conflicting Breakpoints
Breakpoints at Execution
Modifying a Breakpoint
Setting Breakpoints When Using the fork()/execve() Functions
Debugging Processes That Call the fork() Function
Debugging Processes that Call the execve() Function
Example: Multi-process Breakpoint
Evalpoints
Setting an Evalpoint
Creating a Pending Evalpoint
Modifying an Evalpoint
Creating Conditional Breakpoints
Patching Programs
Branching Around Code
Adding a Function Call
Correcting Code
Using Programming Language Constructs
Watchpoints
Creating Watchpoints
Displaying, Deleting, or Disabling Watchpoints
Modifying Watchpoints
Watching Memory
Triggering Watchpoints
Using Multiple Watchpoints
Performance Impact of Copying Previous Data Values
Using Watchpoint Expressions
Using Watchpoints on Different Architectures
Barrier Points
About Barrier Breakpoint States
Setting a Barrier Breakpoint
Creating a Satisfaction Set
Hitting a Barrier Point
Releasing Processes from Barrier Points
Changing Settings and Disabling a Barrier Point
Using Barrier Points
Barrier Point Illustration
Controlling an Action Point’s Width
About an Action Point’s Width: Group, Process or Thread
Setting the Action Point’s Width
Action Point Width and Process/Thread State
Managing and Diving on Action Points
Sorting
Diving
Deleting, Disabling, and Suppressing
Saving and Loading Action Points
More on Action Points Using the CLI
Breakpoints
Breakpoints
Evalpoints
Watchpoints
Barrier Points
Saving Action Points to a File Using the CLI
Suppressing and Unsuppressing Action Points
Chapter 6 Examining and Editing Data
Overview
About Expressions
Using C++
The Call Stack and Local Variables Views
The Call Stack View
The Local Variables View
Viewing Call Stack Data
Viewing Data in Fortran
Viewing Modules and Their Data
Common Blocks
Fortran 90 User-Defined Types
Fortran 90 Deferred Shape Array Types
Fortran 90 Pointer Types
Fortran Parameters
The Data View
Adding Variables to the Data View
Add to the Data View from the Local Variables View
Move an Expression or Variable from the Source View to the Data View
Creating a New Expression by Diving on a Variable in the Data View
Entering a New Expression into the Data View
Editing an Expression
Dereferencing a Pointer
Changing the Value of Data
Casting to Another Type
Displaying Arrays
Customizing the Data View
Duplicating the Data View
The Data View Drawer
Searching for Program Elements
Using the CLI to Examine Data
Changing the Display of Data
Displaying Variables
Chapter 7 Debugging Python
Overview
Python Debugging Requirements
Python Version
Limitations and Extensions:
Starting a Python Debugging Session
Debugging Python and C/C++ with TotalView
Transforming the Stack
Viewing and Comparing Python and C/C++ Variables
Leveraging Other Debugging Technologies for Python debugging
Supported Python Extension Technologies for Stack Transformations
Chapter 8 Using the Command Line Interface (CLI)
Access to the CLI
Introduction to the CLI
About the CLI and Tcl
Integration of the CLI and NextGen TotalView for HPC
Invoking CLI Commands
Starting the CLI in a Terminal Window
Startup Example
Starting Your Program
About CLI Output
‘more’Processing
Using Command Arguments
Using Namespaces
About the CLI Prompt
Using Built-in and Group Aliases
How Parallelism Affects Behavior
Types of IDs
Controlling Program Execution
Advancing Program Execution
Using Action Points
Examples of Using the CLI
Setting the CLI EXECUTABLE_PATH Variable
Initializing an Array Slice
Printing an Array Slice
Writing an Array Variable to a File
Automatically Setting Breakpoints
Chapter 9 Controlling fork, vfork, and execve Handling
The exec_handling and fork_handling Command Options and State Variables
Exec Handling
Fork Handling
Example
Chapter 10 ReplayEngine
How ReplayEngine Works
Play It Backwards
The Process of Recording and Playback
System Resource Issues
Replaying Your Program
Threads and Processes
Attaching to Running Programs
Saving and Loading the Execution History
Using ReplayEngine
Enabling and Disabling ReplayEngine
Enabling ReplayEngine at Program Load
Enabling and Disabling ReplayEngine for a Loaded Program
Examining Program State and History
Replay Bookmarks
Creating bookmarks
Activating bookmarks
Setting Preferences for ReplayEngine
CLI Support
Known Limitations and Issues
Limitations
Performance Issues
Chapter 11 Reverse Connections
About Reverse Connections
Reverse Connection Environment Variables
TV_REVERSE_CONNECT_DIR
TV_CONNECT_OPTIONS
Starting a Reverse Connect Session
Listening for Reverse Connections
Reverse Connect Examples
CLI Example
MPI Batch Script Example
Troubleshooting Reverse Connections
Stale Files in the Reverse Connect Directory
Directory Permissions
User ID Issues
Reverse Connect Directory Environment Variable
Chapter 12 Preferences
About Preferences
Action Points
Display Settings
Tool Bar
Search Path
Parallel Configuration
PART III Using the CUDA Debugger
Chapter 13 About the TotalView CUDA Debugger
Overview
Installing the CUDA SDK Tool Chain
Directive-Based Accelerator Programming Languages
Chapter 14 CUDA Debugging Model and Unified Display
The TotalView CUDA Debugging Model
Pending and Sliding Breakpoints
Unified Source View and Breakpoint Display
Chapter 15 CUDA Debugging Tutorial
Compiling for Debugging
Compiling for Fermi
Compiling for Fermi and Tesla
Compiling for Kepler
Compiling for Pascal
Compiling for Volta
Starting a TotalView CUDA Session
Controlling Execution
Viewing GPU Threads
CUDA Thread IDs and Coordinate Spaces
Single-Stepping GPU Code
Halting a Running Application
Displaying CUDA Program Elements
GPU Assembler Display
GPU Variable and Data Display
Managed Memory Variables
About Managed Memory
How TotalView Displays Managed Variables
CUDA Built-In Runtime Variables
Type Casting
PTX Registers
Enabling CUDA Memory Checker Feature
GPU Core Dump Support
GPU Error Reporting
Chapter 16 CUDA Problems and Limitations
Hangs or Initialization Failures
CUDA and ReplayEngine
Chapter 17 Sample CUDA Program
PART IV Appendices
Appendix A More on Expressions
Overview
Calling Functions: Problems and Issues
Using Built-in Variables and Statements
Using TotalView Variables
Using Built-In Statements
Using Programming Language Elements
Using C and C++
Using Fortran
Fortran Statements
Fortran Intrinsics
Appendix B Compiling for Debugging
Overview
Compiling with Debugging Symbols
Maintaining Debug Information Separate from an Executable
Controlling Separate Debug Files
Searching for the Debug Files
Appendix C Platform-Specific Topics
Overview
Swap Space
Shared Libraries
Changing Linkage Table Entries and LD_BIND_NOW
Linking with the dbfork Library
Linux or Mac OS X
Appendix D Resources
TotalView for HPC Documentation
Conventions
Contacting Us
NextGen TotalView for HPC Reference Guide
About this Guide
About this Guide
Overview
Resources
PART I Using the CLI
Chapter 1 CLI Command Summary
Chapter 2 CLI Commands
Command Overview
General CLI Commands
CLI Initialization and Termination Commands
Program Information Commands
Execution Control Commands
Action Points
Platform-Specific CLI Commands
Other Commands
alias
capture
dactions
dassign
dattach
dbarrier
dbreak
dcache
dcalltree
dcheckpoint
dcont
dcuda
ddelete
ddetach
ddisable
ddlopen
ddown
denable
dexamine
dflush
dfocus
dga
dgo
dgroups
dhalt
dheap
dhistory
dhold
dkill
dlappend
dlist
dload
dmstat
dnext
dnexti
dout
dprint
dptsets
drerun
drestart
drun
dsession
dset
dskip
dstacktransform
dstatus
dstep
dstepi
dunhold
dunset
duntil
dup
dwait
dwatch
dwhat
dwhere
dworker
exit
help
quit
spurs
stty
unalias
Chapter 3 CLI Namespace Commands
Command Overview
Accessor Functions
Helper Functions
actionpoint
dec2hex
dll
errorCodes
expr
focus_groups
focus_processes
focus_threads
group
hex2dec
process
read_symbols
respond
scope
source_process_startup
symbol
thread
type
type_transformation
Chapter 4 Batch Debugging Using tvscript
Overview
tvscript Command Syntax
tvscript Options
tvscript External Script Files
Logging Functions API
Process Functions API
Thread Functions API
Action Point API
Event API
Example tvscript Script File
Chapter 5 TotalView Variables
Overview
Top-Level (::) Namespace
TV:: Namespace
TV::MEMDEBUG:: Namespace
TV::GUI:: Namespace
PART II Transformations
Chapter 6 Creating Type Transformations
Overview
Why Type Transformations
Creating Structure and Class Transformations
Transforming Structures
build_struct_transform Function
Type Transformation Expressions
Using Type Transformations
C++View
Writing a Data Display Function
TV_ttf_type_ascii_string
TV_ttf_type_int
Templates
Precedence - Searching for TV_ttf_display_type
TV_ttf_add_row
TV_ttf_ec_ok
TV_ttf_ec_not_ active
TV_ttf_ec_invalid_characters
TV_ttf_ec_buffer_exhausted
Return values from TV_ttf_display_type
TV_ttf_format_ok
TV_ttf_format_ok_elide
TV_ttf_format_ failed
TV_ttf_format_ raw
TV_ttf_format_ never
Elision
Other Constraints
Safety
Memory Management
Multithreading
Tips and Tricks
Core Files
Using C++View with ReplayEngine
C
Compiling and linking tv_data_display.c
C++View Example Files
Limitations
Licensing
PART III Running TotalView
Chapter 7 TotalView Command Syntax
Overview
Command-Line Syntax
Command-Line Options
PART IV Platforms and Operating Systems
Chapter 8 Platforms and Compilers
Overview
Compiling with Debugging Symbols
Apple Running Mac OS X
IBM AIX on RS/6000 Systems
IBM Blue Gene
IBM Power Linux
Linux Running on an x86 Platform
Linux Running on an x86-64 Platform
Linux Running on an ARM64 Platform
Sun Solaris
Maintaining Debug Information Separate from an Executable
Controlling Separate Debug Files
Searching for the Debug Files
Linking with the dbfork Library
dbfork on IBM AIX on RS/6000 Systems
Linking C++ Programs with dbfork
Linux or Mac OS X
SunOS 5 SPARC
Compiling and Linking Split DWARF
Using GNU DebugFission Split DWARF on Linux
Using Split DWARF on Solaris
Chapter 9 Operating Systems
Operating Systems
Supported Operating Systems
Troubleshooting Mac OS X Installations
Problem Description
For Mac OS X Versions 10.8 (Mountain Lion) or Later
For Mac OS X Versions 10.11 (Capitan) or Later
Remotely Debugging without Console Access
Mounting the /proc File System
Mounting /proc with SunOS 5
Swap Space
Swap Space on IBM AIX
Swap Space on Linux
Swap Space on SunOS 5
Shared Libraries
Changing Linkage Table Entries and LD_BIND_NOW
Debugging Your Program’s Dynamically Loaded Libraries
dlopen Options for Scalability
Filtering dlopen Events
Handling dlopen Events in Parallel
Known Limitations
Remapping Keys
Expression System
Expression System on IBM AIX-Power and Blue Gene/Q
Chapter 10 Architectures
Overview
AMD and Intel x86-64
x86-64 General Registers
x86-64 Floating-Point Registers
x86-64 FPCR Register
Using the x86-64 FPCR Register
x86-64 FPSR Register
x86-64 MXCSR Register
Power Architectures
Power General Registers
Blue Gene Power Registers
Blue Gene/Q QPX Floating-Point Registers
Power MSR Register
Power Floating-Point Registers
Power FPSCR Register
Using the Power FPSCR Register
ARM64
ARM64 General Registers
ARM64 Floating-Point Registers
ARM64 FPCR Register
ARM64 FPSR Register
Intel x86
Intel x86 General Registers
Intel x86 Floating-Point Registers
Intel x86 FPCR Register
Using the Intel x86 FPCR Register
Intel x86 FPSR Register
Intel x86 MXCSR Register
Sun SPARC
SPARC General Registers
SPARC PSR Register
SPARC Floating-Point Registers
SPARC FPSR Register
Using the SPARC FPSR Register
NextGen TotalView for HPC Installation Guide
Chapter 1 Installing NextGen TotalView for HPC
Overview
Installing NextGen TotalView for HPC
Configuring Your Environment
Installing a Demo License
Setting the License File Variable
Chapter 2 License Installation Quick Setup
Overview
Updating an Existing License
FlexNet Update
No FlexNet Update
Installing a New License
Chapter 3 License Setup: Linux PowerLE & ARM64
Overview
Prerequisites
Installing a Demo or Node-Locked License
Renewing a Node-Locked License
Installing a License Server
Renewing a Served License
Uninstalling an FNE License Server
Chapter 4 Installing or Changing a Regular License
Overview
Is My License Valid for this Release?
Planning for a Regular License
Choosing a License Server
Finding a License Server Host ID
Creating a FlexNet User Account
Installing your Permanent License
Accessing a License Server
Starting the License Manager Manually
Stopping the License Manager
Testing the License Manager
Starting the License Manager When the System Boots
Installing FlexNet Standalone
Starting the Install Program
More FlexNet Information
Chapter 5 Installing License Keys
Overview
Step 1: Locating the FlexNet License Server Directory
Step 2: Deleting Older License Keys
Step 3: Adding New License Keys
Adding a New license.dat File
Revising Your license.dat File
Step 4: Starting and Stopping the License Server
Step 5: Configuring NextGen TotalView for HPC to Use the License Server
Reporting Problems with License Installation
Chapter 6 Uninstalling NextGen TotalView for HPC
The Three-Step Removal Process
New in NextGen TotalView for HPC 2019.3Explore NextGen TotalView for HPC
Chapter 6 Uninstalling NextGen TotalView for HPC
GettingStartedPartIntro
GettingStartedFullInterface
GettingStartedCustomizingInterface
Preferences
GettingStartedPreferences
GettingStartedResizing
GettingStartedDockingUndocking
GettingStartedSourceViews
GettingStartedToolbars
GettingStartedLogger
GettingStartedStartPage
SessionsOverview
SessionsSettingUp
SessionsLoadingfromEditor
SessionsStarting
SessionsDebugaProgram
SessionsDebugParallel
SessionsAttachtoProcess
SessionsAttachtoProcessFields
SessionsAttachtoProcessSearch
SessionsAttachtoProcessDebugOptions
SessionsCoreorReplayFile
SessionsLoadRecent
SessionsEditPrevious
SessionsLoadingProgramwithCLI
SessionsOptionsReverseDebugging
SessionsProgramEnvironment
SessionsStandardInandOut
SessionsManagingSessions
SessionsManageSessionsWindow
SessionsManagerDisplaySessionData
SessionsManagerIconsDescription
BasicDebugging
OrganizingProcessesandThreads
TheProcessesandThreadsView
ProcessesandThreadsTreeView
ProcessesandThreadsInteractions
DebuggingCommandWidth
ProcessesandThreadsAttributes
AttributeListButtons
ActionPoints
ActionPointProperties
ActionPointsSetting
ActionPointsWidth
ActionPointsManagingandDiving
ActionPointsSavingToaFileUsingTheCLI
CallStackandData
LocalVariablesView
CallStackandDataViewing
DataViewGeneral
DataViewEditingExpressions
DataViewValue
DataViewCasting
LookupView
CommandViewandCLI
ReplayEngineChapter
ReplayingYourProgram
ReplayBookmarks
ReverseConnect
CudaToolbars
TypeTransformations
This site works best with JavaScript enabled