GPU Error Reporting

By default, TotalView reports GPU exception errors as "signals". Continuing the application after these errors can lead to application termination or unpredictable results.

Table 2 lists reported errors, according to these platforms and settings:

Exception codes Lane Illegal Address and Lane Misaligned Address are detected using all supported SDK versions when CUDA memcheck is enabled, on supported Tesla and Fermi hardware.

All other CUDA errors are detected only for GPUs with sm_20 or higher (for example Fermi) running SDK 3.1 or higher. It is not necessary to enable CUDA memcheck to detect these errors.

 

Table 2: CUDA Exception Codes

Exception code	Error Precision	Error Scope	Description
CUDA_EXCEPTION_0: “Device Unknown Exception”	Not precise	Global error on the GPU	An application-caused global GPU error that does not match any of the listed error codes below.
CUDA_EXCEPTION_1: “Lane Illegal Address”	Precise (Requires memcheck on)	Per lane/thread error	A thread has accessed an illegal (out of bounds) global address.
CUDA_EXCEPTION_2: “Lane User Stack Overflow”	Precise	Per lane/thread error	A thread has exceeded its stack memory limit.
CUDA_EXCEPTION_3: “Device Hardware Stack Overflow”	Not precise	Global error on the GPU	The application has triggered a global hardware stack overflow, usually caused by large amounts of divergence in the presence of function calls.
CUDA_EXCEPTION_4: “Warp Illegal Instruction”	Not precise	Warp error	A thread within a warp has executed an illegal instruction.
CUDA_EXCEPTION_5: “Warp Out-of-range Address”	Not precise	Warp error	A thread within a warp has accessed an address that is outside the valid range of local or shared memory regions.
CUDA_EXCEPTION_6: “Warp Misaligned Address”	Not precise	Warp error	A thread within a warp has accessed an incorrectly aligned address in the local or shared memory segments.
CUDA_EXCEPTION_7: “Warp Invalid Address Space”	Not precise	Warp error	A thread within a warp has executed an instruction that attempts to access a memory space not permitted for that instruction.
CUDA_EXCEPTION_8: “Warp Invalid PC”	Not precise	Warp error	A thread within a warp has advanced its PC beyond the 40-bit address space.
CUDA_EXCEPTION_9: “Warp Hardware Stack Overflow”	Not precise	Warp error	A thread within a warp has triggered a hardware stack overflow.
CUDA_EXCEPTION_10: “Device Illegal Address”	Not precise	Global error	A thread has accessed an illegal (out of bounds) global address. For increased precision, enable memcheck.
CUDA_EXCEPTION_11: “Lane Misaligned Address”	Precise (Requires memcheck on)	Per lane/thread error	A thread has accessed an incorrectly aligned global address.