GPU Error Reporting
By default, TotalView reports GPU exception errors as "signals." Continuing the application after these errors can lead to application termination or unpredictable results.
Table 4 lists reported errors, according to these platforms and settings:
Exception codes Lane Illegal Address and Lane Misaligned Address are detected using all supported SDK versions when CUDA memcheck is enabled, on supported Tesla and Fermi hardware.All other CUDA errors are detected only for GPUs with sm_20 or higher (for example Fermi) running SDK 3.1 or higher. It is not necessary to enable CUDA memcheck to detect these errors.Exception code | Error Precision | Error Scope | Description |
|---|---|---|---|
CUDA_EXCEPTION_0: “Device Unknown Exception” | Not precise | Global error on the GPU | An application-caused global GPU error that does not match any of the listed error codes below. |
CUDA_EXCEPTION_1: “Lane Illegal Address” | Precise (Requires memcheck on) | Per lane/thread error | A thread has accessed an illegal (out of bounds) global address. |
CUDA_EXCEPTION_2: “Lane User Stack Overflow” | Precise | Per lane/thread error | A thread has exceeded its stack memory limit. |
CUDA_EXCEPTION_3: “Device Hardware Stack Overflow” | Not precise | Global error on the GPU | The application has triggered a global hardware stack overflow, usually caused by large amounts of divergence in the presence of function calls. |
CUDA_EXCEPTION_4: “Warp Illegal Instruction” | Not precise | Warp error | A thread within a warp has executed an illegal instruction. |
CUDA_EXCEPTION_5: “Warp Out-of-range Address” | Not precise | Warp error | A thread within a warp has accessed an address that is outside the valid range of local or shared memory regions. |
CUDA_EXCEPTION_6: “Warp Misaligned Address” | Not precise | Warp error | A thread within a warp has accessed an incorrectly aligned address in the local or shared memory segments. |
CUDA_EXCEPTION_7: “Warp Invalid Address Space” | Not precise | Warp error | A thread within a warp has executed an instruction that attempts to access a memory space not permitted for that instruction. |
CUDA_EXCEPTION_8: “Warp Invalid PC” | Not precise | Warp error | A thread within a warp has advanced its PC beyond the 40-bit address space. |
CUDA_EXCEPTION_9: “Warp Hardware Stack Overflow” | Not precise | Warp error | A thread within a warp has triggered a hardware stack overflow. |
CUDA_EXCEPTION_10: “Device Illegal Address” | Not precise | Global error | A thread has accessed an illegal (out of bounds) global address. For increased precision, enable memcheck. |
CUDA_EXCEPTION_11: “Lane Misaligned Address” | Precise (Requires memcheck on) | Per lane/thread error | A thread has accessed an incorrectly aligned global address. |
CUDA_EXCEPTION_12 : "Warp Assert" | Precise | Per warp | Any thread in the warp has hit a device side assertion. |
CUDA_EXCEPTION_13: "Lane Syscall Error" | Precise (Requires memcheck on) | Per lane/thread error | A thread has corrupted the heap by invoking free with an invalid address (for example, trying to free the same memory region twice) |
CUDA_EXCEPTION_14 : "Warp Illegal Address" | Not precise | Per warp | A thread has accessed an illegal (out of bounds) global/local/shared address. For increased precision, enable the CUDA memcheck option. See Enabling CUDA MemoryChecker Feature. |
CUDA_EXCEPTION_15 : "Invalid Managed Memory Access" | Precise | Per host thread | A host thread has attempted to access managed memory currently used by the GPU. |
