Debugging Memory Problems with MemoryScape : Chapter 1 Locating Memory Problems : Batch Scripting and Using the CLI : dheap : Red Zones Bounds Checking: dheap –red_zones

Red Zones Bounds Checking: dheap –red_zones
The Red Zones feature helps catch bounds errors and use-after-free errors. The basic idea is that each allocation is placed in its own page. An allocation is positioned so that if an overrun, that is, an access beyond the end of the allocation, is to be detected, the end of the allocation corresponds to the end of the page.
The page following that in which the allocation lies is also allocated, though access to this page is disabled. This page is termed the fence. Should the application attempt to access a location beyond the end of the allocation, that is, in the fence, the operating system sends the target a segment violation signal. This is caught by a signal handler installed by the HIA. The HIA examines the address that caused the violation. If it it lies in the fence, then the HIA raises an overrun bounds error using the normal event mechanism.
If, however, the address does not lie in any region that the HIA "owns," the HIA attempts to replicate what would have happened if the HIA’s signal handler were not in place. If the application had installed a signal handler, then this handler is called. Otherwise, the HIA attempts to perform the default action for the signal. It should be clear from this that the HIA needs to interpose the signal’s API to ensure that it always remains the installed handler as far as the operating system is concerned. At the same time, it needs to present the application with what it expects.
Underruns, or errors where the application attempts to read before the start of an allocation, are handled in a similar way. Here, though, the allocation is positioned so that its start lies at the start of the page, and the fence is positioned to precede the allocation.
One complication that arises concerns overrun detection. The architecture or definition of the allocation routines may require that certain addresses conform to alignment constraints. As a consequence, there may be a conflict between ensuring that the allocation’s start address has the correct alignment, and ensuring that the allocation ends at the end of the page.
Use-after-free errors can also be detected. In this case, when the block is deallocated, the pages are not returned to the operating system. Instead, the HIA changes the state of the allocation’s table entry to indicate that it is in the deallocated state, and then disables access to the page in which the allocation lies. This time, should the application attempt to access the block after it has been deallocated, a signal will be raised. Again, the HIA examines the faulting address to see what it knows about the address, and then either raises an appropriate event for TotalView, or forwards the signal on.
A key feature distinguishing TotalView Red Zones is that they can be engaged and disengaged at will during the course of the target's execution. The settings can be adjusted so that new allocations have different properties from existing allocations. Since Red Zones can be turned on or off, some of the application's requests can be satisfied by the Red Zones allocator, and others by the standard heap manager. The HIA keeps track of which allocator is responsible for, or owns, each block.
The dheap -red_zones [-status [-all]] option displays the current HIA Red Zone settings. By default, dheap -red_zones displays only those settings that can vary in the current mode, so that, for example, in overrun mode, the settings for fences and end positioning are not shown. The dheap
-red_zones -status -all command will cause all settings to be shown, including those that are overridden for the current mode.
Please note that the abbreviation -rz can be used in the CLI for -red_zones.
The dheap -red_zones -stats [<start_addr [<end addr]] option shows statistics relating to the HIA’s Red Zones allocator for the optionally specified address range. If no range is specified the statistics are shown for the entire address space. These are:
*
*
*
*
The same set of statistics are also shown for deallocated blocks. In addition, the space used for each category is also shown as a percentage of the overall space used for Red Zones.
dheap -red_zones -info [<start_addr [<end addr]] shows the Red Zones entries for allocations (and deallocations) lying in the optionally specified range. If no range is specified the entries are shown for the entire address space.
Red Zones is enabled using dheap -red_zones -set on, and disabled with dheap -red_zones -set off. dheap -red_zones -reset allows the HIA to determine its setting using the usual rules.
dheap -red_zones -set_mode sets the HIA in one of several Red Zone modes. When a new allocation is requested, the HIA will override the actual settings for some of the individual controls, and instead use values that correspond to that mode. The settings that are affected are: pre-fence, post-fence, and end-positioning. The other settings, like use-after-free, exit value, and alignment, take their values from the actual settings of those controls.
The Red Zone modes are:
*
dheap -red_zones -set_mode overrun
The settings used are those that allow overruns to be detected. These are: no for pre-fence, yes for post-fence, and yes for end positioned.
*
dheap -red_zones -set_mode underrun
The settings used are those that allow underruns to be detected. These are: yes for pre-fence, no for post-fence, and no for end positioned.
*
dheap -red_zones -set_mode unfenced
The settings used are those that allow use_after_frees to be detected. These are: no for pre-fence, no for post-fence. End positioned is determined from the control's setting.
*
dheap -red_zones -set_mode manual
All settings are determined from their actual values.
Use the dheap -red_zones -set_pre_fence (on | off) | -reset_pre_fence commands to adjust the pre-fence control. However, the setting is ignored unless the mode is manual.
Use the the dheap -red_zones -set_post_fence (on | off) | -reset_post_fence commands to adjust the post-fence control. However, the setting is ignored unless the mode is manual.
To enable the use-after-free control, enter dheap -red_zones -set_use_af-ter_free on. To disable the control enter “off”. If enabled, any subsequent allocations will be tagged such that the allocation and its fences are retained when the block is deallocated. Access to the block is disabled when it is deallocated to allow attempts to access the block to be detected.
The alignment control dheap -red_zones -set_alignment <integer> regulates the alignment of the start address of a block issued by the Red Zones allocator. An alignment of zero indicates that the default alignment for the platform should be used. An alignment of two ensures that any address returned by the Red Zones allocator is a multiple of two. In this case, if the length of the block is odd, then the end of the block will not line up with the end of the page containing the allocation. An alignment of one would be necessary for the end of the block to always correspond to the end of the page.
Adjusting the fence size is done through the dheap -red_zones -set_fence-_size <integer> command. A fence size of zero indicates that the default fence size of one page should be used. If necessary, the fence size is rounded up to the next multiple of the page size. In most cases it should not be necessary to adjust this control. One instance where it may be useful, however, is where it is suspected that a bounds error is a consequence of a badly coded loop, and the stride of the loop is large. In such a case, a larger fence may be helpful.
dheap -red_zones -set_end_aligned (on | off) controls whether the allocation is positioned at the end of the containing page or at its start. The control in the HIA is always updated, though the actual value is ignored in overrun and underrun modes.
Use dheap -red_zones -set_exit_value <integer> to adjust the exit value used if the HIA terminates the target following detection of a Red Zone error. Generally, the application would fail if it is allowed to continue after a Red Zone error has been detected. In order to allow some control over the application’s exit code, the HIA will call exit when an error is detected. The value it passes to exit as a termination code can be controlled, so that if the application is run from scripts the cause for the termination can be determined.
The dheap -red_zones -size_ranges ... option for Red Zones allows the user to restrict the use of Red Zones to allocations of specified sizes. If Red Zones are engaged and size ranges are enabled, the Red Zones allocator will be used if the size of the request lies in one of the defined size ranges. A value is deemed to lie in a range if start <= size <= end.
To make typing a bit easier, -size_ranges can be abbreviated to -sr.
A range having an end of 0 is interpreted as having no upper limit. Thus if the end is 0, the size matches the range if it is at least as large as the start.
The HIA supports a number of size ranges. This allows particular ranges of sizes to be included or excluded. The Red Zones allocator is used if the size of the request lies in any one of these ranges. The HIA does not check to see that ranges don't overlap or are otherwise consistent.
The determination of whether the Red Zones allocator should be used is made at the time of the original allocation. Thus, once an allocator has taken ownership of a block, that allocator is used for the remainder of the block's life. In particular, all realloc operations are handled by the same allocator, irrespective of the size range settings at the time of reallocation.
There are two attributes associated with each range. The first is the in_use” attribute. This is ignored by the HIA, and is provided for the benefit of TotalView. The motivation here is to allow TotalView to keep the state that would otherwise be lost if the target is detached, and then reattached to later.
The second attribute is the “active” attribute. This indicates if the size range is active, and therefore whether it is used by the HIA when determining whether the Red Zones allocator should be used.
The TotalView cli command dheap -red_zones -size_ranges -set on enables size ranges, and dheap -red_zones -size_ranges -set off disables size ranges. If size ranges are disabled, but Red Zones are enabled, the Red Zones allocator will be used for all allocations.
dheap -red_zones -size_ranges -reset unsets the TotalView setting for the enable/disable control.
The dheap -red_zones -size_ranges -status [-all <id_range> command shows the current settings of the size ranges. The absence of an <id_range> is equivalent to an id_range of “0:0”. By default, only “in_use” size ranges are displayed. To display all known ranges, specify -all. <id_range> must be in one of the following formats:
x:y = id's from x to y
:y = id's from 1 to y
x: = id of x and higher
x = id is x
To set a size range identified by <id> to a particular size range the dheap
-red_zones -size_ranges -set_range <id> <size_range> command is used. <size_range> must be in one of the following formats:
x:y = allocations from x to y
:y = allocations from 1 to y
x: = allocations of x and higher
x = allocations of x
To reset an id or range of ids use dheap -red_zones -size_ranges -reset_range <id_range>. <id_range> must be in a format defined above.
The dheap -red_zones -size_ranges -set_in_use (on | off) <id_range> option adjusts the “in_use” attribute of all the size ranges whose ids lie within <id_range>. dheap -red_zones -size_ranges -set_active (on | off) <id_range> adjusts the “active” attribute of all the size ranges whose ids lie within <id_range>.
The following Red Zones command options unset the TotalView settings for these controls:
dheap -red_zones -reset_mode
dheap -red_zones -reset_pre_fence
dheap -red_zones -reset_post_fence
dheap -red_zones -reset_use_after_free
dheap -red_zones -reset_alignment
dheap -red_zones -reset_fence_size
dheap -red_zones -reset_exit_value
dheap -red_zones -reset_end_aligned
When the above commands are entered, the HIA will determine its settings using the values in the TVHEAP_ARGS environment variable, the HIA configuration file, or its default values.

Rogue Wave Software, Inc.
Voice: (303) 473-9118
rwonlinedocs@roguewave.com