mirror of
https://github.com/ioacademy-jikim/debugging
synced 2025-06-21 14:56:21 +00:00
401 lines
16 KiB
XML
401 lines
16 KiB
XML
<?xml version="1.0"?> <!-- -*- sgml -*- -->
|
|
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
|
|
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
|
|
[ <!ENTITY % vg-entities SYSTEM "../../docs/xml/vg-entities.xml"> %vg-entities; ]>
|
|
|
|
|
|
<chapter id="dh-manual"
|
|
xreflabel="DHAT: a dynamic heap analysis tool">
|
|
<title>DHAT: a dynamic heap analysis tool</title>
|
|
|
|
<para>To use this tool, you must specify
|
|
<option>--tool=exp-dhat</option> on the Valgrind
|
|
command line.</para>
|
|
|
|
|
|
|
|
<sect1 id="dh-manual.overview" xreflabel="Overview">
|
|
<title>Overview</title>
|
|
|
|
<para>DHAT is a tool for examining how programs use their heap
|
|
allocations.</para>
|
|
|
|
<para>It tracks the allocated blocks, and inspects every memory access
|
|
to find which block, if any, it is to. The following data is
|
|
collected and presented per allocation point (allocation
|
|
stack):</para>
|
|
|
|
<itemizedlist>
|
|
<listitem><para>Total allocation (number of bytes and
|
|
blocks)</para></listitem>
|
|
|
|
<listitem><para>maximum live volume (number of bytes and
|
|
blocks)</para></listitem>
|
|
|
|
<listitem><para>average block lifetime (number of instructions
|
|
between allocation and freeing)</para></listitem>
|
|
|
|
<listitem><para>average number of reads and writes to each byte in
|
|
the block ("access ratios")</para></listitem>
|
|
|
|
<listitem><para>for allocation points which always allocate blocks
|
|
only of one size, and that size is 4096 bytes or less: counts
|
|
showing how often each byte offset inside the block is
|
|
accessed.</para></listitem>
|
|
</itemizedlist>
|
|
|
|
<para>Using these statistics it is possible to identify allocation
|
|
points with the following characteristics:</para>
|
|
|
|
<itemizedlist>
|
|
|
|
<listitem><para>potential process-lifetime leaks: blocks allocated
|
|
by the point just accumulate, and are freed only at the end of the
|
|
run.</para></listitem>
|
|
|
|
<listitem><para>excessive turnover: points which chew through a lot
|
|
of heap, even if it is not held onto for very long</para></listitem>
|
|
|
|
<listitem><para>excessively transient: points which allocate very
|
|
short lived blocks</para></listitem>
|
|
|
|
<listitem><para>useless or underused allocations: blocks which are
|
|
allocated but not completely filled in, or are filled in but not
|
|
subsequently read.</para></listitem>
|
|
|
|
<listitem><para>blocks with inefficient layout -- areas never
|
|
accessed, or with hot fields scattered throughout the
|
|
block.</para></listitem>
|
|
</itemizedlist>
|
|
|
|
<para>As with the Massif heap profiler, DHAT measures program progress
|
|
by counting instructions, and so presents all age/time related figures
|
|
as instruction counts. This sounds a little odd at first, but it
|
|
makes runs repeatable in a way which is not possible if CPU time is
|
|
used.</para>
|
|
|
|
</sect1>
|
|
|
|
|
|
|
|
|
|
<sect1 id="dh-manual.understanding" xreflabel="Understanding DHAT's output">
|
|
<title>Understanding DHAT's output</title>
|
|
|
|
|
|
<para>DHAT provides a lot of useful information on dynamic heap usage.
|
|
Most of the art of using it is in interpretation of the resulting
|
|
numbers. That is best illustrated via a set of examples.</para>
|
|
|
|
|
|
<sect2>
|
|
<title>Interpreting the max-live, tot-alloc and deaths fields</title>
|
|
|
|
<sect3><title>A simple example</title></sect3>
|
|
|
|
<screen><![CDATA[
|
|
======== SUMMARY STATISTICS ========
|
|
|
|
guest_insns: 1,045,339,534
|
|
[...]
|
|
max-live: 63,490 in 984 blocks
|
|
tot-alloc: 1,904,700 in 29,520 blocks (avg size 64.52)
|
|
deaths: 29,520, at avg age 22,227,424
|
|
acc-ratios: 6.37 rd, 1.14 wr (12,141,526 b-read, 2,174,460 b-written)
|
|
at 0x4C275B8: malloc (vg_replace_malloc.c:236)
|
|
by 0x40350E: tcc_malloc (tinycc.c:6712)
|
|
by 0x404580: tok_alloc_new (tinycc.c:7151)
|
|
by 0x40870A: next_nomacro1 (tinycc.c:9305)
|
|
]]></screen>
|
|
|
|
<para>Over the entire run of the program, this stack (allocation
|
|
point) allocated 29,520 blocks in total, containing 1,904,700 bytes in
|
|
total. By looking at the max-live data, we see that not many blocks
|
|
were simultaneously live, though: at the peak, there were 63,490
|
|
allocated bytes in 984 blocks. This tells us that the program is
|
|
steadily freeing such blocks as it runs, rather than hanging on to all
|
|
of them until the end and freeing them all.</para>
|
|
|
|
<para>The deaths entry tells us that 29,520 blocks allocated by this stack
|
|
died (were freed) during the run of the program. Since 29,520 is
|
|
also the number of blocks allocated in total, that tells us that
|
|
all allocated blocks were freed by the end of the program.</para>
|
|
|
|
<para>It also tells us that the average age at death was 22,227,424
|
|
instructions. From the summary statistics we see that the program ran
|
|
for 1,045,339,534 instructions, and so the average age at death is
|
|
about 2% of the program's total run time.</para>
|
|
|
|
<sect3><title>Example of a potential process-lifetime leak</title></sect3>
|
|
|
|
<para>This next example (from a different program than the above)
|
|
shows a potential process lifetime leak. A process lifetime leak
|
|
occurs when a program keeps allocating data, but only frees the
|
|
data just before it exits. Hence the program's heap grows constantly
|
|
in size, yet Memcheck reports no leak, because the program has
|
|
freed up everything at exit. This is particularly a hazard for
|
|
long running programs.</para>
|
|
|
|
<screen><![CDATA[
|
|
======== SUMMARY STATISTICS ========
|
|
|
|
guest_insns: 418,901,537
|
|
[...]
|
|
max-live: 32,512 in 254 blocks
|
|
tot-alloc: 32,512 in 254 blocks (avg size 128.00)
|
|
deaths: 254, at avg age 300,467,389
|
|
acc-ratios: 0.26 rd, 0.20 wr (8,756 b-read, 6,604 b-written)
|
|
at 0x4C275B8: malloc (vg_replace_malloc.c:236)
|
|
by 0x4C27632: realloc (vg_replace_malloc.c:525)
|
|
by 0x56FF41D: QtFontStyle::pixelSize(unsigned short, bool) (qfontdatabase.cpp:269)
|
|
by 0x5700D69: loadFontConfig() (qfontdatabase_x11.cpp:1146)
|
|
]]></screen>
|
|
|
|
<para>There are two tell-tale signs that this might be a
|
|
process-lifetime leak. Firstly, the max-live and tot-alloc numbers
|
|
are identical. The only way that can happen is if these blocks are
|
|
all allocated and then all deallocated.</para>
|
|
|
|
<para>Secondly, the average age at death (300 million insns) is 71% of
|
|
the total program lifetime (419 million insns), hence this is not a
|
|
transient allocation-free spike -- rather, it is spread out over a
|
|
large part of the entire run. One interpretation is, roughly, that
|
|
all 254 blocks were allocated in the first half of the run, held onto
|
|
for the second half, and then freed just before exit.</para>
|
|
|
|
</sect2>
|
|
|
|
|
|
<sect2>
|
|
<title>Interpreting the acc-ratios fields</title>
|
|
|
|
|
|
<sect3><title>A fairly harmless allocation point record</title></sect3>
|
|
|
|
<screen><![CDATA[
|
|
max-live: 49,398 in 808 blocks
|
|
tot-alloc: 1,481,940 in 24,240 blocks (avg size 61.13)
|
|
deaths: 24,240, at avg age 34,611,026
|
|
acc-ratios: 2.13 rd, 0.91 wr (3,166,650 b-read, 1,358,820 b-written)
|
|
at 0x4C275B8: malloc (vg_replace_malloc.c:236)
|
|
by 0x40350E: tcc_malloc (tinycc.c:6712)
|
|
by 0x404580: tok_alloc_new (tinycc.c:7151)
|
|
by 0x4046C4: tok_alloc (tinycc.c:7190)
|
|
]]></screen>
|
|
|
|
<para>The acc-ratios field tells us that each byte in the blocks
|
|
allocated here is read an average of 2.13 times before the block is
|
|
deallocated. Given that the blocks have an average age at death of
|
|
34,611,026, that's one read per block per approximately every 15
|
|
million instructions. So from that standpoint the blocks aren't
|
|
"working" very hard.</para>
|
|
|
|
<para>More interesting is the write ratio: each byte is written an
|
|
average of 0.91 times. This tells us that some parts of the allocated
|
|
blocks are never written, at least 9% on average. To completely
|
|
initialise the block would require writing each byte at least once,
|
|
and that would give a write ratio of 1.0. The fact that some block
|
|
areas are evidently unused might point to data alignment holes or
|
|
other layout inefficiencies.</para>
|
|
|
|
<para>Well, at least all the blocks are freed (24,240 allocations,
|
|
24,240 deaths).</para>
|
|
|
|
<para>If all the blocks had been the same size, DHAT would also show
|
|
the access counts by block offset, so we could see where exactly these
|
|
unused areas are. However, that isn't the case: the blocks have
|
|
varying sizes, so DHAT can't perform such an analysis. We can see
|
|
that they must have varying sizes since the average block size, 61.13,
|
|
isn't a whole number.</para>
|
|
|
|
|
|
<sect3><title>A more suspicious looking example</title></sect3>
|
|
|
|
<screen><![CDATA[
|
|
max-live: 180,224 in 22 blocks
|
|
tot-alloc: 180,224 in 22 blocks (avg size 8192.00)
|
|
deaths: none (none of these blocks were freed)
|
|
acc-ratios: 0.00 rd, 0.00 wr (0 b-read, 0 b-written)
|
|
at 0x4C275B8: malloc (vg_replace_malloc.c:236)
|
|
by 0x40350E: tcc_malloc (tinycc.c:6712)
|
|
by 0x40369C: __sym_malloc (tinycc.c:6787)
|
|
by 0x403711: sym_malloc (tinycc.c:6805)
|
|
]]></screen>
|
|
|
|
<para>Here, both the read and write access ratios are zero. Hence
|
|
this point is allocating blocks which are never used, neither read nor
|
|
written. Indeed, they are also not freed ("deaths: none") and are
|
|
simply leaked. So, here is 180k of completely useless allocation that
|
|
could be removed.</para>
|
|
|
|
<para>Re-running with Memcheck does indeed report the same leak. What
|
|
DHAT can tell us, that Memcheck can't, is that not only are the blocks
|
|
leaked, they are also never used.</para>
|
|
|
|
<sect3><title>Another suspicious example</title></sect3>
|
|
|
|
<para>Here's one where blocks are allocated, written to,
|
|
but never read from. We see this immediately from the zero read
|
|
access ratio. They do get freed, though:</para>
|
|
|
|
<screen><![CDATA[
|
|
max-live: 54 in 3 blocks
|
|
tot-alloc: 1,620 in 90 blocks (avg size 18.00)
|
|
deaths: 90, at avg age 34,558,236
|
|
acc-ratios: 0.00 rd, 1.11 wr (0 b-read, 1,800 b-written)
|
|
at 0x4C275B8: malloc (vg_replace_malloc.c:236)
|
|
by 0x40350E: tcc_malloc (tinycc.c:6712)
|
|
by 0x4035BD: tcc_strdup (tinycc.c:6750)
|
|
by 0x41FEBB: tcc_add_sysinclude_path (tinycc.c:20931)
|
|
]]></screen>
|
|
|
|
<para>In the previous two examples, it is easy to see blocks that are
|
|
never written to, or never read from, or some combination of both.
|
|
Unfortunately, in C++ code, the situation is less clear. That's
|
|
because an object's constructor will write to the underlying block,
|
|
and its destructor will read from it. So the block's read and write
|
|
ratios will be non-zero even if the object, once constructed, is never
|
|
used, but only eventually destructed.</para>
|
|
|
|
<para>Really, what we want is to measure only memory accesses in
|
|
between the end of an object's construction and the start of its
|
|
destruction. Unfortunately I do not know of a reliable way to
|
|
determine when those transitions are made.</para>
|
|
|
|
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Interpreting "Aggregated access counts by offset" data</title>
|
|
|
|
<para>For allocation points that always allocate blocks of the same
|
|
size, and which are 4096 bytes or smaller, DHAT counts accesses
|
|
per offset, for example:</para>
|
|
|
|
<screen><![CDATA[
|
|
max-live: 317,408 in 5,668 blocks
|
|
tot-alloc: 317,408 in 5,668 blocks (avg size 56.00)
|
|
deaths: 5,668, at avg age 622,890,597
|
|
acc-ratios: 1.03 rd, 1.28 wr (327,642 b-read, 408,172 b-written)
|
|
at 0x4C275B8: malloc (vg_replace_malloc.c:236)
|
|
by 0x5440C16: QDesignerPropertySheetPrivate::ensureInfo (qhash.h:515)
|
|
by 0x544350B: QDesignerPropertySheet::setVisible (qdesigner_propertysh...)
|
|
by 0x5446232: QDesignerPropertySheet::QDesignerPropertySheet (qdesigne...)
|
|
|
|
Aggregated access counts by offset:
|
|
|
|
[ 0] 28782 28782 28782 28782 28782 28782 28782 28782
|
|
[ 8] 20638 20638 20638 20638 0 0 0 0
|
|
[ 16] 22738 22738 22738 22738 22738 22738 22738 22738
|
|
[ 24] 6013 6013 6013 6013 6013 6013 6013 6013
|
|
[ 32] 18883 18883 18883 37422 0 0 0 0
|
|
[ 36] 5668 11915 5668 5668 11336 11336 11336 11336
|
|
[ 48] 6166 6166 6166 6166 0 0 0 0
|
|
]]></screen>
|
|
|
|
<para>This is fairly typical, for C++ code running on a 64-bit
|
|
platform. Here, we have aggregated access statistics for 5668 blocks,
|
|
all of size 56 bytes. Each byte has been accessed at least 5668
|
|
times, except for offsets 12--15, 36--39 and 52--55. These are likely
|
|
to be alignment holes.</para>
|
|
|
|
<para>Careful interpretation of the numbers reveals useful information.
|
|
Groups of N consecutive identical numbers that begin at an N-aligned
|
|
offset, for N being 2, 4 or 8, are likely to indicate an N-byte object
|
|
in the structure at that point. For example, the first 32 bytes of
|
|
this object are likely to have the layout</para>
|
|
|
|
<screen><![CDATA[
|
|
[0 ] 64-bit type
|
|
[8 ] 32-bit type
|
|
[12] 32-bit alignment hole
|
|
[16] 64-bit type
|
|
[24] 64-bit type
|
|
]]></screen>
|
|
|
|
<para>As a counterexample, it's also clear that, whatever is at offset 32,
|
|
it is not a 32-bit value. That's because the last number of the group
|
|
(37422) is not the same as the first three (18883 18883 18883).</para>
|
|
|
|
<para>This example leads one to enquire (by reading the source code)
|
|
whether the zeroes at 12--15 and 52--55 are alignment holes, and
|
|
whether 48--51 is indeed a 32-bit type. If so, it might be possible
|
|
to place what's at 48--51 at 12--15 instead, which would reduce
|
|
the object size from 56 to 48 bytes.</para>
|
|
|
|
<para>Bear in mind that the above inferences are all only "maybes". That's
|
|
because they are based on dynamic data, not static analysis of the
|
|
object layout. For example, the zeroes might not be alignment
|
|
holes, but rather just parts of the structure which were not used
|
|
at all for this particular run. Experience shows that's unlikely
|
|
to be the case, but it could happen.</para>
|
|
|
|
</sect2>
|
|
|
|
</sect1>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<sect1 id="dh-manual.options" xreflabel="DHAT Command-line Options">
|
|
<title>DHAT Command-line Options</title>
|
|
|
|
<para>DHAT-specific command-line options are:</para>
|
|
|
|
<!-- start of xi:include in the manpage -->
|
|
<variablelist id="dh.opts.list">
|
|
|
|
<varlistentry id="opt.show-top-n" xreflabel="--show-top-n">
|
|
<term>
|
|
<option><![CDATA[--show-top-n=<number>
|
|
[default: 10] ]]></option>
|
|
</term>
|
|
<listitem>
|
|
<para>At the end of the run, DHAT sorts the accumulated
|
|
allocation points according to some metric, and shows the
|
|
highest scoring entries. <varname>--show-top-n</varname>
|
|
controls how many entries are shown. The default of 10 is
|
|
quite small. For realistic applications you will probably need
|
|
to set it much higher, at least several hundred.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry id="opt.sort-by" xreflabel="--sort-by=string">
|
|
<term>
|
|
<option><![CDATA[--sort-by=<string> [default: max-bytes-live] ]]></option>
|
|
</term>
|
|
<listitem>
|
|
<para>At the end of the run, DHAT sorts the accumulated
|
|
allocation points according to some metric, and shows the
|
|
highest scoring entries. <varname>--sort-by</varname>
|
|
selects the metric used for sorting:</para>
|
|
<para><varname>max-bytes-live </varname> maximum live bytes [default]</para>
|
|
<para><varname>tot-bytes-allocd </varname> total allocation (turnover)</para>
|
|
<para><varname>max-blocks-live </varname> maximum live blocks</para>
|
|
<para>This controls the order in which allocation points are
|
|
displayed. You can choose to look at allocation points with
|
|
the highest maximum liveness, or the highest total turnover, or
|
|
by the highest number of live blocks. These give usefully
|
|
different pictures of program behaviour. For example, sorting
|
|
by maximum live blocks tends to show up allocation points
|
|
creating large numbers of small objects.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
</variablelist>
|
|
|
|
<para>One important point to note is that each allocation stack counts
|
|
as a seperate allocation point. Because stacks by default have 12
|
|
frames, this tends to spread data out over multiple allocation points.
|
|
You may want to use the flag --num-callers=4 or some such small
|
|
number, to reduce the spreading.</para>
|
|
|
|
<!-- end of xi:include in the manpage -->
|
|
|
|
</sect1>
|
|
|
|
</chapter>
|