Next batch of Doxygen updates. (#1180)
* Sketch of the H5S life cycle. * Committing clang-format changes * Fix H5S_UNLIMITED snafu. * Updated RM template and RM page. * Added H5S life cycle. * Committing clang-format changes * Added H5T life cycle. * Committing clang-format changes * Cleaner layout (?) * Cleaned the H5F life cycle. Called out unfinished biz. * Committing clang-format changes * Remaining life cycle skeletons. * Committing clang-format changes * Committing clang-format changes * Added H5Z life cycle. * Committing clang-format changes * Added H5G life cycle. * Committing clang-format changes * H5 and H5I life cycle updates. * Committing clang-format changes * Added H5PL life cycle. * Committing clang-format changes * Added H5L life cycle. * Committing clang-format changes * Fix for Chris' comment. * Add a variable for Doxygen pre-processor definitions. * Forgot to add the H5M API. * Clarify the H5Z life cycle. * Committing clang-format changes * Add H5Zdevelop.h to Doxygen.in. Added H5I life cycle. * Committing clang-format changes * Clarified introduction and fixed missing label declaration. * Added H5O life cycle. * Committing clang-format changes * H5O cleanup, part 1. * Committing clang-format changes * Cleaned up some of the endless repetition in H5O. * Committing clang-format changes * Cookbook & RFC draft layouts. * Updated manifest. * Updated the manifest, the example paths, and sketched the 1st recipe. * Committing clang-format changes * Outlined two more recipes. * Committing clang-format changes * More recipes and RFCs. * Committing clang-format changes * Draft of templatized RFC references. * Another batch of RFC changes. * Another batch of RFCs. * Fixed reference. * RFCs in reverse chronological order. * First cut of RFCs. * Fixed reference. * Updated recipes. * Updated recipes. * More RFCs. * Updated D*PL comments. * Added H5P descriptions. * Committing clang-format changes * H5R life-cycle snapshot. * Committing clang-format changes * H5R life-cycle. Added line numbers to life-cycle examples. * Committing clang-format changes * Fixed formatting for H5Dchunk_iter(). * Added comment on collective mode requirement w/ compression. * Simplified API compat. macro dox. * More API vers. updates. * Hide the async macro entrails. * Latest VFD SWMR RFC. * Create a tag file for permalinks. * Added TODOs for metadoc. * Removed duplication. * Revised RM landing page. * Trimmed more duplication. * Committing clang-format changes * Revised H5D. * Committing clang-format changes * Updated survey link. * Added Doxygen RM entry template link. * Added the "Multi-Thread HDF5" RFC. * Added DOXYGEN_TAG_FILE. * Added selection I/O RFC. * Added the VFD Sub-filing RFC. * Updated meta-documentation and added two old presentations. * Added a few more RFCs (4). * Fixed MANIFEST. * Updated meta-documentation. * Added Filters technical note. * Fixed MANIFEST. * Restore the path stripper. * Experimental full-text search via Google. * Better full-text search integration. * Whoops. Forgot this one. * Oh boy. * Make CMake happy. * Added "Debugging HDF5 Applications" technical note. * Another batch of RFCs. Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
This commit is contained in:
Gerd Heber
GitHub
392
doxygen/examples/DebuggingHDF5Applications.html
Normal file
392
doxygen/examples/DebuggingHDF5Applications.html
Normal file
@@ -0,0 +1,392 @@
|
||||
<html>
|
||||
<head>
|
||||
<title>Debugging HDF5 Applications</title>
|
||||
|
||||
<h2>Introduction</h2>
|
||||
|
||||
<p>The HDF5 library contains a number of debugging features to
|
||||
make programmers' lives easier including the ability to print
|
||||
detailed error messages, check invariant conditions, display
|
||||
timings and other statistics, and trace API function calls and
|
||||
return values.
|
||||
|
||||
</p><dl>
|
||||
<dt><b>Error Messages</b>
|
||||
</dt><dd>Error messages are normally displayed automatically on the
|
||||
standard error stream and include a stack trace of the library
|
||||
including file names, line numbers, and function names. The
|
||||
application has complete control over how error messages are
|
||||
displayed and can disable the display on a permanent or
|
||||
temporary basis. Refer to the documentation for the H5E error
|
||||
handling package.
|
||||
|
||||
<br><br>
|
||||
</dd><dt><b>Invariant Conditions</b>
|
||||
</dt><dd>Unless <code>NDEBUG</code> is defined during compiling, the
|
||||
library will include code to verify that invariant conditions
|
||||
have the expected values. When a problem is detected the
|
||||
library will display the file and line number within the
|
||||
library and the invariant condition that failed. A core dump
|
||||
may be generated for post mortem debugging. The code to
|
||||
perform these checks can be included on a per-package bases.
|
||||
|
||||
<br><br>
|
||||
</dd><dt><b>Timings and Statistics</b>
|
||||
</dt><dd>The library can be configured to accumulate certain
|
||||
statistics about things like cache performance, datatype
|
||||
conversion, data space conversion, and data filters. The code
|
||||
is included on a per-package basis and enabled at runtime by
|
||||
an environment variable.
|
||||
|
||||
<br><br>
|
||||
</dd><dt><b>API Tracing</b>
|
||||
</dt><dd>All API calls made by an application can be displayed and
|
||||
include formal argument names and actual values and the
|
||||
function return value. This code is also conditionally
|
||||
included at compile time and enabled at runtime.
|
||||
</dd></dl>
|
||||
|
||||
<p>The statistics and tracing can be displayed on any output
|
||||
stream (including streams opened by the shell) with output from
|
||||
different packages even going to different streams.
|
||||
|
||||
</p><h2>Error Messages</h2>
|
||||
|
||||
<p>By default any API function that fails will print an error
|
||||
stack to the standard error stream.
|
||||
|
||||
</p><p>
|
||||
</p><center>
|
||||
<table border="" align="center" width="100%">
|
||||
<tbody><tr>
|
||||
<td>
|
||||
<p><code></code></p><pre><code>
|
||||
HDF5-DIAG: Error detected in thread 0. Back trace follows.
|
||||
#000: H5F.c line 1245 in H5Fopen(): unable to open file
|
||||
major(04): File interface
|
||||
minor(10): Unable to open file
|
||||
#001: H5F.c line 846 in H5F_open(): file does not exist
|
||||
major(04): File interface
|
||||
minor(10): Unable to open file
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
</center>
|
||||
|
||||
<p>The error handling package (H5E) is described
|
||||
<a href="./group___h5_e.html">elsewhere</a>.
|
||||
|
||||
</p><h2>Invariant Conditions</h2>
|
||||
|
||||
<p>To include checks for invariant conditions the library should
|
||||
be configured with <code>--disable-production</code>, the
|
||||
default for versions before 1.2. The library designers have made
|
||||
every attempt to handle error conditions gracefully but an
|
||||
invariant condition assertion may fail in certain cases. The
|
||||
output from a failure usually looks something like this:
|
||||
|
||||
</p><p>
|
||||
</p><center>
|
||||
<table border="" align="center" width="100%">
|
||||
<tbody><tr>
|
||||
<td>
|
||||
<p><code></code></p><pre><code>
|
||||
Assertion failed: H5.c:123: i<NELMTS(H5_debug_g)
|
||||
IOT Trap, core dumped.
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
</center>
|
||||
|
||||
<h2>Timings and Statistics</h2>
|
||||
|
||||
<p>Code to accumulate statistics is included at compile time by
|
||||
using the <code>--enable-debug</code> configure switch. The
|
||||
switch can be followed by an equal sign and a comma-separated
|
||||
list of package names or else a default list is used.
|
||||
|
||||
</p><p>
|
||||
</p><center>
|
||||
<table border="" align="center" width="80%">
|
||||
<tbody><tr>
|
||||
<th>Name</th>
|
||||
<th>Default</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">a</td>
|
||||
<td align="center">No</td>
|
||||
<td>Attributes</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">ac</td>
|
||||
<td align="center">Yes</td>
|
||||
<td>Meta data cache</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">b</td>
|
||||
<td align="center">Yes</td>
|
||||
<td>B-Trees</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">d</td>
|
||||
<td align="center">Yes</td>
|
||||
<td>Datasets</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">e</td>
|
||||
<td align="center">Yes</td>
|
||||
<td>Error handling</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">f</td>
|
||||
<td align="center">Yes</td>
|
||||
<td>Files</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">g</td>
|
||||
<td align="center">Yes</td>
|
||||
<td>Groups</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">hg</td>
|
||||
<td align="center">Yes</td>
|
||||
<td>Global heap</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">hl</td>
|
||||
<td align="center">No</td>
|
||||
<td>Local heaps</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">i</td>
|
||||
<td align="center">Yes</td>
|
||||
<td>Interface abstraction</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">mf</td>
|
||||
<td align="center">No</td>
|
||||
<td>File memory management</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">mm</td>
|
||||
<td align="center">Yes</td>
|
||||
<td>Library memory managment</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">o</td>
|
||||
<td align="center">No</td>
|
||||
<td>Object headers and messages</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">p</td>
|
||||
<td align="center">Yes</td>
|
||||
<td>Property lists</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">s</td>
|
||||
<td align="center">Yes</td>
|
||||
<td>Data spaces</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">t</td>
|
||||
<td align="center">Yes</td>
|
||||
<td>Datatypes</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">v</td>
|
||||
<td align="center">Yes</td>
|
||||
<td>Vectors</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">z</td>
|
||||
<td align="center">Yes</td>
|
||||
<td>Raw data filters</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
</center>
|
||||
|
||||
<p>In addition to including the code at compile time the
|
||||
application must enable each package at runtime. This is done
|
||||
by listing the package names in the <code>HDF5_DEBUG</code>
|
||||
environment variable. That variable may also contain file
|
||||
descriptor numbers (the default is `2') which control the output
|
||||
for all following packages up to the next file number. The
|
||||
word <code>all</code> refers to all packages. Any word my be
|
||||
preceded by a minus sign to turn debugging off for the package.
|
||||
|
||||
</p><p>
|
||||
</p><center>
|
||||
<table border="" align="center" width="100%">
|
||||
<caption align="top"><b>Sample debug specifications</b></caption>
|
||||
<tbody><tr valign="top">
|
||||
<td><code>all</code></td>
|
||||
<td>This causes debugging output from all packages to be
|
||||
sent to the standard error stream.</td>
|
||||
</tr>
|
||||
<tr valign="top">
|
||||
<td><code>all -t -s</code></td>
|
||||
<td>Debugging output for all packages except datatypes
|
||||
and data spaces will appear on the standard error
|
||||
stream.</td>
|
||||
</tr>
|
||||
<tr valign="top">
|
||||
<td><code>-all ac 255 t,s</code></td>
|
||||
<td>This disables all debugging even if the default was to
|
||||
debug something, then output from the meta data cache is
|
||||
send to the standard error stream and output from data
|
||||
types and spaces is sent to file descriptor 255 which
|
||||
should be redirected by the shell.</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
</center>
|
||||
|
||||
<p>The components of the <code>HDF5_DEBUG</code> value may be
|
||||
separated by any non-lowercase letter.
|
||||
|
||||
</p><h2>API Tracing</h2>
|
||||
|
||||
<p>The HDF5 library can trace API calls by printing the
|
||||
function name, the argument names and their values, and the
|
||||
return value. Some people like to see lots of output during
|
||||
program execution instead of using a good symbolic debugger, and
|
||||
this feature is intended for their consumption. For example,
|
||||
the output from <code>h5ls foo</code> after turning on tracing,
|
||||
includes:
|
||||
|
||||
</p><p>
|
||||
</p><center>
|
||||
<table border="" align="center" width="100%">
|
||||
<tbody><tr>
|
||||
<td>
|
||||
<code><pre>
|
||||
H5Tcopy(type=184549388) = 184549419 (type);
|
||||
H5Tcopy(type=184549392) = 184549424 (type);
|
||||
H5Tlock(type=184549424) = SUCCEED;
|
||||
H5Tcopy(type=184549393) = 184549425 (type);
|
||||
H5Tlock(type=184549425) = SUCCEED;
|
||||
H5Fopen(filename="foo", flags=0, access=H5P_DEFAULT) = FAIL;
|
||||
HDF5-DIAG: Error detected in thread 0. Back trace follows.
|
||||
#000: H5F.c line 1245 in H5Fopen(): unable to open file
|
||||
major(04): File interface
|
||||
minor(10): Unable to open file
|
||||
#001: H5F.c line 846 in H5F_open(): file does not exist
|
||||
major(04): File interface
|
||||
minor(10): Unable to open file
|
||||
</pre></code>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
</center>
|
||||
|
||||
<p>The code that performs the tracing must be included in the
|
||||
library by specifying the <code>--enable-trace</code>
|
||||
configuration switch (the default for versions before 1.2). Then
|
||||
the word <code>trace</code> must appear in the value of the
|
||||
<code>HDF5_DEBUG</code> variable. The output will appear on the
|
||||
last file descriptor before the word <code>trace</code> or two
|
||||
(standard error) by default.
|
||||
|
||||
</p><p>
|
||||
</p><center>
|
||||
<table border="" align="center" width="100%">
|
||||
<tbody><tr>
|
||||
<td>To display the trace on the standard error stream:
|
||||
<code><pre>$ env HDF5_DEBUG=trace a.out
|
||||
</pre></code>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>To send the trace to a file:
|
||||
<code><pre>$ env HDF5_DEBUG="55 trace" a.out 55>trace-output
|
||||
</pre></code>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
</center>
|
||||
|
||||
<h3>Performance</h3>
|
||||
|
||||
<p>If the library was not configured for tracing then there is no
|
||||
unnecessary overhead since all tracing code is excluded.
|
||||
However, if tracing is enabled but not used there is a small
|
||||
penalty. First, code size is larger because of extra
|
||||
statically-declared character strings used to store argument
|
||||
types and names and extra auto variable pointer in each
|
||||
function. Also, execution is slower because each function sets
|
||||
and tests a local variable and each API function calls the
|
||||
<code>H5_trace()</code> function.
|
||||
|
||||
</p><p>If tracing is enabled and turned on then the penalties from the
|
||||
previous paragraph apply plus the time required to format each
|
||||
line of tracing information. There is also an extra call to
|
||||
H5_trace() for each API function to print the return value.
|
||||
|
||||
</p><h3>Safety</h3>
|
||||
|
||||
<p>The tracing mechanism is invoked for each API function before
|
||||
arguments are checked for validity. If bad arguments are passed
|
||||
to an API function it could result in a segmentation fault.
|
||||
However, the tracing output is line-buffered so all previous
|
||||
output will appear.
|
||||
|
||||
</p><h3>Completeness</h3>
|
||||
|
||||
<p>There are two API functions that don't participate in
|
||||
tracing. They are <code>H5Eprint()</code> and
|
||||
<code>H5Eprint_cb()</code> because their participation would
|
||||
mess up output during automatic error reporting.
|
||||
|
||||
</p><p>On the other hand, a number of API functions are called during
|
||||
library initialization and they print tracing information.
|
||||
|
||||
</p><h3>Implementation</h3>
|
||||
|
||||
<p>For those interested in the implementation here is a
|
||||
description. Each API function should have a call to one of the
|
||||
<code>H5TRACE()</code> macros immediately after the
|
||||
<code>FUNC_ENTER()</code> macro. The first argument is the
|
||||
return type encoded as a string. The second argument is the
|
||||
types of all the function arguments encoded as a string. The
|
||||
remaining arguments are the function arguments. This macro was
|
||||
designed to be as terse and unobtrousive as possible.
|
||||
|
||||
</p><p>In order to keep the <code>H5TRACE()</code> calls synchronized
|
||||
with the source code we've written a perl script which gets
|
||||
called automatically just before Makefile dependencies are
|
||||
calculated for the file. However, this only works when one is
|
||||
using GNU make. To reinstrument the tracing explicitly, invoke
|
||||
the <code>trace</code> program from the hdf5 bin directory with
|
||||
the names of the source files that need to be updated. If any
|
||||
file needs to be modified then a backup is created by appending
|
||||
a tilde to the file name.
|
||||
|
||||
</p><p>
|
||||
</p><center>
|
||||
<table border="" align="center" width="100%">
|
||||
<caption align="top"><b>Explicit Instrumentation</b></caption>
|
||||
<tbody><tr>
|
||||
<td>
|
||||
<code><pre>
|
||||
$ ../bin/trace *.c
|
||||
H5E.c: in function `H5Ewalk_cb':
|
||||
H5E.c:336: warning: trace info was not inserted
|
||||
</pre></code>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
</center>
|
||||
|
||||
<p>Note: The warning message is the result of a comment of the
|
||||
form <code>/*NO TRACE*/</code> somewhere in the function
|
||||
body. Tracing information will not be updated or inserted if
|
||||
such a comment exists.
|
||||
|
||||
</p><p>Error messages have the same format as a compiler so that they
|
||||
can be parsed from program development environments like
|
||||
Emacs. Any function which generates an error will not be
|
||||
modified.</p>
|
||||
|
||||
</body></html>
|
||||
Reference in New Issue
Block a user