Task for HDFFV-7862 - Select data by chunk direction to improve performance in h5repack
Description:
h5repack sometimes became very slow when handling big chunked datasets in
certain cases. (when chunk boundary doesn't match with a hyperslab boundary.)
The main issue was from figuring out a hypeslab without considering chunk
boundary to read from and write to such datasets.
The update was made to figure out a better hyperslab unit with considering
chunk boundary to improve performance for such cases prior to the update.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows
Fix for HDFFV-7836 h5diff: displays error stack message for comparing the two dangling symlink with follow-symlinks option
Description:
While ago, to improve performance, skipping same object checking
(h5tools_is_obj_same()) was added.
However the checking function doesn't understand about the dangling link and
caused the issue.
Since handling dangling link code section already implemented, move the
checking function after handling dangling-links to address the problem.
Test was added and tagged with jira#.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), Cmake (jam)
HDFFV-7770 - h5diff segmentation fault on JPSS files
Description:
Fixed segfault over non-comparable attribute with different dimention or
rank, along with '-c' option to display details.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows, Cmake
Description:
Added new H5SL_TYPE_GENERIC skip list type, which uses void *'s as keys and a
client-supplied callback for key comparison. This was added to support the
upcoming "merge named datatype" feature for H5Ocopy, but may be used in other
places as well. Also added testing.
Also fixed a potential bug with the H5SL_TYPE_OBJ implementation, and added
testing for that.
Tested: jam, koala, heiwa (h5committest), durandal
Added a funtion to reset dataset & hyperslab buffer size for h5repack from an
environment variable.
This is performance debugging purpose for now.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), Windows (32-LE), cmake
HDFFV-7693 - h5diff produces different behavior between versions: 1.8.3 and 1.8.6
Description:
There were two issues on this.
One was not displaying all the comparable objects if non-comparable
object/attribute exist and compared before comparables ones. This issue
occurred after 1.8.4 release. This is the issue from user.
The other issue was not displaying all the comparable attributes,
if non-comparable object/attribute exist and compared ahead. This issue
were exist even before 1.8.4 release. So it's possible some attribute
comparison may have not displayed correctly in the past, if non-comparable
data were exist in common object.
Fixed h5diff to display all the comparable object and attribute regardless
of non-comparables.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Cmake (jam, Windows)
tempint = (tempint >> packed_data_offset) & packed_data_mask;
The right shift may cause undefined behavior if PACKED_DATA_OFFSET is
32-bit or more. For every kind of native integers, I changed the code
to make it zero if PACKED_DATA_OFFSET is greater than or equal to the
size of integer.
Tested on Mac OS Lion with CLANG compiler and jam, koala, and linew.
Ran bin/reconfigure to update the Makefile.in in directories not part of the fortran directory check=in. Updates Makefile.in due to changes made in configure.in for the Fortran 2003 additions.
Tested on all platforms run under daily tests.
Work for:
HDFFV-7515 - GMQS: h5unjam - incorrect help page and should display when
no file is given.
and
HDFFV-5941 - h5jam: specifying a user block file with a proper HDF5 magic
number will result in a corrupted HDF5 file.
Description:
- Revised command help pages of h5jam and h5unjam. The descriptions
were not up to date and some were missing. (HDFFV-7515)
- Fixed h5jam not to allow specifying an HDF5 formatted file as input
file for -u (user block file) option, because the original HDF5 file
will not be accessible if allows. (HDFFV-5941)
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Cmake (jam, Wondows)
Switch from H5P_DATASET_ACCESS_DEFAULT to H5P_DEFAULT for calls to
H5Rdereference2().
Tested on:
Mac OS X/32 10.6.8 (amazon) w/debug
(too minor to require h5committest)
- h5repack: h5repack failed to copy dataset if the layout is changed from chunked with
unlimited dims to contiguous. (PC -- 2011/07/15)
- h5diff: "--delta" option considers two NaN of the same type are different, which is wrong
based on http://www.hdfgroup.org/HDF5/doc/RM/Tools.html#Tools-Diff. (PC -- 2011/07/15)
HDFFV-5878 - GMQS: h5dump - incorrect output format for array type with multiple elements in an index
Description:
Fixed h5dump to display the first line of each element into correct
position for multiple dimention array type.
Before this fix, the first line of each element in array were displayed
after the last line of previous element without moving to the next
line (+indentation)
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE)
Bring Coverity changes back to trunk:
r20340:
Address coverity issue 248. Set freed pointer to NULL if realloc fails, then check for NULL before attempting to close array of handles stored in memory addressed by pointer.
r20341:
Added initialization udata.path = NULL; udata.visited = NULL; to prevent unintialized use after done: label.
Do C-style initialization AFTER all variable definitions.
r20342:
Fix valgrind error in H5DUMP-XML-tall.h5
r20343:
Addressed Coverity issue 591 - check the return value of H5S_SELECT_ITER_RELEASE.
r20344:
Added initialization udata.path = NULL; to prevent uninitialized used after done: label
r20347:
Purpose: Fix memory leak in overhead.c
Description: Fixed leak in test() in overhead.c where array "had" was never
freed.
r20348:
Addressed Coverity issue 705 - added prototype for resize_configs_are_equal to cache_common.h.
r20392:
Purpose: Fix coverity issues 1677 and 1678
Description: Added return value checking for 4 funciton calls in
H5tools_get_symlink_info().
r20394:
Purpose: Fix coverity issues 1674 and 1675
Description: Modified H5Epush2() and H5E_printf_stack() to keep track of whether
va_start() has been called, and to call va_end() at the end if so.
r20395:
Should be H5MM_calloc(count * sizeof(void*)). Fixed --gh
r20396:
Should be H5MM_calloc(count * sizeof(void*)). Fixed. --gh
Tested on:
Mac OSX/32 10.6.7 (amazon) w/debug & production
(h5committested on branch)
HDFFV-7594 - GMQS: h5dump shows wrong values for H5T_STD_I8LE dataset on Blue-gene (ppc64 linux BE +@)
Description:
For the STD_I8 type data, updated to use 'signed char' instead of 'char' as
'char' could be defined as unsigned on a certain system, which cause h5dump
to display data values incorrectly.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE)
h5dump (and h5stat, h5jam/h5unjam) help option not tested. Added h5dump parse_command_line technique for h5stat and h5jam/unjam and created testfiles.
Tested: local linux
