HDFFV-8214 - h5repack failed converting small chunked dataset (size < 1K) to contiguous layout.
Description:
h5repack failed converting small chunked dataset (size < 1K) to contiguous layout.
The first case was when chunk dim is bigger than the dataset dim (at leat one), h5repack failed with displaying error stacks.
The other case is when chunk dim is smaller than the dataset dim, h5repack failed to change layout.
Merged from HDF5 trunk 23407
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), emu (solaris-BE), fred (mac64-LE), Windows (32-LE cmake), cmake (jam)
HDFFV-8012 - h5repack changes max dims and cause failure if only "-f none" is used without changing layout for chunked dataset when a chunk dim is bigger than a dataset dim
Description:
"h5repack -f <obj>:NONE <file.h5> out.h5" command failed if source file contains chunked dataset and a chunk dim is bigger than a dataset dim.
Another issue is that the command changed max dims if chunk dim is smaller than the dataset dim.
These issue occurred when dataset size is smaller than 64k (compact size limit)
Fixed them.
Merged from HDF5 trunk r22805
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), cmake (jam)
Convert configure.in to configure.ac.
Align TESTS usage with autotools convention.
Update all references from configure.in to configure.ac
Tested: h5committest
Bring r22646 from trunk to 1.8 branch:
Changes resulting from Klocwork static analysis tool, from Mark Miller
@ LLNL (miller86@llnl.gov).
Tested on:
Mac OS X/64 10.7.4 (amazon) w/debug, C++ & FORTRAN, using gcc 4.7.x
(too minor to require h5committest)
HDFFV-8107 testh5diff will fail if build/test in HDF5 source tree
Description:
This is sub-task for "HDFFV-8105 testh5diff.sh uses the wrong operator (-a) in an if statement."
From the HDFFV-8105's update, h5diff test failed if build&test is performed in HDF5 source tree because 'cp' try to copy test files to self dir.
It's addressed by skipping if cp's src dir and dest dir is same.
Also this applied for all other tools under src/tools dir.
No change to the CMakeLists.txt files because CMake cautions/demands that in-source builds be avoided.
Merged from HDF5 trunk r22617.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), some manual tests as well
Fix h5repacktst (test file generating binary) to address HDFFV-8016 h5repacktst failed in AIX 32bit mode
Description:
Segmentation fault occurred when h5repacktst try to generate a dataset when
system memory is limited to 256MB.
Add error handling to display apporiate message to exit and adjust the test
dataset according to the original size which didn't causing out of memory.
Tested: test was done by limit memory in csh.
jam (linux32-LE), ostrich (linuxppc64-BE)
Fix for HDFFV-7993 - h5repack fails with error "chunk size must be <= maximum dimension size for fixed-sized dimensions"
Description:
Fixed a failure when change the chunk size of a specified chunked dataset with unlimited max dims.
Also took care of converting to contiguous and compact from the dataset.
Test cases were added and tagged with jira#.
Merged from HDF5 trunk r22277.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), Windows (32-LE cmake), Cmake (jam)
Bring r22180, r22181 and r22214 from trunk to 1.8 branch:
Add new "metadata block size" command line option ('-M <x>' or
'--metadata_block_size=<x>') for h5repack.
[With release managers blessing]
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug)
(h5committest upcoming)
Task for HDFFV-7862 - Select data by chunk direction to improve performance in h5repack
Description:
h5repack sometimes became very slow when handling big chunked datasets in
certain cases. (when chunk boundary doesn't match with a hyperslab boundary.)
The main issue was from figuring out a hypeslab without considering chunk
boundary to read from and write to such datasets.
The update was made to figure out a better hyperslab unit with considering
chunk boundary to improve performance for such cases prior to the update.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows
Merge 1.8 and h5dump/tools and tests based on tools library from trunk.
Reduced warnings.
HDFFV-7949:
Remove duplicated functions in h5ls
Tested: local linux,h5committest
HDFFV-7602 - GMQS: HDF5 command tools: Provide framework for reusable test files among tools
Stage2 working for h5repack cmake test.
Description:
Update cmake script to clean up correctly.
Revisit h5repack cmake test script updated from r21870.
Also sync with HDF5 trunk. (Jira 2605)
Tested:
cmake test for Windows (32-LE) and jam (linux32-LE)
Fix for HDFFV-7840 h5repack: memory leak over one of the h5diff test file
Description:
Turned out that there were two causes of memory leaks.
1. for handling variable length string in attribute.
2. for handling compound type with non-reference members.
The first issue is fixed in copy_attr() which is updated to use h5tools_detect_vlen to take care of vlen string as well as vlen data.
The second is fixed in copy_refs_attr() of compound handling code.
Merged from HDF5 trunk r21869.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), Cmake (jam)
h5dump was used in test script but was not invoked by RUNSERIAL.
THis does not work in batch machines like Blue Gene in LLNL.
Solution:
Added $RUNSERIAL to invoke $H5DUMP_BIN.
Tested:
LLNL BlueGene (udawn)
HDFFV-5932 - h5repack breaks files with dimension scales
Description:
- Fixed h5repack to update values of references(object and region) of
attributes in h5repack for 1) references, 2) ARRAY of references,
3) VLEN of references, and 4) COMPOUND of references.
- Merged from HDF5 trunk 21393, 21382, 21386, 21389. (support Peter)
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE)
HDFFV-2511 - GMQS: h5repack: some tests failed in -q32 AIX
Description:
- Need to be Merged from HDF5 trunk r21283 also, which was missed from previous checkin.
- Previously Merged from HDF5 trunk r21265
- Added macro to handle failure in H5Dread/write when memory allocation
failed inside the library. Add graceful error message when lack of
memory error occurs from HDF5 lib. (XCAO)
Tested:
jam (linux32-LE), koala (linux64-LE)
HDFFV-2511 - GMQS: h5repack: some tests failed in -q32 AIX
Description:
- Merged from HDF5 trunk r21265
- Added macro to handle failure in H5Dread/write when memory allocation
failed inside the library (XCAO)
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE)
Merged from HDF5 trunk r21105
Fixed two bugs:
- h5repack: h5repack failed to copy dataset if the layout is changed from c
hunked with unlimited dims to contiguous. (PC -- 2011/07/15)
- h5diff: "--delta" option considers two NaN of the same type are different
, which is wrong based on http://www.hdfgroup.org/HDF5/doc/RM/Tools.html#Tools-Diff. (PC -- 2011/07/15)
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE)
Bring half of r21125 from trunk to 1.8 branch:
Correct char arrays that were too small for strings defined.
Tested on:
FreeBSD/32 8.2 (loyalty) w/debug
(h5committested on trunk)
Work for HDFFV-7602 - HDF5 command tools: Provide framework for reusable
test files among tools
Description:
Merged from HDF5 trunk r21057
Provide framework to share test files among tools for tools test.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE)
Fix HDFFV-7592
"Remove PACKEDBITS option from autotools files"
Description:
The code for h5dump's packed bits feature was
conditionally compiled in via this option during its
'beta' release in 1.8.5. Now that the feature is fully
supported, h5dump has since been changed to always
compile the packed bits code (having removed the #ifdef
H5_HAVE_H5DUMP_PACKED_BITS checks), so this option in
configure is no longer needed.
This commit removes the --enable-h5dump-packed-bits
configure option, including the help summary from
'./configure --help' and the libhdf5.settings output
summary, the PACKED_BITS Makefile variable, and the
definition of H5_HAVE_H5DUMP_PACKED_BITS define in
appropriate pubconf header files (which has already
been removed from use in source code as of r20722).
Note that this option was never added to the trunk,
thus this commit only need occur on the 1.8 branch.
Tested:
- Configure output, help, & summary verified on jam
- h5committested
