Fix for Bug1975 h5diff - support recursive comparison on group when specified as an object
Description:
Merged from hdf5 trunk (r19400).
Compare member objects and groups recursively when two files or groups are specified to be compared. Support parallel diff and handling symbolic links accordingly.
Tested:
jam, amani
Tested with h5committest on amani, jam, and heiwa, and on linew.
-This line, and those below, will be ignored--
M hdf5_1_8/test/Makefile.in
A hdf5_1_8/m4
AM hdf5_1_8/m4/ltsugar.m4
AM hdf5_1_8/m4/libtool.m4
AM hdf5_1_8/m4/ltversion.m4
AM hdf5_1_8/m4/lt~obsolete.m4
AM hdf5_1_8/m4/ltoptions.m4
M hdf5_1_8/configure
M hdf5_1_8/Makefile.in
M hdf5_1_8/testpar/Makefile.in
M hdf5_1_8/configure.in
M hdf5_1_8/src/Makefile.in
M hdf5_1_8/tools/Makefile.in
M hdf5_1_8/tools/misc/Makefile.in
M hdf5_1_8/tools/h5dump/Makefile.in
M hdf5_1_8/tools/h5repack/Makefile.in
M hdf5_1_8/tools/h5jam/Makefile.in
M hdf5_1_8/tools/h5diff/Makefile.in
M hdf5_1_8/tools/lib/Makefile.in
M hdf5_1_8/tools/h5copy/Makefile.in
M hdf5_1_8/tools/h5import/Makefile.in
M hdf5_1_8/tools/h5stat/Makefile.in
M hdf5_1_8/tools/h5ls/Makefile.in
M hdf5_1_8/hl/test/Makefile.in
M hdf5_1_8/hl/tools/Makefile.in
M hdf5_1_8/hl/tools/gif2h5/Makefile.in
M hdf5_1_8/hl/Makefile.in
M hdf5_1_8/hl/src/Makefile.in
M hdf5_1_8/hl/c++/test/Makefile.in
M hdf5_1_8/hl/c++/Makefile.in
M hdf5_1_8/hl/c++/src/Makefile.in
M hdf5_1_8/hl/c++/examples/Makefile.in
M hdf5_1_8/hl/fortran/test/Makefile.in
M hdf5_1_8/hl/fortran/Makefile.in
M hdf5_1_8/hl/fortran/src/Makefile.in
M hdf5_1_8/hl/fortran/examples/Makefile.in
M hdf5_1_8/hl/examples/Makefile.in
M hdf5_1_8/config/commence.am
M hdf5_1_8/MANIFEST
M hdf5_1_8/c++/test/Makefile.in
M hdf5_1_8/c++/Makefile.in
M hdf5_1_8/c++/src/Makefile.in
M hdf5_1_8/c++/examples/Makefile.in
M hdf5_1_8/bin/ltmain.sh
M hdf5_1_8/bin/reconfigure
M hdf5_1_8/perform/Makefile.in
M hdf5_1_8/fortran/test/Makefile.in
M hdf5_1_8/fortran/testpar/Makefile.in
M hdf5_1_8/fortran/Makefile.in
M hdf5_1_8/fortran/src/Makefile.in
M hdf5_1_8/fortran/examples/Makefile.in
M hdf5_1_8/aclocal.m4
M hdf5_1_8/examples/Makefile.in
Corrected use/name of source folder aliases.
Duplicated FindMPI.cmake so that non-c++ compiler is found first (recommemded commands did not work).
Bring r19325 from trunk
Tested: local linux with mpich
Bring r19252 & 19253 from trunk to 1.8 branch:
Bring Coverity changes from branch to trunk:
r19161:
Fixed the part for matching the subset info with dataset
r19189:
BZ1646: h5dump does not check number of dimensions for subsetting parameters against the dataset
Changed subset_t structure from holding hsize_t pointers to holding new subset_d pointers, which hold the original hsize_t pointer + len. this len is then checked against dataset ndims in the handle_dataset function of h5dump.
Changed all references to use new data structure.
Added tests for each subset parameter.
r19190:
Added new h5dump ddl files
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & production
(h5committested on Coverity branch)
Problem:
some launching command like yod (and sometimes mpirun), do not pass
back the exit code of the executables back to the shell. It always returns 0
if it can launch the executable. This cause tests that expect non-zero exit
code to fail incorrectly.
Short term Solution:
Added the feature that if $RUNSERIAL_NOEXITCODE is defined, do not
test conditions that rely on non-zero exit code.
Tests:
dove-linux using RUNSERIAL=thgyod to see the failures and using
$RUNSERIAL_NOEXITCODE=yes to bypass the errors.
Also h5committested.
Rename '--follow-links' to '--follow-symlinks'
Description:
The '--follow-links' option is to follow symbolic links (soft and external).
Make the name more intuitive and specific to the feature.
Merged from hdf5 trunk r18873.
Tested:
jam
Improve configure's large-file support control.
Description:
Modified configure to now attempt to add defines necessary for
supporting largefiles on all systems, instead of solely on linux. This
is in response to user requests to enable largefile support on Solaris
by default, as well as to give extra control on AIX (instead of just
jamming the necessary flag into the config files).
The old --enable-linux-lfs flag is being deprecated in favor of the
--enable-largefile flag (enabled by default), which can be used on all
platforms. --disable-linux-lfs can still be used to disable largefile
support (on linux) when the --enable-largefile flag is not specified.
On systems where large files cannot be supported in this manner,
configure will report as such.
Tested:
h5committest
AIX (NCSA's blue_print machine)
duty, liberty, and linew.
Autotool Upgrade / Libtool Bug Fix
Description:
Updated autotools referenced in bin/reconfigure as follows:
Autoconf 2.64 --> Autoconf 2.65
Automake 1.11 --> Automake 1.11.1
Libtool 2.2.6a --> Libtool 2.2.6b-mcg
The referenced libtool version is a custom version of 2.2.6b. It
has been tweaked to fix a bug in libtool that occurs
when using PGI 10.0 compilers. A check incorrectly categorizes
the C++ compiler as version 1.0 instead of 10.0, and the link
line is subsequently set up incorrectly and fails to compile.
A patch has been made available and will be included in the next
release of libtool, but in the meantime I've applied the patch to a custom
installation as indicated above. This bin/reconfigure now references
the custom installation, and the resulting configure script will correctly
categorize the PGI 10.0 C++ compiler.
Ran bin/reconfigure to regenerate configure and makefiles.
Tested:
jam and amani with PGI 10.0 compilers.
h5committest
Work for bugzilla 1754: h5diff: support comparing through links.
Description:
merged from hdf5 trunk (18164, 18165, 18166, 18167, 18170, 18266, 18273)
Tested:
h5committest (jam, amani and linew)
Additional fix relted to the fix of bug1672.
Description:
The fix of bug1672 caused some changes of output which required modifying
some test cases. After some discussion, restoring the changes of the test
cases was decided. After many experiments for the solution, this fix was
made so the code which improved performance can stay.
Tested on Jam.
The mixed use of RUNTEST (original) and RUNTESTS (new) caused
confusion. E.g., the timings in test/ was still using the old
$RUNTEST. It made more sense to use $RUNTEST which is used
by the dejagnu feature of automake. So, I changed all
$RUNTEST or $RUNTESTS to $RUNEXEC.
config/commence.am & config/conclude.am are the two files
that got changes. Also fixed an error in test/Makefile.am.
The rest are changed by bin/reconfigure.
Tested: h5committested.
Fix Bug1672 - Display garbage value on LE machine for BE data.
Description:
Casuing by calling diff_dataset twice when -r or no option was given.
Change to call it once which fix the problem.
It also improves the performance. (diffing time in half)
According further test, it also occurred on BE machine as well and it
seems occruing only with the BE data attached to Bugzilla.
Don't know how the file was created. Anyway this fix will prevent
from diffing with garbage values in similiar potential case.
Tested on:
linux32 (jam) , linux64 (almani), solaris (linew)
Merge from Trunk
Description:
Merging all of my CFLAGS-related changes from trunk to 1.8
This includes revision #s: 17616, 17625, 17627, 17639, and 17643.
Tested:
h5committest and some additional checks on our FreeBSD as well
as NCSA's machines.
The ph5diff tests printout incorrectly reporting they are running
h5diff when it actually is running ph5diff.
Fixed.
Tested: Jam (parallel), linew (serial)
No H5committest since it is a shell script and Jam tests both
serial and parallel h5diff for Linux while Linew tests Big
endian platform.
" Use "--use-system-epsilon" for system EPSILON
" Use "-p" or "-d" for whatever user's choice of epsilon
" Use "-p 0" or "-d 0" for strict equality (same as default)
Merge from trunk
Description:
Merged revisions 17228 and 17440 from trunk to 1.8
Specifically, this brings support of --enable-static-exec flag into 1.8
Tested:
Manually on jam, linew, smirom, liberty, plus h5committest.
Currently, there is no automatic regression test that exists due to
portability issues. Behavior is both different and undefined on certain
systems (and while 'nm' command seems to exist on all machines, behavior
is confirmed to be different on Mac, possibly others). Solution will be
to set up some sort of framework in daily tests to build statically,
remove shared paths, and verify executables can function.
Updating autotools
Description:
Bring revisions 17345 and 17357 from trunk to 1.8, specifically:
- Automake upgraded to 1.11
- Autoconf upgraded to 2.64
- bin/reconfigure script edited to use new versions (on jam), and
run to generate new configure script and Makefile.in's.
- configure.in script edited to add "_cv_" to all AC_CACHE_VAL strings
(in order to comply with new autoconf standard).
- bin/install-sh script replaced with new version as provided
by automake.
Tested:
- All issues on trunk were resolved, so only tested on jam and linew. Any
other outliers should be caught by daily tests.
Add a run to the h5repack shell script to read a family file
The file used for input is located in the common source tools for testfiles, in tools/testfiles
Modified the h5repack shell script to read files from this location (h5repack reads its input files from a dedicated testfiles location in h5repack/testfiles)
Changed the h5diff open file call to use h5tools_fopen, so that it can open all file drivers
Tested: linux
Skipped the test of
TESTING $H5DIFF -v $SRCFILE9 $SRCFILE10
TOOLTEST h5diff_100.txt -v $FILE9 $FILE10
again because they still hanged.
Do not turn it back on until it is proven fixed.
Also, want to verify if this is the only tests that hang or
if other tests may hang or if any non-thg machines may hang.
Tested:
jam-pp since it is a simple shell-script change.
- Updated bin/reconfigure to use latest version of automake (1.10.2).
Regenerated Makefile.in's by running bin/reconfigure.
- Added libtool version numbers to c++, fortran, hl, hl c++, and hl fortran
libraries.
Tested:
jam, liberty, smirom
1. #1501 (B1) tools bug if dataset is larger than H5TOOLS_BUFSIZE limit.
ISSUE : the tools use the following formula to read by hyperslabs: hyperslab_size[i] = MIN( dim_size[i], H5TOOLS_BUFSIZE / datum_size) where H5TOOLS_BUFSIZE is a constant defined of 1024K. This is OK as long as the datum_size does not exceed 1024K, otherwise we have a hyperslab size of 0 (since 1024K/(greater than 1024K) = 0). This affects h5dump. h5repack, h5diff
SOLUTION: add a check for a 0 size and define as 1 if so.
TEST FOR H5DUMP: Defined a case in the h5dump test generator program of such a type (an array type of doubles with a large array dimension, that was the case the user reported). Since the written file commited in svn would be around 1024K, opted for not writing the data (the part of the code where the hyperslab is defined is executed, since h5dump always reads the files). Defined a macro WRITE_ARRAY to enable such writing if needed. Added a run on the h5dump shell script. Added 2 new files to svn: tools/testfiles/tarray8.ddl, tools/testfiles/tarray8.h5. NOTE: while doing this I thought of adding this dataset case to an existing file, but that would add the large array output to those files (the ddls). The issue is that the file list is increasing.
TEST FOR H5DIFF: for h5diff the check for reading by hyperslabs is H5TOOLS_MALLOCSIZE (128 * H5TOOLS_BUFSIZE) or 128 Mb. This makes it not possible to add such a file to svn, so used the same method as h5dump (only write the dataset if WRITE_ARRAY is defined). As opposed to h5dump, the hyperslab code is NOT executed when the dataset is empty (dataset is not read). Added the new dataset to existing files and shell run (tools/h5diff/testfiles/h5diff_dset1.h5 and tools/h5diff/testfiles/h5diff_dset2.h5 and output in tools/h5diff/testfiles/h5diff_80.txt).
TEST FOR H5REPACK: similar issue as h5diff with the difference that the hyperslab code is run. Added a run to the shell script (with a filter, otherwise the code uses H5Ocopy).
FURTHER ISSUES: the type in question ("double") has a different output cross platforms (e.g on liberty some garbage number is printed at some array locations)
SOLUTION: defined an "int" type for this test. However the printing of such an array has a bogus output at least in one platform (FreeBsd), so eliminated the test run altogether and filed a bug report on this
Bug Fix
Description:
Fixing BZ #1381. The --includedir=DIR configure option, which is used to
spceify installation location of C header files, did not work correctly as
the path was hard-coded into config/commence.am. I'm presuming this is
because an older version of automake didn't know where to put C header
files. In any case, removing this line now defaults the includedir to the
same directory that it is currently hard-coded to, and also fixes the
configure flag which allows for customization of this value.
Tested:
jam, liberty
have 2 expected outputs for 2 h5ls runs depending if run on a big or little endian machine. Configure.in was modified to export a variable carrying endianess information to testh5ls.sh. This script then compares the current run with 2 expected outputs, one for a big-endian machine (linew was used to generate the output), other for little endian (jam was used to generate the output)
the way h5ls prints types, it starts searching for NATIVE types first. One solution would be h5ls not to detect these native types, using for example the same print datatype function that h5dump does, that would make the output look the same on all platforms ("32-bit little-endian integer" would be printed instead). Drawback, this "native" information would not be available. Other solution is to have not one but 2 expected outputs and make the shell script detect the endianess and compare with one output or other
tested: jam, linew
