The ph5diff tests printout incorrectly reporting they are running
h5diff when it actually is running ph5diff.
Fixed.
Tested: Jam (parallel), linew (serial)
No H5committest since it is a shell script and Jam tests both
serial and parallel h5diff for Linux while Linew tests Big
endian platform.
" Use "--use-system-epsilon" for system EPSILON
" Use "-p" or "-d" for whatever user's choice of epsilon
" Use "-p 0" or "-d 0" for strict equality (same as default)
Merge from trunk
Description:
Merged revisions 17228 and 17440 from trunk to 1.8
Specifically, this brings support of --enable-static-exec flag into 1.8
Tested:
Manually on jam, linew, smirom, liberty, plus h5committest.
Currently, there is no automatic regression test that exists due to
portability issues. Behavior is both different and undefined on certain
systems (and while 'nm' command seems to exist on all machines, behavior
is confirmed to be different on Mac, possibly others). Solution will be
to set up some sort of framework in daily tests to build statically,
remove shared paths, and verify executables can function.
Updating autotools
Description:
Bring revisions 17345 and 17357 from trunk to 1.8, specifically:
- Automake upgraded to 1.11
- Autoconf upgraded to 2.64
- bin/reconfigure script edited to use new versions (on jam), and
run to generate new configure script and Makefile.in's.
- configure.in script edited to add "_cv_" to all AC_CACHE_VAL strings
(in order to comply with new autoconf standard).
- bin/install-sh script replaced with new version as provided
by automake.
Tested:
- All issues on trunk were resolved, so only tested on jam and linew. Any
other outliers should be caught by daily tests.
Add a run to the h5repack shell script to read a family file
The file used for input is located in the common source tools for testfiles, in tools/testfiles
Modified the h5repack shell script to read files from this location (h5repack reads its input files from a dedicated testfiles location in h5repack/testfiles)
Changed the h5diff open file call to use h5tools_fopen, so that it can open all file drivers
Tested: linux
Skipped the test of
TESTING $H5DIFF -v $SRCFILE9 $SRCFILE10
TOOLTEST h5diff_100.txt -v $FILE9 $FILE10
again because they still hanged.
Do not turn it back on until it is proven fixed.
Also, want to verify if this is the only tests that hang or
if other tests may hang or if any non-thg machines may hang.
Tested:
jam-pp since it is a simple shell-script change.
- Updated bin/reconfigure to use latest version of automake (1.10.2).
Regenerated Makefile.in's by running bin/reconfigure.
- Added libtool version numbers to c++, fortran, hl, hl c++, and hl fortran
libraries.
Tested:
jam, liberty, smirom
1. #1501 (B1) tools bug if dataset is larger than H5TOOLS_BUFSIZE limit.
ISSUE : the tools use the following formula to read by hyperslabs: hyperslab_size[i] = MIN( dim_size[i], H5TOOLS_BUFSIZE / datum_size) where H5TOOLS_BUFSIZE is a constant defined of 1024K. This is OK as long as the datum_size does not exceed 1024K, otherwise we have a hyperslab size of 0 (since 1024K/(greater than 1024K) = 0). This affects h5dump. h5repack, h5diff
SOLUTION: add a check for a 0 size and define as 1 if so.
TEST FOR H5DUMP: Defined a case in the h5dump test generator program of such a type (an array type of doubles with a large array dimension, that was the case the user reported). Since the written file commited in svn would be around 1024K, opted for not writing the data (the part of the code where the hyperslab is defined is executed, since h5dump always reads the files). Defined a macro WRITE_ARRAY to enable such writing if needed. Added a run on the h5dump shell script. Added 2 new files to svn: tools/testfiles/tarray8.ddl, tools/testfiles/tarray8.h5. NOTE: while doing this I thought of adding this dataset case to an existing file, but that would add the large array output to those files (the ddls). The issue is that the file list is increasing.
TEST FOR H5DIFF: for h5diff the check for reading by hyperslabs is H5TOOLS_MALLOCSIZE (128 * H5TOOLS_BUFSIZE) or 128 Mb. This makes it not possible to add such a file to svn, so used the same method as h5dump (only write the dataset if WRITE_ARRAY is defined). As opposed to h5dump, the hyperslab code is NOT executed when the dataset is empty (dataset is not read). Added the new dataset to existing files and shell run (tools/h5diff/testfiles/h5diff_dset1.h5 and tools/h5diff/testfiles/h5diff_dset2.h5 and output in tools/h5diff/testfiles/h5diff_80.txt).
TEST FOR H5REPACK: similar issue as h5diff with the difference that the hyperslab code is run. Added a run to the shell script (with a filter, otherwise the code uses H5Ocopy).
FURTHER ISSUES: the type in question ("double") has a different output cross platforms (e.g on liberty some garbage number is printed at some array locations)
SOLUTION: defined an "int" type for this test. However the printing of such an array has a bogus output at least in one platform (FreeBsd), so eliminated the test run altogether and filed a bug report on this
Bug Fix
Description:
Fixing BZ #1381. The --includedir=DIR configure option, which is used to
spceify installation location of C header files, did not work correctly as
the path was hard-coded into config/commence.am. I'm presuming this is
because an older version of automake didn't know where to put C header
files. In any case, removing this line now defaults the includedir to the
same directory that it is currently hard-coded to, and also fixes the
configure flag which allows for customization of this value.
Tested:
jam, liberty
have 2 expected outputs for 2 h5ls runs depending if run on a big or little endian machine. Configure.in was modified to export a variable carrying endianess information to testh5ls.sh. This script then compares the current run with 2 expected outputs, one for a big-endian machine (linew was used to generate the output), other for little endian (jam was used to generate the output)
the way h5ls prints types, it starts searching for NATIVE types first. One solution would be h5ls not to detect these native types, using for example the same print datatype function that h5dump does, that would make the output look the same on all platforms ("32-bit little-endian integer" would be printed instead). Drawback, this "native" information would not be available. Other solution is to have not one but 2 expected outputs and make the shell script detect the endianess and compare with one output or other
tested: jam, linew
Bug fix: for compound types, the not comparable test for members was not done
Solution: for compound types, recursively apply that check
Two new cases are added
1) the compound type has a different number of members. Message printed is
<obj1> has X members <obj2> has Y members
Where X and Y are the number of members of each compound type being compared
2) the compound type has not comparable types (for example a double and an int at the same index)
In this case the message
Comparison not possible: object1 is of class1 and object2 is of class2
Is replaced with
Comparison not possible: object1 has a class1 and object2 has a class2
Modified the test generator program to have these 2 cases
Added a shell run for these 2 cases
Tested: h5committest
bug fix
a new line was not inserted at the end of output, causing diff to complain between linux and frebsd
tested: linux (freebsd tested on the trunk)
#1368 (E1) h5diff: implement "not comparable" messages. Implemented RFC. The new option is <-c, --compare List objects that are not comparable>
added some test cases
tested: windows, linux
The test "h5diff -v h5diff_hyper1.h5 h5diff_hyper2.h5" sometimes hangs in THG
machines. Skip it until its hanging is resolved.
Tested: Jam pp only since it is a simple script change.
Added an option to avoid dealing with NaNs
-N, --nan Avoid NaNs detection
Note: there is no shell script run for datasets with NaN because the output is non portable (different results and NaN strings for different systems)
Tested: windows, linux
Bug fix
PG compiler complains about array out of bounds (a rank of zero was not checked)
Adding a scalar dataset to the test generator program. this case is run on a previous existing run, the case was added to 2 existing files
Tested: windows, linux
Description: Libtool wasn't working on linew because the script was
not working with the Solaris Bourne shell. Libtool has built-
in detection to ensure that it uses an appropriate shell,
but our config/commence.am was hard coding this shell to be
/bin/sh. Removing this line allows the shell to be picked up
by configure, thus allowing libtool to use the correct shell
on linew (bash). This was initially added to correct for a problem
on an old machine (janus), so shoudn't be needed for machines
we currently support.
Tested: full make check install on kagiso and linew, compile only
on smirom, duty, and liberty.
Description:
1) configure now sets LD_LIBRARY_PATH before checking for presence of
SZIP encoder, so user does not need to set this. The path is then
saved and sent to Makefiles, and used when "make check-install" is
invoked, so user doesn't need to set this manually.
2) Upgraded libtool to version 2.2.6a
3) Rearranged tools build order as h5dump depends on existence of h5diff
and h5import. Since h5dump is a sister directory as opposed to a parent
of either, it doesn't know about the build rules of these tools, so
setting any sort of explicit dependency confuses automake as it won't
know how to build the tools. Instead, setting CONFIG = ordered forces
an in-order traversal of each tools subdirectory. (without it will also
traverse in order by default, but this should prevent gmake's -j option
from jumping ahead as well). 'make check install' should now be able to
be invoked after 'configure' without causing a dependency failure.
4) Removed H5_HAVE_LARGE_HSIZET macro from vms/src/h5pubconf.h, as this macro
has now been out of the code for some time. (vms pubconf needs to be
updated manually, which is why it was still hanging around).
Tested: kagiso, smirom, liberty
Note: h5diff looks to be causing failures in h5copy and h5dump tools tests,
though these were present before any of my changes. My changes only
affect build order and configuration setup, and shouldn't prevent
fixes for these failures coming in after this checkin. Other than these,
tests pass fine.
Remove trailing whitespace from C/C++ source files, with the following
script:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Tested on:
Mac OS X/32 10.5.5 (amazon)
No need for h5committest, just whitespace changes...
Summary: when using h5diff to compare the results of h5repack (or other tools that copy one HDF5 file to another), a new option is needed to allow h5diff to make an "absolute" comparison of the 2 files. This is the "contents" mode explained in the usage below.
If this mode is present, objects in both files must match (must be exactly the same). If this does not happen, the tool returns an error code of 1 (instead of the success code of 0)
Changes to the h5repack test script: the call to h5diff was changed to include -c (maintaining the previous -q).
tested: windows, linux
bug fixes.
Description:
Added code to create an empty hdf5 (named h5diff_empty.h5) in order to test
if h5diff compares correctly an empty hdf5 vs. a non-empty one.
Tested:
Tested in kagiso of h5diffgentest itself.
Verified by h5dump that h5diff_empty.h5 was indeed empty.
Then "h5diff h5diff_empty.h5 h5diff_basic1.h5" returned 0 (should have
returned non-zero).
Description:
The fortran Makefile.am used HDF_FORTRAN to indicate it is part of the
Fortran API source so that conclude.am will give fortran api prefix in the
test output. The symbox HDF_FORTRAN is also used in configure for a different
purpose (indicated --enable-fortran). They conflicted.
Similar problem for the symbol HDF_CXX.
Solution:
Changed all the involved Makefile.am to use "FORTRAN_API" instead. It is
a more appropriate name. Same for CXX_API.
Along the way, discovered that the Makefile.am of hl/fortran/test and
hl/cxx/test did not have those symbols at all. Added them in.
Platform tested:
Kagiso serial and h5committested (kagisopp, smirom, linew).
Improvement.
Description:
src/libhdf5.settings was the initial configure summary and is installed.
Then configure is changed to dump a summary of the configure settings to
the output and also append it to src/libhdf5.settings. That created
two different output formats and duplicated information. This is the
initial attempt to clean up this confusion and unify the output format.
It is decided to use the src/libhdf5.settings template as the unified means.
This requires more macros symbols be defined. The following symbols are
all related to generating the src/libhdf5.settings file.
AC_SUBST(EXTERNAL_FILTERS)
AC_SUBST(MPE) MPE=no
AC_SUBST(STATIC_EXEC) STATIC_EXEC=no
AC_SUBST(HDF_FORTRAN) HDF_FORTRAN=no
AC_SUBST(FC) HDF_FORTRAN=no
AC_SUBST(HDF_CXX) HDF_CXX=no
AC_SUBST(CXX) HDF_CXX=no
AC_SUBST(HDF5_HL) HDF5_HL=yes
AC_SUBST(GPFS) GPFS=no
AC_SUBST(LINUX_LFS) LINUX_LFS=no
AC_SUBST(INSTRUMENT) INSTRUMENT=no
AC_SUBST(CODESTACK) CODESTACK=no
AC_SUBST(HAVE_DMALLOC) HAVE_DMALLOC=no
AC_SUBST(DIRECT_VFD) DIRECT_VFD=no
AC_SUBST(THREADSAFE) THREADSAFE=no
AC_SUBST(STATIC_SHARED)
AC_SUBST(enable_shared)
AC_SUBST(enable_static)
AC_SUBST(UNAME_INFO) UNAME_INFO=`uname -a`
The src/libhdf5.settings.in has CONDITIONAL's added to it too. The
untrue conditions turned into a "#" and these lines are cleaned by the
post processing script.
Platform tested:
h5committest on kagiso, smirom and linew. (smirom had failures not due to
my changes.)
Description: Updated bin/reconfigure script to reflect the new versions of
libtool and automake in the /home1/packages/ directory.
Rearranged configure.in script. For some reason, when using
libtool 2.2, the libtool script doesn't generate until
later in the configuration process, so I had to move
a test that parsed through the libtool script to a point
after where it was actually being generated.
Ran libtoolize on the project, and ran bin/reconfigure to
regenerate configure and Makefile.in's throughout.
Tested: kagiso, smirom, linew, tg-login
Correct the prototype for H5Sselect_elements() to take an 'hsize_t *' for
the coordinates, instead of 'hsize_t **'.
Tested on:
Mac OS X/32 10.5.1 (amazon)
Add work-around to allow reading files that were produced with a buggy
earlier version of the library, which could create objects with the wrong
object header message count. There is now a configure flag
"--enable-strict-format-checks" which triggers a failure on reading a file
with this sort of corruption (when enabled) and allows the object to be read
(when disabled). The default value for the "strict-format-checks" flag is
yes when the "debug" flag is enabled and no when the "debug" flag is disabled.
Note that if strict format checks are disabled (allowing objects with
this particular kind of corruption to be read) and the file is opened with
write access, the library will re-write the object header for the corrupt
object with the correct # of object header messages.
This closes bugzilla bug #1010.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.4.10 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
