Ported shape same test express test mods from trunk to 1.8 branch.
This was mostly a matter of copying testpar/t_shapesame.c from trunk
to the 1.8 branch, as the root versions of the files were almost
identical.
Tested parallel on Jam only, as changes have been well tested on the
trunk.
t_cache.c - define needed for windows
t_posix_compliant.c - #ifdef around include <unistd.h>
t_shapesame.c - HD prefix needed for stat function
Tested: local linux
Fix HDFFV-7592
"Remove PACKEDBITS option from autotools files"
Description:
The code for h5dump's packed bits feature was
conditionally compiled in via this option during its
'beta' release in 1.8.5. Now that the feature is fully
supported, h5dump has since been changed to always
compile the packed bits code (having removed the #ifdef
H5_HAVE_H5DUMP_PACKED_BITS checks), so this option in
configure is no longer needed.
This commit removes the --enable-h5dump-packed-bits
configure option, including the help summary from
'./configure --help' and the libhdf5.settings output
summary, the PACKED_BITS Makefile variable, and the
definition of H5_HAVE_H5DUMP_PACKED_BITS define in
appropriate pubconf header files (which has already
been removed from use in source code as of r20722).
Note that this option was never added to the trunk,
thus this commit only need occur on the 1.8 branch.
Tested:
- Configure output, help, & summary verified on jam
- h5committested
Bring r20119 from trunk to 1.8 branch:
Clean up MPI resource leaks in parallel tests, along with a bunch of
compiler warnings.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
Mac OS X/32 10.6.6 (amazon) in debug mode
Mac OS X/32 10.6.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
When $HDF5ExpressTest is NOT set or when it is set to 1 or 0, it does not
skip test.
When $HDF5ExpressTest is set other than values above, it may skip tests.
The following message is printed:
Test skipped
when some tests are really skipped.
This is a temporary patch so that v186 can be tested. A more permanent fix
is needed, later.
Tested: h5commitest.
This continues the previous work and this one breaks the
checker_board_hyperslab_dr_pio_test() into 4 smaller
sub-tests.
Tested: h5committest plus jam serial.
The shape same tests ran too long. Break them into smaller subtests
so that they can finish sub-test in a shorter time. Easier to tell
which one sub-test is taking too much time and/or errors occur in
one fo the sub-tests.
This one breaks the contig_hyperslab_dr_pio_test() into 4 smaller
sub-tests.
Tested: h5committest
error and wanted to exit the test program. This was not good since if only a
subset of processes called MPI_Finalize(), the other processes will likely
hang. That happened in AIX that it would waited till the alarm signal to kill
the processes. Definitely a waste of time.
Solution: Changed it to call MPI_Abort.
That showed another problem. HDF5 has setup atexit post-process to try to close
unclose objects, release resources, etc. But if the MPI processes have
encountered an error and has been aborted, it is not likely any more MPI calls
can function properly. E.g., it would attempt to free some communicators in
the HDF5 MPIO file handle. It would again hang.
Solution: need to call H5dont_atexit() to disable any atexit post-processing.
This must be done early, like before calling H5open. This is added to each
parallel test main program.
testphdf5.h:
Changed macros VRFY and MESG. Added comments too.
testphdf5.c:
t_mpi.c:
t_cache.c:
t_shapesame.c:
Added H5dont_atexit.
Tested: h5committest.
not find t_shapesame in daily test. Turned out the mpiexec launcher is
working like real shell and the daily test signon (hdftest) does not have
"." in its $PATH. So, it could not automatically look for executables in
the current directory.
Solution:
Change the executable to an explicit ./t_shapesame. Now mpiexec can "find"
it.
Tested by hand in Amani.
Moved the two shape same tests from testphdf5 to a separated executables,
named t_shapesame. The shape same tests runs too long for testphdf5.
In a separated executalbe, it will be easier to separate any errors in
testphdf5 sub-tests from the shape same tests.
t_shapesame.c:
Contains the shape same tests (cloned from t_rank_projection.c) plus
a duplicate of "testphdf5.c" for now. After verifying it is correct, more
cleanup is needed.
testphdf5.c:
Removed the two shape same tests (chsssdrpio & cbhsssdrpio).
Makefile.am:
Makefile.in:
Added t_shapesame as a new test executable.
Removed t_rank_projections.c from part of testphdf5.
testph5.sh.in:
Temporary added the "t_shapesame -p" test for testing shape same tests
with MPIO-Posix VFD.
Tested: h5committested, plus serial jam.
Checked in fix for failure in shape same tests that appeared after
Quincy's recent massage of the test code. The problem was a race
condition created when Quincey re-worked the code selecting either
collective or independant I/O.
Previously, when independant I/O was selected in the test, I had
used H5Pset_dxpl_mpio() and H5Pset_dxpl_mpio_collective_opt() to
select collective semantics with independant I/O going on under
the hood. Quincey modified this to call H5Pset_dxpl_mpio() when
collective I/O was selected, and do nothing in the independant I/O
case. As a result, processes were able to race ahead and
modify the initial values of the data set before some processes
had verified that the initialization was correct.
Solved the problem by adding barriers, and making all barriers
dependant on independant I/O being selected.
Tested parallel on amani and phoenix. h5committested.
Bring r19743-19744 from trunk to 1.8 branch:
Correct tests to use native datatypes consistently, and also to use
"normal" methods for performing collective I/O. Also, minor cleanups for
zeroing out buffers, etc.
Tested on:
FreeBSD/32 6.3 (duty) w/debug
(h5committested on trunk)
Bug fix for failure in the round robin metadata write ojbect header
metadata confusion test that appeared after Albert modified the test.
Cursory commit test. No test on Abe as that system is down, the
fix is very minor, and it seems to work in the 1.8.6 branch
John Mainzer fixed the bug and added a test which wrote file and flush a few
time; close the file then open it by serial and read simple structure. I
changed the test to two parallel running parts of ..._writer and ..._reader
and have the reader verify the file after every flush by the writer.
Tested: parallel in Jam and Amani.
Replaced calls to H5Dcreate() and H5Acreate() with calls to H5Dcreate2()
and H5Acreate2() respectively in t_mdset.c.
This was done to repair a compile failure that occured on a build
with the --with-default-api-version=v16 config option
Cursory commit test
Modified test code in t_mdset to use H5Dopen2() instead of H5Dopen1().
This should fix the compile failure when we used --disable-deprecated-symbols
Cursory commit test
of the H5Ocache.c code to update its image of the on disk representation
of the object header on a call to clear callback.
This wasn't an issue as long as all flushes of the object header were
made from the same process, but if an object header is modified, and
then flushed on one process and cleared on the rest, the changes were
not be reflected in the images of the on disk representation on all
processes where the object header was cleared rather than flushed.
If one of these processes did the next flush, the changes were lost in
the on disk representation.
Fixed this by causing all dirty messages and to be written to the copy
of the on disk image maintained by the object header code on both flush
and clear.
Also added associated test code in t_mdset.c.
Also checking in some cache debug code developed while chasing this bug.
Commit tested and tested (parallel) on phoenix.
log entry reproduced below:
--------------------------------------------------------------
Modified shape same tests to deal with the performance problem on Abe.
Problem appears to have been caused by file system contention.
In the chunked dataset case, reshaping the chunks so that only one
process would touch each chunk and setting the alignment equal to the
default Lustre block size more or less dealt with the problem.
For contiguous datasets, the problem was a bit more difficult, as
re-working the test to avoid contention would have been very time
consuming.
Instead, I added code to time one execution of each type of shape
same test, and skip additional tests of that type if the duration
of the test exceeded some threshold
In all cases, I set up code to turn off the above fixes if express
test is 0.
Tested on Abe and commit tested. On the commit test, the configure
test failed -- probably because I was h5committest from heiwa due
to some ssh wierdness. In any case a manual reconfigure run on
jam seemed to work fine.
Also, in h5committest, I ran into some data conversion warnings.
I didn't worry about them as the only code I changed was in testpar.
has been removed. According to the official description of DESTDIR by Gnu at
http://www.gnu.org/prep/standards/html_node/DESTDIR.html, DESTDIR is
prepended to the normal and complete install path that it precedes for the
purpose of installing in a temporary directory which is useful for building
rpms and other packages. The '/' after ${DESTDIR} will be followed by another
'/' at the beginning of the normal install path. When DESTDIR is empty the
path then begins with '//', which is incorrect and causes problems at least for
Cygwin.
The change made in config/commence.am affects many Makefile.*s when bin/reconfigure
is run. EXAMPLEDIRs had to be corrected individually.
Also the scripts to run examples after they are installed have checks added to see
that the fortran and c++ directories are present before trying to run the scripts
in them.
Tested with h5committest on amani, heiwa and jam.
Tested with h5committest on amani, jam, and heiwa, and on linew.
-This line, and those below, will be ignored--
M hdf5_1_8/test/Makefile.in
A hdf5_1_8/m4
AM hdf5_1_8/m4/ltsugar.m4
AM hdf5_1_8/m4/libtool.m4
AM hdf5_1_8/m4/ltversion.m4
AM hdf5_1_8/m4/lt~obsolete.m4
AM hdf5_1_8/m4/ltoptions.m4
M hdf5_1_8/configure
M hdf5_1_8/Makefile.in
M hdf5_1_8/testpar/Makefile.in
M hdf5_1_8/configure.in
M hdf5_1_8/src/Makefile.in
M hdf5_1_8/tools/Makefile.in
M hdf5_1_8/tools/misc/Makefile.in
M hdf5_1_8/tools/h5dump/Makefile.in
M hdf5_1_8/tools/h5repack/Makefile.in
M hdf5_1_8/tools/h5jam/Makefile.in
M hdf5_1_8/tools/h5diff/Makefile.in
M hdf5_1_8/tools/lib/Makefile.in
M hdf5_1_8/tools/h5copy/Makefile.in
M hdf5_1_8/tools/h5import/Makefile.in
M hdf5_1_8/tools/h5stat/Makefile.in
M hdf5_1_8/tools/h5ls/Makefile.in
M hdf5_1_8/hl/test/Makefile.in
M hdf5_1_8/hl/tools/Makefile.in
M hdf5_1_8/hl/tools/gif2h5/Makefile.in
M hdf5_1_8/hl/Makefile.in
M hdf5_1_8/hl/src/Makefile.in
M hdf5_1_8/hl/c++/test/Makefile.in
M hdf5_1_8/hl/c++/Makefile.in
M hdf5_1_8/hl/c++/src/Makefile.in
M hdf5_1_8/hl/c++/examples/Makefile.in
M hdf5_1_8/hl/fortran/test/Makefile.in
M hdf5_1_8/hl/fortran/Makefile.in
M hdf5_1_8/hl/fortran/src/Makefile.in
M hdf5_1_8/hl/fortran/examples/Makefile.in
M hdf5_1_8/hl/examples/Makefile.in
M hdf5_1_8/config/commence.am
M hdf5_1_8/MANIFEST
M hdf5_1_8/c++/test/Makefile.in
M hdf5_1_8/c++/Makefile.in
M hdf5_1_8/c++/src/Makefile.in
M hdf5_1_8/c++/examples/Makefile.in
M hdf5_1_8/bin/ltmain.sh
M hdf5_1_8/bin/reconfigure
M hdf5_1_8/perform/Makefile.in
M hdf5_1_8/fortran/test/Makefile.in
M hdf5_1_8/fortran/testpar/Makefile.in
M hdf5_1_8/fortran/Makefile.in
M hdf5_1_8/fortran/src/Makefile.in
M hdf5_1_8/fortran/examples/Makefile.in
M hdf5_1_8/aclocal.m4
M hdf5_1_8/examples/Makefile.in
Corrected use/name of source folder aliases.
Duplicated FindMPI.cmake so that non-c++ compiler is found first (recommemded commands did not work).
Bring r19325 from trunk
Tested: local linux with mpich
Initialize loop variable that caused failures in certain circumstances.
Also clean up compiler warnings and release MPI datatype.
Tested on:
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
Rename H5AC_set() to H5AC_insert_entry()
Get rid of H5C_set_skip_flags() & related flags
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug
(too simple to require h5committest)
Bring r19092 from trunk to 1.8 branch:
Bring "shape same" changes from LBL branch to trunk. These changes
allow shapes that are the same, but projected into dataspaces with different
ranks to be detected correctly, and also contains code to project a dataspace
into greater/lesser number of dimensions, so the I/O can proceed in a faster
way.
These changes also contain several bug fixes and _lots_ of code
cleanups to the MPI datatype creation code.
Many other misc. code cleanup are included as well...
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
(h5committested on trunk)
Bring r18917 from trunk to 1.8 branch:
Bring r18911 (plus some adaptions to match the code on the trunk)
from the metadata journaling "merging" branch to the trunk:
More general changes to align trunk with eventual changes from
metadata journaling branch.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Bring r18864 from trunk to 1.8 branch:
General cleanups to align metadata journaling branch and trunk
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(h5committested on trunk)
