Bring r23529 from trunk to 1.8 branch:
Stop aliasing property to indicate internal collective metadata operations
with property to perform collective raw data operations from the application.
Tested on:
Mac OSX/64 10.8.3 (amazon) w/parallel
(daily tests on trunk)
HDFFV-8146 - Remove "multi-chunk IO without optimization" sub-feature from MPI I/O optimization for chunked dataset feature
Description:
The “multi-chunk IO without optimization” feature is removed and made the related xfer property (H5FD_MPIO_CHUNK_MULTI_IO) go directly to “multi-chunk-io” feature.
Also update/fix/cleanup testings (chunk collective IO and actual chunk opt mode) accordingly.
Merged from HDF5 trunk r22992.
Tested:
jam-pp (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), fred (mac64-LE), linew (solaris-BE), koala-pp, wallby-pp, cmake (jam-pp), Windows (32-LE cmake)
HDFFV-8143 Provide a routine(s) for telling the user why the library broke collective data access
Description:
Changed to use H5Dopen2() instead of H5Dopen(), which caught by daily test on jam.
This is based on previous commit r22780.
Tested:
jam-pp (linux32-LE)
HDFFV-8143 Provide a routine(s) for telling the user why the library broke collective data access
Description:
Added H5Pget_mpio_no_collective_cause() function that retrive reasons why the collective I/O was broken during Read/Write IO access.
Reasons to break collective I/O:
- SET_INDEPENDENT
- DATATYPE_CONVERSION
- DATA_TRANSFORMS
- MPIPOSIX
- NOT_SIMPLE_OR_SCALAR_DATASPACES (NULL Space)
- POINT_SELECTIONS
- NOT_CONTIGUOUS_OR_CHUNKED_DATASET (Compact or External-Storage)
- FILTERS
Merged from HDF5 trunk r22735, r22741 , 22743, 22744, 22763.
Tested:
jam-pp (linux32-LE), koala-pp (linux64-LE), wallaby-pp
Bring r22758 from trunk to 1.8 branch:
Bring generic improvements from encode/decode property list branch to
the trunk. This includes a better version of the property list comparison
routine, cleaned up compiler warnings, and some cleaned up property list
callbacks. Also, started on changes to clean up parallel test output, so that
it doesn't report successful tests from each process.
Tested on:
Mac OSX/64 10.7.4 (amazon) w/debug, GCC 4.7.x, FORTRAN, C++, threadsafe and parallel
(h5committested on trunk)
Convert configure.in to configure.ac.
Align TESTS usage with autotools convention.
Update all references from configure.in to configure.ac
Tested: h5committest
Bring r22287 from trunk to 1.8 branch:
Clean up more FUNC_ENTER/FUNC_LEAVE macros and move H5D & H5T code toward
the final design (as exemplified by the H5EA & H5FA code).
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug & parallel
Bring r22244 from trunk to 1.8 branch:
Correct use of deprecated API routines in test routine.
Tested on:
Linux/PPC 2.6 (ostrich) w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-ia64 2.6 (ember) w/Intel compilers, w/C++ & FORTRAN, parallel,
in production mode
Bring r22219 from trunk to 1.8 branch: (with release manager's blessing)
Merge "file image" changes from feature branch back to trunk.
Tested on:
h5committested
Bring r21442 and r21529 from trunk to 1.8 branch: Add new API routines
for querying if collective raw data I/O was actually performed collectively.
Tested:
Linux/32 2.6 (jam)
(h5committested on trunk)
HDFFV-7639
Purpose:
Remove H5_MPI_SPECIAL_COLLECTIVE_IO_WORKS and
H5_MPI_COMPLEX_DERIVED_DATATYPE_WORKS #defines from source.
Description:
Two advanced parallel functionalities, special collective IO and
complex derived datatypes, are not supported by older
implementations of mpi, and thus our code limits the use of these
features with #ifdefs and has checks in configure to set them (or
not). Unfortunately, configure can't actually run a parallel check
to see if these features are working (nor not) so it resorts to
looking in the config files where they are explicity enabled or
disabled based on versions of mpi, sytems being built on, or for
no documented reason at all (i.e. just set to on or off as some
'default'). Overriding these settings is easy if need be, provided
it is known that it needs to be done to get improved performance,
and oftentimes it is not.
Most new MPI implementations successfully handle the functionality
requested when these #defines are set, and many of the "turn these
features off" cases in the config files are for old (> 5 years)
versions of MPI and retired systems (such as NCSA's tungsten).
Therefore, the decision has been made to remove the support for
these old versions of MPI and systems that cannot handle these
behaviors. The #ifdefs and supporting setup in the config/ files
and configure script has been removed, and the code executed when
these options were not set removed from the source.
In passing, this commit also cleans up some whitespace issues in
both t_mpi.c and H5Dmpio.c. Furthermore, in t_mpi.c, the special
collective IO test was not getting regularly run due to it being
written to work only with four processes (we regularly test with
six, previously with three), and thus it failed when actually run
due to an out of bounds data buffer assignment. It has been
modified to run at any number of processes greater than four, and
the memory problem has been fixed so the test passes.
Tested:
Jam, h5committest, Ember
Ported shape same test express test mods from trunk to 1.8 branch.
This was mostly a matter of copying testpar/t_shapesame.c from trunk
to the 1.8 branch, as the root versions of the files were almost
identical.
Tested parallel on Jam only, as changes have been well tested on the
trunk.
t_cache.c - define needed for windows
t_posix_compliant.c - #ifdef around include <unistd.h>
t_shapesame.c - HD prefix needed for stat function
Tested: local linux
Fix HDFFV-7592
"Remove PACKEDBITS option from autotools files"
Description:
The code for h5dump's packed bits feature was
conditionally compiled in via this option during its
'beta' release in 1.8.5. Now that the feature is fully
supported, h5dump has since been changed to always
compile the packed bits code (having removed the #ifdef
H5_HAVE_H5DUMP_PACKED_BITS checks), so this option in
configure is no longer needed.
This commit removes the --enable-h5dump-packed-bits
configure option, including the help summary from
'./configure --help' and the libhdf5.settings output
summary, the PACKED_BITS Makefile variable, and the
definition of H5_HAVE_H5DUMP_PACKED_BITS define in
appropriate pubconf header files (which has already
been removed from use in source code as of r20722).
Note that this option was never added to the trunk,
thus this commit only need occur on the 1.8 branch.
Tested:
- Configure output, help, & summary verified on jam
- h5committested
Bring r20119 from trunk to 1.8 branch:
Clean up MPI resource leaks in parallel tests, along with a bunch of
compiler warnings.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
Mac OS X/32 10.6.6 (amazon) in debug mode
Mac OS X/32 10.6.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
When $HDF5ExpressTest is NOT set or when it is set to 1 or 0, it does not
skip test.
When $HDF5ExpressTest is set other than values above, it may skip tests.
The following message is printed:
Test skipped
when some tests are really skipped.
This is a temporary patch so that v186 can be tested. A more permanent fix
is needed, later.
Tested: h5commitest.
This continues the previous work and this one breaks the
checker_board_hyperslab_dr_pio_test() into 4 smaller
sub-tests.
Tested: h5committest plus jam serial.
The shape same tests ran too long. Break them into smaller subtests
so that they can finish sub-test in a shorter time. Easier to tell
which one sub-test is taking too much time and/or errors occur in
one fo the sub-tests.
This one breaks the contig_hyperslab_dr_pio_test() into 4 smaller
sub-tests.
Tested: h5committest
error and wanted to exit the test program. This was not good since if only a
subset of processes called MPI_Finalize(), the other processes will likely
hang. That happened in AIX that it would waited till the alarm signal to kill
the processes. Definitely a waste of time.
Solution: Changed it to call MPI_Abort.
That showed another problem. HDF5 has setup atexit post-process to try to close
unclose objects, release resources, etc. But if the MPI processes have
encountered an error and has been aborted, it is not likely any more MPI calls
can function properly. E.g., it would attempt to free some communicators in
the HDF5 MPIO file handle. It would again hang.
Solution: need to call H5dont_atexit() to disable any atexit post-processing.
This must be done early, like before calling H5open. This is added to each
parallel test main program.
testphdf5.h:
Changed macros VRFY and MESG. Added comments too.
testphdf5.c:
t_mpi.c:
t_cache.c:
t_shapesame.c:
Added H5dont_atexit.
Tested: h5committest.
not find t_shapesame in daily test. Turned out the mpiexec launcher is
working like real shell and the daily test signon (hdftest) does not have
"." in its $PATH. So, it could not automatically look for executables in
the current directory.
Solution:
Change the executable to an explicit ./t_shapesame. Now mpiexec can "find"
it.
Tested by hand in Amani.
Moved the two shape same tests from testphdf5 to a separated executables,
named t_shapesame. The shape same tests runs too long for testphdf5.
In a separated executalbe, it will be easier to separate any errors in
testphdf5 sub-tests from the shape same tests.
t_shapesame.c:
Contains the shape same tests (cloned from t_rank_projection.c) plus
a duplicate of "testphdf5.c" for now. After verifying it is correct, more
cleanup is needed.
testphdf5.c:
Removed the two shape same tests (chsssdrpio & cbhsssdrpio).
Makefile.am:
Makefile.in:
Added t_shapesame as a new test executable.
Removed t_rank_projections.c from part of testphdf5.
testph5.sh.in:
Temporary added the "t_shapesame -p" test for testing shape same tests
with MPIO-Posix VFD.
Tested: h5committested, plus serial jam.
