Some collective chunk IO macro names are confusing, change them to more meaningful
names.
Description:
H5Pset_dxpl_mpio_chunk_opt will set a flag so that the library can do one linked IO or multi-chunk IO with collective in chunking storage directly. That is, the library won't do analyses to determine this.
The flags for the enum type we used before are:
H5FD_MPIO_OPT_ONE_IO
H5FD_MPIO_OPT_MULTI_IO
They are not good names because of the following two reasons:
1. It doesn't reflect chunking storage
2. OPT is kind of redundant and misleading,
Solution:
We change the names to
H5FD_MPIO_CHUNK_ONE_IO
H5FD_MPIO_CHUNK_MULTI_IO
Platforms tested:
Since only macro names are changed, no need to test with h5committest.
heping(mpich 1.2.6)
Misc. update:
Description: Ran /bin/h5vers to change suffix to alpha4 after creating hdf5-1.8.0-alpha3
tar ball and tagging the release with hdf5_1_8_0_alpha3 tag
Solution:
Platforms tested:
Misc. update:
1) Check in potential fix to unreproduceable bug in t_cache observed on
Cobalt
2) Check in code supporting added pinned entry features in the metadata
cache.
Description:
1) Elena encountered a bug in t_cache when running on Cobalt. On
investigation I found a potential race condition in t_cache that
could explain her observation.
2) Quincey requested additions to the metadata cache allowing renaming
of pinned entryies, and marking of pinned or protected entries as
dirty.
Solution:
1) Modified t_cache.c to prevent the potential race condition. Elena
was unable to reproduce the bug on Cobalt, so we don't know if my
fix actually addressed the problem.
2) Added code supporting the requested functions. The changes were
relatively trivial, but required substantial new test code.
Platforms tested:
h5committest, serial test on heping, parallel test on phoenix.
Misc. update:
Code checkpoint
Description:
Add a few minor tweaks & corrections to the fractal heap code.
Platforms tested:
Linux 2.4 (chicago)
Too minor to require h5committest
Bug fix
Description:
When destroying the cache for a file, don't disconnect the cache from the
file structure until after the cache is shut dow, to allow pinned entries to
be unpinned.
Platforms tested:
Linux 2.4 (chicago)
Too minor to require h5committest
Code checkpoint
Description:
Update fractal heap code to allow objects to be deleted
Fix bugs in fractal heap object insertion
Improve free space manager code to allow sections to be merged and
shrink the container
Another try at making the Windows compilers happy...
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OS X.4/PPC (amazon)
Linux 2.4 (chicago & heping)
Solaris 2.9 (shanti)
AIX 5.? (copper) w/parallel
Bug fix.
Description:
The ${TR}, though avoid the error, is inconvenient. It is needed
because configure.in use the char range in the style of 'a-z'.
The other style of '[a-z]' is more commonly accepted by all tr
but autoconf tends to strip away [], making the syntax rather
clumsy.
Solution:
Learned from autoconf that it avoids the use of character range
by just spell all the letters out. Changed our tr commands to
use those variables defined by autoconf. Also removed the
definition of ${TR} since it will cause inconsistancy when
autoconf also use plain 'tr' in its generated code.
The Makefile.in are changed because the elimination of ${TR} from
configure triggered its removal from all Makefil.in. That is okay
because ${TR} is not used at all in Makefile.
Platforms tested:
h5committested. (sol failed to connected). also tested in shanti
using both /usr/ucb/tr and /bin/tr (the bad one before.)
Misc. update:
Updated both INSTALL and RELEASE files.
Code checkpoint
Description:
Checkpoint fractal heap improvements, as well as move the free space
manager code that it's using into a separate package.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4/64 (mir) w/C++ & FORTRAN
Linux 2.4/32 (heping)
Solaris 2.9 (shanti)
AIX 5.? (copper) w/FORTRAN & parallel
Bug fix.
Description:
Some machines (e.g., HPUX) will have off64_t defined but no support
for stat64 unless certain LargeFile macro is turned on. This caused
the last fix on H5private not quite right which caused compiling failures.
Solution:
Added stat64 and fstat64 tests and then will use them only if they
are supported.
Platforms tested:
Tested on kelgia, copper and heping (pll).
(Compiling alright but tests for kelgia and heping are not done.
expect no problems.)
Description: Function prototype H5E_walk_t and structure H5E_error_t wasn't backward
compatible.
Solution: Make them compatible with v1.6 and provide new definitions of H5E_walk_stack_t
and H5E_error_stack_t.
Platforms tested: fuss and h5committest.
Bug fix for collective chunk IO
Description:
Several options have been provided inside HDF5 library for
obtaining chunk addresses when doing collective chunk IO.
One option is to obtain chunk addresses for one process and broadcast
to other processes. This option needs all processes to participate.
If using link-chunked IO without any optimizations,
sometimes this is not true due to the random initialization for one variable at mpich 1.2.7.
This is a bug inside the collective chunk IO code.
Solution:
1.Initalize all the variables to some safe numbers,
2. Avoid using MPI broadcast to obtain the chunk address if possible
until more performance studies have been done.
3. Seems okay to obtain chunk addresses individually for each processor.
This option may cover most cases.
Platforms tested:
h5committest(copper is not usable)
NCSA teragrid (mpich 1.2.5)
mir 64-bit linux (mpich 1.2.6)
Misc. update:
bug fix.
Description:
Previous H5private.h would attempt to use the off64_t/stat64/etc
by checking just sizeof __INT64 being 8. This caused errors in
some machines like alpha cluster which is a 64bit machine. Its
compiler supports the type __int64 but it does not support
off64_t since its off_t is already 64bits big. The above code
casued the compiler to complain about the unknown off64_t and
such.
Solution:
H5private.h:
Changed the code to use the pseudo standard of off64_t/stat64/etc
only if sizeof(off_t) is not 64 bits AND sizeof(off64_t) exists
and is 64 bits. This assumes if off64_t is defined, all those
stat64, fstat64, etc are supported too.
configure.in:
configure:
Added the testing of sizeof(off64_t) to support the above solution.
Platforms tested:
h5committested. Also tested at LANL QSC.
Purpose:
Rename feature
Description:
Rename the "function stack" debugging feature to "call stack", so the H5FS*
prefix can be used for free space management code.
Platforms tested:
Not yet, but soon..
Code checkpoint
Description:
More progress on fractal heap, can now re-open an existing heap, although
the free space algorithm still needs work.
Also, use the new "pinned entry" metadata cache code.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (heping)
Solaris 9 (shanti)
Linux 2.4/64 (mir)
Code cleanup/feature twist
Description:
Adjust recent H5AC routines to take H5F_t pointers instead of H5C_t
pointers, to match the rest of the H5AC routines.
This change propagated into a few of the tests, which also had some
compiler warnings cleaned up...
Platforms tested:
FreeBSD 4.11 (sleipnir) w/parallel
Linux 2.4/64 (mir) w/C++ & FORTRAN
Add pinned entry capability to cache.
Description:
For frequently accessed cache entries, the protect/unprotect overhead
is sometimes a bottleneck.
Solution:
Allow entries to be pinned in the cache. Pinned entries can't be
evicted, but can be flushed or modified.
Platforms tested:
h5committested -- minus one small typo in test/cache.c whose fix was
tested on copper and heping only.
Misc. update:
Several bug fixes
Description:
Added config file for Cray X1 (previous file was misnamed)
Simplified some code in hyperslab.c test that seemed to have been
confusing compiler on Cray X1.
Fixed typo in config/commence.am
Cleaned up hl/src/Makefile.am
Solution:
All four fixes should be straightforward. The failure on Cray was
very difficult to debug, but involved arithmetic errors.
This change seems to fix it.
Platforms tested:
heping, copper, sol, some Cray X1 (more testing when system comes back up)
Description:
The file size test in C++ library failed on Copper because the
value returned by h5_get_file_size was intepreted incorrectly
due to different interger sizes.
Solution:
H5private.h: Added check to use stat64 and off64_t where appropriate.
h5test.c and h5test.h: used h5_stat_size_t in place of off_t.
tattr.cpp: used h5_stat_size_t in place of off_t.
Platforms tested:
Linux 2.4 (heping)
AIX 5.1 (copper)
SunOS 5.8 64-bit (sol) - still on going
Description: Here is what I did:
modified MANIFEST according Quincey's instructions to fix
the problem exposed by bin/release (compilation fails
because some header files were not included in the distribution)
fixed small typo and modified include_HEADERS line in
src/Makefile.am to address the problem exposed by bin/release
ran bin/h5vers to change version number to 1.8.0-alpha0
ran bin/release to release tar ball to the ftp server
Solution:
Platforms tested: mir MANY,MANY,MANY times
Misc. update:
Improvement
Description:
Modified the warning messages from the Version mismatch checking to suggest
the users to try recompiling or checking the shared lib setting.
Platforms tested:
Tested in heping only since it was just a simple text string changes.
(I did moved the strings around a bit.)
Code checkpoint
Description:
Add more regression tests for odd insertion patterns
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (heping)
Linux 2.4/64 (mir)
Solaris 2.9 (shanti)
Code checkpoint
Description:
Add in more fractal heap support for odd allocation patterns (entries that
are too large for various block sizes, etc.)
Broke out fractal heap code into more source code modules.
Refactored fractal heap test to make it more understandable and
maintainable for the long term.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (heping)
Linux 2.4/64 (mir)
Solaris 2.9 (shanti)
Anti-feature
Description:
Revert changes to H5G_stat_t struct, to make it compatible with the 1.6.x
branch again. The information that was added to the H5G_stat_t struct will
be reported through other API routines.
Platforms tested:
FreeBSD 4.11 (sleipnir) w/C++
Linux 2.4/64 (mir) w/C++ & Fortran
Solaris 2.9 (shanti)
new feature
Description:
1) separated the HL library into "public" and "private" header files, with the same caracteristics as the basic library
2) added the public headers to hdf5.h (with a conditional include macro, defined in configure.in)
3) added the path to HL in all Makefile.am 's , because of the inclusion in hdf5.h
Solution:
Platforms tested:
linux 32, 64
AIX
solaris
with fortran and c++
(one packet table example fails)
Misc. update:
"Hide" file format changes (for now)
Description:
Add ifdef's (controlled by the --enable-group-revision configure flag)
to disable group revision changes to the file format, in order to allow alpha
release to go ahead without releasing an unsupported version into the wild.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 32-bit (heping)
Linux 2.4 64-bit (mir)
Solaris 2.9 (shanti)
Code cleanup
Description:
Don't include empty public header files.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 32-bit (heping)
Linux 2.4 64-bit (mir)
Solaris 2.9 (shanti)
Bug fix
Description:
Correct the handling of the group creation property list when copying
objects with the "create intermediate groups" flag.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest (once I pay attention to the compiler
warnings... :-)
Code checkpoint
Description:
Update fractal heap code to allow skipping direct blocks (that are too
small to hold object) when the root indirect block is created.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
New/expanded features
Description:
Check in Peter's changed for the object copy code:
- Allow/fix copying datasets using named variable-length datatypes
- Start adding framework for property list to control how object
copying occurs.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Code checkpoint
Description:
Checkpoint recent additions to fractal heap code, which allow for better
support of objects that don't fit into current direct block, requiring
skipped direct blocks to be tracked in free section list.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago & heping)
Linux 2.4 (mir) w/64-bit & FORTRAN & C++
Mac OSX (amazon)
Adding parallel tests for optional collective chunk APIs
Description:
Three new APIs
"H5Pset_dxpl_mpio_chunk_opt_ratio
H5Pset_dxpl_mpio_chunk_opt_num
H5Pset_dxpl_mpio_chunk_opt"
for optional optimization choices from users
have been added to the libraries.
This check-in adds six tests to verify the funcationality and correctedness
of these APIs.
These tests need to be verified with 3 or more processors and with MPI-IO driver only.
Solution:
Using H5Pinsert, H5Pget, H5Pset to verify that the library indeed goes into the branch we hope for.
Using H5_HAVE_INSTRUMENT macro to isolate these changes so that it won't affect or be misused by the application.
Platforms tested:
h5committest(shanti still refused to be connected)
Parallel tests on heping somehow are skipped. Manually testing at heping. Have checked
1,2,3,4,5 processes.
Misc. update:
Code checkpoint
Description:
Check in fractal heap code to add basic support for skipping direct blocks
when an object is too large to fit in one.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
Code checkpoint
Description:
Checkpoint fractal heap code at a stable point in development. It appears
that adding objects to heaps and reading them back out is now working
properly (as long as there are no gaps in the heap and no standalone objects
are created).
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
Linux 2.4 (mir) w/64-bit & FORTRAN & C++
Solaris 9 (shanti) w/64-bit
Code optimization
Description:
Tweak file free space allocation routines to optimize them a bit.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Bug fix
Description:
Previous checkin did a bad thing; 'make clean' failed in example directories.
Solution:
Fixed commence.am so that examples no longer break, and fixed a mistake
in conclude.am.
Platforms tested:
heping (minor makefile change)
Misc. update:
Bug fix
Description:
make check-clean didn't clean results of example tests
Solution:
Fixed Makefiles so that check-clean recurses into example directories.
Also a little Makefile cleanup.
Platforms tested:
mir, modi4, heping, copper
portibility issue to windows
Description:
setvbuf on visual studio 2005 needs the size to be greater or equal to 2.
<description of setvbuf from msdn library>
Buffer size in bytes. Allowable range: 2 <= size <=
INT_MAX (2147483647). Internally, the value supplied
for size is rounded down to the nearest multiple of 2.
h5import used linebuffer option of setvbuf for stderr and stdout and the size is
0 by default. This causes core dump with visual 2005 on windows 64-bit.
Solution:
1. Use HDsetvbuf inside h5import.c,
2. Define HDsetvbuf inside windows H5pubconf.h for windows to work around
the core dump of h5import test. This is probably a bug inside visual studio 2005.
Platforms tested:
h5committest
windows 6.0 at XP
visual 2005 at XP-64bit
Misc. update:
Description: Unix remove function removes only the latest version of a file
on VMS. Some of our tests create multiple versions of the testfiles
and as a result, test programs may be confused, give false negative result,
etc.
Solution: Created HDremove_all function for VMS that removes all versions of the files.
HDremove on VMS is an alias to HDremove_all.
Platforms tested: VMS server and heping (to check that nothing is borken on UNIX side)
Misc. update:
