Bring r16848 from revise_chunks branch to trunk:
Add test (and bugfixes) for detecting if a filter makes a chunk size
larger than can be encoded in a 32-bit variable (in the file).
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(h5committest not needed, multi-platform testing done on branch)
Bring r16846 from revise_chunks branch back to trunk:
Fix broken (for how long?) H5_ASSIGN_OVERFLOW macro to actually detect
overflows during assignments, along with several errors it [now] detected.
Cleaned up a fix minor warnings and/or pieces of code also.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(h5committest not needed - multi-platform test performed on branch)
In Up (AIX system), the insert_libhdf5_settings() infinite looped because
it never found EOF. Man getc in another system said:
"If the integer value returned by getc() is stored into a variable of
type char and then compared against the integer constant EOF, the com-
parison may never succeed, because sign-extension of a variable of type
char on widening to integer is implementation-defined."
Solution:
Changed inchar from type char to int fixed the problem.
Morale of the story: RTFM or HTCW (heed the compiler warnings).
Tested:
Up (AIX 5.3).
Embed the content of libhdf5.settings into the hdf5 executables so that an
"orphaned" executables can display (via the Unix strings command, for example)
the library settings used to build the executables.
This is a prototype implementation. Much improvement is needed.
configure.in:
Added the --disable-embedded-libinfo option to disable this feature.
configure:
src/H5config.h.in:
Generated by autotools like automake.
src/H5detect.c:
Implement insert_libhdf5_settings() to insert the contents of
libhdf5.settings into the library as an extern string variable so that it
is included in all HDF5 executable. Much improvement is needed.
fortran/src/Makefile.in:
Auto-generated by bin/reconfigure. (i.e., I did make any changes to cause
its direct regeneration).
Tested:
Jam serial, using default and --disable-embedded-libinfo, configure options.
Description:
Previously, there was no versioning for H5Z_class_t. This prevented applications
written for 1.6 using custom filters from being able to use the 1.8 library.
There is now an H5Z_class1_t and H5Z_class2_t to enable compatibility. H5Zregister is
*not* versioned, it determines which version of the struct has been passed in by the
value of the first field (id or version, both are ints).
Tested: jam, linew, smirom (h5committest), jam (--with-default-api-version=v16)
Description:
When writing data to a dataset, the data transform was performed after type conversion.
This caused an error if the file type was non-native. This has been changed so data
transforms are always performed on the memory type.
Tested: jam, linew. smirom (h5committest)
- Updated bin/reconfigure to use latest version of automake (1.10.2).
Re-generated Makefile.in's by running bin/reconfigure.
- Added libtool version numbers to c++, fortran, hl, hl c++, and hl fortran
libraries.
Tested:
jam, liberty, smirom
Description:
The H5L interface was not marked as uninitialized when H5L_term_interface was
called. This caused the interface to not initialize itself later on. This
prevented external links from working after calling H5close, because they were
never re-registered.
Tested: jam, smirom, linew (h5committest)
Description:
Changed H5G_mkroot to be tolerant of files with symbol table information cached
but no symbol table in the root group. Also changed H5G_mkroot to properly
clean up in case of an error, and changed H5G_stab_is_valid to properly detect
errors in H5O_msg_read.
Tested: jam, linew, smirom (h5committest), jam (--disable-strict-format-checks)
Description:
Versions of the library between 1.3.0 and 1.6.3 have a bug which prevents them
from opening any file that does not have the root group's symbol table
information cached in the root group's entry in the superblock. Prior to 1.8
this was not an issue as this information was always cached. However, 1.8.0
stopped writing this information (which is not required by the file format
specification), and these older versions can therefore not read files created or
last written by versions 1.8.0 to 1.8.2. This fix modifies the library to once
again add this information to the superblock (when using the old file format).
Tested: jam, linew, smirom (h5committest)
Description:
Factory free lists were formerly implemented as block free lists. This was
inefficient as factories can only be one size, and implementing them as blocks
(which can be variable size) wastedd computation and space. They have been
rewritten with a separate implementation, which is simlar to regular free lists
except they can be dynamically created and destroyed.
Tested: jam, linew, smirom (h5committest)
Description:
Changed Skip list package to use a deterministic skip list. This allows the
skip list package to avoid calling rand() and srand(), even on machines without
rand_r(). There is no longer a p-value or maximum level for skip lists.
Tested: jam, smirom, linew (h5committest)
Description:
Previously, H5Lcopy and H5Lmove would (through H5L_move) improperly apply the
"create intermediate groups" property to the source path traversal, and not the
destination. Fixed it to apply the property to the destination and not the
source. Also fixed H5Lcreate_ud to reject internal link classes without
throwing an assertion.
Tested: linew, jam, smirom (h5committtest)
some argument check in the internal function H5I_find_id and took out the assertion check.
I also removed the argument check in H5Iis_valid because it's in H5I_find_id now.
Tested on jam - simple change. Tested v1.8 already.
Pass some user data down into the extensible array client context creation
callback.
Tested on:
FreeBSD/32 6.3 (duty)
Too minor to require h5committest
Use metadata cache flush dependencies (and 'notify' callback) to give
extensible arrays SWMR (single-writer/multiple-reader) access capability.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Remove unnecessary 'udata' parameter from H5AC_set() and 'notify' callback
(introduced with checkin that added the 'notify' callback)
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Add 'notify' callback to metadata cache clients, so that they can be
aware when the cache has taken/will take certain actions. Notifications are
urrently limited to when an entry has successfully entered the cache (via an
insertion or a load from the file) and when an entry is about to be evicted
from the cache.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Add mechanism for querying if an entry in the metadata cache is the
parent or child in a flush dependency relationship.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Revise how code for array statistics works.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Modify metadata cache flush dependency feature to allow it to work
with entries that are pinned through the cache API calls.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Description:
Due to a bug in H5F_super_read, every time a file with a user block was opened
it would grow by the size of the user block. The bug has been fixed, and
comments have been added to clarify when an eoa address should be relative and
when it should be absolute.
Tested:
jam, linew (h5committest)
v1.8 library, v1.8 library writes the driver info block in the superblock. But v1.6
doesn't write it. This caused the data after the superblock to be overwritten. The
solution is to use a flag to indicate when the original file doesn't have the driver
info, v1.8 library doesn't write it either.
Tested on jam. I have some trouble to access smirom and linew using h5committest.
Add H5AC_create_flush_dependency() and H5AC_destroy_flush_dependency()
wrappers for similarly named routines in H5C layer.
Tested on:
FreeBSD/32 6.3 (duty)
Too minor to require h5committest
Ensure that metadata cache entries with flush dependencies are written
to the file in the correct order (for serial I/O, at least).
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Description:
Some files apparently exist in the wild which have corrupt symbol table messages
on the root group. These files can be opened by 1.6 (which uses the cached
information in the superblock) but not by 1.8. This patch fixes 1.8 and 1.9 so
they can now open these files, and will correct them if necessary. Also fix
some potential (rare) problems with array datatype versions.
Tested: jam (parallel; h5committest not working on linew or smirom)
Add infrastructure & tests for "flush dependencies" in metadata cache,
which allow relationships that specify which order to flush metadata entries in.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Bug Fix
Description:
Fixing BZ #1381. The --includedir=DIR configure option, which is used
to specify the installation location of C header files, did not work
correctly as the path was hard-coded in config/commence.am. I'm presuming
this is because an older version of automake didn't know where to put
c header files. In any case, removing this line now defaults the includedir
to the same directory that it is currently hard-coded to, and also fixes
the configure flag to allow customization of this value.
Tested:
jam, liberty
Remove another call to H5E_clear_stack() from within the library.
Clean up lots of compiler warnings.
Tested on:
Mac OS X/32 10.5.6 (amazon)
(followup on other platforms forthcoming)
Pass the chunk "user data" to H5D_chunk_unlock(), so that chunks with
an address already aren't reallocated.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Description:
In some situations it was possible for the fill value to not be written to parts
of a chunked dataset, particularly when extending and/or shrinking. Prior to
the fix for the chunk cache (1015) these bugs would have been exceedingly rare.
Tested: jam, smirom, linew (h5committest)
the way h5ls prints types, it starts searching for NATIVE types first. One solution would be h5ls not to detect these native types, using for example the same print datatype function that h5dump does, that would make the output look the same on all platforms ("32-bit little-endian integer" would be printed instead). Drawback, this "native" information would not be available. Other solution is to have not one but 2 expected outputs and make the shell script detect the endianess and compare with one output or other
tested: h5committest
creation order in query function but there's no creation order indexed in the file, the library
tried to build and sort a table of all links. To optimize it, let the library use the B-tree for
names of the links.
Tested on jam. I tested the same change for v1.8 with h5committest.
