an object header chunk, where the total amount deleted was greater than
64K, an error would occur due to an off by one error in the code that
handled that case. Fixed this and added a test case.
* commit '54957d37f5aa73912763dbb6e308555e863c43f4':
Commit copyright header change for src/H5PLpkg.c which was added after running script to make changes.
Add new files in release_docs to MANIFEST. Cimmit changes to Makefile.in(s) and H5PL.c that resulted from running autogen.sh.
Merge pull request #407 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:hdf5_1_10_1 to hdf5_1_10_1
Change copyright headers to replace url referring to file to be removed and replace it with new url for COPYING file.
Clean up more warnings: drop the warning count from ~1310 down to ~940,
with only 31 types of warnings in 148 files (down from 38 types in 167 files).
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial & parallel
(h5committest forthcoming)
Complete revamp of package initialization/shutdown mechanism in the library.
Each package now has a single init/term routine.
This new way should avoid packages being re-initialized during library
shutdown and is also be _much_ more proactive about giving feedback for
resource leaks internal to the library.
Introduces a new "module" header file for packages in the library
(e.g src/H5Fmodule.h) which sets up some necessary package configuration macros
for the FUNC_ENTER/LEAVE macros. (The VFL drivers have their own slightly
modified version of this header, src/H5FDdrvr_module.h)
Also cleaned up a bunch of resources leaks all across the library and tests,
along with addressing many warnings, as I encountered them.
Tested on:
MacOSX/64 10.10.5 (amazon) w/serial & parallel
Linux/64 3.10.x (kituo) w/serial & parallel
Linux/64 2.6.x (ostrich) w/serial
Renamed a few "h5tri_t extended" variables to "h5tri_t was_extended".
Apparently, "extended" is a typedef name someplace in the headers used
on Solaris, which causes potentially confusing warnings to be emitted.
Tested on: h5committest
Solaris w/ Solaris Studio (emu)
Switch propert list/class iteration from internal to external form of
iteration, cleaning up and simplifying the code a bit.
Bring other general improvements from plist_encode_decode branch back to
trunk.
Clean up many warnings.
Tested on:
Mac OSX/64 10.7.4 (amazon) w/gcc 4.7, debug and C++ & FORTRAN
(too minor to require h5committest)
Description:
When using the new object header format, it was possible for corruption to occur
if the first object header chunk changed size such that the lenght of the "chunk
0 size" field changed. This only occurred if there were messages that had not
been decoded. The original algorithm that changed the object header chunk size
marked all messages as dirty, causing those that had not been decoded to have
both the raw and native form invalidated. Changed the algorithm to avoid
marking messages dirty and added assertions to catch the case where messages
are dirtied without being decoded (or recently created) first.
Tested: jam, koala, ostrich (h5committest), durandal
Description:
When using the new object header format and adding an attribute with a size near
64K, it was possible for file corruption to occur. This happened only if the
first object header chunk was smaller than 256 bytes and then grew to larger
than 64K after the attribute was added.
Tested: ostrich, jam, koala (h5committest), durandal
Refactor function name macros and simplify the FUNC_ENTER macros, to clear
away the cruft and prepare for further cleanups.
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug, production & parallel
Bring revisions from Coverity branch back to trunk:
r19044:
Coverity #449 - Line 1560 called function H5O_chunk_protect for 2 pointers to allocate. But when there's failure on the second one, the first wasn't freed
(H5O_chunk_unprotect). We fixed it by freeing the pointers when an error happens.
r19045:
Fixed coverity issue # 319. Free sec_node in done if it is not NULL.
r19046:
Add intended but missing assignments to initialize pointers to NULL (coverity issue fixes).
r19049:
Hdf5_1_8_coverity branch was recreated from hdf5_1_8 branch in revision 18839 without fix for Coverity issue #84 having been propagated to the hdf5_1_8 branch. This revision adds the fix again.
r19060:
added parentheses to see if they will keep subversion from getting confused
r19061:
Fix coverity item 139. Fixed incorrect condition for freeing buffer on error.
Fix coverity items 20 and 21. Removed unused NTESTS facility from dtypes.c.
Cleanup in H5Shyper.c.
r19062:
Fix coverity item 450. Check to see if chk_proxy has been allocated before
attempting to free it.
Fix coverity item 454. Check to see if allocation of buf failed in
H5D_fill_refill_vl.
Fix coverity items 455-457. Initilize hid_t's to -1, check their value before
attempting to close them, and check if the close failed.
r19063:
New fix to address coverity issue #84. Check that pointers in H5Z_xform_find_type are not NULL before passing them to H5T_cmp.
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & production
(Too minor to require h5committest)
Remove the size parameter from H5[A]C_unprotect() and the
H5[A]C__SIZE_CHANGED_FLAG, switching all clients & tests over to using
H5[A]C_resize_entry appropriately.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Bring r18720 from metadata journaling merging branch to trunk:
Bring changes from metadata journaling branch to 'merging' branch:
Rename H5[A]C_mark_pinned_or_protected_entry_dirty() to
H5[A]C_mark_entry_dirty() and get rid of H5[A]C_mark_pinned_entry_dirty().
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Bring r18715 from metadata journaling merging branch to trunk:
Changes to further align trunk with state of metadata journaling branch.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Add new internal object header routines to query the header chunk that
a message is in, and to lock/unlock a message into a chunk (so it doesn't get
moved into another chunk during allocation/free events).
Also, move dataspace message to beginning of object header messages
added to a dataset's object header, so it can be locked into chunk #0 (when
performing SWMR operations).
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Increment reference count on object header when "fake" object header
chunk proxy is returned from H5O_chunk_protect(). Also, clean up the
interface to H5O_chunk_unprotect by removing the unnecessary 'oh' parameter.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Bring r18542 from metadata journaling "merging" branch to trunk:
Bring new object header pin/unpin & protect/unprotect routines and
split-out object header chunk proxy changes from metadata_journaling branch to
"merging" branch, along with some other minor tweaks to clean up compiler
warnings, etc.
Also: clean up chunk protect/unprotect calls when allocating or freeing
space in a chunk, optimize metadata accumulator code to avoid some re-reading
of information from the file, refactor H5O_pin/H5O_unpin from way they are done
on the merging branch back to way they were previously done on trunk, other
minor code cleanups, etc.
Tested on
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Bring r18035 from merge_metadata_journaling branch to trunk:
More "brush clearing" convergence between metadata_journaling branch
and the trunk.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.2 (amazon) in debug mode
Mac OS X/32 10.6.2 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
More "brush clearing" convergence between metadata_journaling branch
and the trunk.
Tested on:
Mac OS X/32 10.6.2 (amazon) debug & prod
(h5committest not required on this branch)
Bring r18030 from merge_metadata_journaling branch to trunk:
Bring "brush clearing" changes (whitespace & style issues, mostly) from
metadata_journaling branch to the "merging" branch, to converge the trunk and
the metadata_journaling branch.
Also, some other minor cleanups along the way.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.2 (amazon) in debug mode
Mac OS X/32 10.6.2 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Bring "brush clearing" changes (whitespace & style issues, mostly) from
metadata_journaling branch to the "merging" branch, to converge the trunk and
the metadata_journaling branch.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.2 (amazon) in debug mode
Mac OS X/32 10.6.2 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Bring back various minor code cleanups from the file free space branch
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.8 (amazon) in debug mode
Mac OS X/32 10.5.8 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Description:
Since the new object header format, it has been possible for a situation to be
created where none of the messages are large enough to hold a continuation
message and there are no null messages to merge with. This makes it impossible
to add a new object header chunk. This case will now be handled by moving every
message in the last chunk to the newly allocated one, except for null messages
which are deleted.
Tested: jam, smirom (h5committest)
Description: Added a new function, H5O_alloc_shrink_chunk, which removes all
null messages from a chunk and shrinks the chunk appropriately. Modified
H5O_merge_null to call this function when a null message is created with a size
>= 64k. Tests added for this functionality.
Tested: kagiso (h5committest on 1.8 version)
Bring file free space branch changes through r15795 into trunk, which
includes a fair bit of code cleanup & rearrangement along with a couple of
bug fixes also.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Centralize all macros for declaring "magic numbers"/signatures for
objects in the file into src/H5Fprivate.h, so it's easier to know what values
have already been defined, etc.
Tested on:
Mac OS X/32 10.5.4 (amazon) in debug mode
Mac OS X/32 10.5.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Remove trailing whitespace from C/C++ source files, with the following
script:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Tested on:
Mac OS X/32 10.5.5 (amazon)
No need for h5committest, just whitespace changes...
datatype versions are encountered.
Description: The library now recognizes some problems with datatype versions in
H5O_decode_helper(), and, if not performing strict format checks, automatically
corrects them. Framework added for other message decode routines to
automatically correct file errors. Datatype version information added to
h5debug.
Tested: kagiso, smirom, linew (h5committest)
Move H5Pinsert() out of old "compat v1.6" section and into API
versioning, switching internal usage to H5Pinsert2().
Add regression test for H5Pinsert1()
Clean up some other minor warnings.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.4.10 (amazon) in debug mode
Fix bug with moving object header messages forward into null messages
of exactly the same size and located in an object header chunk with a gap.
(Exposed by netCDF-4 test)
Tested on:
FreeBSD/32 6.2 (duty)
Fix possible file corruption when using "new" format object headers and the
size of chunk #0 for an object header transitions between needing 1->2->4->8-
byte encoding for the size and there are "clean" messages in the object header
already. (Usually triggered by flushing the file while adding attributes to
an object)
Tested on:
Mac OS X/32 10.4.9 (amazon)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Check in "unique, but sharable" optimization to ISOHM code, which
allows object header messages that are only used in one object to remain in
the sole user's header, but migrates messages that are used in more than one
header into the ISOHM heap.
Tested on:
Mac OS X/32 10.4.9 (amazon)
FreeBSD/32 6.2 (duty)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Move ref. count of # of links to an object out of the object header's
prefix and make it a header message instead (since it's a "rare" occurence),
eliminating some more space for each object in the file.
Inserting this "ref. count" message exposed a flaw in the library's
mechanism for locating a message to promote to another chunk and replace
with a continuation message, which required some additional work to fix.
It's still not completely robust, but it's working for more cases now and
detects failures robustly.
Reduced the minimum size of an object header chunk to just enough to
contain a header message prefix and continuation message.
Tested on:
FreeBSD/32 6.2 (duty)
Fix mis-calculation of chunk #0 size encoding and also catch a few more
places which adjust the size of chunk and have them adjust the encode flags
also. Add some assertions to object header sanity checking routine.
Tested on:
FreeBSD/32 6.2 (duty)
Eliminate message count from new version of object header prefix -
it can be computed when the header is loaded and the table of messages is
built.
Tested on:
FreeBSD/32 6.2 (duty)
