Description:
Previously, it was possible for a chunk to be flushed due to chunk operations in
the callback from H5B_iterate in H5D_chunk_prune_by_extent. Because flushing
the chunk can force it to be reallocated if it is filtered, this can change the
contents of the chunk b-tree in the middle of H5B_iterate. Because H5B_iterate
uses a locally cached copy of the b-tree, this causes subsequent operations
to be passed incorrect data. Rewrote H5D_chunk_prune_by_extent to avoid
H5B_iterate entirely.
Also fixed a bug in the dataset expand algorithm that could cause extra chunks
to be created.
Tested: jam, linew, smirom (h5committest)
Bring r18740 from trunk to 1.8 branch:
Bring r18738 from metadata journaling 'merging' branch to trunk:
Switch H5AC_resize_pinned_entry() to work on protected entries as well,
and rename to H5AC_resize_entry()
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
(h5committested on trunk)
Bring r18725 from trunk to 1.8 branch:
Bring r18724 from metadata journaling 'merging' branch to trunk:
Rename H5[A]C_rename to H5[A]C_move_entry.
Tested on:
FreeBSD/32 6.3 (duty) w/debug
(h5committested on trunk)
Bring r18721 from trunk to 1.8 branch:
Bring r18720 from metadata journaling merging branch to trunk:
Bring changes from metadata journaling branch to 'merging' branch:
Rename H5[A]C_mark_pinned_or_protected_entry_dirty() to
H5[A]C_mark_entry_dirty() and get rid of H5[A]C_mark_pinned_entry_dirty().
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(h5committested on trunk)
Improve configure's large-file support control.
Description:
Modified configure to now attempt to add defines necessary for
supporting largefiles on all systems, instead of solely on linux. This
is in response to user requests to enable largefile support on Solaris
by default, as well as to give extra control on AIX (instead of just
jamming the necessary flag into the config files).
The old --enable-linux-lfs flag is being deprecated in favor of the
--enable-largefile flag (enabled by default), which can be used on all
platforms. --disable-linux-lfs can still be used to disable largefile
support (on linux) when the --enable-largefile flag is not specified.
On systems where large files cannot be supported in this manner,
configure will report as such.
Tested:
h5committest
AIX (NCSA's blue_print machine)
duty, liberty, and linew.
Bring r18702 from trunk to 1.8 branch:
Bring r18672 from metadata journaling "merging" branch to trunk:
Mostly changes to move to only using one 'user data' parameter for
calls to H5AC_protect(), along with some minor reformatting code cleanups.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
the data conversion test from unsigned long long to long double. The GCC on Cygwin seems to
have rounding problem when doing conversion from unsigned long long to long double (Please bug
#1813).
Tested on jam. I'll test it on Cygwin after this checkin.
Bring r18664 from trunk to 1.8 branch:
Add a few H5Tequal() calls to make certain that the native datatype
constructed with H5Tget_native_type() is identical to the memory datatype
created from the struct in memory.
Tested on:
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
(h5committested on trunk)
Added check in H5Rcreate for space_id = -1 and is region reference type.
Added test of H5R package API parameters to testhdf5 test, these test only the public API parameter checks.
Tested: local linux
Bring r18627 from trunk to 1.8 branch:
More progress toward addressing Bz#1398: add 'volatile' keyword to floating-
point types when detecting their properties. Also, minor code formatting
cleanups.
Tested on:
Mac OS X/32 10.6.3 (amazon) w/debug)
(h5committested on trunk)
Bring r18618 from trunk to 1.8 branch:
Add new internal object header routines to query the header chunk that
a message is in, and to lock/unlock a message into a chunk (so it doesn't get
moved into another chunk during allocation/free events).
Also, move dataspace message to beginning of object header messages
added to a dataset's object header, so it can be locked into chunk #0 (when
performing SWMR operations).
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
(h5committested on trunk)
Autotool Upgrade / Libtool Bug Fix
Description:
Updated autotools referenced in bin/reconfigure as follows:
Autoconf 2.64 --> Autoconf 2.65
Automake 1.11 --> Automake 1.11.1
Libtool 2.2.6a --> Libtool 2.2.6b-mcg
The referenced libtool version is a custom version of 2.2.6b. It
has been tweaked to fix a bug in libtool that occurs
when using PGI 10.0 compilers. A check incorrectly categorizes
the C++ compiler as version 1.0 instead of 10.0, and the link
line is subsequently set up incorrectly and fails to compile.
A patch has been made available and will be included in the next
release of libtool, but in the meantime I've applied the patch to a custom
installation as indicated above. This bin/reconfigure now references
the custom installation, and the resulting configure script will correctly
categorize the PGI 10.0 C++ compiler.
Ran bin/reconfigure to regenerate configure and makefiles.
Tested:
jam and amani with PGI 10.0 compilers.
h5committest
Bring r18571 from trunk to 1.8 branch:
Bring r18542 from metadata journaling "merging" branch to trunk:
Bring new object header pin/unpin & protect/unprotect routines and
split-out object header chunk proxy changes from metadata_journaling branch to
"merging" branch, along with some other minor tweaks to clean up compiler
warnings, etc.
Also: clean up chunk protect/unprotect calls when allocating or freeing
space in a chunk, optimize metadata accumulator code to avoid some re-reading
of information from the file, refactor H5O_pin/H5O_unpin from way they are done
on the merging branch back to way they were previously done on trunk, other
minor code cleanups, etc.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
(h5committested on trunk)
Description:
Attempting to copy an object with NULL references (all bytes zero) with the
H5O_COPY_EXPAND_REFERENCE_FLAG flag set would cause a failure or an assertion
(depending on whether it was in debug mode). Changed copy routine to detect
NULL references (object and region) and avoid attempting to expand the reference
in this case.
Tested: jam, linew, amani (h5committest)
Description:
Modified algorithm for extending a dataset with early allocation so it only
deals with the new chunks. Formerly, it would loop over all chunks, checking
to see if each existed in cache and on disk, causing major performance issues
with large numbers of chunks.
Tested: jam, linew, amani (h5committest)
Bring r18413:18420 from trunk to 1.8 branch:
- unify srcdir query, etc.
- Move libhdf5.settings string out of H5Tinit.c
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.2 (amazon) in debug mode
Mac OS X/32 10.6.2 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Description:
The core file driver previously matched files only based on the name used to
open them, even when a backing store is used. This caused problems in the
failing test when it traversed an external link to itself, and did not recognize
that it was the same file. Similar tests in links.c were skipped when the core
file driver was used. The failure was previously masked by inappropriate file
flushes caused by external link traversal. Added code to the core file driver
to keep track of low level file information of the backings store to allow for
accurate recognition of duplicate files. Also re-enabled all links.c tests for
the core file driver.
Tested: Fedora (core and sec2 drivers; no time for full commit test, will
monitor daily tests)
Bring r18391 from trunk to 1.8 branch:
Clean up compiler warnings, check error values and fix whitespace.
Tested on:
FreeBSD/64 6.3 (duty) w/debug
(too minor to require h5committest)
Bring r18388 from trunk back to 1.8 branch:
Bring changes from Coverity session from branch into trunk:
r18378:
Fixed coverity issues 207 and 322. Pointer hdr was checked for NULL after being
asserted and dereferenced. Check was removed.
r18379:
Fix coverity issues # 88 and # 435.
r18380:
Fixed Coverity issue # 85. Added check of returned pointer for NULL before use.
r18381:
Resolve coverity issues # 214 and # 215
r18382:
Issue 131: Add null checks to allocations and check for free in error handling
r18383:
Issue 421:
Reorganized code to make intention clearer. Also, set local variable fl to NULL
after transfer to tail. Heap->freelist will take care of all allocations
r18384:
Coverity #249 and #250 - STRING_ATT_CHECK wasn't allocated before being used
and freed in function test_write_vl_string_attribute and
test_read_vl_string_attribute.
Tested on:
Mac OS X/32 10.6.2 (amazon) w/debug & prod
(h5committested in daily tests)
Description:
Support for expanding external links was not implemented in H5Ocopy, even though
a flag existed for H5Pset_copy_object to enable this. This patch implements
this feature.
Tested: jam, amani, linew (h5committest); Fedora
Description:
Previously, the library would reopen the source file when traversing an external
link if unable to find the target file otherwise. This has been corrected.
Also moved the call to H5F_flush from H5F_try_close to H5F_dest, so the file is
only flushed when the last identifier for the file is closed. This prevernts
situations where the library could attempt to flush a file with protected
metadata.
Tested: jam, amani, linew (h5committest); Fedora
Bring r18346 from trunk to 1.8 branch:
Bring Coverity fixes back from branch to trunk:
r18336:
Fix coverity issues 275, 276, 277, 323, 432, 433, and 434
r18337:
Fix Coverity issue #106: release free space section node on error
r18338:
Fixed Coverity #94 - In H5P_register, new_class wasn't closed when there's an
error after it's created.
r18339:
Fix Coverity #185 - In test_conv_str_1, BUF wasn't freed when there's an error
in this function.
r18340:
Correct error in r18337 that wasn't releasing indirect fractal heap block
early enough.
r18341:
Close nodes if any failed in the middle of allocating new nodes. Coverity 140
and 141
r18342:
Correct [another] problem w/r18337.
r18343:
Fix coverity items 185, 20, and 21.
r18344:
Fix Coverity 213 - In H5FD_family_close, the double pointer file->memb was
dereferenced without NULL checking
(We believe).
r18345:
Fix Coverity issue # 210; removed NULL check after pointer dereferenced in
H5HFdblock.c. Also assigned NULL to pointer in H5Pint.c to fix segmentation
fault.
Tested on:
FreeBSD/32 6.3 (duty) w/debug)
(h5committested on trunk)
Bring r18300 from trunk to 1.8 branch:
Bring Coverity fixes from branch to trunk:
r18282:
Fix Coverity issue #428 by wrapping testing calls with if(pass) {} block.
r18283:
Fix Coverity issue #425 by wrapping test calls in if(pass) {} block
r18284:
Issue 166: init_error() malloc'd 3 pointers in initialization and never freed
inc ase of errors. Init pointers to NULL, check allocation results and free
allocations in error block
r18285:
Fix Coverity issue #410 by wrapping test calls with if(pass) {} block.
r18286:
Issue 165: custom_print_cb() needed allocations freed in error block.
r18287:
Fix coverity issue # 409
Added if (pass) checks around calls to flush_cache. Additionally,
added a check for file_ptr = NULL after call to setup_cache.
r18288:
Fix coverity# 107 free fh in H5HF_close() correctly before exit the function
even when failure occurs.
r18289:
Fix Coverity issue #429: correct failure return values to match return type
from routine.
r18290:
Fix Coverity issue #103: release allocated indirect section on error
r18294:
Issue 153, 152: Check allocations and free allocations in error block. Also
cleaned up hid_t identifer that were opened in error block.
r18295:
Fix coverity# 101 free new_loc in H5HF_man_iter_start_entry() correctly before
exit the function even when failure occurs
r18296:
Fix coverity# 100 free down_loc in H5HF_man_iter_down() before exit the function
when failure occurs
r18297:
Fixed coverity issues 54, 55 and 216. Correctly handle the various ways that
allocation of attr_name can fail in test_attr_basic_write.
r18298:
Fix coverity# 119 free object in H5HG_read() before exit the function when
failure occurs
r18299:
Fix coverity issue #112:
Add cleanup during error handling of H5MP_create.
Tested on:
Mac OS X/32 10.6.2 (amazon) w/debug & production
Misc. Linux configurations (on original checkins)
Description:
In certain cases, removal of an object in a v1 b-tree would cause the leftmost
key in the right neighbor to be overwritten. While this did not pose a problem
for group b-trees, with chunked datasets it would overwrite the offset value
of the neighbor's leftmost child, causing corruption. Reworked the code to
differentiate between b-trees whose children are fundamentally associated with
their left key and those who are associated with their right key.
Tested: jam, linew, amani (h5committest)
Bring r18256 from trunk to 1.8 branch:
Bring changes from Coverity fixing branch to trunk:
r18235:
Fixed coverity 114:
if (NULL ==_dest)
H5MM_free(dest);
r18236:
Close Coverity issue #28 (again :-) by working through the logic of the
routine more thoroughly to eliminate the goto statements. (LK & QK)
r18237:
fixed coverity 133:
if (NULL==_dest && NULL==ret_value && NULL != dest)
H5MM_free(dest);
r18238:
Fix coverity items 421 and 422. Added assertion that the heap's free list is
NULL when entering H5HL_fl_deserialize, guarateeing that the free list will
always be linked in even on failure.
r18239:
Fix coverity item 268. Changed H5MM_xfree(read_buf) to read_buf =
H5MM_xfree(read_buf) so that read_buf isn't manipulated after it's been freed.
r18241:
coverity fix: use correct free functon H5FL_FREE()from the previous fix
r18242:
Coverity fix 139: Free dst correctly in H5O_sdspace_copy().
r18243:
Fix Coverity issue #417 by checking for NULL return value from
setup_cache() (LK & QK)
r18244:
Coverity Fix 132: free dest correctly in H5O_efl_copy()
r18245:
Issue 121: H5S_hyper_make_spans() cannot deal with counts of 0. However,
H5Sselect_hyperslab() API does allow a count of 0.
Therefore, simply throw an error if this function encounters a count of 0.
r18246:
Check file_ptr to address coverity issue # 418
r18247:
Fixed coverity 113:
if (sequence)
H5MM_xfree(sequence);
r18248:
Coverity issue #414 by checking for NULL return from setup_cache() (LK & QK)
r18249:
fixed coverity 274: moved H5FL_FREE(H5A_t, attr); to a line above so that if
attr is null, it will not free it.
r18250:
Fix coverity issue #86.
Check return of malloc function to ensure non-null before continuing.
Tested on:
Mac OS X/32 10.6.2 (amazon) w/debug & production)
(h5committested in daily tests on branch)
Bring r18252 from trunk to 1.8 branch:
Rename test macros that are [nearly] identical to source code macros to
have "TEST" in their names, to avoid conflicts during compilation. (And add
comment to originals in source code, to keep them in sync).
Tested on:
Mac OS X/32 10.6.2 (amazon) w/debug & production
up041 machines (at Sandia)
Bring r18230 from trunk to 1.8 branch:
Bring r18229 from metadata journaling "merging" branch into trunk:
Bring r18227 from metadata_journaling branch to the merging branch:
Small cleanups and move v1 cache maintenance macros into package header
file, to better align with changes to v2 cache.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(h5committested on trunk)
Bring r18244 from trunk to 1.8 branch:
Bring revisions from Coverity session from hdf5_1_8_coverity branch to
trunk:
r18214:
Issue 423: cache_ptr was not initialized. if global 'pass' was false,
cache_ptr could be used. Extended scope of if(pass) block to include whole
function-flush_cache().
r18215:
Fix coverity item 317. Removed unreachable code in H5B2_hdr_alloc.
r18216:
Fixed Coverity issues 57 & 67. Checked oid_count > 0 before memory allocation
and oid_list != NULL before dereferencing.
Tested on:
Mac OS X/32 10.6.2 (amazon) w/debug & production
(daily tested on Coverity branch already)
Bring r18205 & r18222 from trunk to 1.8 branch:
Add new H5Oexists_by_name() routine.
Tested on:
Mac OS X/32 10.6.2 (amazon) w/debug & production
(daily tested on trunk already)
a bug report (#1608) saying h5repack failed to do that sometimes. But I couldn't reproduce it.
So I added this test and closed the report. The test is in the function test_copy_dcpl in dsets.c.
Tested on jam, amani, and linew.
The previous implementation did not work for WIndows.
Moved the macro definition of HDGetTestVerbosity from testhdf5.h
to h5test.h.
Removed as much as possible macro definitions that use HDGetTestVerbosity
directly.
Tested: Jam (serial and parallel). Did not run h5committest since this
is a trival change that is portable.
Bring r18171 from trunk to 1.8 branch (with appropriate tweaks):
Bring r18169 from 'merge_metadata_journaling' branch back to trunk:
Converge changes on metadata_journaling branch and trunk:
- Remove H5F_t* parameter from H5HL_unprotect()
- Remove H5F_t* parameter from H5HL_dirty()
- Remove H5F_t* parameter from H5O_unprotect()
- Bring changes to metadata caching routines back:
- H5AC_pin_protected_entry()
- H5AC_resize_pinned_entry()
- H5AC_unpin_entry()
- H5AC_mark_pinned_entry_dirty()
- H5AC_mark_pinned_or_protected_entry_dirty()
- Revise internal H5C routines to drop H5F_t* and/or H5C_t* as
parameter where possible
- Revise tests to follow internal routine changes
Also, other minor changes to speedup the 'cache' test.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.2 (amazon) in debug mode
Mac OS X/32 10.6.2 (amazon) w/C++ & FORTRAN, w/threadsafe,
Bring r18159 from trunk to 1.8 branch:
Bring Coverity fixes from 1/22/10 session to trunk:
r18137:
219: Initialized hid_t to -1 and added close to error block.
189-191: Initialized line to NULL and added free line, and close fp to error block.
r18138:
19: Moved code block for printing that the number of enums is empty to the error block. (Would never have been executed otherwise)
r18139:
Fix coverity item 58. Moved code related to displaying the parent of a repeated
group to the else(isRoot) section, as the root group has no parent.
r18140:
218: Initialized ret_value variable to -1. Because of throw Exception in default case of switch, the coverity problem would not have executed anyway. Good pratice is to initialize variables.
r18141:
Fix coverity item 92. Added code to H5E_register_class to free cls in case of
an error.
r18142:
Fix coverity item 91. Added code to H5E_create_msg to free msg in case of an
error.
r18143:
fixed issue 14,
took away "if" and used #ifndef_xxx.
r18144:
Fix coverity item 110. Added code to H5Eget_minor to free msg_str in case of
an error.
r18145:
fixed coverity #18 removed "aligned", it is always NULL.
r18146:
Fix coverity item 109. Added code to H5Eget_major to free msg_str in case of
an error.
r18147:
Fixed coverity #81 and #82, Check for bad pointer(s), but can't issue error, just leave
r18148:
Fix coverity item 97. Added code to H5FD_fapl_open to free copied_driver_info
in case of an error.
r18149:
Fix coverity item 96. Added code to H5FD_dxpl_open to free copied_driver_info
in case of an error.
r18150:
Fix Coverity issue #29: Protected cache_ptr dereferences with "if(pass)"
block
r18151:
Fix coverity item 93. Added code to H5FL_fac_init to free factory and new_node
in case of an error.
r18152:
Fix coverity items 98 and 99. Added code free allocated space in case of
error.
r18155:
124: Freed head pointer before jumping to done. There was no error handling block and normal exit used same path out.
120-123: Freed list of lists in error handling block.
r18156:
Fix coverity issues 179, 180, 181, 182, 183, 184, 186, 320, 407. These were resource leak issues where allocated memory was not freed, generally in the case of tests that failed.
Tested on:
Mac OS X/32 10.6.2 (amazon) debug & production
Bring r18157 from trunk to 1.8 branch:
Bring back changes from Coverity session on 1/15/10:
r18111:
Fix Coverity issue #130: make certain that the cache gets freed on error.
r18112:
Fix Coverity issue #43 by making cache testing calls protected by 'pass'
variable.
r18113:
Fix Coverity issue #129 by releasing the cache on error.
r18115:
Coverity #45 fix: patched an error check in H5Screate_simple to prevent future dereferencing of a NULL point.
Added a verification in test/th5s.c.
r18116:
Fix Coverity issue #43 by releasing cache on error.
r18117:
Coverity #362,363 by adding HGOTO_DONE, freeing allocations and associated changes. REsolving coverity results #364-368, 369, 370-372, 377, 379, and 380.
r18118:
Fix Coverity issue #42: assert that cache & test specification pointer are
valid.
r18122:
Coverity #362,363 by adding HGOTO_DONE and freeing allocations. This also takes care of #357,358.
r18123:
Coverity #359-361, 373-376: Added HGOTO_DONE(FAIL) statement after checking allocation for NULL. Verified allocation is freed in done block.
r18128:
Fixed coverity issue #10 -- removed dead code.
Tested on:
Mac OS X/32 10.6.2 (amazon)
GetTestVerbosity() is called many times during tests. Implemented a more
efficient macro version HDGetTestVerbosity.
Tested: Jam (serial and parallel). Did not run h5committest since this
is a trival change that is portable, I am 99% sure.
