Bring r19352 from trunk to 1.8 branch:
Align w/changes on revise_chunks branch
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & production
(too minor to require h5committest)
Bring r19297 from trunk to 1.8 branch:
Whack a few more memory errors exposed by valgrind.
Make the debugging dump output a little bit prettier.
Tested on:
FreeBSD/32 6.3 (duty) w/debug
(h5committested on trunk)
fill values.
Description:
Previously, H5D_chunk_allocate would, if a dataset were filtered and had a non-default VL fill value, allocate a new buffer for every chunk. Changed
H5D_chunk_allocate to reuse the existing buffer if possible. Also other
miscellaneous cleanup.
Tested: jam, linew, amani (h5committest)
Description:
Previously, it was possible for a chunk to be flushed due to chunk operations in
the callback from H5B_iterate in H5D_chunk_prune_by_extent. Because flushing
the chunk can force it to be reallocated if it is filtered, this can change the
contents of the chunk b-tree in the middle of H5B_iterate. Because H5B_iterate
uses a locally cached copy of the b-tree, this causes subsequent operations
to be passed incorrect data. Rewrote H5D_chunk_prune_by_extent to avoid
H5B_iterate entirely.
Also fixed a bug in the dataset expand algorithm that could cause extra chunks
to be created.
Tested: jam, linew, smirom (h5committest)
Description:
Modified algorithm for extending a dataset with early allocation so it only
deals with the new chunks. Formerly, it would loop over all chunks, checking
to see if each existed in cache and on disk, causing major performance issues
with large numbers of chunks.
Tested: jam, linew, amani (h5committest)
Description:
In certain cases, removal of an object in a v1 b-tree would cause the leftmost
key in the right neighbor to be overwritten. While this did not pose a problem
for group b-trees, with chunked datasets it would overwrite the offset value
of the neighbor's leftmost child, causing corruption. Reworked the code to
differentiate between b-trees whose children are fundamentally associated with
their left key and those who are associated with their right key.
Tested: jam, linew, amani (h5committest)
Description:
In some situations it was possible for the fill value to not be written to parts
of a chunked dataset, particularly when extending and/or shrinking. Prior to
the fix for the chunk cache (1015) these bugs would have been exceedingly rare.
Tested: jam, smirom, linew (h5committest)
bug fix: TESTING2 macro introduced recently for the test suite (it adds an extra space at the beginning of output, for indentation) was already available for HL and caused compilation problems on AIX. Replaced the name with TESTING_2
tested: h5committest
#1449 Change set_extent test to use standard standalone test framework
Call h5_fixname (with an array of test filenames) for generating the
filename to create and then call h5_cleanup() when the tests pass, to delete the files
created and close the FAPL from h5_fileaccess().
Defined a macro
#define TESTING2(WHAT) {printf(" Testing %-62s",WHAT); fflush(stdout);}
Similar to TESTING, except that it has a initial indentation space.
The effect is for nested loop tests
Testing with old file format:
Testing with fill value, no compression PASSED
tested: windows, linux
Bring r16417 from trunk to 1.8 branch:
Bring closer to standard standalone test format, add checks for using
the latest file format and close dataset ID leaked.
Tested on:
FreeBSD/32 6.3 (duty)
(too minor to require h5committest)
#1449 Change set_extent test to use standard standalone test framework
use a fapl obtained by h5_fileaccess to the call of H5Fcreate
and redo the test calls with
h5_reset();
fapl = h5_fileaccess();
nerrors += do_ranks( fapl ) < 0 ? 1 : 0;
nerrors += test_external( fapl ) < 0 ? 1 : 0;
nerrors += do_layouts( fapl ) < 0 ? 1 : 0;
tested: windows, linux
Modified H5Dset_extent so that it fails when called for compact datasets and contiguous with no external storage
new test program: It adds tests for several ranks, use of fill value or not, compression, different fill value allocation times, use of different storage layouts, and external files
tested: windows, linux
Test the non deletion of a chunk for the case chunk_offset == dimension by shrinking first, then expanding to original size and compare buffers of initialized values
Tested: windows, linux
close & re-open the file for the test of non-default B-tree internal 'K' value (There was a bug where a non-default B-tree internal 'K' value was not being stored in the file)
this is a rewrite of the old set_extent.c test program
tested: windows, linux
16132
some chunks were not deleted for some cases, the comparison to check for chunk offsets outside of the new dimensions was
chunk_offset[ i ] > dimension [ i ]
and it must be
chunk_offset[ i ] >= dimension [ i ]
16133
H5TBdelete_record was not handlong correctly records at the end of the table
added a ckeck that avoids to read these records if they are not needed (for pushing down the table)
16143
add a test for recent fixes of H5Dset_extent
tested: windows, linux
Bring back revision 15131 from trunk:
Finish omnibus chunked dataset I/O refactoring, to separate general
actions on chunked datasets from actions that are specific to using the v1
B-tree index.
Cleaned up a few bugs and added some additional tests also.
Tested on:
Mac OS X/32 10.5.3 (amazon)
Linux/32 2.4 (chicago)
Make H5Dopen versioned and change all internal usage to use H5Dopen2
Add simple regression test for H5Dopen1
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.4.10 (amazon) in debug mode
Tested platform:
Kagiso only since it is only a comment block change. If it works in one
machine, it should work in all, I hope. Still need to check the parallel
build on copper.
Code cleanup
Description:
Trim trailing whitespace, which is making 'diff'ing the two branches
difficult.
Solution:
Ran this script in each directory:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Code optimization
Description:
Don't recompute the internal index value for looking up the chunk in the
hash table, just use the value already computed from iterating through the
chunks.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir) w/parallel
Bug fix
Description:
When a non-default indexed storage B-tree internal 'K' value is set by the
user, the chunked datasets created in that file (until it is closed) use the
user's 'K' value and the data can be accessed correctly, but the 'K' value is
not stored in the file.
However, once the file is closed and re-opened, the non-default 'K' value
is lost and the data in the chunked datasets will not be able to be accessed
correctly.
Solution:
Store the indexed storage B-tree internal 'K' value in the superblock.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
New feature/Bug fix
Description:
Add new fill time value - H5D_FILL_TIME_IFSET which writes the fill value
to a dataset if the user has defined one, otherwise not writing the fill value
to the dataset.
Platforms tested:
FreeBSD 4.8 (sleipnir) serial & parallel
h5committest
New feature/enhancement
Description:
Chunked datasets are handled poorly in several circumstances involving
certain selections and chunks that are too large for the chunk cache and/or
chunks with filters, causing the chunk to be read from disk multiple times.
Solution:
Rearrange raw data I/O infrastructure to handle chunked datasets in a much
more friendly way by creating a selection in memory and on disk for each chunk
in a chunked dataset and performing all of the I/O on that chunk at one time.
There are still some scalability (the current code attempts to
create a selection for all the chunks in the dataset, instead of just the
chunks that are accessed, requiring portions of the istore.c and fillval.c
tests to be commented out) and performance issues, but checking this in will
allow the changes to be tested by a much wider audience while I address the
remaining issues.
Platforms tested:
h5committested, FreeBSD 4.8 (sleipnir) serial & parallel, Linux 2.4 (eirene)
Update
Description:
Updated the Copyright statement
Platforms tested:
Linux (This change is only in the comments, so I just check that the
modules still compile)
Misc. update:
Code cleanup
Description:
Clean up a few warnings which were showing up with --enable-production
turned on.
Platforms tested:
FreeBSD 4.6 (sleipnir) serial & parallel
Purpose:
added more tests to the H5set_extent function
Description:
there was a bug in the H5S_select_fill call when the fill value was not defined
added 2 more set of tests that call H5set_extent without the fill value being defined
there are now 4 sets of tests:
Test H5Dset_extent with chunks on the raw data cache, with and without fill value
Test H5Dset_extent with chunks written to file (b-tree routines), with and without fill value
Platforms tested:
w2000
linux
Bug Fix.
Description:
When the fill value for a dataset is not set, the size is set to zero,
causing problems with filling unused areas in previously defined chunks
(i.e. when the dataset shrinks in size)
Solution:
Use the size of the elements in the dataset directly, instead of relying on
the size of the fill value (which must be the same size).
Platforms tested:
FreeBSD 4.5 (sleipnir)
Purpose:
test for the H5Dset_extent API function
Description:
there are 2 types of tests:
Test H5Dset_extent with chunks on the raw data cache
Test H5Dset_extent with chunks written to file
Platforms tested:
Windows 2000
SUN( arabica)
LINUX (eirene)
IRIX64 (modi4)
