Purpose:
HDF5 now supports SZIP with no encoder.
Description:
SZIP can be configured to have both encoder and decoder or just to have the decoder. HDF5 can now query the configuration of any filter, and will throw errors if users try to write using a filter with encoding disabled.
Solution:
Added H5Zget_filter_info function, changed API for H5Pget_filter and H5P_get_filter_by_id. See SZIP RFC.
Platforms tested:
Copper (fortran, C++, parallel), Sleipnir (C++), Arabica (fortran, C++), Verbena (fortran, C++)
Misc. update:
Code cleanup & minor optimization
Description:
Re-work the way interface initialization routines are specified in the
library to avoid the overhead of checking for them in routines where there is
no interface initialization routine. This cleans up warnings with gcc 3.4,
reduces the library binary size a bit (about 2-3%) and should speedup the
library's execution slightly.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/gcc34
h5committest
Code optimization
Description:
Avoid memcpy() when setting up new chunk coordinates
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
Code optimization
Description:
Eliminate memcpy() when using default DXPL by pointing at existing
default object, instead of copying it.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
Code optimization
Description:
Use 'size_t' instead of 'hsize_t' to track the number of elements in
memory buffers, especially for type conversion.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
Code optimization
Description:
Revised dataspace selections to use a more "object oriented" mechanism
to set the function pointers for each selection and selection iterator. This
reduces the amount and number of times that dataspace selection info has to
be copied.
Additionally, change hyperslab selection information to be dynamically
allocated instead of an inline struct.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
Code optimization
Description:
Don't allocate conversion buffer larger than the user's buffer.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel
too minor to require h5committest
Refactor code
Description:
Move chunk and contiguous cached raw data from file information to dataset
information. This simplifies a number of internal interfaces, aligns the
code with it's purpose better and should allow more optimizations to the
chunked data I/O performance.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir)
h5committest
Misc. update:
Code optimization
Description:
Don't recompute the internal index value for looking up the chunk in the
hash table, just use the value already computed from iterating through the
chunks.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir) w/parallel
Code optimization & bug fix
Description:
When dimension information is being stored in the storage layout message
on disk, it is stored as 32-bit quantities, possibly truncating the dimension
information, if a dimension is greater than 32-bits in size.
Solution:
Fix the storage layout message problem by revising file format to not store
dimension information, since it is already available in the dataspace.
Also revise the storage layout data structures to be more compartmentalized
for the information for contiguous, chunked and compact storage.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel
Solaris 2.7 (arabica)
h5committest
Code optimization
Description:
Expand the use of macros to inline trivial function pointer lookup and
calls to reduce the overall number of functions invoked during normal operation
of the library.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir) w/parallel
Too minor to require h5committest
Code optimization
Description:
Defer creating the span trees for hyperslab selections until they are
actually needed (which may be never, in certain circumstances).
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir) w/parallel
Code optimization
Description:
Further reduce the number of copies we make of a hyperslab selection for
chunked I/O, especially when we are only going to throw the old selection away
for a new one.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir) w/parallel
Bug fix.
Description:
Fixed handling of trivial data transform expressions (like 5/3 + 3) and some
data coversion fixes. Also added more tests to dtransform.c
Solution:
Added some more checks in the H5Z_xform_reduce_tree function to see if perhaps
the transform expression is complicated and is a non-trivial reduction.
Added tests for data conversion to dtransform as well as tests for a trivial
data transform expression.
Platforms tested:
h5committest'ed, except used arabica instead of sol and didn't do on copper
b/c no logon there. Problem noted with mtime test...doesn't appear to be
related to anything having to do with data transforms.
Code cleanup
Description:
Refactored data transform code to reduce amount of symbols in the global
scope and also cleaned up & simplified the code a bit.
Platforms tested:
h5committest (minus copper, plus serial modi4)
FreeBSD 4.9 (sleipnir) w & w/o parallel
New Feature
Description:
Add the data transform function, H5Pset_transform().
Platforms tested:
"h5committested".
Copper was down. Ran parallel tests in sol instead.
Misc. update:
Code optimization
Description:
Reduce the number of times the number of elements in a selection is
computed.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir) w/parallel
too minor to require h5committest
Code optimization
Description:
Move the element size for the selection into the selection iterator instead
of always passing it as a parameter.
Also, eleminate another 64-bit multiply for "all" selections.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Code optimization
Description:
Avoid dividing the chunk coordinates at the top levels of the chunk I/O
routines, only to multiply them at the bottom of the routines.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Code cleanup
Description:
Clean up lots of warnings based on those reported from the SGI compilers
as well as gcc.
Platforms tested:
SGI O3900, IRIX64 6.5 (Cheryl's SGI machine)
FreeBSD 4.9 (sleipnir) w/ & w/o parallel
h5committest
Code cleanup
Description:
Update null dataspace changes to try to write older version of dataspace
information whenever possible.
Refactor common code to only one location.
Allow I/O operations to succeed on null dataspaces.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Code optimization
Description:
Remove redundant function call by re-using value we already know.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Code optimization
Description:
Remove another dataspace copy, in certain circumstances.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Code optimization
Description:
Avoid another extraneous dataspace copy.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir) w/parallel
too minor to require h5committest
Code optimization
Description:
Change algorithm to directly use coordinates describing a chunk's position
in a dataspace instead of creating a dataspace with the chunk's position
selected. This reduces the number of copies of dataspaces we need to keep
around.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir) w/parallel
too minor to require h5committest
Code optimization
Description:
Check for only performing I/O on a single chunk and re-use memory dataspace,
instead of re-creating the same dataspace & selection the hard way.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir) w/parallel
too minor to require h5committest
Code optimization
Description:
Eliminate more dataspace copying, etc.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir) w/parallel
too minor to require h5committest
Code optimization
Description:
Reduce the number of dataspace copies made when performing I/O on chunked
datasets.
Platforms tested:
Solaris 2.7 (arabica)
too minor to require h5committest
Code cleanup/optimization
Description:
Query property list values once, at the beginning of the I/O routines,
instead of querying the property list values multiple (lots!) of times in
lower level routines.
Solution:
Create "property list caches" for internal library queries of the property
list values.
Platforms tested:
IBM p690 (copper) w/parallel & fphdf5
h5committest
Bug fix
Description:
Correct two problems with variable-length datatypes in datasets:
- When overwriting an entire dataset, writing the fill value to the
file would be skipped, causing problems for VL datatypes when
objects in the file had been unlinked (and thus the space in the
file was not all zeros)
- When an application has set a fill-value for a dataset and the
dataset's datatype contained a VL datatype, the library was filling
space on disk with the memory form of the VL information, instead
of the disk form.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug fix.
Description:
Using a selection offset with hyperslab selections in chunked datasets
was getting into an infinite loop and hanging the application.
Solution:
Apply the selection offset to the hyperslab selection properly.
Platforms tested:
FreeBSD 4.9 (sleipnir) w & w/o parallel
h5committest
Code cleanup
Description:
Clean up compiler warnings, especially the 'FUNC' variable not used which
comes out in production mode.
Solution:
Had to add a new FUNC_ENTER_NOAPI_NOINIT_NOFUNC macro for those non-API
functions which don't need the 'FUNC' variable defined. (This will be _so_
much easier when C99 is standard on all our supposed platforms, since it has a
__FUNC__ macro... )
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor for h5committest (although there were lots of files changed, the
change was minor in each one)
Bug fix
Description:
Variable length strings and sequences with NULL pointers were not handled
by library, causing problems access the data. This also affected fill values
for variable-length datatypes.
Solution:
Address the issues in the library by detecting NULL sequences/strings
and avoid trying to convert them.
Patched up dumper to display NULL sequences/strings.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Fix serial build
Description:
I accidentally put the "use_par_opt_io" variable in an #ifdef PARALLEL
section.
Solution:
Hoist it out of parallel section
Platforms tested:
Eyeballed it - very trivial
Code cleanup
Description:
Straighten out more goofiness in the MPI code dealing with collective I/O
transfers - mostly make certain that a view is set if-and-only-if collective
I/O is occurring on raw data (and vice versa for views and independent I/O)
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel & FPHDF5
too minor to repquire h5committest
Code cleanup
Description:
More refactoring on setup of MPI-IO parameters: hoisted the code to change
the transfer mode from collective to independent up into the H5D_read and
H5D_write routines, instead of duplicating it inside each of the chunk/contig
read/write routines.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Code cleanup
Description:
Clean up setup code for collective I/O transfers to make the logic more
obvious.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor for h5committest
Bug/feature fix.
Description:
Relax restriction on parallel writing to compact datasets to allow partial
I/O.
Updates to reference manual mentioning the issues involved are delayed until
reference manual 'lock' is removed later this week.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug fix
Description:
H5Dwrite was only checking the "top level" of the datatype to stop parallel
I/O on variable-length datatypes.
Solution:
Make checks "deeper".
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug fix
Description:
I/O on chunked datasets with a scalar dataspace for the memory dataspace
was not working correctly.
Solution:
Translate the scalar dataspace into a n-dimensional (where n is the number
of dimensions of the file's dataspace) dataspace of dimensions 1x1x1...
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest (modulo modi4 which is not working correctly)
Bug fix
Description:
I/O on chunked datasets with point selections was not working correctly.
Solution:
Re-wrote some parts of raw data I/O routines that build the selections for
each chunk to correctly handle point selections.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
Bug fix
Description:
H5T_BKG_TEMP was accidentally removed from library code, but is used by
application's datatype conversion routines.
Solution:
Revert removal of H5T_BKG_TEMP.
Platforms tested:
h5committest
Refactoring code
Description:
Refactored "IS_H5FD_MPIO || IS_H5FD_MPIPOSIX || IS_H5FD_FPHDF5" combination
of macros in many places into single IS_H5FD_MPI macro, which has the same
definition, but should be easier to maintain.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
Bug fix
Description:
The dataset's modification time was getting set whenever raw data was
written with H5Dwrite. Unfortunately, this is a metadata change (which are
required to be performed collectively) and H5Dwrite may be called independently
from a parallel program, resulting in metadata cache corruption and/or program
hangs.
Solution:
Don't update the modification time when raw data it written. :-(
Platforms tested:
FreeBSD 4.8 (sleipnir) w/parallel
h5committest
Misc. update:
Noted in release notes and also sent to Frank for updating the docs.
