Purpose:
Bugfix
Description:
The Stream VFD was leaking memory on every opened file.
Solution:
In H5FD_stream_close(), finally free the file structure used to describe
the closed file.
Platforms tested:
Linux, SGI
Rearrange code
Description:
The data sieve buffering code for contiguously stored datasets was
wedged in the H5F_arr_read/H5F_arr_write routines.
Solution:
Created a new H5Fcontig.c to hold I/O routines for contiguously stored
datasets (like H5Fistore.c for chunked dataset I/O routines) and moved
data sieving code into those routines.
Platforms tested:
Solaris 2.6 (i.e. baldric)
Code Optimization.
Description:
The optimized routines for copying regular hyperslabs in memory have been
using the same matrix routines to copy their hyperslab pieces as the
routines for irregularly shaped hyperslabs. This ends up imposing lots of
extra overhead on the optimized routine, since it basically "knows" all the
matrix information it needs.
Solution:
Keep track of the [small] amount of matrix information necessary to perform
the regular hyperslab copies in the optimized routines themselves instead of
using the matrix routines. This improves the performance for the benchmark
I'm running from ~18 seconds to ~12 seconds and should apply to parallel
I/O situations also.
Platforms tested:
Solaris 2.6 (i.e. baldric)
Code Optimization
Description:
The matrix operations are currently the hot-spot in the library code
for regular hyperslab operations.
Solution:
Unrolled loops for 3 of the more heavily used functions
(H5V_stride_optimize2, H5V_hyper_stride & H5V_hyper_copy) for the common
cases (i.e. up to 3-D datasets). This squeezes some more blood out of
the stone (turnip? :-) and improves the h5hypers.c benchmark on baldric
by another 20-25%.
Platforms tested:
Solaris 2.6 (i.e. baldric)
Bug Fix
Description:
The core and log VFL drivers were leaking small amounts of memory when they
were used.
Solution:
Free the appropriate memory block (for the core driver) and don't allocate
a block (for the log driver).
Platforms tested:
Solaris 2.6 (i.e. baldric)
Implemented new feature
Description:
Added data sieve buffering code to raw I/O data path. This is enabled for
all the VFL drivers except the mpio & core drivers. Also added two new
API functions to control the sieve buffer size: H5Pset_sieve_buf_size() and
H5Pget_sieve_buf_size().
Platforms tested:
Solaris 2.6 (i.e. baldric)
Fix compiler warning
Description:
"HUGE_VAL" (a double value) was being put into a float type and generating
a warning during compile time.
Solution:
Replaced "HUGE_VAL" with "FLT_MAX"
Platforms tested:
FreeBSD 4.1
Small Code Cleanup
Description:
Code to optimize adjacent (i.e. contiguous) hyperslab was ugly and used too
many temporary variables.
Solution:
Computed the optimized hyperslabs slightly differently and got rid of
unnecessary temporary variables.
Platforms tested:
FreeBSD 4.1
Bug fix (sorta)
Description:
When the stride and block size of a hyperslab selection are equal, the
blocks that are selected are contiguous in the dataset. Prior to my
hyperslab optimizations, this situation used to be detected and somewhat
optimized to improve performance. I've added more code to optimize for
this situation and integrated it with the new hyperslab optimization that
weren't very efficient for that case as they should have been.
Solution:
Detect contiguous hyperslab selections (i.e. block size in a dimension is
the same as the stride in that dimension) and store the optimized,
contiguous version of that hyperslab. We also store the original, un-
optimized version of the hyperslab to give back to the user if they query
the hyperslab selection they just made.
Platforms tested:
FreeBSD 4.1
Bug Fix
Description:
The prototype for the H5Pregister function has a variable named
`class'. This is a reserved word in C++ and causes the C++
compiler to freak.
Solution:
This variable's name was changed to cls_id in the .c file, so I
changed it in the header file to cls_id to match.
Platforms tested:
Linux
Bug Fix.
Description:
An assertion in the local heap code was mistakenly checking against too
large of a value for the size of new local heap created. When used with
larger-sized (>10KB) variable-length objects, it was failing the check.
Solution:
Corrected to check against the actual size of the heap allocated, without
the heap header.
Platforms tested:
FreeBSD 4.1
Restore file
Description:
It appears that Robb's checkin earlier today erroneously overwrote this
file with an older version... *grumble*
Solution:
Found another copy of newest version, verified that it is operating
correctly and re-checked it in.
Platforms tested:
FreeBSD 4.1
H5FDstream.h needs to be installed.
Description:
H5FDstream.h is included in the hdf5.h file and needs to be
installed with the other public headers.
Solution:
Added it to the rest of the install headers.
Fix Irix pmake bugs
Description:
Build fails on Irix when builddir != srcdir
Solution:
* acconfig.h
* src/H5config.h.in [REGENERATED]
Added definition for HAVE_STREAM
* config/conclude.in
* config/depend1.in
* config/depend2.in
* config/depend3.in
* config/depend4.in
The `Dependencies' file is located in the source
tree. This fixes bugs for Irix pmake when compiling
outside the source tree. Hopefully it still preserves
Albert's changes which allow concurrent compilations
to not stomp on each other's Dependencies files.
* examples/Dependencies [REGENERATED]
* src/Dependencies [REGENERATED]
* test/Dependencies [REGENERATED]
* tools/Dependencies [REGENERATED]
Regenerated for testing purposes.
Platforms:
i686-pc-linux
mips-sgi-irix6.5
sparc-sun-solaris2.6
Add the Stream VFD sources to the appropriate makefile variables.
Description:
Added H5FDstream.c to the LIB_SRC variable and H5FDstream.h
to the PUB_HDR variable for building the Stream VFD.
Define HAVE_STREAM.
Description:
If the Stream VFD was configured the configured script
will expand this into
'#define HAVE_STREAM 1' in H5config.h and
'#define H5_STREAM 1' in H5pubconf.h.
Added source files for the Stream Virtual File Driver.
Description:
The Stream VFD allows users to stream complete HDF5 files
via socket connections between different applications.
Files which were created anew are flushed to any connected client
on each H5Fflush() or H5Fclose() operation.
Files which are opened as read-only will be read from a socket
on a H5Fopen() call.
The driver's H5FDset_fapl_stream() routine allows to pass in
several parameters such as an external socket descriptor,
some socket options, and flags for broadcasting a received file.
If an external socket is provided the Stream VFD would use that
for the socket calls. Otherwise it parses the filename argument
in H5Fcreate()/H5Fopen() for a 'hostname::port' parameter.
All files processed by the Stream VFD are kept in memory
(same way as the core VFD does).
Platforms:
Tested so far under Linux, Irix 32/64bit, OSF1, Solaris, Cray Unicos,
Hitachi SR8000, IBM AIX.
Not tested under Windows yet.
Fix last couple of errors from introducing "regular" hyperslab feature
into the library.
Description:
Code was blindly dereferencing data structures which aren't defined when
operating on regular hyperslabs.
Solution:
Check for regular hyperslab defined and retrieve information from regular
hyperslab info instead of mucking about in other hyperslab information.
Platforms:
Solaris 2.6
they are about 4-5 times faster than before. We no longer generate "general"
hyperslab data structures for regular hyperslabs, the general data structures
are only generated when needed for irregular hyperslabs.
Still fixing a couple of nook-and-cranny functions to understand the new
information for the regular hyperslabs, so the tests aren't completely passing,
but I wanted to get this checked in for Elena's benchmarks. I should have
more/all tests passing later today.
multi-dereferenced pointer chains. This buys us another ~20% improvement in
the hyperslab I/O speed. (From ~30 seconds to ~25 seconds on the h5hypers
benchmark)
Bug fixes
Description:
All tests were core=dumping in IRIX64. The bug is in Generic
property list creation in which malloc asked for 2*64-1 bytes
due to coding bug. The object creation failed but the return
code was not checked. Program eventually crashed.
Solution:
H5F.c:
Check the return code from new file object creation and flag
error accordingly.
H5FL.c:
H5FL_arr_free is a replacement for H5MM_xfree which accepts
null value as a legal argument value. H5FL_arr_free assert
on it. Since other parts of the code have been passing null
value to H5MM_xfree, H5FL_arr_free must accept it too until
all the calling routines are changed to not pass Null.
H5P.c:
some routine passes in 0 as the hashsize value which is uintn.
The expression (hashsize-1) underflows to the largest unsigned
int for some machines. Thus the calloc failed. Cast hashsize
to unsigned int first (this assumes hashsize stays within the
signed int data range.
H5Smpio.c:
Added the extra parameter because the H5FD_write has been redefined.
Platforms tested:
IRIX64 -64 and -n32
hyperslab boundaries after adding them all, instead of maintaining the sorted
order during each addition. This boosts performance for sub-sampled (i.e.
strided) hyperslabs by about a factor of 10! :-)
added code to allow metadata to be allocated out of a more contiguous block
("metadata aggregation") and also code for "catching" small metadata write
calls and building a buffer of the small pieces of metadata for later writing
as one, larger, block ("metadata accumulation"). These features are enabled
on a per VFL driver basis with the new VFL 'query' call and both currently
enabled for the sec2, family and stdio drivers. The mpio VFL driver enables
only the "metadata aggregation" code, not the "metadata accumulation" code.
All the other drivers have these features turned off.
