Add the Stream VFD sources to the appropriate makefile variables.
Description:
Added H5FDstream.c to the LIB_SRC variable and H5FDstream.h
to the PUB_HDR variable for building the Stream VFD.
Define HAVE_STREAM.
Description:
If the Stream VFD was configured the configured script
will expand this into
'#define HAVE_STREAM 1' in H5config.h and
'#define H5_STREAM 1' in H5pubconf.h.
Added source files for the Stream Virtual File Driver.
Description:
The Stream VFD allows users to stream complete HDF5 files
via socket connections between different applications.
Files which were created anew are flushed to any connected client
on each H5Fflush() or H5Fclose() operation.
Files which are opened as read-only will be read from a socket
on a H5Fopen() call.
The driver's H5FDset_fapl_stream() routine allows to pass in
several parameters such as an external socket descriptor,
some socket options, and flags for broadcasting a received file.
If an external socket is provided the Stream VFD would use that
for the socket calls. Otherwise it parses the filename argument
in H5Fcreate()/H5Fopen() for a 'hostname::port' parameter.
All files processed by the Stream VFD are kept in memory
(same way as the core VFD does).
Platforms:
Tested so far under Linux, Irix 32/64bit, OSF1, Solaris, Cray Unicos,
Hitachi SR8000, IBM AIX.
Not tested under Windows yet.
Fix last couple of errors from introducing "regular" hyperslab feature
into the library.
Description:
Code was blindly dereferencing data structures which aren't defined when
operating on regular hyperslabs.
Solution:
Check for regular hyperslab defined and retrieve information from regular
hyperslab info instead of mucking about in other hyperslab information.
Platforms:
Solaris 2.6
they are about 4-5 times faster than before. We no longer generate "general"
hyperslab data structures for regular hyperslabs, the general data structures
are only generated when needed for irregular hyperslabs.
Still fixing a couple of nook-and-cranny functions to understand the new
information for the regular hyperslabs, so the tests aren't completely passing,
but I wanted to get this checked in for Elena's benchmarks. I should have
more/all tests passing later today.
multi-dereferenced pointer chains. This buys us another ~20% improvement in
the hyperslab I/O speed. (From ~30 seconds to ~25 seconds on the h5hypers
benchmark)
Bug fixes
Description:
All tests were core=dumping in IRIX64. The bug is in Generic
property list creation in which malloc asked for 2*64-1 bytes
due to coding bug. The object creation failed but the return
code was not checked. Program eventually crashed.
Solution:
H5F.c:
Check the return code from new file object creation and flag
error accordingly.
H5FL.c:
H5FL_arr_free is a replacement for H5MM_xfree which accepts
null value as a legal argument value. H5FL_arr_free assert
on it. Since other parts of the code have been passing null
value to H5MM_xfree, H5FL_arr_free must accept it too until
all the calling routines are changed to not pass Null.
H5P.c:
some routine passes in 0 as the hashsize value which is uintn.
The expression (hashsize-1) underflows to the largest unsigned
int for some machines. Thus the calloc failed. Cast hashsize
to unsigned int first (this assumes hashsize stays within the
signed int data range.
H5Smpio.c:
Added the extra parameter because the H5FD_write has been redefined.
Platforms tested:
IRIX64 -64 and -n32
hyperslab boundaries after adding them all, instead of maintaining the sorted
order during each addition. This boosts performance for sub-sampled (i.e.
strided) hyperslabs by about a factor of 10! :-)
added code to allow metadata to be allocated out of a more contiguous block
("metadata aggregation") and also code for "catching" small metadata write
calls and building a buffer of the small pieces of metadata for later writing
as one, larger, block ("metadata accumulation"). These features are enabled
on a per VFL driver basis with the new VFL 'query' call and both currently
enabled for the sec2, family and stdio drivers. The mpio VFL driver enables
only the "metadata aggregation" code, not the "metadata accumulation" code.
All the other drivers have these features turned off.
placed on how much memory is used by the free lists before they are garbage
collected. The default is to have no limit, with garbage collection only
occurring when they cannot allocate memory.
lists. Each kind of list one has hard-coded limits on when to garbage collect,
which will be replaced with user-controllable knobs (through property list
settings, I think) once I finish debugging some related performance problems.
