Bug Fix, Code Cleanup, Code Optimization, etc.
Description:
Fold in the hyperslab speedups, clean up compile warnings and change a
few things from using 'unsigned' or 'hsize_t' to use 'size_t' instead.
Platforms tested:
FreeBSD 4.3 (hawkwind), Solaris 2.7 (arabica), Irix64 6.5 (modi4)
Document bug fix
Description:
IMPORTANT! IMPORTANT! IMPORTANT!
A case where metadata in a file could get corrupted in certain unusual
sitations was detected and fixed.
In certain circumstances, metadata could get cached in the raw data cache,
and if that particular piece of metadata was updated on disk while
incorrectly cached, the new metadata would get overwritten with the stale
metadata from the raw data cache when it was flushed out.
Additionally, I've patched up the raw data cache to be smarter about how
much it caches and how much I/O it triggers, leading to some speedups.
Solution:
Changed the raw data I/O routines which perform caching to require a
parameter with the size of the dataset being accessed and limited the
cache to no more than that many bytes.
Platforms tested:
FreeBSD 4.3 (hawkwind)
Update
Description:
Changed
#include <hdf_file.h>
construct to
#include "hdf_file.h"
so that the GNU compiler can more easily pick up the dependencies
which it places in the .depend and Dependencies files. Also
regenerated the Dependencies to go along with this.
Platforms tested:
Linux
Code cleanup.
Description:
Fixed _lots_ (I mean _tons_) of warnings spit out by the gcc with the
extra warnings. Including a few show-stoppers for compression on IRIX
machines.
Solution:
Changed lots of variables' types to more sensible and consistent types,
more range-checking, more variable typecasts, etc.
Platforms tested:
FreeBSD 4.2 (hawkwind), IRIX64-64 (modi4)
Backward compatibility code
Description:
Add in code to allow the library to emulate the v1.2 API and behavior.
Platforms tested:
FreeBSD 4.2 (hawkwind)
Feature symmetry
Description:
A while ago I needed to get the 'type' of data being accessed during writes
to the VFL driver, so I put in code to get the information down there.
Albert asked for the same information during reads, so I've added that in.
Tested:
FreeBSD 4.1.1 (hawkwind)
Purpose:
Parallel Bug Fixes
Description:
Was out of sync with header file re-arrangements I checked in last night.
Solution:
Fixed to use new header files, etc.
Platforms tested:
O2K (modi4)
Maintainance & performance enhancements
Description:
Re-arranged header files to protect private symbols better.
Changed optimized regular hyperslab I/O to compute the offsets more
efficiently from previous method of using matrix operations.
Added sequential I/O operations at a more abstract level (at the same level
as H5F_arr_read/write), to support the optimized hyperslab I/O.
Platforms tested:
Solaris 2.6 (baldric) & FreeBSD 4.1.1 (hawkwind)
Rearrange code
Description:
The data sieve buffering code for contiguously stored datasets was
wedged in the H5F_arr_read/H5F_arr_write routines.
Solution:
Created a new H5Fcontig.c to hold I/O routines for contiguously stored
datasets (like H5Fistore.c for chunked dataset I/O routines) and moved
data sieving code into those routines.
Platforms tested:
Solaris 2.6 (i.e. baldric)
Implemented new feature
Description:
Added data sieve buffering code to raw I/O data path. This is enabled for
all the VFL drivers except the mpio & core drivers. Also added two new
API functions to control the sieve buffer size: H5Pset_sieve_buf_size() and
H5Pget_sieve_buf_size().
Platforms tested:
Solaris 2.6 (i.e. baldric)
but I fixed lots of misc. compiler warnings in other code and also tracked down
the memory overwrite bug that was causing the development branch to core dump
on most machines.
----------------------
This extensive change is the virtual file layer implementation. I've
ported and tested the sec2, family, and core drivers and only ported
the mpio driver (Albert will test it). So if you need MPIO I would
recommend sticking with the previous version for a while.
You will get a few compile warnings about split and stdio drivers not
being implemented and possibly tracing information not inserted in
some of the drivers. You can safely ignore them but I plan to fix
them.
I'm still working on the split driver because I just realized that it
needs a part of the VFL that isn't written yet.
Documentation is being updated also because there were some minor
changes (mostly just name changes). It should be available on my web
site later this week.
./MANIFEST
./src/Makefile.in
./src/hdf5.h
./src/H5Flow.c [REMOVED]
./src/H5Fstdio.c [REMOVED]
./src/H5Fsec2.c [REMOVED]
./src/H5Fsplit.c [REMOVED]
./src/H5Fmpio.c [REMOVED]
./src/H5Ffamily.c [REMOVED]
./src/H5Fcore.c [REMOVED]
./src/H5MFpublic.h [REMOVED]
./src/H5FD.c [NEW]
./src/H5FDcore.c [NEW]
./src/H5FDcore.h [NEW]
./src/H5FDfamily.c [NEW]
./src/H5FDfamily.h [NEW]
./src/H5FDmpio.c [NEW]
./src/H5FDmpio.h [NEW]
./src/H5FDprivate.h [NEW]
./src/H5FDpublic.h [NEW]
./src/H5FDsec2.c [NEW]
./src/H5FDsec2.h [NEW]
Removed/added files for virtual file layer.
./bin/trace
./src/H5.c
Removed unused public datatypes and added new VFL public
datatypes.
Changed an error message.
./config/BlankForm
./config/dec-flags
./config/gnu-flags
./config/hpux10.20
./config/hpux9.03
./config/irix5.x
./config/irix6.x
./config/solaris2.x
./config/unicosmk
Removed the H5F_OPT_SEEK and H5F_LOW_DFLT constants from the
configuration since they're no longer applicable. The default
file driver is always the sec2 driver and it always optimizes
calls to lseek() or lseek64().
./config/depend.in
C preprocessor errors generated during automatic dependency
building are sent to /dev/null to prevent them from appearing
twice in the make output.
./src/H5AC.c
./src/H5B.c
./src/H5D.c
./src/H5F.c
./src/H5G.c
./src/H5Gent.c
./src/H5Gnode.c
./src/H5HG.c
./src/H5HL.c
./src/H5O.c
./src/H5Oattr.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Oshared.c
./src/H5T.c
./src/H5detect.c
./test/ohdr.c
Changed H5F_ADDR_UNDEF to HADDR_UNDEF to be more consistent
with the `haddr_t' datatype which is now a public type.
./src/H5D.c
./src/H5P.c
./src/H5Ppublic.h
./src/H5Tconv.c
./test/cmpd_dset.c
./test/dsets.c
./test/overhead.c
./test/tselect.c
./test/tvltypes.c
The H5P_DATASET_XFER constant was changed to H5P_DATA_XFER
because the properties apply to all types of I/O operations,
not just datasets.
./src/H5B.c
./src/H5Bprivate.h
./src/H5D.c
./src/H5Dpublic.h
./src/H5F.c
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5Gnode.c
./src/H5Gpkg.h
./src/H5HG.c
./src/H5HL.c
./src/H5O.c
./src/H5R.c
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Smpio.c
./src/H5Spoint.c
./src/H5Sprivate.h
./test/big.c
./test/h5test.c
./test/istore.c
./testpar/t_dset.c
./testpar/t_file.c
./tools/h5debug.c
./tools/h5ls.c
Modified to work with the virtual file layer by calling H5FD_*
functions instead of H5F_low_* functions and by passing file
access and data transfer properties by object ID instead of
pointer.
Changed H5D_transfer_t to H5FD_mpio_xfer_t since the
COLLECTIVE vs. INDEPENDENT transfer mode is specific to the
MPIO file driver.
Moved MPIO-specific stuff into the MPIO driver.
./src/H5B.c
./src/H5D.c
./src/H5Fprivate.h
The H5F_mpio_* private functions were renamed and placed in
the H5FDmpio driver except those which appeared in H5Smpio.c.
./src/H5E.c
./src/H5Epublic.h
Added major error number H5E_VFL for virtual file layer
related errors.
./src/H5F.c
./src/H5Fprivate.h
Changed the logic that controls whether the boot block is
written. Instead of assuming that the first call to write the
boot block is only to allocate space, I've added a function
argument which makes this explicit.
Changed the way files are compared so that a driver-defined
comparison function can be called. Files which belong to
different drivers are always considered different.
Removed H5F_driver_t since file drivers are now identified by
object ID instead of a special non-user-extendible datatype.
Removed all the hard-coded low-level file properties which
have been replaced by the various file drivers.
./src/H5I.c
./src/H5Iprivate.h
Added the H5I_inc_ref() which was removed a few months ago
since we finally have a use for it.
./src/H5Ipublic.h
Added the H5I_VFL object ID type to identify file drivers in
the virtual file layer.
./src/H5MF.c
./src/H5MFprivate.h
Moved all the allocation/deallocation code into the virtual
file layer which allows file drivers to override much of it.
./src/H5P.c
./src/H5Ppublic.h
Moved file driver-specific code into the various file driver
files.
The H5Pcopy() and H5Pclose() functions make calls into the
virtual file driver to manage the memory for driver-specific
file access and data transfer properties.
./src/H5private.h
./src/H5public.h
The `haddr_t' type is now public.
./test/tfile.c
Added a few more comments.
----------------------
./src/H5B.c
./src/H5D.c
./src/H5F.c
./src/H5Farray.c
./src/H5Fcore.c
./src/H5Ffamily.c
./src/H5Fistore.c
./src/H5Flow.c
./src/H5Fmpio.c
./src/H5Fprivate.h
./src/H5Fsec2.c
./src/H5Fsplit.c
./src/H5Fstdio.c
./src/H5Gnode.c
./src/H5HG.c
./src/H5HL.c
./src/H5O.c
Changed H5F_block_read/write() to take a file transfer
property list instead of just a parallel I/O mode. This will
allow us to pass additional parameters to the I/O functions
without having so many arguments to worry about.
This fixes a bug I introduced last Friday for parallel HDF5
./src/H5B.c
Fixed decoding of B-tree keys when iterating through the
leaves of the B-tree. This fixes a bug for applications that are
adding new entries to groups and also listing the groups.
----------------------
./config/depend.in
Fixed automatic dependencies. We were storing dependencies for
*.o files instead of *.lo files after shared libraries were
added.
./config/gnu-flags
./config/linux-gnulibc1
Moved `-march=pentiumpro -mcpu=pentiumpro -malign-double' from
the linux file to this file and caused it to depend on the CPU
name. This fixes one of Elena's bugs.
./src/H5B.c
./src/H5Bprivate.h
./src/H5D.c
./src/H5Dprivate.h
./src/H5Dpublic.h
./src/H5F.c
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fmpio.c
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5Gnode.c
./src/H5P.c
./src/H5RA.c
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Smpio.c
./src/H5Spoint.c
./src/H5Sprivate.h
./src/H5Tpublic.h
./test/istore.c
Added an H5Dget_storage_size() function that reports the
amount of storage allocated for raw data in a dataset.
Changed H5D_xfer_* to H5F_xfer_* because these properties are
more general than datasets. This also allows some of the
lower-level I/O functions to get this information easier.
./src/H5S.c
./src/H5Sall.c
Added two new functions H5S_all_read() and H5S_all_write()
which are optimizations that copy data directly between file
and memory without having to go through the scatter gather
step. This knocks quite a bit of time off the I/O and
reading/writing entire datasets is a fairly common operation.
./tools/h5ls.c
Reports the logical size of data, the allocated size of data,
and the percent utilization.
./MANIFEST
Removed old pablo files, added new files. Snapshots should now
start to work again.
./src/H5D.c
./src/H5Fmpio.c
Removed two warnings signed vs. unsigned comparisons and check
for overflow.
----------------------
./src/H5.c
./src/H5A.c
./src/H5AC.c
./src/H5B.c
./src/H5D.c
./src/H5E.c
./src/H5F.c
./src/H5Farray.c
./src/H5Fcore.c
./src/H5Ffamily.c
./src/H5Fistore.c
./src/H5Flow.c
./src/H5Fmpio.c
./src/H5Fsec2.c
./src/H5Fsplit.c
./src/H5Fstdio.c
./src/H5G.c
./src/H5Gent.c
./src/H5Gnode.c
./src/H5Gstab.c
./src/H5HG.c
./src/H5HL.c
./src/H5I.c
./src/H5Iprivate.h
./src/H5MF.c
./src/H5MM.c
./src/H5O.c
./src/H5Oattr.c
./src/H5Ocomp.c
./src/H5Ocont.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Ofill.c
./src/H5Olayout.c
./src/H5Omtime.c
./src/H5Oname.c
./src/H5Osdspace.c
./src/H5Oshared.c
./src/H5Ostab.c
./src/H5P.c
./src/H5R.c
./src/H5RA.c
./src/H5S.c
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Smpio.c
./src/H5Snone.c
./src/H5Spoint.c
./src/H5Sselect.c
./src/H5T.c
./src/H5TB.c
./src/H5Tbit.c
./src/H5Tconv.c
./src/H5V.c
./src/H5Z.c
./src/H5detect.c
./src/H5private.h
Most of these changes are because the `interface_initialize_g'
variable change from hbool_t to int. It's a one line change.
Changed the way the library is closed so we have more control
over the order the interfaces are shut down. Instead of
registering an atexit() function for every interface in some
haphazard order we just register one: H5_term_library() which
then calls the H5*_term_interface() functions in a
well-defined order.
If the library is closed and then reopened repeatedly by
calling H5close() and H5open() in a loop we only add one copy
of the library termination functions with atexit().
Termination is a two-step process in order to help detect
programming errors that would cause an infinite loop caused by
the termination of one interface waking up some other
previously terminated interface. The first step terminates
the interface and *marks it as unusable*. After all
interfaces are terminated then we mark them all as usable
again. The FUNC_ENTER() macro has been modified to return
failure or to dump core (depending on whether NDEBUG is
defined) if we try to call an interface while it's shutting
down.
./src/H5.c
The H5dont_atexit() function returns failure if it's called
more than once or if it's called too late. However, the error
stack is not automatically printed on failure because the
library might not be initialized yet
./test/chunk.c
./test/flush1.c
./test/flush2.c
./test/iopipe.c
./test/overhead.c
./test/ragged.c
Changed the extra cast for Win32 so we do floating point
division again -- it was just confusion about precedence and
associativity of casting and the C coercion rules. Removed
extra carriage returns inserted by broken operating system.
./src/H5Ffamily.c
Fixed an bug where H5F_fam_write() lowered the EOF marker for
one of the family members causing H5F_fam_read() to read
zeros.
./test/h5test.h [NEW]
./test/h5test.c [NEW]
./test/Makefile.in
./test/bittests.c
./test/cmpd_dset.c
./test/dsets.c
./test/dtypes.c
./test/extend.c
./test/external.c
Support library for test files. This isn't done yet but
Katie's contractions are ~10 minutes apart so I figured I
better back this stuff up just in case I'm not here next
week...
Eventually all test files will understand HDF5_DRIVER to name
the low level file driver and parameters so we can easily test
various drivers. They will also understand HDF5_PREFIX to
prepend to the beginning of file names which is necessary for
testing ROMIO with various drivers. Also, the cleanup function
will know how to use the file name prefix and will understand
different file driver naming schemes like file families. I'm
not sure they'll understand the `gsf:' type prefixes yet.
Note, the external test is completely commented out because
I'm in the middle of modifying it. It will still compile and
run but it doesn't test anything at the moment.
----------------------
./INSTALL.parallel [NEW]
We're beginning to unify some of the parallel installation
steps. This file will contain general information for
installing the parallel library. It's not complete yet.
./configure.in
./configure [REGENERATED]
./src/H5config.h.in [REGENERATED]
Check for xdr_int() in libnsl required on Solaris when linking
with hdf4. It's found on the Irix system I tested which
complains that `-lnsl' didn't resolve any symbols. Oh well.
Fixed the order of searching for libdf and libmfhdf for hdf4
linking.
./configure.in
./configure [REGENERATED]
./src/H5config.h.in [REGENERATED]
./src/H5Z.c
Check for compress() in libz in order to find older versions
of the library that will still work for hdf4. Added a
separate check for compress2() that hdf5 will use.
./configure.in
./configure [REGENERATED]
./src/H5config.h.in [REGENERATED]
./src/H5.c
./src/H5private.h
./src/H5A.c
./src/H5B.c
./src/H5Bprivate.h
./src/H5D.c
./src/H5F.c
./src/H5Farray.c
./src/H5Fcore.c
./src/H5Ffamily.c
./src/H5Fistore.c
./src/H5Flow.c
./src/H5Fmpio.c
./src/H5Fprivate.h
./src/H5Fsec2.c
./src/H5Fsplit.c
./src/H5Fstdio.c
./src/H5Gent.c
./src/H5Gnode.c
./src/H5Gpkg.h
./src/H5Gprivate.h
./src/H5HG.c
./src/H5HL.c
./src/H5O.c
./src/H5Oattr.c
./src/H5Ocomp.c
./src/H5Ocont.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Ofill.c
./src/H5Olayout.c
./src/H5Omtime.c
./src/H5Oname.c
./src/H5Oprivate.h
./src/H5Osdspace.c
./src/H5Oshared.c
./src/H5Ostab.c
./src/H5R.c
./src/H5RA.c
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Snone.c
./src/H5Spoint.c
./src/H5Sprivate.h
./src/H5Sselect.c
./src/H5T.c
./src/H5Tbit.c
./src/H5Tconv.c
./src/H5Tpkg.h
./src/H5V.c
./test/bittests.c
./test/gheap.c
./test/hyperslab.c
./test/istore.c
./test/tmeta.c
./test/trefer.c
./test/tselect.c
./tools/h5debug.c
./tools/h5tols.c
Added checks for Posix.1g types like `int8_t'. If not defined
then H5private.h defines them. Changed all `int8' etc. to
`int8_t'.
./src/H5A.c
./src/H5D.c
./src/H5F.c
./src/H5G.c
./src/H5I.c
./src/H5P.c
./src/H5R.c
./src/H5RA.c
./src/H5S.c
./src/H5T.c
./src/H5TB.c
./src/H5Z.c
Calling H5*_term_interface() resets interface_initialize_g to
FALSE so a subsequent call to H5open() (implied or explicit)
reinitializes global variables properly.
./src/H5private.h
./src/H5Oefl.c
./src/H5S.c
Changed MAX_SIZET, MAX_SSIZET, MAX_HSIZET, and MAX_HSSIZET to
SIZET_MAX, SSIZET_MAX, HSIZET_MAX, and HSSIZE_MAX to they
match the Posix.1 constants in <limits.h>.
./src/H5T.c
./src/H5Tconv.c
./src/H5Tpkg.h
./src/H5Tprivate.h
./src/H5detect.c
Added 36 more integer hardware conversion functions to the
type conversion table for conversions to/from `long long' and
`unsigned long long'. The `long long' names will be changed
shortly to make them portable to Win32.
Changed H5T_init() to H5T_native_open() and added an
H5T_native_close() to open and close the predefined native
data types.
Increased the initial size of the type conversion table from
64 to 128 entries.
Reordered the 90 new integer conversion functions so the names
that are printed favor `int' over `short' or `long' when two
of them are the same.
./test/dtypes.c
Added hardware and software integer conversion tests for the
56 functions I added recently but not the additional 36
checked in this time. That will come next.
Call H5close() after each test so type conversion statistics
are easier to follow. Try this: $ HDF5_DEBUG=t ./dtypes
Added more debugging output for when things go wrong.
./src/H5private.h
Removed trailing carriage-returns inserted by broken operating
system ;-)
Changes since 19980930
----------------------
./MANIFEST
./doc/html/H5.format.html
./src/H5D.c
./src/H5Dprivate.h
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fprivate.h
./src/H5O.c
./src/H5Ofill.c [NEW]
./src/H5Oprivate.h
./src/H5P.c
./src/H5Ppublic.h
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Spoint.c
./src/H5Sprivate.h
./src/Makefile.in
Added fill values as documented in previous e-mails. They only
work for chunked datasets so far.
./src/H5E.c
./src/H5Epublic.h
./src/H5P.c
Changed H5E_TEMPLATE to H5E_PLIST.
./src/H5Omtime.c
./src/H5V.c
Fixed a typo in a comment.
./src/H5Tconv.c
Fixed a bug in compound type conversions that caused an
assertion to fail.
----------------------
./MANIFEST
Changed freebsd2.2.1 to freebsd2.2.7 to match change in file name.
Added a few missing files. Are all of these supposed to be
distributed?
./tools/testfiles/h5dumptst.c
+ ./tools/testfiles/tall-1.ddl
+ ./tools/testfiles/tall-2.ddl
+ ./tools/testfiles/tall-3.ddl
./tools/testfiles/tall.ddl
./tools/testfiles/tall.h5
+ ./tools/testfiles/tattr-1.ddl
+ ./tools/testfiles/tattr-2.ddl
+ ./tools/testfiles/tattr-3.ddl
+ ./tools/testfiles/tattr-4.ddl
./tools/testfiles/tattr.ddl
./tools/testfiles/tattr.h5
+ ./tools/testfiles/tdset-1.ddl
+ ./tools/testfiles/tdset-2.ddl
+ ./tools/testfiles/tdset-3.ddl
+ ./tools/testfiles/tdset-4.ddl
./tools/testfiles/tdset.ddl
./tools/testfiles/tdset.h5
+ ./tools/testfiles/tgroup-1.ddl
+ ./tools/testfiles/tgroup-2.ddl
+ ./tools/testfiles/tgroup-3.ddl
./tools/testfiles/tgroup.ddl
./tools/testfiles/tgroup.h5
+ ./tools/testfiles/tslink-1.ddl
+ ./tools/testfiles/tslink-2.ddl
./tools/testfiles/tslink.ddl
./tools/testfiles/tslink.h5
./src/H5B.c
./src/H5Bprivate.h
./src/H5D.c
./src/H5Dprivate.h
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fprivate.h
./src/H5Gstab.c
./src/H5P.c
./src/H5Ppublic.h
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Spoint.c
./src/H5Sprivate.h
./test/istore.c
Implemented split ratios as documented in previous
e-mails. Frank, the new API functions are H5Pget_btree_ratios()
and H5Pset_btree_ratios().
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Spoint.c
./src/H5Sprivate.h
Added more type checking for the data transfer property list
passed to these functions.
./src/H5D.c
Added a dataset transfer property list as an argument to
H5D_allocate() for the parallel version in order to pass split
ratios down to H5F_istore_lock(). Eventually we won't need
the split ratios for H5D_allocate() because we'll build a
completely full B-tree from the leaves up, but it might be
useful to have other transfer properties at that level
anyway. I always caled H5D_allocate() with &H5D_dflt_xfer.
./MANIFEST
./test/Makefile.in
./test/overhead.c [NEW]
Added an `overhead' test run by `make timings'.
----------------------
./bin/release
Added ./Makefile to the distribution again -- it got lost in
the changes last week although it isn't all that important a
file since it gets clobbered by configure anyway.
./bin/trace
./doc/html/Filters.html
./doc/html/H5.format.html
./doc/html/H5.user.html
./src/H5.c
./src/H5D.c
./src/H5Dprivate.h
./src/H5E.c
./src/H5Epublic.h
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fprivate.h
./src/H5O.c
./src/H5Ocomp.c
./src/H5Oprivate.h
./src/H5P.c
./src/H5Ppublic.h
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Spoint.c
./src/H5Sprivate.h
./src/H5Ssimp.c
./src/H5Z.c
./src/H5Zprivate.h
./src/H5Zpublic.h
./src/hdf5.h
./test/dsets.c
./tools/h5ls.c
Added the data filter pipeline, a generalization of the
compression stuff which allows things like checksums,
encryption, compression, performance monitoring, etc. See
./doc/html/Filters.html for details -- it replaces the
Compression.html doc.
./src/H5T.c
Cleaned up debugging output.
./config/linux
Added checks for egcs and pgcc and changed optimization flags
for the compilers.
./src/H5G.c
./tools/h5dump.c
Fixed compiler warnings in these files and others.
./configure.in
./src/H5private.h
./test/mtime.c
Added a check for difftime() and defined HDdifftime() to do
something else on systems that don't have difftime().
API defined in the html/Dataspaces.html document. This code does not include
support for strides, merging selections, or permutations of coordinates yet,
but it's a drop-in replacement for the existing API with the same features.
mode to use for data transfer. Currently, INDEPENDENT transfer
is the default mode. Collective is supported for files opened
via the MPIO driver. Specific changes to individual files follow.
H5D.c:
Replaced previous kludge of passing xfer_mode via the file access_mode
with xfer_mode function parameter all the way to the low level I/O
drivers.
H5Dprivate.h:
The H5D_xfer_t.xfer_mode is now used in serial and parallel modes.
Removed the "HAVE_PARALLEL" macro condition to support it.
H5Dpublic.h:
Added H5D_XFER_DFLT as a new value for H5D_transfer_t and removed
the HAVE_PARALLEL condition macro to make the H5D_transfer_t
definition available to both serial and parallel version.
H5B.c:
H5F.c:
H5Farray.c:
H5Fcore.c:
H5Ffamily.c:
H5Fistore.c:
H5Flow.c:
H5Fmpio.c:
H5Fsec2.c:
H5Fsplit.c:
H5Fstdio.c:
H5Gnode.c:
H5HG.c:
H5HL.c:
H5O.c:
H5Sprivate.h:
H5Ssimp.c:
debug.c:
Added xfer_mode to the argument lists of the following routines to
specify data transfer mode. Then updated calls to these routines
with appropriate transfer mode. Whenever cases that collective
mode is inappropriate (e.g. object header read-write, flush),
H5D_XFER_DFLT is used. The H5D_XFER_DFLT mode is currently
acting the same as the independent mode.
H5F_block_read
H5F_block_write
H5F_arr_read
H5F_arr_write
H5F_core_read
H5F_core_write
H5F_fam_read
H5F_fam_write
H5F_mpio_read
H5F_mpio_write
H5F_low_write
H5F_low_read
H5F_sec2_read
H5F_sec2_write
H5F_split_read
H5F_split_write
H5F_stdio_read
H5F_stdio_write
H5S_simp_fgath
H5S_simp_fscat
H5S_simp_read
H5S_simp_write
H5Fprivate.h:
H5P.c:
Removed access_mode field from struct H5F_access_t type.
The access-mode was the initial design to control independent
or collective I/O mode. It is now supported by the data
transfer mode.
----------------------
./html/Datatypes.html
Updated documentation for named data types and type sharing.
./src/H5Farray.c
Split a couple long lines.
./src/H5T.c
The H5T_copy() demotes immutable types to read-only types so
they're cleaned up properly and memory is not leaked.
./test/dtypes.c
Modified to be more consistent with other tests. Removed all
internal header files, constants, types, and functions calls
since the API is now complete enough to test
everything. Temporary files are now removed.
----------------------
./html/Compression.html [NEW]
./html/Datasets.html
./html/H5.format.html
./html/H5.user.html
Documented compression. A couple of the H5P functions aren't
quite implemented yet but they're coming soon...
./src/H5Dprivate.h
./src/H5E.c
./src/H5Epublic.h
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fprivate.h
./src/H5MF.c
./src/H5MFprivate.h
./src/H5O.c
./src/H5Ocomp.c [NEW]
./src/H5Oprivate.h
./src/H5P.c
./src/H5Ppublic.h
./src/H5Sprivate.h
./src/H5Ssimp.c
./src/H5Z.c [NEW]
./src/H5Zprivate.h [NEW]
./src/H5Zpublic.h [NEW]
./src/Makefile.in
./src/hdf5.h
./test/dsets.c
./test/istore.c
Compression is now mostly working. Don't try to open a
compressed dataset though because the compression message
won't be read.
./html/Datatypes.html
./html/H5.api.html
./src/H5.c
./src/H5private.h
./src/H5D.c
./src/H5T.c
./src/H5Tconv.c
./src/H5Tpkg.h
./src/H5Tprivate.h
./src/H5Tpublic.h
Added timing support. When compiled with H5T_DEBUG defined
the library will print conversion bandwidths when the library
closes. The H5Tregister functions take a string as the first
argument so the statistics output is meaningful.
./MANIFEST
Added new files.
./configure.in
./src/H5config.h.in
Check for getrusage(). Check for compress2() in libz.a and
the zlib.h header file. Added `z' to the debug list.
./src/H5B.c
./src/H5Bprivate.h
./src/H5Gnode.c
./src/debug.c
Cleaned up some indentation and added support to print istore
B-trees. From the debugger, give the B-tree address and the
dimensionality from the layout message of the object header.
./src/h5ls.c
The oid is printed as w:x:y:z where w and x are the file ID
and y and z are the OID within the file. You can give z or
y*2^32+z as an argument to the debugger to print the object
header for the object.
./src/H5AC.c
Cleaned up statistics and made them match those reported by
H5T and H5Z.
./src/H5MM.c
./src/H5MMprivate.h
./src/H5Fistore.c
Finally got rid of a couple of long-standing const cast
warnings.
used to open an HDF5 file with MPIO access. The design has changed
to provide access-mode controll (independent or collective access)
as an argument in the H5D read/write calls.
Also removed the macros H5FACC_INDEPENDENT and H5FACC_COLLECTIVE.
The original thinking was to define how a file is independent or
collective access. They are replaced by the enum H5D_transfer_t
with values of H5D_XFER_INDEPENDENT and H5D_XFER_COLLECTIVE.
These values are used in the H5Pset_xfer to setup a data transfer
property list to be used during the H5D read/write calls.
The original access_mode field in the File access structure is
still there. It would be removed soon once the transfer control
can be passed down to the low level I/O routines.
H5D.c
Changed H5D_write and H5D_read to recognize collective transfer
requests. Current algorithm puts the xfer-mode in the file->mpio
structure. Not thread safe. Need to revise it to pass transfer
parameter as a function parameters to the lower levels.
H5Dprivate.h
H5Dpublic.h
Added xfer_mode to the transfer property list structure.
Added defination of transfer modes, H5D_XFER_INDEPENDENT and
H5D_XFER_COLLECTIVE.
H5Farray.c
Added code to handle collective access requests. Currently
works for one contiguous block at a time. Need to revise
it with the MPIO low level calls to combine all blocks as
one truely collective call.
H5P.c
H5Ppublic.h
Added H5Pset_xfer and H5Pget_xfer routines that can set and get
the transfer mode of a data transfer property list.
----------------------
./src/H5B.c
./src/H5D.c
./src/H5Dprivate.h
./src/H5Dpublic.h
./src/H5F.c
./src/H5Farray.c
./src/H5Fcore.c
./src/H5Ffamily.c
./src/H5Fistore.c
./src/H5Flow.c
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5Fsec2.c
./src/H5Fsplit.c
./src/H5Fstdio.c
./src/H5G.c
./src/H5Gent.c
./src/H5Gnode.c
./src/H5HG.c
./src/H5HL.c
./src/H5MF.c
./src/H5MFprivate.h
./src/H5O.c
./src/H5Ocont.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Olayout.c
./src/H5Oname.c
./src/H5Oprivate.h
./src/H5Osdspace.c
./src/H5Oshared.c
./src/H5Ostab.c
./src/H5P.c
./src/H5Ppublic.h
./src/H5S.c
./src/H5Sprivate.h
./src/H5Spublic.h
./src/H5Ssimp.c
./src/H5Tconv.c
./src/H5Tpkg.h
./src/H5V.c
./src/H5Vprivate.h
./src/H5private.h
./src/H5public.h
./src/h5ls.c
./test/cmpd_dset.c
./test/dsets.c
./test/extend.c
./test/external.c
./test/hyperslab.c
./test/iopipe.c
./test/istore.c
./test/shtype.c
./test/tfile.c
./test/th5s.c
Anything having to do with the size of a dataset now uses the
types `hsize_t' and `hssize_t' which must be the same size and
at least as large as `size_t'. This isn't fully tested yet,
so hsize_t and hssize_t are defined as size_t and ssize_t in
H5public.h. Setting them to larger values will trip up gcc
versions less than 2.8.1 on x86 platforms.
Documented unused function formals with `__unused__' before
the formal name. This also has the effect of supressing
warning messages for gcc since it's defined to be
`__attribute__((unused))' in the H5private.h file.
./src/debug.c
./src/h5ls.c
If the file name contains a `%' then the file is opened as a
file family with H5P_DEFAULT for the file member access
property list.
./src/h5ls.c
The group name is optional, defaulting to `/'.
./src/hdf5.h
Added some missing public header files.
----------------------
./configure.in
Moved setting of compiler warning switches earlier in the file.
Turned on more warning switches to gcc.
./config/linux
Prints a warning if the gcc version is less than 2.8.1 since
that version has problems with register allocation for `long
long'.
./html/Datatypes.html
Documented sharing of data types between datasets.
./src/H5G.c
./src/H5Gpublic.h
Implemented H5Gmove(), H5Glink() and H5Gunlink() for hard
links. Still have soft links to do.
./src/H5AC.c
./src/H5ACprivate.h
./src/H5D.c
./src/H5E.c
./src/H5Eprivate.h
./src/H5F.c
./src/H5Farray.c
./src/H5Fcore.c
./src/H5Ffamily.c
./src/H5Fistore.c
./src/H5Flow.c
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5Fsec2.c
./src/H5Fstdio.c
./src/H5G.c
./src/H5Gent.c
./src/H5Gnode.c
./src/H5Gpkg.h
./src/H5Gprivate.h
./src/H5HG.c
./src/H5HL.c
./src/H5HLprivate.h
./src/H5I.c
./src/H5Iprivate.h
./src/H5MM.c
./src/H5MMprivate.h
./src/H5O.c
./src/H5Oefl.c
./src/H5Oprivate.h
./src/H5Osdspace.c
./src/H5Oshared.c
./src/H5Ostab.c
./src/H5P.c
./src/H5S.c
./src/H5Ssimp.c
./src/H5T.c
./src/H5Tconv.c
./src/H5Tprivate.h
./src/H5Tpublic.h
./src/H5V.c
./src/H5Vprivate.h
./src/H5detect.c
./src/h5ls.c
./test/cmpd_dset.c
./test/dsets.c
./test/external.c
./test/hyperslab.c
./test/iopipe.c
./test/istore.c
./test/shtype.c
./test/tstab.c
Fixed comparisons between signed and unsigned values. Fixed
warnings about unused function arguments.
----------------------
./MANIFEST
New files
./html/extern1.gif [NEW]
./html/extern1.obj [NEW]
./html/extern2.gif [NEW]
./html/extern2.obj [NEW]
./html/Datasets.html
Documented external raw data storage.
./html/Datasets.html
./html/chunk1.gif [NEW]
./html/chunk1.obj [NEW]
Added a picture and better description for chunking since it
seems that people are thinking chunks are fixed-size at the
file level instead of at the array level.
./src/H5D.c
./src/H5Dpublic.h
./test/external.c
Changed H5Dget_create_parms() to H5Dget_create_plist().
./src/H5D.c
./src/H5Farray.c
./src/H5Fprivate.h
./src/H5Oefl.c
./src/H5Oprivate.h
./src/H5Sprivate.h
./src/H5Ssimp.c
./test/external.c
./test/istore.c
External raw data file I/O for contiguous storage is now
working. The library supports segments of various sizes which
can be scattered throughout multiple files in any order.
./test/Makefile.in
Remove more temporary files.
----------------------
./html/Dataspaces.html
./html/Errors.html
./html/Files.html
./html/H5.api.html
./html/review1.html
./src/H5private.h
./src/H5public.h
./test/dsets.c
./test/dtypes.c
Removed all the types like `int32' and `intn' into private
headers since they violate the naming scheme and pollute
application name space. Besides, our test files only use them
in a handful of places and it's probably useless to export
them to the app. The app is always written in terms of
standard numeric types or its own numeric types and probably
never in terms of HDF5 numeric types. If it were, then the
user would have to copy from their type to hdf5 type for
almost every hdf5 API function call! Same goes for return
values. I also removed SUCCEED/FAIL from the API since apps
should be checking against zero anyway.
if (FAIL==(space=H5Screate_simple(...))) /*wrong*/
if ((space=H5Fcreate_simple(...)<0)) /*right*/
./src/H5.c
Changed arguments of H5version() from `uintn' to `unsigned'.
./src/H5Tpublic.h
./src/H5T.c
Changed return type of H5Tget_nmembers() from `intn' to `int'
./src/H5A.c
./src/H5Aprivate.h
./src/H5Apublic.h
Changed `H5Asearch_func_t' to `H5A_search_func_t' and moved
its definition from the public to the private header file.
./html/H5.format.html
Documented changes made to the external file list (H5O_EFL)
message.
./src/H5D.c
./src/H5Dprivate.h
./src/H5E.c
./src/H5Epublic.h
./src/H5O.c
./src/H5Oefl.c
./src/H5Oprivate.h
./src/H5P.c
./src/H5Ppublic.h
Added partial support for external raw data files. HDF5 can
now describe external raw data files by listing the file
names, offsets, and size for a dataset. However, we will
restrict a dataset to be stored "contiguously" when the
external file list is viewed as a single address space. The
current implementation is unable to read/write to external
files--that will come later this week as will documentation.
For now, take a look at ./test/external.c, particularly the
calls to H5Pset_external().
./test/Makefile.in
./test/external.c [NEW]
./MANIFEST
Added tests for external storage. Note: the read test is
supposed to fail at this point since reading external datasets
is not implemented yet. There is no write test.
./src/H5S.c
./src/H5Sprivate.h
./src/H5Ssimp.c
Added H5S_get_npoints_max() to return the maximum possible
number of data points in a data space.
Added an extra argument to H5S_get_dims() which returns the
maximum dims.
./src/H5F.c
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5M.c [DEPRICATED]
./src/H5Mpublic.h [DEPRICATED]
Changed `template' to `property list' in lots of places.
./src/H5Osdspace.c
Removed an extra `\n' from a print statement.
./src/H5S_public.h
Changed H5S_UNLIMITED to the maximum size_t value.
./test/extend.c
"Extendable" is spelled "extendible".
./src/H5Farray.c
./src/H5V.c
./src/H5Vprivate.h
./test/hyperslab.c
Strides are now type ssize_t instead of int. These have
nothing to do with the sample granularity arguments for
hyperslabs, which are also called "strides" in the code.
./test/tstab.c
Changed assumptions about default address and length sizes.
while he is visiting LLNL. I changed the default creation template
offset and length to 4. Will fix the problem later.
Changes since 19980205
----------------------
./src/H5H.c
./src/H5Hprivate.h
./src/H5O.c
./src/H5Ocont.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Olayout.c
./src/H5Oname.c
./src/H5Onull.c
./src/H5Oprivate.h
./src/H5Odspace.c
./src/H5Ostab.c
./src/debug.c
./html/H5.format.html
Added an extra 4-byte field after the heap magic number for
alignment on the DEC alpha. Changed object header message
alignment to 8-bytes.
./src/H5F.c
./src/H5Farray.c
./src/H5Ffamily.c
./src/H5Fistore.c
./src/H5Flow.c
./src/H5Fprivate.h
./src/H5Fsec2.c
./src/H5Fstdio.c
./src/H5Gnode.c
./src/H5O.c
./src/H5Odtype.c
./src/H5P.c
./src/H5Pprivate.h
./src/H5T.c
./src/H5Tconv.c
./src/H5Tpkg.h
./src/H5Tpublic.h
./src/H5V.c
./src/H5detect.c
./test/cmpd_dset.c
./test/dsets.c
./test/dtypes.c
./test/extend.c
./test/hyperslab.c
./test/istore.c
./test/th5p.c
./test/theap.c
Fixed a few irix64 warnings regarding size_t vs. int,
variables set but not used, printf formats
./config/irix64
Added `-woff 1196' to get rid of errors about __vfork() being
implicitly defined in a system header file.
./src/H5B.c
Fixed a stack alignment problem.
Changed interface to the H5P..hyperslab functions to 'int' instead of 'size_t'.
Cleaned up lots of warnings on IRIX 6.2 platform. Minor other tweaks to get
to a mostly clean build on the SGI. It still whines about 'long long' being
non-standard and some "pointless comparison of unsigned with 0" but those
aren't big problems.
----------------------
./Makefile.in
Added more dependencies to .PHONY.
./src/H5D.c
The write side of the I/O pipeline is implemented now too.
Things are looking good for the prototype and it's just a
matter of populating the library with the data type and data
space conversion functions.
./src/H5Farray.c
./src/H5Fprivate.h
./test/istore.c
Changed the order of the arguments for H5F_arr_read() and
H5F_arr_write().
./src/H5P.c
./src/H5Pprivate.h
Changed the names of the arguments of H5P_find(). Fleshed out
the mgath and fscat callback types.
./src/H5Psimp.c
Added stubs for H5P_simp_mgath() and H5P_simp_fscat() that
operate on the entire data space. Quincey, once you have the
data space hyperslab stuff in place let me know and I'll
finish the H5P_simp_*() functions to do partial I/O. Or you
can take a look at it too if you like; there's some comments
in there for you.
./src/H5V.c
./src/H5Vprivate.h
Changed dimensionality arguments from `size_t' to `intn' to be
consistent with the rest of the library and to get rid of
warnings on 64-bit Irix.
----------------------
./src/H5AC.c
./src/H5ACprivate.h
./src/H5B.c
./src/H5D.c
./src/H5Farray.c
./src/H5Fprivate.h
./src/H5V.c
./src/H5Vprivate.h
Fixed indent oopses.
./src/H5D.c
./src/H5Fprivate.h
./src/H5Farray.c
./test/istore.c
We can now perform partial I/O on contiguous storage
transferring between a hyperslab of file storage and a
hyperslab of memory. However, partial I/O hasn't been added
to the I/O pipeline yet in H5D.c
----------------------
./html/Datasets.html
Removed some archaic comments about data spaces. Fixed example
code.
./MANIFEST
./html/H5.format.html
./src/H5O.c
./src/H5Oprivate.h
./src/H5Ocstore.c [DELETED]
./src/H5Oistore.c [DELETED]
./src/H5Olayout.c [NEW]
./src/Makefile.in
./test/istore.c
Replaced H5O_CSTORE and H5O_ISTORE messages with a more
general H5O_LAYOUT message.
./src/H5D.c
./src/H5Dprivate.h
./src/H5Dpublic.h
A little more work on the pipeline. Access to the file data
is through the new H5F_arr_read() and H5F_arr_write() which do
I/O on hyperslabs of byte arrays and don't depend on data
layout. This should simplify the I/O pipeline quite a bit.
I also added another argument to H5Dread() and H5Dwrite() to
describe the hyerslab of the file array on which I/O is
occuring. We discussed this at last week's meeting.
./src/H5Farray.c [NEW]
Added functions that sit on top of H5F_block_read() and
H5F_istore_read() and implement a common set of functions
between all layouts. This means I/O of hyperslabs of
byte-arrays in the file to arrays of bytes in memory. When
operating on arrays of elements (>1byte) then we just add
another dimension. That is, a 10x20 array of int32 becomes a
10x20x4 array of bytes.
[This is the area I'll be working on most of next week to
implement partial I/O for contiguous data and to improve
performance for chunked data.]
./src/H5Fistore.c
./src/H5Fprivate.h
Replaced the H5F_istore_t data type with the layout message
H5O_layout_t which looks almost the same. Eventually I'd like
to rename `istore' to `chunked' everywhere and use `istore'
for 1-d storage where the chunks are all different sizes like
in the small object heap where each object is a chunk.
./src/H5V.c
Changed ISTORE to LAYOUT in one place.
./test/dsets.c
Fixed for extra argument to H5Dread() and H5Dwrite().
