Tested platform:
Kagiso only since it is only a comment block change. If it works in one
machine, it should work in all, I hope. Still need to check the parallel
build on copper.
Add API calls allowing user control of the metadata cache.
Description:
Prior to this update, the metadata cache was not configurable
from outside the library.
Solution:
Add API calls allowing the user to configure the metadata cache
either at file open time, or for any open file. Also added calls
permitting the user to monitor cache size and hit rate. These
latter facilities are needed for "manual" cache size control
Platforms tested:
h5committested
Misc. update:
Description:
Replaced the old metadata cache with a cache with a modified LRU
replacement policy. This should improve the hit rate.
Solution:
Since we want to flush cache entries in increasing address order, I
used the threaded binary B-tree code to store the cache entries.
There is a fair bit of overhead here, so we may want to consider
other options.
While the code is designed to allow the support of other replacement
algorithms, at present, only a modified version of LRU is supported.
The modified LRU algorithm requires that a user selectable portion
of the cache entries be clean. The clean entries are evicted first
when writes are not permitted. If the pool of clean entries is used
up, the cache grows beyond its user specified maximum size. The
cache can also exceed its maximum size if the combined size of the
protected (or locked) entries exceeds the maximum size of the cache.
Platforms tested:
eirene (serial, parallel, fp), h5committested
Misc. update:
----------------------
./html/Files.html
./src/H5C.c
./src/H5Cpublic.h
./src/H5Ffamily.c
./src/H5Fprivate.h
./src/H5Fsplit.c
./test/tstab.c
Added file-access property functions.
The split driver takes file extensions as properties.
./src/H5A.c
./src/H5Aprivate.h
Added some comments. Changed H5A_destroy() to call the free
function on all the atoms that are destroyed. This fixes a bug
where the file boot block isn't updated if the file isn't
closed before calling exit().
Removed extra `*' and `&' from some places.
./src/H5AC.c
Replaced an occurrence of NO_ADDR with NULL.
./src/H5Odtype.c
./src/H5T.c
./src/H5Tconv.c
./src/H5Tpkg.h
Data types of compound members are pointers.
./H5private.h
Some changes to make lseek64() work on Irix 5.3 where header
files don't realize that `long long' works.
./acconfig.h
./configure.in
Removed definition for PHDF5 since it looks like everyone is
useing HAVE_PARALLEL now.
./configure.in
./src/H5detec.c
Added checks for gethostname() and getpwuid().
----------------------
./configure.in
./src/H5Fprivate.h
./src/H5Fsec2.c
We now detect and use lseek64() on systems that have it (e.g.,
Irix64) and are able to generate >2GB files on Irix XFS file
systems (and anything else that supports large files). This
change also removed some warning messages from the Irix `-64'
compiler.
> $ ls -l istore.h5
> -rw-r--r-- 1 matzke meshtv 8605436856 Feb 17 14:03 istore.h5
./configure.in
./src/H5Fprivate.h
./src/H5Fstdio.h
We now detect and use fseek64() on systems that have it (e.g.,
Irix64) and are able to generate >2GB files on Irix XFS file
systems (and anything else that supports large files). This
change also removed some warning messages from the Iris `-64'
compiler.
./src/H5E.c
./src/H5Epublic.h
Added the H5E_OVERFLOW error to signal file address overflow.
./src/H5Fpublic.h
./examples/h5_chunk_read.c
./examples/h5_compound.c
./examples/h5_extend_write.c
./examples/h5_group.c
./examples/h5_read.c
./examples/h5_write.c
./html/Datasets.html
./html/Files.html
./html/H5.api.html
./html/H5.intro.html
./html/H5.sample_code.html
./html/ph5example.c
./html/review1.html
./test/cmpd_dset.c
./test/dsets.c
./test/extend.c
./test/tfile.c
./test/th5p.c
./test/theap.c
./test/tohdr.c
./test/tstab.c
./testpar/phdf5.c
Renamed file access constants to follow the naming scheme.
Also changed the base names a little to be more accurate as to
what they do. The old names H5ACC_WRITE and H5ACC_OVERWRITE
will temporarily work.
H5ACC_DEFAULT --> H5F_ACC_RDONLY for H5Fopen()
H5ACC_DEFAULT --> H5F_ACC_EXCL for H5Fcreate()
H5ACC_WRITE --> H5F_ACC_RDWR
H5ACC_OVERWRITE --> H5F_ACC_TRUNC
Albert or Kim: The H5ACC_INDEPENDENT and H5ACC_COLLECTIVE
macros in H5Fpublic.h should be an enum typedef and have names
more like H5F_MPIO_INDEPENDENT and H5F_MPIO_COLLECTIVE. Also
change the access_mode argument of H5Cset_mpio().
H5Fcreate() and H5Fopen() are more strict now about which
flags are acceptable for the operation.
./src/H5Fprivate.h
./src/H5F.c
./src/H5C.c
Changed the file access template to make it more general. A
union contains a struct for each type of low-level driver and
the default template is initialized at run-time.
./src/H5Fpublic.h
./src/H5F.c
Added H5Fget_access_template() and cleaned up
H5Fget_create_template().
./src/H5C.c
The H5Cset_mpi() no longer trashes the file access template
when an error is detected. We check for errors and *then*
update the file access template.
Added H5C_close() so Albert and Kim have a place to release
the MPI communicator and info from the file access property
list.
Kim or Albert: I notice in H5Cset_mpi() you copy the MPI
communicator. Do you need to do something similar in
H5C_copy()?
./src/H5F.c
Added more error checking for the file creation and access
property lists because it used to be possible to make the
library dump core by swapping the creation and access property
list ID numbers of H5Fcreate().
./test/istore.c
./test/tfile.c
./test/th5p.c
./test/theap.c
./test/tohdr.c
./test/tstab.c
./testpar/testphdf5.c
One must pass H5C_DEFAULT as the file creation or access
property list in order to get the default property list. It
is no longer possible to pass zero or any other arbitrary bad
object ID.
./src/H5Fcore.c
./src/H5Ffamly.c
./src/H5Flow.c
./src/H5Fmpio.c
./src/H5Fsec2.c
./src/H5Fsplit.c
./src/H5Fstdio.c
./src/H5MF.c
The file access property list is passed to all H5F_low_...()
functions and to the drivers.
./src/H5Fcore.c
The block size can be set at run time on a per-file basis
instead of at compile time across all files. The "5000 items in
a group test" now takes 1.6 seconds.
./src/H5private.h
Removed inclusion of mpi.h and mpio.h since they're included
from H5public.h.
./src/H5Cpublic.h
./src/H5C.c
Added H5Cset_stdio(), H5Cset_sec2(), H5Cset_core(),
H5Cset_split(), and H5Cset_family() in addition to the
H5Cset_mpio() that Kim and Albert already wrote. We still
need the H5Cget_driver() and an H5Cget...() counterpart for
each of those functions. The split and family drivers still
need a little work but I'm checking this in anyway.
----------------------
./INSTALL
Added instructions for which C flags to set for debugging.
./src/H5C.c
./src/H5Cpublic.h
H5Cset_chunk() takes const pointer.
./src/H5D.c
./src/H5Dprivate.h
./src/H5Dpublic.h
Added H5Dextend() to extend the dimensions of a dataset.
./src/H5Osdspace.c
./src/H5P.c
./src/H5Pprivate.h
./src/H5Ppublic.h
./test/cmpd_dset.c
./test/dsets.c
./test/th5p.c
Added the optional `maxdims' argument to H5Pcreate_simple()
and defined constant H5P_UNLIMITED which can appear in the
maxdims. Added `const' to arguments.
Implemented H5Pcopy()
Removed the unused file argument from H5P_modify.
Added H5P_extend().
Removed the `flags' field from simple data types and we
determine if the `max' or `perm' arrays are valid by looking
at the pointer. Cleaned up the H5O_sdspace_debug output.
./src/H5T.c
Fixed a printf format.
./MANIFEST
./test/Makefile.in
./test/extend.c [NEW]
Added a test for multi-dimensional unlimited dimensions.
----------------------
./src/*.h
Fixed indentation where indent(1) screwed up. This isn't by
any means the final say, but it's better than it was.
./src/H5A.c
./src/H5Aprivate.h
./src/H5Apublic.h
./src/H5C.c
./src/H5D.c
./src/H5E.c
./src/H5F.c
./src/H5G.c
./src/H5M.c
./src/H5P.c
./src/H5T.c
./src/H5Tconv.c
./src/debug.c
./test/dtypes.c
./test/istore.c
./test/theap.c
./test/tohdr.c
./test/tstab.c
Removed some atom functions from the API and made them
library-scope. Also changed some names by removing the
redundant `atom' from the name and by adding a `_' after the
`H5A'.
----------------------
./config/freebsd2.2.1
./config/linux
Added -UH5O_DEBUG to the debug flags. Turn this on to get
lots of lines on stderr to show what objects are opened and
closed.
./src/H5C.o
./src/H5Cpublic.h
./test/dsets.c
./test/tfile.c
Split H5Cget_prop() and H5Cset_prop() into functions for each
property.
./src/H5D.c
./src/H5Dpublic.h
./src/H5Gstab.c
./src/H5O.c
./src/H5Ocont.c
./src/H5Ocstore.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Oistore.c
./src/H5Oname.c
./src/H5Onull.c
./src/H5Oprivate.h
./src/H5Osdspace.c
./src/H5Ostab.c
./src/H5P.c
./src/istore.c
./test/tohdr.c
Object header functions now understand constant
vs. non-constant messages.
./src/H5F.c
./src/H5Fprivate.h
The file OID can be closed before other OID's.
./src/H5Flow.c
H5F_addr_defined() is a macro in this file.
./src/H5G.c
./src/H5Gpublic.h
./test/tstab.c
A current working group cannot be deleted.
./src/H5Gent.c
./src/H5Gpkg.h
Removed unused functionality.
./src/H5public.h
Includes <sys/types.h> for size_t.
Changes to error handling.
./src/H5B.c
Increased size of internal static buffers.
./src/H5C.c
Fixed syntax error when NDEBUG is defined.
./src/H5E.c
./src/H5Eprivate.h
./src/H5Epublic.h
Errors can now be printed with H5Eprint(). Other minor
changes to names and arg types.
./src/H5F.c
The base address is now stored in the boot block. The user
block size and the base address are synonyms.
./src/H5Fstdio.c
Fixed a bug with a return value from fseek().
./src/H5H.c
Added alignment constraints to get rid of unaligned access
errors on the DEC alpha having to do with the heap free list.
./src/H5P.c
./src/H5Ppublic.h
Changed some size arguments from int to size_t and fixed
memory allocation calls.
./src/H5T.c
./src/H5Tpublic.h
Changed the order of functions so all the public ones are at
the top of the file. Other minor changes.
./src/H5detect.c
Added a newline to a string constant.
./src/H5Cprivate.h
Changed H5Csetparm() to H5Cset_prop() and changed it to pass
property values by value instead of reference. Changed
H5Cgetparm() to H5Cget_prop(). Changed data types of file
creation properties so the application can use `int' or
`size_t' for most properties. Properties and their types are
documented in H5Cget_prop().
Changed H5C_class() to H5Cget_class() to make the naming more
uniform across packages.
./src/H5Cpublic.h
Changed names of properties to make them more uniform across
packages: H5F_USERBLOCK_SIZE became H5F_SIZEOF_USERBLOCK,
H5F_OFFSET_SIZE became H5F_SIZEOF_ADDR, H5F_LENGTH_SIZE became
H5F_SIZEOF_SIZE.
./src/H5D.c
./src/H5Dprivate.h
Added support for chunked data storage. There still isn't any
support for data type conversion or data space conversion, so
chunked storage doesn't do much at this time.
./src/H5E.c
./src/H5Epublic.h
Added the H5E_TEMPLATE major error number. I'm about the
change the HGOTO_ERROR() and HRETURN_ERROR() macros to take an
error string as an additional argument instead of specifying
it as a comment.
./src/H5Fistore.c
Added `const' to the `buf' argument of H5F_istore_write()
although this temporarily causes a compiler warning.
./src/H5Fprivate.h
Changed data types of certain file-related variables from
types like `uint8' to `uintn' since the library data types
should not depend on the file data types.
./src/H5P.c
./src/H5Pprivate.h
./src/H5Ppublic.h
Moved H5Pget_lrank() to H5Pget_ndims(), and H5Pget_ldims() to
H5Pget_dims() to make naming more uniform. Dimensions should
always be of type `size_t'. Similarly for H5P_get_lrank() and
H5P_get_ldims().
Removed the interface initialization argument from
FUNC_ENTER() and made it a locally-defined preprocessor
symbol, INTERFACE_INIT.
Changed `offset' to `address' and `length' to `size' in
documentation so it's more consistent. `Offset' still appears
occassionally when it refers to a byte offset within some
other data structure.
Moved interface termination function prototypes from public
header files to .c files and made them static.
./src/H5.c
./src/H5public.h
Added H5init() because it's possible that the predefined data
types are not initialized. This happens only if the first
call to the hdf5 library passes a predefined data type symbol
as an argument. There should be some way to fix this...
./src/H5A.c
./src/H5Aprivate.h
./src/H5Apublic.h
The free_func returns SUCCEED or FAIL, although the return
value is ignored by H5A. This is so we can use the various
H5*_close() functions to free things.
H5Ainc_ref() and H5Adec_ref() are no longer public. Many of
the other atom functions should also be made private, but I'll
save that for later...
Added additional template groups called H5_TEMPLATE_0 through
H5_TEMPLATE_7 that are used by the various template
subclasses.
Increased the number of bits used for atom groups to prevent
negative atoms.
./src/H5AC.c
./src/H5ACprivate.h
Changed H5AC_new() to H5AC_create() to make names more consistent.
./src/H5B.c
./src/H5Bprivate.h
Changed H5B_new() to H5B_create() to make names more consistent.
./src/H5C.c
./src/H5Cprivate.h
./src/H5Cpublic.h
Now supports multiple subclasses of templates, although it's
done with big switch statements. The default values for
templates are defined in the source file to which that
template belongs. This got rid of lots of needless
preprocessor constants.
Added H5Ccreate() to create a new template. Changed
H5C_release() to H5Cclose() to make the naming more
consistent.
./src/H5D.c
./src/H5Dprivate.h
./src/H5Dpublic.h
Enhanced to use the new dataset interface, and uses the enhanced
data type and data space interfaces, which haven't been
completely implemented. The dataset interface doesn't handle
non-contiguous storage, compression, or data type and space
conversions yet.
./src/H5F.c
./src/H5Fprivate.h
./src/H5Fpublic.h
Removed H5Fflush() since just calls H5F_flush(), which doesn't
do what the user would probably think it does, namely, flush
everything. It only flushes those things sitting in the H5AC
cache and the boot block.
Changed the `file_create_parms' field of H5F_low_t to just
`create_parms' since the `file' part is obvious.
./src/H5Fistore.c
Added some support for external files. Mostly just in the
file format and not supported much by the library yet. I need
to finish some dataset functions first.
Changed H5F_istore_new() to H5F_istore_create() to make names
more uniform across packages.
./src/H5Flow.c
Flushing a file causes the file to be physically extended to
the logical eof. This prevents H5F_open() from thinking a
file has been truncated. Most of the time the file will
already be that large, and when it isn't Unix will often just
allocate the final block anyway.
./src/H5G.c
./src/H5Gent.c
./src/H5Gnode.c
./src/H5Gpkg.h
./src/H5Gprivate.h
./src/H5Gstab.c
Removed H5G_basename()
Removed (temporarily) data type information from symbol table
entries and renamed H5G_CACHED_SDATA to H5G_CACHED_SDSPACE to
reflect that it's a simple data space and has nothing to do
with raw data.
Changed H5G_node_new() to H5G_node_create() and H5G_stab_new()
to H5G_stab_create() to make names more uniform across
packages.
Fixed an undefined address bug that happens when H5G_node_debug()
program doesn't pass enough info to H5G_node_load().
./src/H5H.c
./src/H5Hprivate.h
Changed H5H_new() to H5H_create() to make the names more
uniform across packages.
./src/H5M.c
./src/H5Mprivate.h
./src/H5Mpublic.h
Nulled all the create functions. Most of the other callbacks
are to public functions. Removed H5Mcreate().
Changed hobjtype_t to group_t since it has to be the same
thing anyway.
./src/H5O.c
./src/H5Oprivate.h
./src/H5Osdim.c
./src/H5Osdtyp.c
Changed H5O_SIM_DIM to H5O_SDSPACE (simple data space) since
`simple data space' is its official name, not `simple
dimensions'. Will eventually add H5O_CDSPACE for comples data
spaces. Changed _sim_dim_ to _dspace_.
Replaced H5O_SIM_DTYPE and the compound data type messages
with a single H5O_DTYPE message. Changed _sim_dtype_ to _dtype_.
Changed H5O_STD_STORE to H5O_CSTORE (contiguous storage) since
contiguous storage is not necessarily standard. Changed
_std_store_ to _cstore_ in H5Ocstore.c
Added the H5O_EFL (external file list) message.
Changed H5O_new() to H5O_create() to make names more uniform
across packages.
./src/H5Oefl.c NEW
External file list message for specifying which non-hdf5 files
contain raw data for a dataset.
./src/H5P.c
./src/H5Pprivate.h
./src/H5Ppublic.h
Renamed and moved data structures to make the names conform to
our naming scheme.
./src/H5T.c
./src/H5Tprivate.h
./src/H5Tpublic.h
./src/H5Tpkg.h NEW
Data structures redesigned to be more flexible. The interface
was redesigned to make it more regular and to make some names
more uniform across packages.
./src/H5detect.c
Output was changed to produce a file that conforms to the hdf5
coding standard.
./src/Makefile.in
Generates H5Tinit.c by running H5detect.
./src/debug.c
Moved command argument processing.
----------------------
./config/depend.in
Fixed backslashes in sed script because the H5Gnode.c
dependency info was disappearing. You'll have to rerun
config.status to rebuild the Makefiles unless you use gnu
make.
./config/conclude.in
Also removes emacs backup files, TAGS, and svf backup files.
./config/linux
Grouped gcc flags and added provisions for debugging vs. production.
./html/H5.format.html
Updated messages 0x0008, 0x0009, and 0x000A.
./html/storage.html
Documentation describing storage schemes.
./src/Makefile.in
./test/Makefile.in
New source files.
./src/H5A.c
./src/H5Apublic.h
./src/H5C.c
Changed VOIDP to void* in a couple places.
./src/H5AC.c
./src/H5ACprivate.h
./src/H5B.c
./src/H5Bprivate.h
./src/H5G.c
./src/H5Gnode.c
./src/H5Gprivate.h
./src/H5H.c
./src/H5O.c
Removed `const' from some variables because H5G_node_found() wanted
to modify it's udata argument. Removing const there caused it
to cascade to these other locations.
./src/H5AC.c
./src/H5ACprivate.h
./src/H5B.c
./src/H5Gnode.c
./src/H5Gstab.c
./src/H5H.c
./src/H5O.c
Added an extra argument to H5AC_find_f() and H5AC_protect(). This
arg gets passed to the load() method. Also added an extra
argument to the H5AC_find() macro.
./src/H5B.c
./src/H5Bprivate.h
./src/H5Gnode.c
Extra argument passed to the sizeof_rkey() method.
./src/H5Fprivate.c
./src/H5Fistore.c (new)
Added indexed I/O operations.
./src/H5G.c
./src/H5Gnode.c
./src/H5Gprivate.h
Beginning to add H5G_open/close and related bug fixes.
./src/H5Oprivate.h
./src/H5Oistore.c (new)
Added the H5O_ISTORE messsage (0x0008) for indexed storage of
objects.
./src/H5private.h
Added extra braces around both sides of the FUNC_ENTER() and
FUNC_LEAVE() macros so FUNC_ENTER() can appear before
declarations or after executable statements the second case is
used by H5G_namei() to initialize output arguments to sane
values before FUNC_ENTER() might return failure.
int f ()
{
int decl1;
printf ("This happens before FUNC_ENTER()\n");
FUNC_ENTER (...);
int another_declaration;
./src/H5B.c
./src/H5Bprivate.h
./src/H5Gnode.c
Extra arguments for key encoding and decoding.
./src/H5E.c
./src/H5Epublic.h
./src/H5Fistore.c
./src/H5Oistore.c
./src/H5Oprivate.h
Indexed, chunked, sparse storage (not ready for general consumption
yet).
./src/H5V.c (new)
./src/H5Vprivate.h (new)
./test/hyperslab.c (new)
Vector, array, and hyperslab functions.
./src/H5B.c
./src/H5Bprivate.h
./src/H5Fistore.c
./src/H5Gnode.c
./src/H5V.c
./src/H5Vprivate.h
./test/hyperslab.c
Added functionality for indexed storage.
./src/H5F.c
Fixed problems with seek optimizing. Recommend we disable it
until we can implement it in the file/address class since all
of HDF5 must be aware of it.
./src/H5O.c
Fixed comeent speling erorr :-)
./MANIFEST
Added new files.
./config/conclude.in
Added the word `Testing' to the test cases. So if a test
program is called hyperslab then the make output will contain
the line `Testing hyperslab'.
./config/linux
The default file I/O library is Posix section 2 on my linux
machine so I can do some I/O performance testing.
./src/H5C.c
./src/H5Cprivate.h
./src/H5Cpublic.h
Added ability to set size of indexed-storage B-tree.
./src/H5D.c
./src/H5E.c
./src/H5Epublic.h
./src/H5F.c
./src/H5Fprivate.h
./src/H5G.c
./src/H5Gnode.c
./src/H5Gpkg.h
./src/H5Gprivate.h
./src/H5Gpublic.h
./src/H5Gshad.c
./src/H5Gstab.c
./test/stab.c
Changed `directory' to `group' in numerous places.
./src/H5private.h
The FILELIB constant can be set on the compile command-line.
./src/istore.c NEW
Tests for indexed storage.
./src/H5Aproto.h
./src/H5Bproto.h
./src/H5Cproto.h
./src/H5Dproto.h
./src/H5Eproto.h
./src/H5Fproto.h
./src/H5Gproto.h
./src/H5Hproto.h
./src/H5MFproto.h
./src/H5MMproto.h
./src/H5Mproto.h
./src/H5Oproto.h
./src/H5Pproto.h
./src/H5Tproto.h
./src/H5proto.h
These files were removed from the library and renamed by
changing `proto' to `public'.
./src/H5ACpublic.h NEW
./src/H5Apublic.h NEW
./src/H5Bpublic.h NEW
./src/H5Cpublic.h NEW
./src/H5Dpublic.h NEW
./src/H5Epublic.h NEW
./src/H5Fpublic.h NEW
./src/H5Gpublic.h NEW
./src/H5Hpublic.h NEW
./src/H5MFpublic.h NEW
./src/H5MMpublic.h NEW
./src/H5Mpublic.h NEW
./src/H5Opublic.h NEW
./src/H5Ppublic.h NEW
./src/H5Tpublic.h NEW
./src/H5public.h NEW
These files came from the old H5*proto.h files.
./src/Makefile
./src/Makefile.in NEW
Removed. Now generated automatically from Makefile.in by
running configure.
./src/h5oplat.h
./src/hdf5fort.h
./src/hdf5gen.h
./src/hdf5glob.h
./src/hdf5lims.h
./src/hdf5meta.h
./src/hdf5pabl.h
./src/hdf5plat.h
./src/hdf5port.h
./src/hdf5type.h
Removed. The contents of these files has moved to other
header files or source files depending on it's nature.
./src/H5.c
./src/H5A.c
./src/H5B.c
./src/H5C.c
./src/H5D.c
./src/H5E.c
./src/H5F.c
./src/H5G.c
./src/H5H.c
./src/H5M.c
./src/H5MF.c
./src/H5MM.c
./src/H5O.c
./src/H5Ocont.c
./src/H5Oname.c
./src/H5Onull.c
./src/H5Ostab.c
./src/H5P.c
./src/H5T.c
Fixed include files. Moved some things from old headers into
these files.
./src/H5ACprivate.h
./src/H5Aprivate.h
./src/H5Bprivate.h
./src/H5Cprivate.h
./src/H5Dprivate.h
./src/H5Eprivate.h
./src/H5Fprivate.h
./src/H5Gprivate.h
./src/H5Hprivate.h
./src/H5MFprivate.h
./src/H5MMprivate.h
./src/H5Mprivate.h
./src/H5Oprivate.h
./src/H5Pprivate.h
./src/H5Tprivate.h
./src/H5private.h
./src/debug.c
Fixed include files.
./src/hdf5.h
This is now the top-level *PUBLIC* include file. It should
never appear in the library *.c files.
