Optimize code
Description:
Eliminate some computations from inner loop, giving about a 5% speedup.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to h5committest
Refactored code
Description:
Switched float->double and double->int conversions to use new, more generic
conversion looping macros.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too small to require h5committest
More refactoring
Description:
Re-wrote inner loop of integer conversion routines with "psuedo-template"
macros, to hoist 'if-else's out of inner loop in an easy to understand way.
This speeds up the integer type conversions by 15-20%.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Code cleanup
Description:
Re-wrote duff's device with some macro substitution to make algorithm more
obvious.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too small to require h5committest
Refactor integer conversion macros
Description:
Turned integer conversion macros "inside" out, using an interlocking
macro technique similar to templates in C++. The macro which actually performs
the conversion is now invoked "genericly" from inside another macro, which
will allow the inner conversion loop to be optimized in a much easier way.
This "psuedo-template" technique could be useful for other semi-repetitious
patterns in the library - possibly the datatype initialization code...
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Code cleanup
Description:
Change field member count and indices for compound and enumerated types from
'int' to 'unsigned' to better reflect actual use.
Cleaned up a few other minor compiler warnings, etc.
Platforms tested:
FreeBSD 4.9 (sleipnir)
Linux 2.4 (verbena)
too minor to require h5committest
Bug fix
Description:
The 'char *' type is one of the "strongly" aligned types on Crays, but
a 'void *' is "weakly" aligned. So, assigning a 'void *' (pointing to a
location to place a 'char *') to a 'char **' can change the pointer value
during the assignment.
Solution:
Don't alias the 'void *' where the variable-length information ('char *'
or 'hvl_t') will go. Use a temporary variable on the stack to build up the
information about the VL string or sequence and then memcpy() the temporary
variable directly to the location pointed to with the 'void *'
Platforms tested:
FreeBSD 4.9 (sleipnir)
Cray SV1 (wind)
specific to Cray problems, h5committest not necessary.
Bug fix
Description:
Correct the size of the buffer needed for the destination value to use
the actual destination type size.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug fix
Description:
The VL type conversion routine attempt to align it's destination buffer to
an offer that will work for both hvl_t and char * types, but the algorithm used
fails to work correctly on Cray machines.
Solution:
Give up on attempting to align the buffer when it's allocated on the stack.
Just dynamically allocate it instead.
Platforms tested:
FreeBSD 4.9 (sleipnir)
Cray SV1 (wind)
Bug fix
Description:
When too many messages were inserted into an object header, the library
had an internal pointer to the "new message" that was pointing to the incorrect
location when the array of messages was re-allocated.
In the worst case, this could cause a file to be corrupted.
Solution:
Update the internal pointer when the array is re-allocated.
Platforms tested:
FreeBSD 4.9 (sleipnir)
Too small to require h5committest
Feature add
Description:
Add a few new fields to the H5G_stat_t structure, to allow more information
about the object header to be retrieved.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Add feature
Description:
Add H5Fget_freespace() routine, to check the amount of free space in a
file. This information is only valid until the file is closed currently,
however (until we start recording the free space information in the file
itself).
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Improved algorithm (bug fix, sorta)
Description:
The internal algorithm for adding new IDs in the ID manager code (H5I) was
adding new IDs to the front of the linked list and never adjusting the order
of the items on the list (unless an ID was deleted). If many new objects were
created, they would push earlier ones _way_ down the list (especially if the
objects were being leaked in the application, as they appear to be in the
current HDF-EOS5 library) and would cause O(n) search time for items on the
list.
The ID caching code in the ID manager was avoiding this behavior sometimes,
but it was adding IDs that were looked up to the very tail of the cache and
they would frequently leave the cache before helping.
Solution:
Implemented a "move to front" scheme for the linked list of IDs, which
improves the lookup situation for frequently accessed objects.
Removed ID caching code now, as the "move to front" algorithm actually
works better.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug/feature fix.
Description:
Relax restriction on parallel writing to compact datasets to allow partial
I/O.
Updates to reference manual mentioning the issues involved are delayed until
reference manual 'lock' is removed later this week.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug fixes and code cleanup
Description:
Lots of changes here:
- Fixed bug #691 - when shared datatypes are used in attributes they
are incorrectly copied into the attribute instead of referring
the the named datatype in the file. This required bumping the
version of the attribute message. The new version of the attribute
message is only written out when a shared datatype is used in
the attribute. [Also, this format change made the size of the
attribute smaller.]
- Added information to attribute debugging routine so that shared
datatypes are displayed correctly with the h5debug tool.
- Refactored the H5O* routines to extract code that was common to
several routines into subroutines to call.
- Added 'link' method for H5O message sub-classes, which increments
the link count on shared objects when a message is created which
shares them.
- Corrected [unreported] bug where the link count was not being
decremented on the shared object when a object header message
with a reference to that object was deleted from the file.
- Reduced size of shared message from 49 bytes (which was incorrect
anyway and should have been 48 bytes) to 10 bytes, which required
bumping the version of "shared" messages.
- Refactored some of the shared datatype routines to allow for easier
queries of "committedness" internally to the library and also
added routine to easily increment/decrement the reference count of
a shared datatype.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug Fix
Description:
The FPHDF5 code couldn't create a dataset then access it. Turns out
that the "O_find_in_ohdr" code was protecting the object header which
pulls it into the cache then unprotecting it. However, this caused
the cache entry to be blown away and THEN we'd try to reread the
entry (via AC_protect) but it didn't have all of the data that the
find_in_ohdr function decodes for us decoded. It was also kind of
unnecessary since we can just protect then call O_find_in_ohdr.
Solution:
Removed the AC_protect and AC_unprotect from O_find_in_ohdr. Called
AC_protect before calling the O_find_in_ohdr function.
Platforms tested:
Linux (Fortran, C++)
IRIX (parallel, Fortran)
Sun (Fortran)
Misc. update:
Code cleanup
Description:
Clean up a few loose ends and warnings for the 1.6 compatibility changes
to the error API.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Code cleanup
Description:
Clean up various warnings and parameter mis-matches, etc.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to need h5committest
Removed Dead Code
Description:
Some of the FPHDF5 code was dead (I thought it'd be useful at one
point, but was wrong).
Solution:
Removed
Platforms tested:
Linux (FPHDF5 specific. No need for h5committest)
Misc. update:
Bug fix
Description:
Buffer for decoding superblock's driver information was too small when
using some VFDs (like the multi-file VFD).
Also made FPH5 code more portable and obvious when it's broadcasting the
superblock from the captain process to the other clients.
Solution:
Allocate the buffer for the driver information dynamicly
Platforms tested:
Copper
No h5committest because it's already working on other platforms.
FPHDF5 Update
Description:
Added locking/unlocking to the AC_protect/AC_unprotect calls. The SAP
does all of the caching so nothing should be saved on the client
side. So, when the client unlocks the metadata, it can destroy it.
The metadata is always read from the SAP during a lock.
Platforms tested:
Linux (FPHDF5 specific change. No need for h5committest)
Update
Description:
A lot of modifications for the FPHDF5 stuff:
H5AC.c
H5ACprivate.h - Removed AC_find (it's replaced with AC_protect
and AC_unprotect). Added flushing if it's an FPHDF5 driver and
we're doing an AC_set or AC_unprotect with the dirty flag set.
H5B.c - Split up the B_flush function into different functions
since the one function was doing serialization which is better
left as a separate entity.
H5D.c - Removed some FPHDF5 code that was incorrect
H5F.c - Split up the F_flush function so that it no longer
allocates file space. Created new functions (F_init_superblock,
F_read_superblock, and F_write_superblock) for greater modularity
and so that the FPHDF5 non-captain processes can read the
superblock after the captain process writes it.
H5FD.c - Error message correction.
H5FDfphdf5.c - Removed MPI barrier call that wasn't needed.
H5FPclient.c
H5FPserver.c - Modified so that if a process requests data that
isn't exactly aligned, we can return it if we have the block that
contains the requested address.
H5G.c
H5Gent.c
H5Gnode.c
H5HL.c
H5HLpkg.h
H5HLprivate.h
H5Oefl.c - Removed the H5HL_peek function since it was doing a
(now unsafe) holding of the information in the cache. Replaced
with protect and unprotect calls.
H5TB.c - Error fix. The TB_dless function wasn't working
properly.
H5Gstab.c - Format change.
H5err.txt
H5Edefin.h
H5Einit.h
H5Epubgen.h
H5Eterm.h - Added new error code.
Platforms tested:
Modi4 (paralle, Fortran)
Sol (Fortran)
Linux (C++, Fortran)
Copper (Parallel, Fortran)
Misc. update:
Bug fix
Description:
The MPI_File_set_size() routine on ASCI Red is not able to extend files
so that they are larger than 2GB.
Solution:
Add an extra macro which controls whether MPI_File_set_size() can handle
>2GB offsets or if our "older" way of reading a byte, then writing a byte at
the appropriate offset should be used.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug fix
Description:
The MPI_File_set_size() routine on ASCI Red is not able to extend files
so that they are larger than 2GB.
Solution:
Add an extra macro which controls whether MPI_File_set_size() can handle
>2GB offsets or if our "older" way of reading a byte, then writing a byte at
the appropriate offset should be used.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug fix
Description:
H5Dwrite was only checking the "top level" of the datatype to stop parallel
I/O on variable-length datatypes.
Solution:
Make checks "deeper".
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Code cleanup & optimization
Description:
Split superblock initialization and space allocation out from writing
the superblock information to disk, which makes the code much cleaner and
easier to understand and also allows FPHDF5 to have a fully completed superblock
to Bcast to other (non-caption) processes.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Update
Description:
The F_open and F_flush functions had a lot of cruft in them. The
F_flush was being used as a way to allocate the superblock. The
F_open had a bunch of code in there to read and serialize the
superblock.
Solution:
Moved these out into their own functions.
Platforms tested:
Modi4 (parallel, Fortran)
Copper (parallel, Fortran)
Verbena (Fortran, C++)
Sol (Fortran)
Misc. update:
Code cleanup, etc.
Description:
Generalize Ray's datatype fixes to handle packing compound datatypes which
are the base type of an array or variable-length type, etc.
Also track "packedness" of a compound datatype from it's creation, instead
of only setting the 'packed' flag after the datatype was explicitly packed.
Updated docs to reflect that a compound datatype is allowed to grow (but
not shrink).
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Code cleanup
Description:
Patch up a few places where the metadata cache could ask for a read lock
instead of a write lock and other minor code cleanups.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to need h5committest
Misc. update:
Update
Description:
Added extra parameter to the H5AC_protect() function that indicates
if the cache being asked for is going to be written to or just read
from. Those AC_protect calls that were H5AC_find calls are now
read-only. The rest are writes.
Platforms tested:
Linux (h5committest not needed due to size and small impact)
Misc. update:
Bug fix.
Description:
Correct bug where a file opened twice, once with read-write permission
and once with read-only permission would cause closing the file with the
read-only file ID to fail because it was trying to flush information out
of the file.
Solution:
Check the permissions on file IDs that are being closed and only flush
when the particular file ID was opened with write permission.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too small to need h5committest
Misc. update:
Removal of H5AC_find()
Description:
The H5AC_find() function is mostly redundant and with the new
Flexible Parallel HDF5 stuff, we need to do locking on metadata
returned from the H5AC_find() anyway. So, all of the locking stuff
will be placed in the H5AC_{un}protect() functions. The H5AC_find()
is no longer needed.
Solution:
Replaced all H5AC_finds with H5AC_protects and H5AC_unprotects.
Platforms tested:
Linux (Fortran & C++)
Solaris (Fortran)
Irix (Parallel & Fortran)
Misc. update:
Code cleanup/bug fix
Description:
Move metadata cache calls around a bit so they protect the usage of the
metadata better.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too small to need h5committest
Fix, of a sort
Description:
Some of the code would get an object from the cache via the
H5AC_find() function and then modify the returned object. This
behavior is incorrect as the pointer returned via the H5AC_find()
function is supposed to be read only.
Solution:
Changed the H5AC_finds to H5AC_protect() instead and added the
appropriate H5AC_unprotect() function.
Platforms tested:
(simulated h5committest by hand since it doesn't work for me)
Linux (Fortran, C++)
Solaris (Fortran)
AIX (Fortran, C++)
SGI (Parallel, Fortran)
