Code cleanup
Description:
Clean up warnings exposed by compiling on O2K. Also, revert some of Bill
and my changes to the H5S_mpi_opt_types_g, etc. and settle them back into their
original location.
Platforms tested:
h5committested.
Description: H5Sselect_hyperslab and H5Sselect_elements didn't check scalar
dataspaces.
Solution: put error detection in those functions.
Platforms tested: eirene(simple change).
Feature change
Description:
HDF5_PARAPREFIX has been default to various values like /tmp,
/var/tmp, ... None of these solve all problems and all of them
created new problems. For LLNL GPFS, '.' is more appropriate since
they can do the build right inside gpfs. Same for NCSA Copper.
The special setting for Janus is still retained since that is a
one of a kind machine and the uses could use all the help.
Platforms tested:
h5committested (using sol instead of arabica) plus eirene(pp).
Misc. update:
New feature/enhancement
Description:
Chunked datasets are handled poorly in several circumstances involving
certain selections and chunks that are too large for the chunk cache and/or
chunks with filters, causing the chunk to be read from disk multiple times.
Solution:
Rearrange raw data I/O infrastructure to handle chunked datasets in a much
more friendly way by creating a selection in memory and on disk for each chunk
in a chunked dataset and performing all of the I/O on that chunk at one time.
There are still some scalability (the current code attempts to
create a selection for all the chunks in the dataset, instead of just the
chunks that are accessed, requiring portions of the istore.c and fillval.c
tests to be commented out) and performance issues, but checking this in will
allow the changes to be tested by a much wider audience while I address the
remaining issues.
Platforms tested:
h5committested, FreeBSD 4.8 (sleipnir) serial & parallel, Linux 2.4 (eirene)
Purpose: Cray T3E maintenance with Raymond's help
Description: fillval test failed for compact dataset since the
size of the dataset was bigger than 64K.
Solution: Reduced the dataspace of the compact dataset to 1024 elements.
Platforms tested: T3E; it was also tested with semi-manual h5committest.
(I had to built and test manually on modi4 parallel because
of some weird failure of h5committest on modi4)
Misc. update:
code warrior fix
Description:
eliminate the CW failure on getname.c
restrict the H5_NO_FILE_SHARING just to the test file_close of testhdf5 test
Solution:
the CW failure on getname.c can be eliminated if a unmount of the file is made before closing it
on tfile.c the failure is due to the file sharing flag of CW, and the test is maintained as before
Platforms tested:
Windows 2000 (octopus)
Linux 2.4 (rockaway)
SunOS 5.7 (arabica)
IRIX 6.5 (modi4)
Misc. update:
Revert last change (at least temporarily)
Description:
Revert last "NO_SHARED_WRITING" ifdefs after reviewing the tests that were
ifdef'ed out. I don't think that these tests should cause a problem, so I
would like them reviewed again to see if there is a file handle leak in the
library (or test).
Platforms tested:
FreeBSD 4.8 (sleipnir)
Triple check not needed.
Code restructure.
Description:
Restructured tests so they shouldn't depend on shared open()'s and can be
tested with CodeWarrior
Platforms tested:
FreeBSD 4.8 (sleipnir)
Triple check not needed.
code warrior port
Description:
added a flag H5_NO_SHARED_WRITING, it is defined only for CW in the H5pubconf.h
this avoids doing some tests for CW that fail on shared writing
Solution:
Platforms tested:
windows (cw and ms)
Linux 2.4 (rockaway)
SunOS 5.7 (arabica)
IRIX 6.5 (modi4)
Misc. update:
Purpase: Fortran Windows DLL support
Description: Fortran test uses functions h5_fixname and h5_cleanup from h5test.c.
Those were not exported, and Fortran could not link while using DLLs.
Solution: Added H5_DLL to functions prototype to export them from C DLL.
Platforms tested: Windows, h5committested.
Misc. update:
removed and change comments
Description:
removed the modification comment about the while loop, it is not sufficiently important to be there
changed the "Id to name" comment in the test description, it was incorrecly about other thing
Solution:
Platforms tested:
none , just comments
Misc. update:
Purpose: Windows maintenance and code cleanup
Description: There were a lot of places in the source code where varibale
was declared but never used.
Solution: Cleaned the source code to avoid warnings on Windows and Unix
platforms.
Platforms tested: arabica, modi4, Windows
Probably my environment is wrong on Linux, cannot
compile at all when fortran is enabled
If daily tests fails tomorrow, this will be my fault
but I want to go home now :-)
Misc. update:
Purpose: Bug fix
Description: on Cray T90IEEE compact storage test (in test_misc8)
fails since dataset dimensions are too big; dataset
will not fit into the message header.
Solution: Reduced dimension sizes from 100 to 50.
Platforms tested: h5committested on arabica and mod4; verbena failed
because of the F90 license problem. I tested on verbena
by hand and C only; Cray T90IEEE
Misc. update:
Purpose: Verify library behavior
Description: A file is first opened as read only. Then it's closed with
an object left open. A second open of the same file for read and
write should fail.
Platforms tested: arabica - simply test to verify correct library behavior.
Code cleanup
Description:
Christi Forsythe reported that the NGROUPS macro in this file was causing
a macro redefinition warning on the QT machine.
Solution:
Rename NGROUPS to UNLINK_NGROUPS
Platforms tested:
Eyeballed, too trivial for triple check.
Bug Fix
Description:
A resource leak happened if the H5Fget_access_plist() function was
called. What was happening: the driver ID and info parts of the
property list copied in H5Fget_access_plist were being overwritten,
but those properties were copied initially, so we lost information.
Added calls to the H5Fget_access_plist function to get the PList and
then immediately close it. It would cause an infinite loop if there
is a resource leak.
Solution:
Before copying over those values, call the H5F_acs_close() function
to close those values.
Platforms tested:
Modi4 (Parallel & Fortran)
Arabica (Fortran)
Verbena (Fortran & C++)
Misc. update:
Bug Fix
Description:
When using the STDIO driver, the dsets test would fail.
Solution:
Need to flush the data to disk before doing an HDopen call.
Platforms tested:
Modi4 (Parallel & Fortran)
Arabica (Fortran)
Verbena (Fortran & C++)
Misc. update:
Purpose: Windows port
Description: Windows cannot convert from _uint64_t to double. One of
the tests (test_set_local) used conversion to fill a data
buffer. Compilation failed on Windows.
Solution: Fixed the code not to use conversion
Platforms tested: CRAY T90IEEE, arabica, modi4 with parallel, Windows 2000
Only static tests (both debug and release) passed on Windows.
DLLs have multiple problems ;-)
Misc. update:
Code cleanup
Description:
Clean up comments, etc.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
New feature.
Description:
Added tests to verify correct operation of re-using file space.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Code cleanup
Description:
Added macro for reporting line #'s of test failures more easily.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Bug fix
Description:
Return correct value (1 instead of -1) on test failure.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Code cleanup
Description:
Switched some system/library calls to use wrappers.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Bug fix
Description:
This fixes a bug in the low-level metadata caching code in the library
which could possibly lose metadata during file I/O when a lot of objects are
inserted into a group.
This also fixes a couple of (similar) fencepost bugs in the B-tree
deletion code.
Solution:
For the metadata bug - call the low-level driver's 'write' routine instead
of H5FD_write.
For the B-tree bug - include the correct number of keys.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN & parallel
(h5committest is still not working for me on burrwhite)
Misc. update:
Code cleanup & new tests
Description:
Updated szip and shuffle testing to reflect changes to API functions.
Added testing for new API functions: H5Pmodify_filter, H5Pget_filter_by_id
and H5Pall_filters_avail.
Added testing for "can apply" and "set local callbacks.
Cleaned up various things, also.
Solution:
Platforms tested:
FreeBSD 4.8 (sleipnir) w/szip
Linux 2.4 (sleipnir) w/szip
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/szip, FORTRAN & parallel
Misc. update:
Purpose: This a fix for bug #921
Description: On platforms where big test runs, it creates a set of
"solid" 1GB files. Test will take forever (modi4) or will fail
if quotas are exceeded.
Solution: Default library behavior is to allocate dataset space earlier and
to write fill values. I changed the test to use late space allocation
and never write fill values.
Platforms tested: True64 (PSC), modi4 (sequential and parallel), arabica with 64-bit
option(test was skipped there). I also enabled szip and it passed :-)
Misc. update:
Code cleanup
Description:
Updated copyright.
Removed szip-specific testing in favor of standard testing, now that bugs
in the szip filter are fixed.
Updated to latest version of H5Pset_szip API
Solution:
Platforms tested:
FreeBSD 4.8 (sleipnir)
IRIX64 6.5 (modi4) w/parallel
Misc. update:
To add tests for szip compression in HDF5
Description:
Three tests were added:
1) szip filter itself
3) szip + shuffle + fletch32
2) fletch32 + shuffle + szip
Solution:
Since H5Dget_storagesize returns 0(Quincey can fix it or it may be gone),
I create another function called test_internal_filter_szip, which is the almost the same
as test_internal_filter but comments out H5Dget_storage_size function.
Platforms tested:
Since there are changes of configure.in and configure,I didn't use h5committest.
I tested with four platforms.
1) Linux 2.4 (eirene) 2) Solaris 2.7(arabic
3) windows 2000(VS 6.0) 4) SGI IRIX6.5-64(modi4)
For test 1)-3), only basic C tests were done For modi4 test, I tested 64-bit C,parallel and fortran.
All tests passed, except a warning message from szip library when checksum is used in some order, which doesn't cause any real problems.
Misc. update:
Update
Description:
Updated the Copyright statement
Platforms tested:
Linux (This change is only in the comments, so I just check that the
modules still compile)
Misc. update:
New test feature
Description:
Added the VFD_LIST definition to support the "make check-vfd" tests.
Platforms tested:
h5committested.
Since check-vfd is not invoked by default, I went to each machine
to invoke them by hand. The target worked fine but there were
errors reported from those driver tests. (Those errors were beyond
the scope of this make target test.)
Misc. update:
New internal feature
Description:
Add internal API for building and working with heaps (H5HP). This will be
used for the LRU algorithm in the new metadata cache code.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir)
Code cleanup
Description:
Added more information to information printed for test failure.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Code/comment cleanup
Description:
Made more information about reference-counted strings private to the
H5RS package by trimming the typedef used in H5RSprivate.h and switching
from using macros for a couple of operations to using functions.
Updated copyright information for a few of these files.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Code cleanup
Description:
Clean up miscellaneous warnings which have crept into the code.
Fix "_POSIX_C_SOURCE not defined" warning on FreeBSD.
Adjust gcc compiler flags to be more concise for production mode.
Refactor the H5O code so that there is a stronger boundary between code
in the H5O package and code in the library which just calls H5O routines.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel and gcc 2.95.4 & gcc 3.2.2
Misc. update:
Update MANIFEST if you add or remove any file.
Bug Fix
Description:
Metadata cache in parallel I/O can cause hangs in applications which
perform independent I/O on chunked datasets, because the metadata cache
can attempt to flush out dirty metadata from only a single process, instead
of collectively from all processes.
Solution:
Pass a dataset transfer property list down from every API function which
could possibly trigger metadata I/O.
Then, split the metadata cache into two sets of entries to allow dirty
metadata to be set aside when a hash table collision occurs during
independent I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Updated release_docs/RELEASE
Bug fix
Description:
The "system scope" for threads isn't supported on all platforms.
Solution:
Add detection of this feature to the configure script and check for
"H5_HAVE_SYSTEM_SCOPE_THREADS" in the appropriate places.
Platforms tested:
modi4 w/threadsafe
New feature for developers.
Description:
Added "function stack" tracing to library. This allows developers (there
is no public API) to call H5FS_print within the library and get a listing
of the functions traversed to reach that point in the library. Eventually,
I may add support for reporting the parameters to each function also...
Mainly for debugging parallel I/O programs, but I think it will come in
handy in other cases also.
The function stack tracking is controlled with a configure switch:
--enable-funcstack, which defaults to enabled currently. When we branch
for 1.6, we should change the default setting on the branch to be disabled.
Also, added a destructor to the thread-specific keys when thread-safety is
turned on in the library. Otherwise, they were leaking memory and causing
difficult to debug errors in threaded programs (like the test/ttsafe test).
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) w/thread-safety enabled.
Misc. update:
Updated MANIFEST with new files added (src/H5FS.c & src/H5FDprivate.h)
Update release_docs/RELEASE with thread-safety bug fix.
