when handling an I/O request on a metadata entry that has been sub-allocated
from a larger file space allocation (i.e. fixed and extensible array), and
that crosses at least one page boundary. .
This required modifying the metadata cache to provide the type of the
metadata cache entry in the current I/O request. For now, this is done
with a function call. Once we are sure this works, it may be appropriate
to convert this to a macro, or to add a flags parameter to the H5F block
read/write calls.
Also updated the metadata cache to report whether a read request is
speculative -- again via a function call. This allowed me to remove
the last address static variable in the H5PB_read() call, which is
necessary to support multiple files opened in VFD SWMR mode.
Also re-wrote the H5PB_remove_entries() call to handle release
of large metadata file space allocations that have been sub-allocated
into multiple metadata entries. Also modified the call to
H5PB_remove_entries() in H5MF__xfree_impl() to invoke it whenever
the page buffer is enabled and the size of the space to be freed is
of page size or larger.
Tested serial / debug on charis and Jelly.
Found a bug in H5MF_xfree_impl(), in which the call to H5PB_remove_entries()
is skipped due to HGOTO_DONE calls earlier in the function. While the
obvious action is to move the call earlier in the function, best to
consult with Vailin first, as there is much going on and it would be
best to avoid making the situation worse. If nothing else, there are
some error management issues.
including the merge of `hdffv/hdf5/develop`, back to the branch that Vailin and
I share.
Now I need to put this branch on a fork with a less confusing name than
vchoi_fork!
(A) #5: Add the "pb_expansion_threshold" field to the "H5F_vfd_swmr_config_t" structure
and update H5Pset_vfd_swmr_config() and H5Pget_vfd_swmr_config() accordingly
(B) #13 bullet 2: Comment H5F_vfd_swmr_config_t in H5Fpublic.h properly
(copied from John's description in the RFC)
(C) Change the field name "vfd_swmr_writer" to "writer" in "struct H5F_vfd_swmr_config_t"
(as indicated on page 11 in the RFC) and all references to it
comment to myself that I need to reduce code duplication with the MPMDE
test.
In vfd_read_each_equals(), print the correct expected value when there
is a discrepancy.
No functional change intended: correct a comment in
vfd_read_each_equals. Fix indentation in the test_raw_data_handling()
header comment.
buffer's treatment of multipage metadata entries (MPMDEs). Mention why
an H5PB_flush() is not necessary for MPMDEs to reach visibility at the
VFD layer.
macro magic. Use namebases and namebase, instead.
Extract a bunch of copy-and-paste VFD SWMR setup into a new subroutine,
swmr_fapl_augment().
Make sure that the metadata reads all-0s until it reads all -1s.
Extract a subroutine, vfd_read_each_equals(), that reads and compares a
region with one of its arguments.
Rename from test_basic_metadata_handling() to
test_metadata_delay_basic(), since that gets at what we're testing
better.
Don't perform an H5PB_flush(), it's not necessary for this test because
H5Fvfd_swmr_end_tick() has done essentially the same thing.
anticipate comparing the written buffer with the read buffer.
Don't initialize variables prematurely so that the compiler has a chance
to warn about variables read before they are written.
Repeatedly flush the page buffer, once each time we end the tick.
Write errors to stdout instead of stderr.
Combine the VFD SWMR and non-VFD SWMR raw-data test into one
routine that takes a bool parameter to switch on VFD SMWR.
Update my description of the to-be-written metadata test for VFD SWMR.
(2) Test files for encoding/decoding property lists
(3) Fix test failures for PB statistics in test/page_buffer.c
(Will double check with John later about PB statistics collection)
Tested serial / debug on Charis and Jelly.
Two known issues:
1) New page buffer seems to expose issues in the accumulator code.
For whatever reason, fheap with the new page buffer exposes corruption
issues if the page buffer uses H5F__accum_read/write(), but the problems
go away if the page buffers uses H5FD_read/write() instead.
Need to either chase this or dis-able page bufffer in combination with
the accumulator.
2) Encountered a number of assertion failures that are explainable by the
free space manager code not telling the page buffer to discard pages
when they are freed.
Wrote code to handle this -- once the free space manager is modified,
this code should be removed and the original assertions restored.
Description:
Applied the typo fixes from user's report.
The previous pull request couldn't be merged because it was too old,
and it was too complicated for me to resolve conflicts.
Platform tested:
Linux/64 (jelly) - very minor
* commit '54957d37f5aa73912763dbb6e308555e863c43f4':
Commit copyright header change for src/H5PLpkg.c which was added after running script to make changes.
Add new files in release_docs to MANIFEST. Cimmit changes to Makefile.in(s) and H5PL.c that resulted from running autogen.sh.
Merge pull request #407 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:hdf5_1_10_1 to hdf5_1_10_1
Change copyright headers to replace url referring to file to be removed and replace it with new url for COPYING file.
1) Added code test/page_buffer.c to verify that page buffering is
disabled in parallel builds.
2) Added code to test/cache_image.c to verify correct interaction
between evict on close and cache image -- in particular management
of a file containing a cache image containing dirty metadata that
has been opened R/O. Also fix for the bug exposed.
3) Added code to testpar/t_cache_image.c to verify expected procedure
for reading cache images, and also supporting stats collection code
needed for the test.
4) Repair of an overactive sanity check in H5C__reconstruct_cache_contents().
5) Other minor tidies in passing.
Tested serial and parallel, debug and production on Jelly.
