Commit Graph

22731 Commits

Author SHA1 Message Date
Dana Robinson
5ceaf920ec VFD SWMR: Warning fixes and minor cleanup (#1847)
* Normalization with develop

* Moves remaining datatype code changes over from develop

* Cleanup in examples files

* Warning cleanup in VFD SWMR code

* Committing clang-format changes

* Warning cleanup

* Warning reduction

* More warning fixes

* Committing clang-format changes

* Even more warning reduction in the VFD SWMR tests

* More warning fixes in the VFD SWMR tests

* Even more VFD SWMR test warning fixes

* Last warning fixes in VFD SWMR

* Committing clang-format changes

* Minor things missed from develop

* Warning fixes from GitHub build failures

* Committing clang-format changes

* Fix for warning due to weird bit shift type promotion

* Yet another attempt at fixing the integral promotion warning

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-05 16:55:18 -07:00
Dana Robinson
ac7bddf2af VFD SWMR: sync with develop (#1825)
* bin directory sync

* doxygen changes

* C++ sync with develop

* Fortran sync with develop

* Sync various docs with develop

* Java sync with develop

* More doxygen sync with develop

* tools sync with develop

* h5test.h testing macros get enclosed in do..while loops (#1721)

* Minor examples normalization with develop

* hl sync with develop

* sprintf to snprintf (#1815)

* Misc sync w/ develop

* Brings some selection I/O bits over from develop

* Brings over some const fixes from develop

* Brings over more const bits from develop

* Minor bits missed in early syncs

* Brings over rest of selection I/O

* Sync of mirror VFD changes w/ develop

* Committing clang-format changes

* Adds missing testpar file

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-06-23 20:16:21 -07:00
vchoi-hdfgroup
50b3fb09a7 Merge pull request #1806 from vchoi-hdfgroup/to_merge_echidna_fixes
To merge echidna fixes
2022-06-08 01:09:28 -05:00
github-actions
3848c783b6 Committing clang-format changes 2022-06-08 06:08:34 +00:00
vchoi
1c7f3aa4db Merge branch 'to_merge_echidna_fixes' of https://github.com/vchoi-hdfgroup/hdf5 into to_merge_echidna_fixes 2022-06-08 01:06:09 -05:00
vchoi
cf2b9d600a (1) Modify test/vfd_swmr_bigset_writer.c to use h5_retry routines instead of "for loop for NUM_ATTEMPTS" times.
(2) Clean up test/vfd_swmr.c
2022-06-08 01:05:04 -05:00
vchoi-hdfgroup
c698497575 Merge pull request #1802 from vchoi-hdfgroup/to_merge_echidna_fixes
To merge echidna fixes
2022-06-07 11:00:23 -05:00
github-actions
a284ec5ece Committing clang-format changes 2022-06-07 03:31:55 +00:00
vchoi
7bbe45e71d Fixes for:
(1) H5O_refresh_metadata(): address failure when make check-passthrough-vol: use H5VL_object_data()
to retrieve the vol object.
(2) H5FD__vfd_swmr_open(): use HADDR_UNDEF as maxaddr when opening the underlying hdf5 file;
otherwise it will casue problem when the hdf5 file is opened with the core driver.
(3) src/H5Fvfd_swmr.c and vfd swmr tests:
--Use long long for constant define in H5Fvfd_swmr.c and when snprintf the updater file's sequence number
--Modify test/vfd_swmr.c to accommodate endianness of machine and also long long for sequence number
--Modify vfd_swmr_bigset_writer.c, vfd_swmr_common.c, vfd_swmr_gfail_writer.c, vfd_swmr_gperf_writer.c
to use long long for constant define
2022-06-06 22:27:40 -05:00
vchoi-hdfgroup
63c777b913 Merge pull request #67 from HDFGroup/feature/vfd_swmr
Feature/vfd swmr
2022-06-06 11:53:58 -05:00
vchoi-hdfgroup
b34aa88f48 Merge pull request #1800 from vchoi-hdfgroup/merge_john_tarball
Merge john tarball
2022-06-03 21:46:43 -05:00
github-actions
86c725c319 Committing clang-format changes 2022-06-03 19:35:52 +00:00
vchoi
22ff522173 Merge branch 'merge_john_tarball' of https://github.com/vchoi-hdfgroup/hdf5 into merge_john_tarball 2022-06-03 14:33:17 -05:00
vchoi
ff77fd7abc Correct spelling errors. 2022-06-03 14:18:45 -05:00
github-actions
fae493e84c Committing clang-format changes 2022-06-03 19:12:40 +00:00
vchoi
c766dc8d88 Merge in VFD SWMR changes by John Mainzer.
Commit log message from John:
    Returned VFD SWMR to using the VFD SWMR reader VFD only in the reader case.
    In passing, added a private VFD SWMR reader VFD fapl entry that is pushed
    and popped off the FAPL during file open, and removed the code that set
    the VFD SWMR reader VFD as the driver in the FAPL when VFD SWMR is
    configured.  This was necessary, as there is no mechanism to prevent the
    user from overwriting this entry on the FAPL before file open.  While
    we don't use it now, it also gives us a mechanism for allowing the
    user to specify an underlying VFD for VFD SWMR.

    Modified code to compare file opens to compare the terminal VFDs, not
    the top level VFDs.  Failure to do this allowed multiple opens of
    the same file with the same VFD but with different pass through
    VFDs to appear to be treated as different files -- with the obvious
    file corruption issues.  To support this, added a new VFD ctl op code
    to return a pointer to the instance of H5FD_t associated with the
    terminal VFD.

    Note that this change does not address the case of the same file being
    opened twice with different terminal VFDs -- that will have to be addressed
    another day.

Overview of major changes from John:
1) Reworked file open so that the VFD SWMR reader VFD is only
   used when a file is opened VFD SWMR reader.  This required
   the following changes:
   a) Removed code to set the driver in H5Pset_vfd_swmr_config()
   b) Added a private fapl entry for the VFD SWMR reader VFD
   c) Modified H5F_open to test for VFD SWMR reader opens, and push the
      vfd swmr reader vfd FAPL entry on the VFD stack if so.  In this case
      the entry is popped off the VFD stack on exit so as to avoid any net
      modification from the supplied FAPL.
2) Removed dedup code, and augmented H5FD_cmp() to provide the
   necessary functionality.  This required the following changes:
   a) Added the get terminal VFD op code to the H5FD ctl call.
      This allows duplicate file opens with the same VFD but different overlying
      pass through VFDs to be recognized.  Updated ctl callback in VFDs as required
      to support the new op code.
   b) Modified H5FD_cmp to use the above ctl op code to allow it to recognize
      duplicate file opens with the same VFD but with different overlying
      passthoguh VFDs.  This is necessary to recognize duplicate VFD SWMR reader
      and regular opens.  Note that this does not allow us to recognize duplicate
      opens with different terminal VFDs.
2022-06-03 14:09:32 -05:00
vchoi-hdfgroup
54fdbd3af2 Merge pull request #66 from HDFGroup/feature/vfd_swmr
Merge pull request #1786 from vchoi-hdfgroup/feature/vfd_swmr
2022-05-25 15:48:57 -05:00
vchoi-hdfgroup
cfdc7f262f Merge pull request #1786 from vchoi-hdfgroup/feature/vfd_swmr
Feature/vfd swmr
2022-05-19 14:40:03 -05:00
vchoi-hdfgroup
64e730c4f1 Merge pull request #65 from vchoi-hdfgroup/new_vds_feature_vfd_swmr
New vds feature vfd swmr
2022-05-16 12:08:27 -05:00
github-actions
b4bcd01292 Committing clang-format changes 2022-05-16 16:20:11 +00:00
vchoi
06d0f73981 Add test to verify multiple opens of the same file with different VFDs. 2022-05-16 11:15:59 -05:00
vchoi-hdfgroup
22c57f1aa1 Merge pull request #64 from vchoi-hdfgroup/feature/vfd_swmr
Feature/vfd swmr
2022-05-13 16:46:06 -05:00
vchoi-hdfgroup
469e8da61e Merge pull request #63 from HDFGroup/feature/vfd_swmr
Feature/vfd swmr
2022-05-13 12:20:39 -05:00
Dana Robinson
b2e1a59c73 Check for parallel HDF5 before running SWMR acceptance tests (#1764) 2022-05-11 16:32:32 -07:00
vchoi-hdfgroup
11a88dbda3 New vds feature vfd swmr (#1743)
* 1) Add feature flag H5FD_FEAT_SUPPORTS_VFD_SWMR to sec2, core and stdio drivers
2) Disable file locking for VFD SWMR reader in H5F_open()
3) Add checking of VFD SWMR feature flag in test/page_buffer.c and test/vfd_swmr.c.
Skip tests if the driver does not support VFD SWMR feature.
4) Add new tests + cleanups to test/vfd_swmr.c.

* Committing clang-format changes

* Fix failing check for Windows:
Add the VFD SWMR feature flag also to test_windows() (which is sec2) in test/vfd.c.

* Committing clang-format changes

Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-05-06 15:04:00 -07:00
github-actions
13d6e10dc6 Committing clang-format changes 2022-05-06 21:32:20 +00:00
vchoi
45b4f9ed10 Merge branch 'new_vds_feature_vfd_swmr' of https://github.com/vchoi-hdfgroup/hdf5 into new_vds_feature_vfd_swmr 2022-05-06 16:30:03 -05:00
vchoi
a8d506ffce Fix failing check for Windows:
Add the VFD SWMR feature flag also to test_windows() (which is sec2) in test/vfd.c.
2022-05-06 16:26:58 -05:00
github-actions
6e043ce6f2 Committing clang-format changes 2022-05-06 19:58:54 +00:00
vchoi
3b312e2606 Merge branch 'new_vds_feature_vfd_swmr' of https://github.com/vchoi-hdfgroup/hdf5 into new_vds_feature_vfd_swmr 2022-05-06 14:56:10 -05:00
vchoi
080b6d0405 1) Add feature flag H5FD_FEAT_SUPPORTS_VFD_SWMR to sec2, core and stdio drivers
2) Disable file locking for VFD SWMR reader in H5F_open()
3) Add checking of VFD SWMR feature flag in test/page_buffer.c and test/vfd_swmr.c.
Skip tests if the driver does not support VFD SWMR feature.
4) Add new tests + cleanups to test/vfd_swmr.c.
2022-05-06 14:54:21 -05:00
vchoi-hdfgroup
702dd6910d Merge pull request #1671 from vchoi-hdfgroup/new_vds_feature_vfd_swmr
New vds feature vfd swmr
2022-04-21 17:27:08 -05:00
github-actions
fd3e7a1781 Committing clang-format changes 2022-04-21 22:03:02 +00:00
vchoi
21c9366d1d Merge branch 'new_vds_feature_vfd_swmr' of https://github.com/vchoi-hdfgroup/hdf5 into new_vds_feature_vfd_swmr 2022-04-21 17:00:00 -05:00
vchoi
9163f6ca9b 1) Fix spelling errors
2) Fix couple routine names due to FUNC_ENTER_PACKAGE.
2022-04-21 15:10:29 -05:00
github-actions
3ed0a0ee54 Committing clang-format changes 2022-04-21 19:00:37 +00:00
vchoi
3e990fed8e Modifications to:
A) Allow automatic generation of metadata file names for VDS support
B) Allow VFD SWMR reader to open an existing HDF5 file either before
the VFD SWMR writer has opened it or after it has closed.
C) Remove the usage of H5P_FILE_ACCESS_ANY_VFD.
D) Add H5FD_FEAT_SUPPORTS_VFD_SWMR feature flag.
E) Do not map H5FD_MEM_GHEAP to H5FD_MEM_DRAW when vfd swmr IS enabled;
   map H5FD_MEM_GHEAP to H5FD_MEM_DRAW when vfd swmr is NOT enabled.

For details regarding #A and #B above, see version 9 of the RFC.
2022-04-21 13:56:59 -05:00
vchoi-hdfgroup
ff8f512871 Merge pull request #62 from vchoi-hdfgroup/feature/vfd_swmr
Feature/vfd swmr
2022-04-21 12:46:54 -05:00
vchoi-hdfgroup
2f86cba3ec Merge pull request #61 from HDFGroup/feature/vfd_swmr
VFD SWMR: Merges from develop (#1670)
2022-04-21 12:09:51 -05:00
Dana Robinson
ec14985c85 VFD SWMR: Merges from develop (#1670)
* Cleans up SWMR test scripts (both legacy and VFD)

* Brings over format source changes for Java

* Normalization of HL, Fortran, C++, Java w/ develop

* Removes the STATIC flavor of FUNC_ENTER macros (#1622)

* Removes the STATIC flavor of FUNC_ENTER macros

* Remove H5_NO_ALIGNMENT_RESTRICTIONS (#1426)

* Do not conditionally compile code that uses a pointer dereference
and assignment to copy a potentially unaligned variable to aligned
automatic storage, or vice versa.  Instead, always use naked `memcpy(3)`s.
Disassembling the generated code reveals that the `memcpy(3)`s optimize
(`-O3`) to a single `mov` instruction for x86_64, which is not strict
about alignment.

This change reduces the size of code and scripts by 143 lines, eases
our way to cross-compilation, and avoids invoking undefined behavior.

* Committing clang-format changes

* Per discussion, use HD and add comments.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Removes unused definitions from module headers (#1624)

* Misc stuff from develop (includes some parallel things)

* Brings over SWMR test quiet mode changes from develop

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-04-20 22:24:35 -07:00
vchoi-hdfgroup
ff7fe9e9af Merge pull request #60 from HDFGroup/feature/vfd_swmr
Feature/vfd swmr
2022-04-04 14:40:16 -05:00
Dana Robinson
bc62da7eb9 VFD SWMR: Normalization with develop and cleanup (#1559)
* Brought over plugin and test script changes

* Removes remaining register keywords (#1481)

* Fixed warnings in the aux process code

* Minor fixes from develop

* Minor changes from develop, fixed format warnings

* Formatted source

* Added HD prefix to timespec_get

* Cleanup in new files

* Removes the MANIFEST file and unused release files (#1497)

* Removes the MANIFEST file and unused release files

* Updated tar command

* checkposix corrections

* More checkposix fixes

* Ripped out unused instrumentation functionality

* Brought over cache tagging changes from develop

* Changes to tagged metadata expulsion iteration

* Fixed typo

* Brought over H5O__free() changes from develop

* Brings (unused) parallel page buffer test in line with develop

* Moved the functionality in supervise.subr to test_vfd_swmr.sh

* Tools VFD parameter updates

* Committing clang-format changes

* H5F VFD SWMR refactoring

* Committing clang-format changes

* Misc changes

* Acceptance test cleanup

* HD prefix
* H5_get_option instead of getopt
* Some bool fixes

* Changes to VFD SWMR acceptance tests

* Minor cleanup

* Use AC_MSG_RESULT/AC_MSG_ERROR/AC_MSG_WARN instead of echo (#1532)

* Merges from develop

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-03-31 00:56:29 -07:00
Dana Robinson
fbb532cd63 VFD SWMR: Normalization with develop (#1506)
* Brought over plugin and test script changes

* Removes remaining register keywords (#1481)

* Fixed warnings in the aux process code

* Minor fixes from develop

* Minor changes from develop, fixed format warnings

* Formatted source

* Added HD prefix to timespec_get

* Cleanup in new files

* Removes the MANIFEST file and unused release files (#1497)

* Removes the MANIFEST file and unused release files

* Updated tar command

* checkposix corrections

* More checkposix fixes

* Ripped out unused instrumentation functionality

* Brought over cache tagging changes from develop

* Changes to tagged metadata expulsion iteration

* Fixed typo

* Brought over H5O__free() changes from develop

* Brings (unused) parallel page buffer test in line with develop

* Moved the functionality in supervise.subr to test_vfd_swmr.sh

* Tools VFD parameter updates

* Committing clang-format changes

* H5F VFD SWMR refactoring

* Committing clang-format changes

* Misc changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-03-23 06:48:05 -07:00
Dana Robinson
7837588248 VFD SWMR: normalization with develop (#1472)
Much normalization with develop. Still needs tools changes wrt VFD plugins.
2022-03-06 15:33:00 -08:00
vchoi-hdfgroup
e3ff70f44a Merge pull request #58 from HDFGroup/feature/vfd_swmr
VFD SWMR: Aux process changes (#1451)
2022-02-18 23:08:41 -06:00
Dana Robinson
7e176db164 VFD SWMR: Aux process changes (#1451)
* Removed the aux process submodule

* Directly added aux process files after submodule delete

* Updated Akadio copyright on appropriate files

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-02-18 15:28:43 -08:00
vchoi-hdfgroup
950b5cbf08 Merge pull request #56 from HDFGroup/feature/vfd_swmr
Feature/vfd swmr
2022-02-17 16:13:53 -06:00
Dana Robinson
e569dc948f Merge pull request #1445 from derobins/vfd_swmr/license
Updated VFD SWMR's Akadio license
2022-02-17 12:20:33 -08:00
Dana Robinson
c9a363fdd9 Updated the submodule to point to the correct commit so the tests pass 2022-02-17 11:19:28 -08:00
Dana Robinson
b4866778a6 Updated VFD SWMR's Akadio license 2022-02-17 10:56:42 -08:00