Commit Graph

766 Commits

Author SHA1 Message Date
David Young
dc97b9a165 Merge branch 'aug-develop' into aug-merge-attempt/feature/vfd_swmr 2020-08-04 10:22:47 -05:00
Dana Robinson
b9de162eae Minor normalizations with 1.12 branch 2020-08-01 10:26:22 -07:00
Allen Byrne
7381bea1e8 OESS-65 add option to use AEC library in place of SZip 2020-05-20 09:37:19 -05:00
Quincey Koziol
d35daa45e9 Trim trailing whitespace 2020-05-20 09:37:19 -05:00
Jacob Smith
b11015c4cc Add Splitter VFD to library.
* "Simultaneous and equivalent" Read-Write and Write-Only channels for
  file I/O.
* Only supports drivers with the H5FD_FEAT_DEFAULT_VFD_COMPATIBLE flag for
  now, preventing issues with multi-file drivers.

Add Mirror VFD to library.

* Write-only operations over a network.
* Uses TCP/IP sockets.
* Server and auxiliary server-shutdown programs provided in a new directory,
  `utils/mirror_vfd`.
* Automated testing via loopback ("remote" of localhost).
2020-05-20 09:34:20 -05:00
Larry Knox
01a152f83c Add \d to list of characters allowed after '-' in versions. 2020-05-20 09:31:58 -05:00
Quincey Koziol
7b119d614b Revise API for H5get_alloc_stats() to take a struct instead of separate values. 2020-05-20 09:31:56 -05:00
Allen Byrne
95c4222cd1 Update examples and fix old version references. 2020-05-20 09:31:55 -05:00
David Young
4cd3dc95d9 Filter all of the new _USED / _UNUSED type modifiers. 2020-05-20 09:31:55 -05:00
Larry Knox
812e7eda57 Max Library "index" should be 7. 2020-05-20 09:31:51 -05:00
Larry Knox
ec3847712c Remove duplicate lines noted on forum by Eric Bavier 2020-05-20 09:31:50 -05:00
Larry Knox
7bbca40652 Add v111 version for H5O functions to enable version1 default for HDF5
1.8-1.10 and version 3 default for HDF5 1.12.  Version 2 functions are
available (deprecated) but will not be the default for any version.
2020-05-20 09:31:50 -05:00
Dana Robinson
3e6a192e9c Squashed commit of the token_refactoring branch: 2020-05-20 09:31:50 -05:00
kmu
1d46e82317 fix intel compile warnings 2020-05-20 09:20:25 -05:00
kmu
fb8f955a73 Revert "fix warnings from Intel compiler"
This reverts commit 8b9338ab57.
2020-05-20 09:20:24 -05:00
kmu
f2028c7568 Revert "using a different MACRO"
This reverts commit fc61b7a9f3.
2020-05-20 09:20:24 -05:00
kmu
9a3a13cacb using a different MACRO 2020-05-20 09:20:23 -05:00
kmu
6725222332 fix warnings from Intel compiler 2020-05-20 09:20:23 -05:00
Quincey Koziol
996e12e796 Refactor all the 'H5VL_*_optional' callbacks to move the type of operation out
of the va_list, so it's at least possible for another connector to know what
the operation is and decide whether to implement it or not.

Added a new VOL sub-class called "introspect" where callbacks that report
information about the connector or container can be placed.  Added an
'opt_query' callback to this sub-class, for a connector to report back
to the library whether a particular optional callback operation is supported.
Also added a 'get_conn_cls' introspection callback, to retrieve the H5VL_class_t
of a connector (either the "current" connector, H5VL_GET_CONN_LVL_CURR, or
the terminal connector, H5VL_GET_CONN_LVL_TERM).

Moved the "post open" operation from a file 'specific' operation to a file
'optional' operation, now that it's possible to detect (with the 'opt_query'
introspection callback) whether a VOL connector implements an optional
operation, without just returning an error.

Added new internal VOL helper routines: H5VL_object_is_native, to determine
if an object is in (or is a) native file, and H5VL_file_is_same, to determine
if two objects are in (or are) the same terminal VOL connector's container.
(And moved the special handling for FILE_IS_EQUAL operation out of internal VOL
callback routine into H5VL_file_is_same)

Made new dataset 'get' operation for H5Dvlen_get_buf_size, aligning it better
with other 'get' operations in API.

Fixed several issues with pass-through connectors, which are now passing the
'make check-passthrough-vol' tests again.

A bunch of warning and style cleanups as well.
2020-05-20 09:20:21 -05:00
David Young
5ad9d9277e Make this script work on hosts where PERL is not installed at
/usr/bin/perl.
2020-05-20 09:20:21 -05:00
Allen Byrne
16b1ee6899 OESS-65 add option to use AEC library in place of SZip 2020-04-22 11:48:56 -05:00
Quincey Koziol
9e5dbf6906 Trim trailing whitespace 2020-04-20 18:12:00 -05:00
Jake Smith
f1893e9404 Merge pull request #2452 in HDFFV/hdf5 from ~JAKE.SMITH/hdf5:feature/vfd_splitter_mirror_a to develop
Add Splitter VFD and Mirror VFD.

* commit '664fc6af47e369152727ff66d68d8532feb0e210':
  Add enable-mirror-vfd flag to configure options.
  Tidying of Mirror VFD.
  Add function header comment for `mirror_writer:run_writer()`.
  Add Splitter VFD to library.
2020-04-11 11:53:41 -05:00
Jacob Smith
075e74e9b2 Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/~jake.smith/hdf5 into feature/vfd_splitter_mirror_a 2020-04-07 10:29:00 -05:00
Larry Knox
8890fa3eb5 Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit 'd3ef866a2963ce93d0b95184534a70849b6fa11d':
  Revise API for H5get_alloc_stats() to take a struct instead of separate values.
  Move the GNU warnings configuration under config/gnu-warnings/ and trim the filename prefixes.  Update config/gnu-flags to suit.
  Fixed the RELEASE.txt note for HDFFV-11057
  Fix shutdown errors when using the HDF5_VOL_CONNECTOR environment variable to set a dynamically loaded plugin as the default VOL connector. Fixes HDFFV-11057
  Fix threadsafe for new test
  Fix issues when deserializing point/all/none selection with version beyond the library's supported version: (1) Verify the decoded version before proceeding further with deserialization (2) Close the dataspace if errors occurred after opening the dataspace
  Very minor comment change in H5VLconnector.h.
  Add routines to query the library's free list sizes and allocation stats.
  By default, demote warnings promoted to errors back to warnings.
  Incorporate the H5_ECFLAGS into the library CFLAGS.  Now a bunch of errors will occur.  Next commit will demote the warnings promoted to errors back to warnings again.
  Break out warnings into more files that autoconf and CMake can share.  This change temporarily disables the warnings that were promoted to errors, but I will add those warnings back as warnings, not errors, in the next commit.
  Avoid a crash, don't check if we're flushing when the file is closing: there's no need to check if the metadata cache is flushing if we already know the file is closing, because the condition we rely on is "closing OR flushing."  Further, the cache may have already gone away, so sometimes calling into the cache to see if it's flushing will crash the program.
2020-03-24 08:22:25 -05:00
Quincey Koziol
b51585a911 Revise API for H5get_alloc_stats() to take a struct instead of separate values. 2020-03-20 16:39:55 -05:00
Jacob Smith
b65405439d Add Splitter VFD to library.
* "Simultaneous and equivalent" Read-Write and Write-Only channels for
  file I/O.
* Only supports drivers with the H5FD_FEAT_DEFAULT_VFD_COMPATIBLE flag for
  now, preventing issues with multi-file drivers.

Add Mirror VFD to library.

* Write-only operations over a network.
* Uses TCP/IP sockets.
* Server and auxiliary server-shutdown programs provided in a new directory,
  `utils/mirror_vfd`.
* Automated testing via loopback ("remote" of localhost).
2020-03-13 17:13:17 -05:00
Larry Knox
d9f93659a2 Add \d to list of characters allowed after '-' in versions. 2020-03-12 09:51:53 -05:00
Allen Byrne
13f5b3aee2 Update examples and fix old version references. 2020-02-26 13:54:34 -06:00
David Young
ac7cd484d8 Filter all of the new _USED / _UNUSED type modifiers. 2020-02-18 16:31:35 -06:00
Larry Knox
edcba1ce2e Merge pull request #2297 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:develop to develop
* commit '30fa2606b45e789ba36f3193130cdea2c12b68a6':
  Max Library "index" should be 7.
2020-01-22 22:13:34 -06:00
Kimmy Mu
8ce59829e4 Merge pull request #2094 in HDFFV/hdf5 from ~KMU/hdf5:bugfix/unused to develop
* commit '145ef3ceee20c28a443bd11507c58858bea3f889':
  more cleanup
  change it back
  pick up missing piece
  fix unused related warnings
  removed unused parameter
  more fix and address comments
  remove unsed var,function,macro, etc
2020-01-22 19:46:53 -06:00
Larry Knox
f3d1b4c6d2 Max Library "index" should be 7. 2020-01-22 15:46:34 -06:00
Larry Knox
793eb6b219 Remove duplicate lines noted on forum by Eric Bavier 2020-01-21 15:42:37 -06:00
Larry Knox
30ccab0fb3 Add v111 version for H5O functions to enable version1 default for HDF5
1.8-1.10 and version 3 default for HDF5 1.12.  Version 2 functions are
available (deprecated) but will not be the default for any version.
2020-01-21 15:08:17 -06:00
kmu
9be5b2842e merge and fix 2020-01-20 21:11:32 -06:00
kmu
628d267162 merge and fix conflict 2020-01-19 08:09:44 -06:00
Dana Robinson
a92c735c9b Squashed commit of the token_refactoring branch: 2020-01-16 13:29:34 -08:00
Kimmy Mu
127f07d64a Merge pull request #2071 in HDFFV/hdf5 from ~KMU/hdf5:bugfix/intel_warnings to develop
* commit '0a2bb11b248df6841daabca3970df5d8504adfc7':
  address problems from comments
  fix and address comments
  change according to previous comments
  add missing piece
  remove unnecessary check macro
  fix intel compile warnings
  Revert "fix warnings from Intel compiler"
  Revert "fix warnings and some text alignment"
  Revert "let hdf5 pick up the right compiler in Intel environment"
  Revert "fix issues from previous PR comments"
  Revert "using a different MACRO"
  using a different MACRO
  fix issues from previous PR comments
  let hdf5 pick up the right compiler in Intel environment
  fix warnings and some text alignment
  fix warnings from Intel compiler
2020-01-06 12:17:10 -06:00
Quincey Koziol
58cf795321 Refactor all the 'H5VL_*_optional' callbacks to move the type of operation out
of the va_list, so it's at least possible for another connector to know what
the operation is and decide whether to implement it or not.

Added a new VOL sub-class called "introspect" where callbacks that report
information about the connector or container can be placed.  Added an
'opt_query' callback to this sub-class, for a connector to report back
to the library whether a particular optional callback operation is supported.
Also added a 'get_conn_cls' introspection callback, to retrieve the H5VL_class_t
of a connector (either the "current" connector, H5VL_GET_CONN_LVL_CURR, or
the terminal connector, H5VL_GET_CONN_LVL_TERM).

Moved the "post open" operation from a file 'specific' operation to a file
'optional' operation, now that it's possible to detect (with the 'opt_query'
introspection callback) whether a VOL connector implements an optional
operation, without just returning an error.

Added new internal VOL helper routines: H5VL_object_is_native, to determine
if an object is in (or is a) native file, and H5VL_file_is_same, to determine
if two objects are in (or are) the same terminal VOL connector's container.
(And moved the special handling for FILE_IS_EQUAL operation out of internal VOL
callback routine into H5VL_file_is_same)

Made new dataset 'get' operation for H5Dvlen_get_buf_size, aligning it better
with other 'get' operations in API.

Fixed several issues with pass-through connectors, which are now passing the
'make check-passthrough-vol' tests again.

A bunch of warning and style cleanups as well.
2019-12-19 22:41:37 -06:00
David Young
c8f533cfc3 Merge all of my changes from merge-back-to-feature-vfd_swmr-attempt-1,
including the merge of `hdffv/hdf5/develop`, back to the branch that Vailin and
I share.

Now I need to put this branch on a fork with a less confusing name than
vchoi_fork!
2019-12-09 10:30:58 -06:00
kmu
132fa33dad fix intel compile warnings 2019-12-03 20:52:55 -06:00
kmu
e0262c8bed Revert "fix warnings from Intel compiler"
This reverts commit 8b9338ab57.
2019-11-26 17:24:48 -06:00
kmu
4c8a2f726a Revert "using a different MACRO"
This reverts commit fc61b7a9f3.
2019-11-26 17:23:34 -06:00
kmu
fc61b7a9f3 using a different MACRO 2019-11-25 15:17:55 -06:00
David Young
4281d101c4 Make this script work on hosts where PERL is not installed at
/usr/bin/perl.
2019-11-13 13:04:58 -06:00
kmu
815611a318 Merge branch 'develop' of https://git.hdfgroup.org/scm/~kmu/hdf5 into develop 2019-11-07 14:51:40 -06:00
kmu
8b9338ab57 fix warnings from Intel compiler 2019-11-07 14:34:13 -06:00
David Young
2dd5bbfe16 Merge pull request #1984 in HDFFV/hdf5 from ~DYOUNG/netbsd:develop to develop
* commit '0b721858e46a317c370a24115032d5be41688f67':
  Make these scripts relocatable again: derive a relative path for the original installation prefix from the examples prefix.  Use that relative path to locate the current installation prefix, always.  Fall back to an absolute installation prefix if the relative path cannot be derived.
  Get the path to prefix right: needs a ../ to back out of subdirectory c/.
  Make this script relocatable again: derive a relative path for the original installation prefix from the examples prefix.  Use that relative path to locate the current installation prefix, always.  Fall back to an absolute installation prefix if the relative path cannot be derived.
  Let us override the examples directory using --with-examplesdir=DIR. This is handy for NetBSD where HDF5 examples are installed by convention in $prefix/share/examples/hdf5/ rather than in ${prefix}/share/hdf5_examples/, which is the HDF5 default.
  Follow longstanding execv convention for compatibility with NetBSD.
  Under the examples directories, always find the installed HDF5 executables and scripts using @prefix@ instead of a relative path, because the number of ../ in the relative path will be different on NetBSD than on other systems.
  Make the HDF5 configure script grok NetBSD.
  For portability, insulate the HDF5 library from some system macros.
  Not every system has perl installed in /usr/bin/, so change the shebang (#!) line to `/usr/bin/env perl` to locate perl on the PATH.
  For portability, use the POSIX sh(1) string-comparison operator `=` instead of `==`.
2019-11-05 15:36:09 -06:00
Jerome Soumagne
ae490016b9 Add new H5R API that abstracts object, region and attribute reference types
Also support references to external files

Add new H5T_REF type and type conversion routines

Support conversion from H5T_REF_OBJ/DSET_REG to H5T_REF

Add H5Treclaim() API to reclaim memory of vlen/reference types

Deprecate H5Dvlen_reclaim()

Fix H5T_vlen_reclaim() and H5T_reclaim() to use private callback

Add H5T_ref_reclaim()

Move previous H5R APIs to H5Rdeprec.c

Clean up H5Ocopy

Separate H5O_copy_expand_ref() to H5Ocopy_ref()

Add support for copying new reference types

Clean up deprecated routines to go through VOL and same code path

Fix return codes in existing trefer.c test

Rename trefer.c to trefer_deprec.c

trefer.c is for new references

Add performance test for trefer

Add additional obj_copy_ref test

Make use of tokens and blobs to store references

Skip blob encoding for object references

Start adding new reference examples
2019-10-08 14:30:24 -05:00