Compare commits

...

428 Commits

Author SHA1 Message Date
Dana Robinson
c92d188c51 Add a BRANCH.md file and placeholder document in doc/arch 2023-01-26 02:47:16 -08:00
kwryankrattiger
509fe962d4 CMake: Find MPI in HDF5 CMake config (#2400) 2023-01-18 16:26:35 -06:00
Allen Byrne
2c9de7a9ae Port VOL connector Guide to doxygen (#2333)
* Port VOL connector Guide to doxygen

* Fix spelling

* Updated VOL UG ref and added release note
2023-01-18 01:04:41 -08:00
Mark Kittisopikul
720e04e196 Include shlwapi.h explicitly on Windows (#2407)
* Include shlwapi.h explicitly

* Add shlwapi library to LINK_LIB when header is detected

Looking for StrStrIA in the shlwapi library is not reliable due
to stdcall on mingw32.
2023-01-17 07:27:25 -08:00
raylu-hdf
5543d6eb36 HDFFV-11208 (OESS-320): H5VLquery_optional had an assertion failure with a committed datatype (#2398)
* HDFFV-11208 (OESS-320): H5VLquery_optional had an assertion failure with a committed datatype.  Added a test case for the fix that Quincey checked in.

* Committing clang-format changes

* Fixed a typo in a comment.

* Fixed a typo in a comment.

* Minor change: changed H5Tcommit to H5Tcommit2.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2023-01-10 09:29:54 -08:00
Quincey Koziol
06c12f97cd Pass-through connector testing flexibility (#2399)
* Correct concurrency bugs when running tests, along with a bugfix & small
warning cleanup.

* Committing clang-format changes

* Allow spaces (and tabs) in VOL connector info string from environment variable.

* Parse connector name from HDF5_PLUGIN_PATH environment variable better

* Correct H5VLquery_optional to use H5VL routine instead of H5I.  Also add an
error message to the failure return value from not finding a plugin.

* Play nice with existing plugin paths

* Use API routine to determine if native connector is terminal.

* Committing clang-format changes

* Make string size larger, to allow for connectors with longer names.

* Be more flexible about testing external pass through connectors, especially if
they have registered new optional operations.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: AWS ParallelCluster user <ec2-user@ip-10-0-0-65.us-east-2.compute.internal>
Co-authored-by: Koziol <qkoziol@88665a374c70.ant.amazon.com>
2023-01-09 17:18:28 -06:00
Quincey Koziol
90d20764f4 Determine if native connector is terminal (#2397)
* Correct concurrency bugs when running tests, along with a bugfix & small
warning cleanup.

* Committing clang-format changes

* Allow spaces (and tabs) in VOL connector info string from environment variable.

* Parse connector name from HDF5_PLUGIN_PATH environment variable better

* Correct H5VLquery_optional to use H5VL routine instead of H5I.  Also add an
error message to the failure return value from not finding a plugin.

* Play nice with existing plugin paths

* Use API routine to determine if native connector is terminal.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: AWS ParallelCluster user <ec2-user@ip-10-0-0-65.us-east-2.compute.internal>
Co-authored-by: Koziol <qkoziol@88665a374c70.ant.amazon.com>
2023-01-09 13:49:29 -08:00
Quincey Koziol
54590f3ebd Play nice with existing plugin paths (#2394)
* Correct concurrency bugs when running tests, along with a bugfix & small
warning cleanup.

* Committing clang-format changes

* Allow spaces (and tabs) in VOL connector info string from environment variable.

* Parse connector name from HDF5_PLUGIN_PATH environment variable better

* Correct H5VLquery_optional to use H5VL routine instead of H5I.  Also add an
error message to the failure return value from not finding a plugin.

* Play nice with existing plugin paths

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: AWS ParallelCluster user <ec2-user@ip-10-0-0-65.us-east-2.compute.internal>
Co-authored-by: Koziol <qkoziol@88665a374c70.ant.amazon.com>
2023-01-06 09:08:16 -08:00
Quincey Koziol
337a882af8 Update FORTRAN VOL connector test for external pass-through testing (#2393)
* Correct concurrency bugs when running tests, along with a bugfix & small
warning cleanup.

* Committing clang-format changes

* Allow spaces (and tabs) in VOL connector info string from environment variable.

* Parse connector name from HDF5_PLUGIN_PATH environment variable better

* Correct H5VLquery_optional to use H5VL routine instead of H5I.  Also add an
error message to the failure return value from not finding a plugin.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: AWS ParallelCluster user <ec2-user@ip-10-0-0-65.us-east-2.compute.internal>
Co-authored-by: Koziol <qkoziol@88665a374c70.ant.amazon.com>
2023-01-05 15:42:33 -08:00
Dana Robinson
423ee69c1e Updates README.md (#2387)
* Updates release schedule for 2023

* Fixes broken build badge

* Another build badge tweak

* Even more badge tweaking

* More branch tweaking

* More badge tweaking

* More badge tweaks

* Even more badge tweaking

* Adds badges for other branches
2023-01-04 12:44:51 -08:00
Quincey Koziol
c2e3c732f0 Correct VOL connector env string parsing issue (#2350)
* Correct concurrency bugs when running tests, along with a bugfix & small
warning cleanup.

* Committing clang-format changes

* Allow spaces (and tabs) in VOL connector info string from environment variable.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: AWS ParallelCluster user <ec2-user@ip-10-0-0-65.us-east-2.compute.internal>
Co-authored-by: Koziol <qkoziol@88665a374c70.ant.amazon.com>
2023-01-04 13:04:07 -06:00
Dana Robinson
83f81a6422 Adds -Warray-temporaries to the developer warnings (#2385)
These generate a LOT of noise that we are not going to address anytime
soon.
2023-01-03 06:18:48 -08:00
Dana Robinson
d326f6fae8 Adds 'make install' and 'make check-install' to the CI (Autotools only) (#2381) 2023-01-03 06:17:56 -08:00
Dana Robinson
c0b3646193 Adds RELEASE.txt notes and updates Doxygen (#2377) (#2379) 2022-12-29 12:58:54 -08:00
Dana Robinson
3da2551adb Adds Doxygen markup for H5FDdriver_query() (#2376) 2022-12-28 13:07:39 -08:00
Larry Knox
d261272ffa Restore line 264 of main.yml: "thread_safety:" (#2374) 2022-12-28 10:30:57 -08:00
Dana Robinson
23ccd7ee3b Purged references to HDF5 1.13.x from the Doxygen documentation (#2365) 2022-12-27 12:33:14 -08:00
Dana Robinson
298c7efe76 Adds missing Doxygen for H5ESpublic.h (#2364)
* Adds missing Doxygen for H5ESpublic.h

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-12-27 11:42:31 -08:00
Dana Robinson
1cf9ecccf5 Fix for Autotools --disable-deprecated-symbols (#2363)
* Fix for Autotools --disable-deprecated-symbols

When we added v116 as a valid option, we left the default as v114 so
using --disable-deprecated-symbols leads configure to complain that you
can't set a default API that was deprecated.

The GitHub action didn't catch this because it explicitly specifies v116

This only affects develop w/ the Autotools when
--disable-deprecated-symbols is requested.

* Added v116 & default to the --with-default-api-version help string

The Autotools --with-default-api-version help string was missing v116.
This has been added, as well as a "default" target so the CI can catch
version problems when we forget to update the defaults upon creating
a new major version.

The GitHub CI also now uses "default" as the target for the deprecated
symbols build.

* Fixes the 1.14 API GitHub CI threadsafe/build_mode

This action still had the bug we fixed where we set the build mode
and threadsafety using an array of size one instead of an object.
2022-12-24 11:36:38 -08:00
Dana Robinson
cfd5059e77 HL GIF tools changes (#2360)
* Fixes the broken Autotools option
* Removes the "build HL tools" option
2022-12-23 16:05:31 -08:00
Larry Knox
9a694db450 Update testh5cc.sh.in for new major version 1.14. (#2349)
* Update testh5cc.sh.in for new major version 1.14.

* iFix sha256sum commands in release script.
2022-12-22 13:38:51 -06:00
Dana Robinson
ab0a778593 Adds Doxygen VOL flag markup (#2340)
* Initial doxygen markup

* Committing clang-format changes

* Adds Doxygen comments to VOL flags

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-12-21 13:44:50 -08:00
Dana Robinson
57af2ff798 Onion VFD header cleanup (#2337)
* Partial work towards cleaning up the onion VFD code

* Committing clang-format changes

* A few minor tweaks to the onion VFD public header

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-12-21 10:46:05 -08:00
Allen Byrne
9f5cf0a457 H5F_LIBVER_LATEST changes for move to 1.15 (#2288)
* H5F_LIBVER_LATEST changes for move to 1.15

* Add new default api check

* Format fixes

* Fix default configure

* fix lib version tests

* Fix another version variable

* Add 1.14 doc link
2022-12-21 10:07:25 -08:00
vchoi-hdfgroup
976e8c9750 Fix up the H5Pencode2 test for virtual layout. (#2325)
* Fix up the H5Pencode2 test for virtual layout.

* Committing clang-format changes

* Remove VERIFY and use TEST_ERROR.

Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-12-20 16:48:56 -08:00
Allen Byrne
c1e44d32e6 Fix doxygen warnings and remove javadocs (#2324)
* Fix doxygen warnings and remove javadocs

* fix typo
2022-12-20 16:59:40 -06:00
Mark Kittisopikul
b9244a85d9 Align arg types of H5D_chunk_iter_op_t with H5Dget_chunk_info (#2074)
* Align arg types of H5D_chunk_iter_op_t with H5Dget_chunk_info

* Modify chunk_info test to for unsigned / hsize_t types

* Fix types in test
2022-12-19 15:13:43 -08:00
Scot Breitenfeld
70cf2c390b Removed idioms and misc. text clean-up (#2320)
* removed idioms and misc. text clean-up, Issue #2135

* removed idioms and misc. text clean-up, Issue #2135
2022-12-16 19:56:52 -08:00
Allen Byrne
149b8e9769 Disable hl tools by default (#2313)
* Disable hl tools by default

* identify the tools

* Only GIF tools are depecated

* Add new option

* Update autotools
2022-12-16 19:56:07 -08:00
Scot Breitenfeld
8102fa8c97 Only document Fortran functions (#2319)
* Only document Fortran functions

* Only document Fortran functions

* Only document Fortran functions

* Only document Fortran functions
2022-12-16 19:52:08 -08:00
Scot Breitenfeld
784061b15e moved onion VFD to FAPL group (#2321) 2022-12-16 19:51:52 -08:00
vchoi-hdfgroup
6b6bcdead6 Hdffv 11052 (#2315)
* Fix for HDFFV-11052: h5debug fails on a corrupted file (h5_nrefs_POC) producing a core dump.
When h5debug closes the corrupted file, the library calls H5F__dest() which performs all the
closing operations for the file "f" (H5F_t *) but just keeping note of errors in "ret_value"
all the way till the end of the routine.  The user-provided corrupted file has an illegal
file size causing failure when reading the image during the closing process.
At the end of this routine it sets f->shared to NULL and then frees "f".
This is done whether there is error or not in "ret_value".
Due to the failure in reading the file earlier, the routine then returns error.
The error return from H5F__dest() causes the file object "f" not being removed from the
ID node table.  When the library finally exits, it will try to close the
file objects in the table.  This causes assert failure when H5F_ID_EXISTS(f) or H5F_NREFS(f).
Fix:
a) H5F_dest(): free the f only when there is no error in "ret_value" at the end of the routine.
b) H5VL__native_file_close(): if f->shared is NULL, free "f"; otherwise, perform closing on "f" as before.
c) h5debug.c main(): track error return from H5Fclose().

* Committing clang-format changes

* Add test and release note info for fix to HDFFV-11052 which is merged via PR#2291.

* Committing clang-format changes

* Add the test file to Cmake.

* Skip test_misc37() for drivers that is not default compatible as it is using a pre-generated file.

* Committing clang-format changes

Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-12-16 13:47:29 -08:00
Dana Robinson
0e76abaf64 Removes duplicated build mode from -Werror GitHub actions (#2314) 2022-12-16 12:05:01 -08:00
Dana Robinson
4bbda3038a Converts the YAML build mode arrays to objects in special runs (#2308)
* Converts the YAML build mode arrays to objects in special runs

* Adds a dump of the matrix context for each test

This would have made it a LOT easier to debug the build_mode issues...

* Disable the mirror VFD in the -Werror checks

We can re-enable this after we fix the warnings, but I don't want to
conflate code and GitHub changes, so this is a better way to get the
CI to pass for now.
2022-12-15 23:03:02 -08:00
Gerd Heber
138daa8846 Merge pull request #2304 from gheber/develop
Update hdf5_header.html
2022-12-15 13:25:18 -06:00
Gerd Heber
10c693a04f Update hdf5_header.html
Use less awkward language.
2022-12-15 08:49:19 -06:00
vchoi-hdfgroup
0cb5808087 Hdffv 11052 (#2303)
* Fix for HDFFV-11052: h5debug fails on a corrupted file (h5_nrefs_POC) producing a core dump.
When h5debug closes the corrupted file, the library calls H5F__dest() which performs all the
closing operations for the file "f" (H5F_t *) but just keeping note of errors in "ret_value"
all the way till the end of the routine.  The user-provided corrupted file has an illegal
file size causing failure when reading the image during the closing process.
At the end of this routine it sets f->shared to NULL and then frees "f".
This is done whether there is error or not in "ret_value".
Due to the failure in reading the file earlier, the routine then returns error.
The error return from H5F__dest() causes the file object "f" not being removed from the
ID node table.  When the library finally exits, it will try to close the
file objects in the table.  This causes assert failure when H5F_ID_EXISTS(f) or H5F_NREFS(f).
Fix:
a) H5F_dest(): free the f only when there is no error in "ret_value" at the end of the routine.
b) H5VL__native_file_close(): if f->shared is NULL, free "f"; otherwise, perform closing on "f" as before.
c) h5debug.c main(): track error return from H5Fclose().

* Committing clang-format changes

* Add test and release note info for fix to HDFFV-11052 which is merged via PR#2291.

* Committing clang-format changes

* Add the test file to Cmake.

Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-12-15 05:32:23 -08:00
Dana Robinson
1b06e868fd Fixes broken CI matrix actions involving -Werror and API versions (#2302) 2022-12-14 18:05:15 -06:00
Dana Robinson
93938b3a8d Adds 'unused' hints for MDS io_info parameters (#2301)
* Adds 'unused' hints for MDS io_info parameters

* Committing clang-format changes

* Fix issue with formatter

* Committing clang-format changes

* Final fix for formatter

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-12-14 14:54:51 -08:00
Scot Breitenfeld
a1c81eda20 added doc. warning for H5Literate_async return value (#2295) 2022-12-14 15:22:37 -06:00
Allen Byrne
b7511c19d0 Update windows worker compilers (#2286)
* Update windows worker compilers

* Update bin and test issues

* Update script and revert java test
2022-12-14 08:58:20 -06:00
Scot Breitenfeld
502b32b0f2 Updated H5ES documenation (#2293) 2022-12-13 10:56:48 -08:00
vchoi-hdfgroup
a9036005c3 Fix for HDFFV-11052: h5debug fails on a corrupted file (h5_nrefs_POC)… (#2291)
* Fix for HDFFV-11052: h5debug fails on a corrupted file (h5_nrefs_POC) producing a core dump.
When h5debug closes the corrupted file, the library calls H5F__dest() which performs all the
closing operations for the file "f" (H5F_t *) but just keeping note of errors in "ret_value"
all the way till the end of the routine.  The user-provided corrupted file has an illegal
file size causing failure when reading the image during the closing process.
At the end of this routine it sets f->shared to NULL and then frees "f".
This is done whether there is error or not in "ret_value".
Due to the failure in reading the file earlier, the routine then returns error.
The error return from H5F__dest() causes the file object "f" not being removed from the
ID node table.  When the library finally exits, it will try to close the
file objects in the table.  This causes assert failure when H5F_ID_EXISTS(f) or H5F_NREFS(f).
Fix:
a) H5F_dest(): free the f only when there is no error in "ret_value" at the end of the routine.
b) H5VL__native_file_close(): if f->shared is NULL, free "f"; otherwise, perform closing on "f" as before.
c) h5debug.c main(): track error return from H5Fclose().

* Committing clang-format changes

Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-12-13 09:40:59 -08:00
Scot Breitenfeld
b9f4c001ad added doc. warning for H5Oflush in parallel (#2289) 2022-12-09 21:31:13 -06:00
Scot Breitenfeld
4a8be219e0 fixed unclosed objects in test (#2290) 2022-12-09 21:30:42 -06:00
Neil Fortner
ef4f1f2bd4 Check for size==0 for all datatypes read from disk, not just compound (#2287)
members
2022-12-08 10:30:29 -06:00
Egbert Eich
0b4e9cf976 Compound datatypes may not have members of size 0 (#2243)
* Compound datatypes may not have members of size 0

A member size of 0 may lead to an FPE later on as reported in
CVE-2021-46244. To avoid this, check for this as soon as the
member is decoded.
This should probably be done in H5O_dtype_decode_helper() already,
however it is not clear whether all sizes are expected to be != 0.

This fixes CVE-2021-46244 / Bug #2242.

Signed-off-by: Egbert Eich <eich@suse.com>

* Rework error recovery code in H5O__dtype_decode_helper() and
H5O__dtype_decode().

* Format changes for src/H5Odtype.c.

Signed-off-by: Egbert Eich <eich@suse.com>
Co-authored-by: Neil Fortner <nfortne2@hdfgroup.org>
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
2022-12-07 16:14:40 -06:00
Allen Byrne
dcccc35526 Correct requires setting for pkgconfig files (#2280)
* Correct requires setting for pkgconfig files

* Add issue number
2022-12-07 16:05:51 -06:00
Dana Robinson
64514392fb Fixes warnings in H5Opline.c error messages (#2281)
* Fixes warnings in H5Opline.c error messages

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-12-06 11:32:39 -08:00
Allen Byrne
0a2006d9c6 Add text about release notes (#2246) 2022-12-06 06:02:21 -08:00
Allen Byrne
2376723d4a Develop jni trans (#2266)
* Add compound and refactor out atomic types

* Add Array String tests back

* Convert Attribute version of compound example

* Update transfer atom8ic read to return object
2022-12-05 22:29:42 -06:00
V. Armando Solé
281984b3e3 H5E_ERR_CLS_g symbol missing when compiling C++ plugins (#2269)
C++ HDF5 filter plugin SZ3 fails to build under windows with failure at linking time unless that extern "C" block is added.
2022-12-05 19:19:53 -06:00
Scot Breitenfeld
c1c131260a return error message related to calling API (#2276) 2022-12-02 14:24:43 -06:00
Egbert Eich
24700e8f06 CVE 2021 46242 develop (#2255)
* When evicting driver info block, NULL the corresponding entry

Since H5C_expunge_entry() called (from H5AC_expunge_entry()) sets the  flag
H5C__FLUSH_INVALIDATE_FLAG, the driver info block will be freed. NULLing the
pointer in  f->shared->drvinfo will prevent use-after-free  when it is used in other
functions (like  H5F__dest()) - as other places will check whether the pointer is
initialized before using its value.

This fixes CVE-2021-46242 / Bug #2254

Signed-off-by: Egbert Eich <eich@suse.com>

* When evicting the superblock, NULL the corresponding entry

The call to H5AC_expunge_entry() will free the corresonding structure,
to avoid a use-after-free, the corrsponding pointer entry will be NULLed.

Signed-off-by: Egbert Eich <eich@suse.com>

Signed-off-by: Egbert Eich <eich@suse.com>
2022-12-02 14:24:14 -06:00
Allen Byrne
88b24c258b Output should only be printed if verbose. (#2273)
* Output should only be printed if verbose.

* Add note
2022-12-02 09:39:49 -08:00
Egbert Eich
4e0277c35a Report error if dimensions of chunked storage in data layout < 2 (#2241)
For Data Layout Messages version 1 & 2 the specification state
that the value stored in the data field is 1 greater than the
number of dimensions in the dataspace. For version 3 this is
not explicitly stated but the implementation suggests it to be
the case.
Thus the set value needs to be at least 2. For dimensionality
< 2 an out-of-bounds access occurs as in CVE-2021-45833.

This fixes CVE-2021-45833 / Bug #2240.

Signed-off-by: Egbert Eich <eich@suse.com>

Signed-off-by: Egbert Eich <eich@suse.com>
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
2022-12-01 23:04:42 -06:00
Allen Byrne
96a4e10102 Add aarch64 toolchain and change dump test to long opt (#2271) 2022-12-01 14:32:34 -08:00
Larry Knox
cf232a757f Develop fix ubuntu 22.04 openmpi install on github (#2268)
* Update install openmpi-bin to also install mpi-default-dev instead of
libopenmpi-dev.  This according to
https://bugs.launchpad.net/ubuntu/+source/openmpi/+bug/1870780.
2022-11-28 13:20:24 -06:00
Egbert Eich
158dff6b50 Make sure testsuite can handle multiple dependent scripts (#2256)
Commit afc54d75a1 to the test suite to eliminate a separate
shell script did not take into account that there may be
multiple dependent test scripts which resulted in the
message:
`/bin/sh: line 7: [: too many arguments`.
when running the test suite.

Beware that this new version still makes use of non-posix GNU
extensions to `test`.

Signed-off-by: Egbert Eich <eich@suse.com>

Signed-off-by: Egbert Eich <eich@suse.com>
2022-11-28 13:19:31 -06:00
Larry Knox
eac2cd54e2 Develop ubuntu 20.04 (#2267)
* Use ubuntu-20.04 for github autotools  parallel tests until parallel
configure is fixed on  ubuntu 22.04.

* Remove inactive developers from CODEOWNERS.

* Temporarily cat config.log for autotools build in github actions.

* Try ubuntu-latest.

* Attempt to cat config.log even when configure fails.

* Fix syntax.

* Try again

* Try again

* Try again.

* Back to original temporary fix.

* Install libopenmpi-dev with openmpi-bin, which was installed as a
dependency prior to ubuntu-22.04.
2022-11-23 12:00:00 -08:00
Larry Knox
3e762b07aa Use ubuntu-20.04 for github autotools parallel tests until parallel (#2264)
* Use ubuntu-20.04 for github autotools  parallel tests until parallel
configure is fixed on  ubuntu 22.04.
2022-11-18 17:30:29 -06:00
raylu-hdf
ee4d92e01b Bug Fix: Segfault in H5CX_get_vol_wrap_ctx when H5VLwrap_register is called from an application (#2248)
* Jira HDFFV-10881: Segfault in H5CX_get_vol_wrap_ctx when H5VLwrap_register is called from an application.  A quick and simple fix to make it fail with a relevant error message.

* Format changes.

* Committing clang-format changes

* Minor change: split one condition check into two for clarity.

* Adding doxygen comment for H5VLwrap_register.

* Minor change: adding a little more detail to the Doxygen comment for H5VLwrap_register.
2022-11-17 16:29:21 -06:00
Ryan Schmidt
9dd36f016a Simplify & fix check for szlib encoder (#2263)
Return the result rather than setting the exit code. "return" is a
language keyword whereas "exit" is a function for which the <stdlib.h>
header has to be included which it wasn't in this test, therefore the
test would previously fail to identify that the encoder was enabled if
"-Werror=implicit-function-declaration" was used, which it is by default
with clang from Xcode 12 and later.

Fixes #2262
2022-11-17 16:28:44 -06:00
Allen Byrne
169896aedb Remove hard-coded version text (#2260)
* Remove hard-coded version text

* Use macOS 11 until accum test is fixed see #2261
2022-11-16 15:13:46 -06:00
Allen Byrne
719d800499 Refactor JNI translate functions to a recursive switch on datatype (#2232) 2022-11-15 22:44:33 -06:00
Scot Breitenfeld
03cc221016 moved quad precision checks (#2252)
* Only check quad sizes if fortran is enabled

* fixed if struct
2022-11-12 12:53:35 -06:00
Egbert Eich
0f94940f1a H5O_dtype_decode_helper: Parent of enum needs to have same size as enum itself (#2237)
The size of the enumeration values is determined by the size of the parent.
Functions accessing the enumeration values use the size of the enumeration
to determine the size of each element and how much data to copy. Thus the
size of the enumeration and its parent need to match.
Check here to avoid unpleasant surprises later.

This fixes CVE-2018-14031 / Bug #2236.

Signed-off-by: Egbert Eich <eich@suse.com>
2022-11-11 00:05:00 -06:00
Egbert Eich
34ec3bb7bc Make sure info block for external links has at least 3 bytes (#2234)
According to the specification, the information block for external links
contains 1 byte of version/flag information and two 0 terminated strings
for the object linked to and the full path.
Although not very useful, the minimum string length for each (with
terminating 0) would be one byte.
Checking this will help to avoid SEGVs triggered by bogus files.

This fixes CVE-2018-16438 / Bug #2233.

Signed-off-by: Egbert Eich <eich@suse.com>
2022-11-10 23:41:53 -06:00
Egbert Eich
1750b4b0af Validate location (offset) of the accumulated metadata when comparing (#2231)
Initially, the accumulated metadata location is initialized to HADDR_UNDEF
- the highest available address. Bogus input files may provide a location
or size matching this value. Comparing this address against such bogus
values may provide false positives. This make sure, the value has been
initilized or fail the comparison early and let other parts of the
code deal with the bogus address/size.
Note: To avoid unnecessary checks, we have assumed that if the 'dirty'
member in the same structure is true the location is valid.

This fixes CVE-2018-13867 / Bug #2230.

Signed-off-by: Egbert Eich <eich@suse.com>
2022-11-10 23:24:56 -06:00
Egbert Eich
659bc99fd1 Make H5O__fsinfo_decode() more resilient to out-of-bound reads. (#2229)
When decoding a file space info message in H5O__fsinfo_decode() make
sure each element to be decoded is still within the message. Malformed
hdf5 files may have trunkated content which does not match the
expected size. Checking this will prevent attempting to decode
unrelated data and heap overflows. So far, only free space manager
address data was checked before decoding.

This fixes CVE-2021-45830 / Bug #2228.

Signed-off-by: Egbert Eich <eich@suse.com>

Additions

Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
2022-11-10 23:20:09 -06:00
Egbert Eich
99487d9e45 H5IMget_image_info(): Make sure to not exceed local array size (#2227)
Malformed hdf5 files may provide more dimensions than the array dim[] in
H5IMget_image_info() is able to hold. Check number of elements first by calling
H5Sget_simple_extent_dims() with NULL for both 'dims' and 'maxdims' arguments.
This will cause the function to return only the number of dimensions.
The fix addresse a stack overflow on write.

This fixes CVE-2018-17439 / HDFFV-10589 / Bug #2226.

Signed-off-by: Egbert Eich <eich@suse.com>

Signed-off-by: Egbert Eich <eich@suse.com>
2022-11-10 23:01:45 -06:00
Scot Breitenfeld
5985d0e0b1 removed uninterpretable line (#2249) 2022-11-10 22:56:53 -06:00
Dana Robinson
a8942c7413 Adds a release note for PR #2210 (CVE-2019-8396) (#2247)
* Adds a release note for PR #2210 (CVE-2019-8396)

* Capitalization issue fixed
2022-11-09 17:03:55 -08:00
Dana Robinson
d93c6fae43 Removes MPE instrumentation support. (#2245)
* Removes MPE instrumentation support.

The Autotools will no longer accept --with-mpe= and the logging commands
have been removed from the FUNC_ENTER macros. CMake has never supported
instrumenting for MPE.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-11-09 11:06:26 -08:00
Egbert Eich
18c438bdf0 H5O__pline_decode() Make more resilient to out-of-bounds read (#2210)
Malformed hdf5 files may have trunkated content which does not match
the expected size. When this function attempts to decode these it may
read past the end of the allocated space leading to heap overflows
as bounds checking is incomplete.
Make sure each element is within bounds before reading.

This fixes CVE-2019-8396 / HDFFV-10712 / github bug #2209.

Signed-off-by: Egbert Eich <eich@suse.com>

Signed-off-by: Egbert Eich <eich@suse.com>
2022-11-09 10:25:11 -08:00
Dana Robinson
adc467992e Moved the contributing.md document to the project root (#2244) 2022-11-09 09:58:32 -08:00
Dana Robinson
0f30852242 Removes dmalloc support (#2239)
This has not been used to debug the library in a very long time. Most
developers use valgrind, -fsanitize=address, or some other memory checker
instead of this library.

This removes:
    * dmalloc.h include from H5private.h
    * --with-dmalloc= Autotools configure option
    * HDF5_ENABLE_USING_DMALLOC CMake option
2022-11-09 09:22:14 -08:00
Gerd Heber
2025d047e2 Multidataset RFC updates (#2238) 2022-11-08 17:46:52 -08:00
raylu-hdf
81a6f6874f Bug fix: the previous checkin only supported the native connector (#2211)
* Bug fix: make sure the test case supports the pass_through connector.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-11-05 15:24:42 -07:00
Dana Robinson
018f093c09 Removes the memory allocation sanity checks feature (#2218)
* Removes the memory allocation sanity checks feature

* Committing clang-format changes

* Removes zero size checks for H5MM_(c|m)alloc()

* Explicitly return NULL when size == 0 in H5allocate_memory()

* Committing clang-format changes

* Format fix

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-11-03 16:30:08 -07:00
Scot Breitenfeld
40a0e3c7b5 correct H5Ropen_object_reference (#2212) 2022-11-03 09:56:53 -07:00
Dana Robinson
55e3f667d4 Fixed a Doxygen param in H5EScancel (#2213) 2022-11-03 09:55:59 -07:00
Larry Knox
6924d590cc Use ctest_test and APPEND to correctly report passed and failed test numbers in CDash (#2208)
* Change how HPC tests are executed

* Move ctest_test commands for serial and parallel tests into cmake files.

* Update checks for uploading Test.xml to CDash.

* Correct cmake scripots.

* Remove extraneous characters.

* Add ctest_start (APPEND).

* Add binary directories to scripts running ctest_test.

* Correct parameters for appending Test.xml.

* Set CTEST_SITE and CTEST_BUILD_NAME.

* Try to pass site and build_name to ctest_test sripts.

* Add site name and build name for ctest_test commands vi environment.

* Revert "Try to pass site and build_name to ctest_test sripts."

This reverts commit d364aaf0fa.

* Correct typo in src/CMakeLists.txt.

* Update batch scripts to run ctest_test scripts.

Co-authored-by: Allen Byrne <byrn@hdfgroup.org>
2022-11-02 09:02:47 -07:00
Larry Knox
e4fb9aa96e Update link for Software Changes in RELEASE.txt. (#2205)
* Update link for Software Changes in RELEASE.txt.
Add RELEASE.txt contents from 1.13.3 release to HISTORY-1_13.txt.

* Reverse order of entries in HISTORY-1_13.txt to match the order in previous HISTORY
files.
2022-11-01 16:14:52 -05:00
Allen Byrne
94119211a7 Correct the CMake link for MPI (#2198) 2022-11-01 16:03:13 -05:00
Larry Knox
11dfa25910 Update copyright headers (#2184)
* Updated source file copyright headers to remove "Copyright by the Board of Trustees
of the University of Illinois", which is kept in the top-level COPYING file.
2022-11-01 16:02:27 -05:00
raylu-hdf
71595e9ea3 Added a test case for H5VLget_connector_name (#2195)
* Added a test case for H5VLget_connector_name.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-10-28 13:36:49 -07:00
Scot Breitenfeld
2b8139f34c added issue with Perlmutter and subfiling (#2202)
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
2022-10-28 09:01:06 -07:00
Larry Knox
9acd203041 Update release.txt (#2201)
* Some Javadoc warning fixes

* Updated javadoc fixes

* # WARNING: head commit changed in the meantime

HDFFV-11229 Fix long double usage in tools and java

Changed h5dump and h5ls to just print 128-bit for long double type.
Added test and file for dataset and attributes with all three float
types.

* Committing clang-format changes

* HDFFV-11229 Add release note

* HDFFV-11229 windows testfile needed

* fix typo

* Remove non supported message text

* HDFFV-11229 - change ldouble test to check both native and general

* HDFFV-11229 add second file compare

* HDFFV-11229 fix reference file

* HDFFV-11229 autotools check two refs

* HDFFV-11229 revert back to removal of NATIVE_LDOUBLE in tools output

* Committing clang-format changes

* Update release note

* Update attribute type of ref file

* Change source of ninja for macs

* try port instead of brew

* Recommended is to use brew.

* Undo non long double changes

* remove unneeded file

* Fix perl and doxygen CMake code

* Add "option" command for clang options

* Rework CMake add_custom to use the BYPRODUCTS argument

* Add stamp files to BYPRODUCTS

* Only one copy of file exists

* Fix custom cmmand depends targets

* Fix fortran custom command DEPENDS

* Add LD_LIBRARY_PATH to tests

* Add custom target for DEPENDS

* Add h5detect conditionaly to generated target DEPENDS

* Correct DEPENDS targets

* Parallel builds need the mpi compiler for pkgconfig scripts.

* install only if MPI build

* Fortran target depends

* Remove incorrect source attribute

* doxygen adjustments

* doxygen build updates

* Correct version

* Correct function version - function has been merged to 1.12

* Correct version string for map functions

* Cleanup warnings for tools debug builds

* TRILAB-227 - fix indexing for h5diff selections

* Correct location of pos to index function call

* TRILAB-227 Enable test

* Quote subset args

* Use MATCHES because of AppleClang

* if blocks needed for build modes

* Update list of DT platforms

* VS2019 correctly displays float values

* revert VS2019 change

* Issue #669 remove version from pkgcfg filename

* remove version from h5cc script

* Java reference functions updated

enabled fortran in cmake gcc action yaml file
java reference test changed to correctly test refs
jni reference functions that create ids changed to use jni id logging

* Correct BYPRODUCTS argument

* Correct more genereated files BYPRODUCTS

* BYPRODUCTS must have unique locations

* Fix typo

* Fix fortran configure checks

* Rework H5_PAC_C_MAX_REAL_PRECISION setting logic

* Add note about fortran configure change

* Adds a quick for for some egregious chunk_info badness (#722)

* Fixes issue with ccmake that prevents building Fortran (#723)

ccmake runs iteratively, and the check_fortran_source_runs
macros were clobbering a single output file that did not
get updated on further configure iterations

* Fix conflicts with merge

* Move MAX_PRECISION back to HDF5UseFortran.cmake

* Use STREQUAL to test macro argument

* Move C language test to ConfigureChecks from HDF5UseFortran

* MAX_PRECISION defines must be defined

* Organize flags and align autotools and cmake

* Fix comment in no-error-general warnings files.

* Flag cleanup and fix typos

* Add comment

* Correct VAR used to find configure time file

* Set the path correctly

* Update missing release note info.

* Update code owners

* Correct JIRA note

* add known problem.

* Use only core library for testing dynamic plugins.

* Committing clang-format changes

* Update main.yml

* Update HISTORY-1_13.txt

Missing release note added for changed location of CMake configuration files.

* Update HISTORY-1_13.txt

Fix typo

* Update main.yml

revert change

* Updated version to 1.13.4-1 after creating hdf5_1_13_3 branch for 1.13.3
release.
Added new version to exceptions in lines 74-5 of src/H5.c

* Cleared RELEASE.txt entries following HDF5 1.13.3 release.

Co-authored-by: Allen Byrne <byrn@hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
2022-10-28 08:10:02 -07:00
Allen Byrne
4d0501f48c Fix for documentation misunderstanding on m1 Intel builds (#2188) 2022-10-26 09:24:51 -07:00
Scot Breitenfeld
a6c1dff09a running on node-16 insteas of Node.js 12 (#2190) 2022-10-26 09:24:12 -07:00
Larry Knox
0173706826 Update develop branch version to 1.13.4-1 (#2189)
* Updated version to 1.13.4-1 after creating hdf5_1_13_3 branch for 1.13.3
release.
Added new version to exceptions in lines 74-5 of src/H5.c
2022-10-26 11:07:17 -05:00
Allen Byrne
9c61f7b1e1 JNI: Improve varlen datatype handling in H5A/H5D read/write functions (#2156)
* Improve JNI VL datatype read/write to handle complex combinations

* Implement VL of VL JNI writes and reads

* Add Java VLofVL test for attributtes

* Changes to address review issues

* Fix H5Aread vl blocks
2022-10-25 23:33:30 -05:00
Neil Fortner
ad2d77397a Add RELEASE.txt note for multi dataset feature (#2187) 2022-10-25 22:13:01 -05:00
Scot Breitenfeld
ba1fe3d1df converted from ISO-8859-1 to UTF-8 (#2185) 2022-10-25 12:29:36 -07:00
Scot Breitenfeld
7d1997ff7b Removed the file, func, line args in the _async APIs fpr doxygen (#2179)
* Removed the: file, func, line args in the _async APIs from the doxygen arguments. Documented H5ESclean.

* format fixes

* fixed DOXYGEN_PREDEFINED

* Fixed the #ifdet logic for doxygen, added missing multi-dataset va_args

* format fixes

* Use H5_DOXYGEN instead of H5_DOXYGEN_FORTRAN

* reordered multidataset declarations

* alt. for H5Aclose_async and H5Acreate_async

* made doxgyen API into seperate blocks

* updated codespell

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-10-25 10:41:38 -07:00
Allen Byrne
23e237e728 Port VOL User Guide to doxygen (#2165)
* Port VOL User Guide to doxygen

* Fix spelling

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-10-25 08:51:08 -07:00
Allen Byrne
ea902f9ace Implement option for using CMake GNUInstallDirs (#2175)
* Implement option for using CMake GNUInstallDirs

* Add release notes
2022-10-25 08:50:43 -07:00
raylu-hdf
f98c93ab19 Added a test case to verify the VOL capacity flags work correctly. (#2178)
* Added a test case to verify the VOL capacity flags work correctly.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-10-25 07:59:03 -05:00
Scot Breitenfeld
4e4c7cf66d fixed error code returned for H5Aexits (#2180) 2022-10-25 07:58:23 -05:00
Neil Fortner
7997b53589 Fix problem with variable length attributes being accessed through multiple file handles (#2181)
* Fix bug with variable length attributes being accessed through multiple
file handles.

* Committing clang-format changes

Co-authored-by: Neil Fortner <nfortne2@localhost.localdomain>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-10-25 07:46:15 -05:00
Neil Fortner
93754cae33 Multi Dataset (#2120)
* Fix bug with cross platform compatibility of references within vlens.
No testing yet.

* Merge from multi_rd_wd_coll_io to a more recent branch from develop.
Untested, probably does not work yet.

* Committing clang-format changes

* Committing clang-format changes

* Fix many bugs in multi dataset branch.  Mostly works, some issues in
SWMR tests.

* Committing clang-format changes

* Disable test in swmr.c that was failing due to bug in HDF5 unrelated to
multi dataset.

* Committing clang-format changes

* Fixed fortran multi-dataset tests

* Fixed xlf errors

* Added benchmark code for multi-datasets

* loops over datasets

* added missing error arg.

* Added gnuplot formatting

* Jonathan Kim original MD benchmarking code

* updated MD benchmarking code

* code clean-up

* Only make files in feature test mode

* misc clean-up

* removed TEST_MDSET_NO_LAST_DSET_2ND_PROC option

* Committing clang-format changes

* Change multi dataset API to use arrays of individual parameters instead
of the parameter struct.

* Committing clang-format changes

* Update to new multi dataset Fortran API and tests. (#1724)

* Update to new multi dataset Fortran API and tests.
* Sync Fortran with develop.
* skipping h5pget_mpio_actual_io_mode_f for now

* Fixed issue with dxpl_id, changed to variable size dim. (#1770)

* Remove "is_coll_broken" field from H5D_io_info_t struct

* Committing clang-format changes

* Minor cleanup in multi dataset code.

* Committing clang-format changes

* Clean up in multi dataset code.

* Committing clang-format changes

* Committing clang-format changes

* Fix speeling

* Fix bug in parallel compression. Switch base_maddr in io_info to be a
union.

* Committing clang-format changes

* Implement selection I/O support with multi dataset.  Will be broken in
parallel until PR 1803 is merged to develop then the MDS branch.

* Committing clang-format changes

* Spelling

* Fix bug in multi dataset that could cause errors when only some of the
datasets in the multi dataset I/O used type conversion.

* Committing clang-format changes

* Integrate multi dataset APIs with VOL layer.  Add async versions of
multi dataset APIs.

* Committing clang-format changes

* Spelling fixes

* Fix bug in non-parallel HDF5 compilation.

* Committing clang-format changes

* Fix potential memory/free list error. Minor performance fix. Other minor
changes.

* Committing clang-format changes

* Fix memory leak with memory dataspace for I/O.

* Committing clang-format changes

* Fix stack variables too large.  Rename H5D_dset_info_t to
H5D_dset_io_info_t.

* Committing clang-format changes

* Remove mem_space_alloc field from H5D_dset_io_info_t.  Each function is
now responsible for freeing any spaces it adds to dset_info.

* Committing clang-format changes

* fixed _multi Fortran declaration

* Refactor various things in (mostly) the serial I/O code path to make
things more maintainable.

* Committing clang-format changes

* updated to array based, doxygen, and examples

* Reinstate H5D_chunk_map_t, stored (via pointer) inside
H5D_dset_io_info_t.

* Change from calloc to malloc for H5D_dset_io_info_t and H5D_chunk_map_t.
Switch temporary dset_infos to be local stack variables.

* Committing clang-format changes

* format cleanup

* format cleanup

* added coll and ind

* Modify all parallel I/O paths to take dset_info instead of assuming
dset_info[0].

* Committing clang-format changes

* fixed output

* Rework parallel I/O code to work properly with multi dataset in more
cases.  Fix bug in parallel compression.

* Committing clang-format changes

* Prevent H5D__multi_chunk_collective_io() from messing up collective opt
property for other datasets in I/O.  Other minor cleanup.  Add new test
case to t_pmulti_dset.c for H5FD_MPIO_INDIVIDUAL_IO, disabled for now
due to failures apparently unrelated to multi dataset code.

* Fix spelling

* Committing clang-format changes

* Replace N log N algorithm for finding chunk in
H5D__multi_chunk_collective_io() with O(N) algorithm, and remove use of
io_info->sel_pieces in that function.

* Committing clang-format changes

* Replace sel_pieces skiplist in io_info with flat array of pointers, use
qsort in I/O routine only when necessary.

* Committing clang-format changes

* Add new test case to mdset.c

* Committing clang-format changes

* Fix spelling

* Very minor fix in H5VL__native_dataset_read()

* Fix bug that could affect filtered parallel multi-dataset I/O.

* Add RM entries for H5Dread_multi(), H5Dread_multi_async(),
H5Dwrite_multi(), and H5Dwrite_multi_async()

* Unskip test in swmr.c

* Committing clang-format changes

* Eliminate H5D__pre_read and H5D__pre_write

* Remove examples/ph5mdsettest.c. Will fix and re-add as a test.

* Enable hyperslab combinations in mdset test

* Committing clang-format changes

* Clarify H5Dread/write_multi documentation.

* Fix bugs in multi-dataset I/O.  Expand serial multi dataset test.
Update macro in parallel multi dataset test.

* Committing clang-format changes

* Spelling

* Remove obsolete entry in bin/trace

* Rework type conversion buffer allocation. Only one buffer is shared
between datasets in mdset mode, and it is malloced instead of calloced.

* Committing clang-format changes

* Fix bug in error handling in H5D__read/write

* added multi-dataset fortran check with optional dataset creation id (#2150)

* removed dup. dll entry

* Address comments from code review.

* Remove spurious changes in H5Fmpi.c

* Fix issue with reading unallocated datasets in multi-dataset mode.
Address other comments from code review.

* Committing clang-format changes

* Delay chunk index lookup from io_init to mdio_init so it doesn't add
overhead to single dataset I/O.

* Committing clang-format changes

* Fix inappropriate use of piece_count

* updated copyright on new file, removed benchmark from testing dir.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: M. Scot Breitenfeld <brtnfld@hdfgroup.org>
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
2022-10-19 09:13:15 -07:00
Pierce Brooks
a898cef6c2 make sure to match the debug library postfix with external cmake projects and allow for zlib link name to be manually overridden (#2166) 2022-10-19 09:06:51 -05:00
Scot Breitenfeld
889dc1e9a7 address test failing when default real and integers are 8 bytes (#2173) 2022-10-19 08:55:06 -05:00
Scot Breitenfeld
0fd867390a changed to referencing C APIs by empty brackets (Fortran) (#2167) 2022-10-17 15:37:19 -05:00
Scot Breitenfeld
afb92a3e22 Fix H5Dfill_f test (#2162)
* change to integer for legacy H5Dfill_f testing

* fixed H5Dfill testing issues with promoted real and integer

* updated build fortran dependencies
2022-10-14 11:35:52 -07:00
vchoi-hdfgroup
5021fd9d4d Fix for HDFFV-10840: Instead of using fill->buf for datatype conversion (#2153)
* Fix for HDFFV-10840: Instead of using fill->buf for datatype conversion
if it is large enough, a buffer is allocated regardless so that the element
in fill->buf can later be reclaimed.
Valgrind is run on test/set_extent.c and there is no memory leak.

* Add information of this fix to release notes.

Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
2022-10-14 11:05:14 -07:00
Dana Robinson
80f80d61db Cleans final production mode warnings from H5Shyper.c (#2163)
* H5S__hyper_free_span_info() returns herr_t (not checked)

* Set up H5S__hyper_free_span_info() error handling

* Set up H5S__hyper_free_span() error handling

* Added error handling to H5S_HYPER_ADVANCE_SPAN

* Handled errors in H5S__hyper_iter_get_seq_list_gen()

* Added a production mode build target for -Werror

* Committing clang-format changes

* Updated names for -Werror checks

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-10-14 06:15:33 -07:00
Scot Breitenfeld
edc3790f5c renamed H5_FORTRAN_C_LONG_DOUBLE_IS_UNIQUE DLL (#2158)
* changed DLL define to a unique name
2022-10-13 11:41:33 -05:00
Scot Breitenfeld
cffaf47c15 Fix for missing h5dfill_c_long_double in dll (#2155)
* added conditionally include h5dfill_c_long_double in dll
2022-10-12 07:38:27 -05:00
Scot Breitenfeld
64e69d9291 Implemented C matching (and general) H5Dfill Fortran wrappers (#2152) 2022-10-11 07:25:05 -05:00
Scot Breitenfeld
306db409d4 Avoid calling H5FD_subfiling_init from h5open_f. (#2149)
* Only call H5FD_SUBFILING if subfiling conditions are met

* clang format
2022-10-07 09:01:42 -07:00
Scot Breitenfeld
e80079fd21 Subfiling Fortran wrapper work. (#2143)
* added C ref. for Fortran constants

* added C ref. for Fortran constants

* move constant paramters to H5* module listing

* added back comment

* Fortran subfiling and ioc FD with tests. H5Pset/get_mpi_params wrappers with tests, misc.. parallel test clean-up.

* misc. fixes

* fixed CMake testpar issues, formatted, misc. updates

* updated tests
2022-10-06 09:08:56 -07:00
Allen Byrne
a322eb6147 Fix static-cast format (#2141)
* Fix static-cast format

* clang format fix
2022-10-04 08:31:46 -05:00
Allen Byrne
c114bd64b0 Fix mostly java warnings, and cpp cast (#2134) 2022-10-03 13:31:32 -05:00
Scot Breitenfeld
1fb83da215 Grouped the Fortran constants in Doxygen (#2136) 2022-10-03 11:13:00 -05:00
Dana Robinson
ce67e4eb05 Adds a first cut at a "getting started with HDF5 development" guide (#2137)
* Added empty "getting started" markdown file

* Added title and brief tour of source code

* Moved platform-independence text to markdown

* Added anatomy of a function documentation

* Adds information on public/private/package calls and macros

* Adds much getting started guide text

* Fixed typos

* Fixed codespell issues
2022-09-30 21:44:39 -07:00
jhendersonHDF
79bdc6183e Subfiling testing fix and documentation (#2132)
* Fix a sporadic failure in Subfiling VFD tests

* Subfiling VFD - add note to H5Pget_fapl_subfiling documentation

Adds note about how H5Pget_fapl_subfiling only returns the original
settings on a FAPL and those settings could have been modified by the
Subfiling VFD's environment variables
2022-09-29 21:06:17 -05:00
Scot Breitenfeld
1bec0ce2d7 Misc. Fortran Doxygen updates (#2129) 2022-09-28 10:52:19 -05:00
Neil Fortner
4184bac2ec Fixed an issue that could occur when combining hyperslab selections (#2122) 2022-09-26 22:51:52 -05:00
Scot Breitenfeld
1c26386c38 Added C references to Fortran constants. (#2126)
* added C references for Fortran APIs
2022-09-26 22:50:09 -05:00
Allen Byrne
dcd332d231 Restructure ref manual presentation (#2123) 2022-09-26 22:49:29 -05:00
Allen Byrne
853a592d2d Add missing windows include/lib configure for StrStrIA (#2114) 2022-09-22 13:52:31 -05:00
Frank Berghaus
100b22e6c2 Use case-insensitive comparison for headers fix #2100 (#2101)
* Use case-inseneitive comparison for headers

HTTP headers should be case-insensitive. Use case-insensitive string
comparisons when working with HTTP header responses to ensure
compatibility.

* Revert "Use case-inseneitive comparison for headers"

This reverts commit a02f591723506b62b7208449be6eef7122120398

* Ignore case when searching HTTP header responses

Looking up the Content-Length in the header returned by S3 storage
endpoints should ignore case. To guarantee portability implement a
function for case-insensitive string search, because it is non-standard.

* Add an _ after H5 for the strcasestr implementation

It is a private function and should sport that underscore.

* Remove author comment from the doc comment

* Use search function defined by system if available

Check whether the system provides a function implementing case
insensitive string searches. Only use the custom implementation if the
system does not provide the functionality.

* Add tests for case-insensitive search

Basic tests:
  - Search for empty string
  - Search with exact match
  - Search with case-insensitive match
  - search with no match

* Enforce clang-format style

Some variable definitions in the th5_system tests did not conform to
clang-format's expectations. Updated the offending lines.

* Correct comment describing test case

* Added some spaces to please clang-format

* Ignore discarding const

Ask the compiler to ignore discarding the const when retunring a match from H5_strcasestr

Co-authored-by: Frank Berghaus <frank.berghaus@mpcdf.mpg.de>
2022-09-21 11:50:17 -05:00
Allen Byrne
d491c33a72 Update examples reference to single-source version 2.0.1 (#2107) 2022-09-20 17:21:15 -05:00
Dana Robinson
6d89634445 Updates VOL capabilities flags (#2105)
* Added feature/vol_cap_flags branch to GitHub actions

* Convert unsigned cap flag field to uint64_t and add flags

* Committing clang-format changes

* Convert unsigned cap flag field to uint64_t and add flags

* Committing clang-format changes

* Updates the VOL struct version number

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-09-17 09:49:38 -07:00
Allen Byrne
a9d8b469f4 Use doxygen snippets for common tables (#2104) 2022-09-16 12:45:06 -05:00
jhendersonHDF
16aa2dbaa0 Subfiling VFD updates (#2106) 2022-09-16 11:17:30 -05:00
Allen Byrne
45178c87a3 develop Merge doxygen from 1.12 branch (#2095) 2022-09-14 15:44:24 -05:00
Scot Breitenfeld
dcf3b54b6e removed H5R interface definition (#2090) 2022-09-14 15:43:27 -05:00
Neil Fortner
b5598575bb Fix memory bug in selection I/O (#2096)
* Fix memory bug in selection I/O

* Change error messages for calls to H5I_remove() in
H5FD_read/write_selection()
2022-09-14 11:10:05 -05:00
Dana Robinson
fe9c07fd90 Adds file locking documentation (#2084)
* Added initial (partial) file locking document

* Almost done with file locking document

* Fix intro

* Cleaned up text

* Updated environment variable verion info

* Fix typo

* Fix typos
2022-09-14 05:53:35 -07:00
Allen Byrne
05a0411140 Remove bitbucket.hdfgroup.org dependency. #2082 (#2086)
*
2022-09-07 17:12:55 -05:00
H. Joe Lee
2d4a96b32c Change test to example. (#2087)
* Remove the commented line.
2022-09-07 17:12:19 -05:00
H. Joe Lee
ecc687c4ff Fix grammar and spelling in comment block. (#2089) 2022-09-07 17:11:40 -05:00
Scot Breitenfeld
99f17c7271 Doxygen Fortran (#2062) 2022-09-07 17:11:20 -05:00
Dana Robinson
1804e0e0c4 Updates release schedule for 2023 (#2085)
* Updates release schedule for 2023

* Updates figure to put 1.13 and 1.14 on the same line
2022-09-04 04:42:25 -07:00
Allen Byrne
c8fdd92cd4 Update CMake code to 3.18 minimum version (#2080)
* Update CMake code to 3.18 minimum version

* Updated release note
2022-09-02 17:31:29 -07:00
Dana Robinson
6e8aaab1b4 Minor changes to CMake comments, etc. (#2078) 2022-08-31 11:03:37 -05:00
Dana Robinson
ccdcec273a Updates onion VFD major error codes (#2076) 2022-08-31 11:01:52 -05:00
Neil Fortner
5de9d63ff4 Address review comments from downstream merge of #2016 (#2073) 2022-08-31 11:01:05 -05:00
jhendersonHDF
5719187a34 Fix Java tests for Developer build modes (#2079) 2022-08-29 09:00:42 -07:00
Dana Robinson
5097ff016f Tidying of configure check C code (#2077)
* Tidying of configure check C code

* Fixed missing endif removal
2022-08-29 08:59:15 -07:00
Dana Robinson
08aebb6d1a Adds documentation for config directory (#2065)
* Adds documentation for config directory

* Fix spelling error
2022-08-24 23:05:28 -07:00
Gerd Heber
e1d6157c5d Added VOL connector feature flag RFC. (#2064) 2022-08-24 10:30:15 -05:00
Dana Robinson
ee55dd777b Fortran module directory improvements (#2060)
* Removes the module directory from the Automake flags as this
  generated a large number of "missing include directory" warnings
  and is unecessary
* Adds the module path to libhdf5.settings
2022-08-23 12:25:07 -07:00
Scot Breitenfeld
88ed5b69d4 removed h5rget_object_type_f module interface (#2057) 2022-08-23 09:09:08 -07:00
Dana Robinson
f020d129b2 Corrects more Fortran warnings (#2059)
* Moves -Wunused-dummy-arguments to the developer warnings since this
  generates a lot of spurious, unfixable noise due to things like
  callback wrappers
* Changes the return cast in h5fget_fileno_c, fixes a comment, and comment
  on potential badness
* Initializes a variable in tH5L_F03.F90 to quiet a 'maybe
  uninitialized' warning
2022-08-23 08:48:14 -07:00
Neil Fortner
21ec337855 Fix inappropriate use of HGOTO_ERROR (#2055) 2022-08-22 11:39:49 -05:00
Dana Robinson
801edb4f52 Quiets Fortran warnings about large arrays (#2053) 2022-08-22 08:28:48 -07:00
Dana Robinson
ec3737ab69 GitHub workflow cleanup (#2047)
* Cleans up main GitHub workflow(s)

* Combines main.yml and pr-check.yml, which differed trivially
* Reduces config redundancy via proper use of the matrix scheme
* Adds build-only configs for:
    - Version 1.6 default API
    - No deprecated symbols
    - Parallel HDF5
* The -Werror config was converted to build-only
* Makes the naming scheme more uniform
* Spaces out the code and adds comments

* Typo fix

* Fix mirror VFD CMake configure typo

* Debugging hack

* Undo debug hack and fix CMake build mode

* Set CC=mpicc for parallel builds

* Another attempt to fix parallel builds

* Updates comments in main.yml

* Fixes a v16 API bug in the mirror VFD test code

* Committing clang-format changes

* Added checks for v1.8 - 1.12 APIs

* Manually specify Windows configs

Java debug builds time out when running tests on Windows. We'll disable
them in debub mode while we investigate.

* Revert "Manually specify Windows configs"

This reverts commit d710d8d219.

* Skip Debug Windows configs while we investigate Java H5L timeouts

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-08-22 08:21:27 -07:00
Dana Robinson
f24dff5157 Quiets Fortran Autotools builds (#2052) 2022-08-22 08:21:06 -07:00
Dana Robinson
3015e3d7ab Quiets Automake rules in src directory (#2051)
* Quiets Automake rules in src directory

* Fix whitespace
2022-08-22 06:25:17 -07:00
Dana Robinson
10eb43d0c2 Removes ST from the list of debug packages (Autotools only) (#2045)
* Removes ST from the list of debug packages

The ST package (ternary search trees) was removed from the library a while ago

* Replaced B2 package

I did some crude timing with and without the B2 debug code enabled
and it doesn't seem to make much of a difference.
2022-08-22 04:12:12 -07:00
Dana Robinson
3e38a001a1 Simplify the CODEOWNERS file (#2049)
Simplify the CODEOWNERS file
2022-08-22 03:01:00 -07:00
jhendersonHDF
2b786ffe5a [WIP] Add Developer build mode to CMake (#1659)
* Add Developer build mode to CMake

* Set a few CMake variables for Developer build modes

* Refactor enabling of debug and developer-level compile definitions

* Convert cache debugging macros to normal ifdef style

Normal ifdef-style instead of if-style allows build system to define macros
without warning about redefining macros with different values (0 vs. 1)

* Add HDF5 Developer compile definitions to testing files

* Temporarily disable -fanalyzer flag for GCC 12+ Developer builds
2022-08-19 17:18:30 -07:00
jhendersonHDF
281db5876e Cleanup some warnings in Java JNI code (#2034) 2022-08-19 09:05:43 -07:00
Dana Robinson
7f6261a677 Removes cmake_ext_mod (files moved to cmake) (#2041)
At one time this content was a git submodule shared between several
projects but that time has long passed.
2022-08-19 07:54:42 -07:00
Dana Robinson
de46019a86 Removes bin/deploy, which just calls h5redeploy under the hood (#2027) 2022-08-19 09:11:22 -05:00
Dana Robinson
8cd97c4793 Updates the warnist script to report gfortran generic warnings (#2040)
These are not associated with a file and line number, like specifying
a non-existant include directory on the compile line.
2022-08-19 09:09:59 -05:00
Larry Knox
9dfcca0bbb Release updates (#2033)
* Update HISTORy-1_13.txt and clean RELEASE.txt entries after 1.13.2 release.
2022-08-18 08:35:21 -05:00
Neil Fortner
a3fd350483 Add note to RELEASE.txt for GitHub #2016 (#2030)
* Fix bug in attribute type conversion where the background buffer would
not be initialized with the destination contents when necessary.  Other
minor simplification.

* Committing clang-format changes

* Fix warnings.

* Address review comments.

* Add RELEASE.txt note for PR #2016

* Add GitHub number to release note

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-08-17 22:08:43 -05:00
Dana Robinson
e6f16c0ec2 Fixes codespell issues (#2032)
* Fixes codespell issues

* Fixed another typo
2022-08-17 16:31:54 -07:00
Neil Fortner
9304f277e8 Add note to RELEASE.txt for HDFFV-11308 (#2029)
* Add note to RELEASE.txt for HDFFV-11308

* Fix formatting error in RELEASE.txt
2022-08-17 13:35:48 -07:00
Dana Robinson
cf482563cc Remove unused dependency files (#2025) 2022-08-17 11:21:03 -07:00
Dana Robinson
d201907a4b Update bin/README.md
Correct missing space after # and ##
2022-08-17 10:46:21 -07:00
Dana Robinson
e4de06057a Adds a TODO section to bin/README.md (#2028)
Mainly to address the semi-broken state of chkcopyright
2022-08-17 10:45:05 -07:00
jhendersonHDF
07efecd9f4 Adjustments to t_bigio.c for >2GiB I/O test (#1992) 2022-08-17 10:44:00 -07:00
Dana Robinson
ecc9bbe4b6 Adds a README.md file to bin that indicates script usage (#2024)
* Adds a README.md file to bin that indicates script usage

* Forgot header row
2022-08-17 09:34:45 -07:00
Dana Robinson
b54d01fcb5 Removes bin/chkconfigure (#2023)
We check autoconf in configure.ac and can check for anything else in
autogen.sh
2022-08-17 09:22:39 -07:00
Dana Robinson
1483eeef34 Removes bin/format_source_patch (#2022)
This differs only trivially from bin/format_source
2022-08-17 09:22:13 -07:00
Dana Robinson
dc9562ee77 Removes the bin/gcov_script file (#2021)
We don't need fragile scripts to run basic profiling programs
2022-08-17 09:21:38 -07:00
Dana Robinson
afc54d75a1 Replaces bin/newer with -nt comparison in Makefiles (#2020) 2022-08-17 09:21:24 -07:00
Dana Robinson
2bba853375 Removes the bin/yodconfigure script which appears unused (#2019) 2022-08-17 08:03:14 -07:00
Dana Robinson
d6927e49c0 Replaced bin/mkdirs w/ mkdir -p -m 755, which is POSIX (#2018) 2022-08-17 08:02:39 -07:00
Dana Robinson
191c27c0a9 Removes the broken bin/errors script (#2017) 2022-08-16 15:42:55 -07:00
bmribler
951566a4ad Fix cpp comment (#2011)
* Fix test failure

Description:
    - Added two H5File constructors to open file with non-default fapl
    - Added non-default fapl to test functions
    - Commented certain renaming attribute tests because of the unexpected
      behavior in renaming an attribute (HDFFV-11327)

Platform tested:
    Linux/64 (jelly)

* Committing clang-format changes

* Update H5File.cpp

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-08-16 14:06:55 -07:00
Neil Fortner
75f4d9b042 Fix bug in attribute type conversion wiith compound types (#2016)
* Fix bug in attribute type conversion where the background buffer would
not be initialized with the destination contents when necessary.  Other
minor simplification.

* Committing clang-format changes

* Fix warnings.

* Address review comments.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-08-16 14:04:48 -07:00
bmribler
10e4dd4ef1 Fix c++ test failure (#2005)
* Fix test failure

Description:
    - Added two H5File constructors to open file with non-default fapl
    - Added non-default fapl to test functions
    - Commented certain renaming attribute tests because of the unexpected
      behavior in renaming an attribute (HDFFV-11327)

Platform tested:
    Linux/64 (jelly)

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-08-12 13:33:30 -05:00
Larry Knox
25ef534039 Update version to 1.13.3-1 after creating hdf5_1_13_2 branch for 1.13.2 (#2004)
release.
Added new version to exceptions in line 75 of src/H5.c.
2022-08-12 08:17:25 -05:00
Larry Knox
029cde6111 Revert "Increment version to 1.13.3-1 after creating hdf5_1_13_2 branch for release"
This reverts commit c58558e872.

H5TEST-tcheck_version-release failed.
2022-08-11 22:45:43 -05:00
Larry Knox
c58558e872 Increment version to 1.13.3-1 after creating hdf5_1_13_2 branch for release
of HDF5 1.13.2.
2022-08-11 22:15:46 -05:00
Allen Byrne
8d5fcbcdce Correct target name for szaec lib (#2000) 2022-08-11 15:35:14 -07:00
jhendersonHDF
b745b54a09 Only use Subfiling VFD in tools if MPI was initialized previously (#1998) 2022-08-11 15:52:25 -05:00
Allen Byrne
ec5100bf26 Fix Cmake depends prop (#1995) 2022-08-11 09:23:16 -07:00
Allen Byrne
29c8937f60 Remove misnamed java vol clear test (#1990) 2022-08-10 14:20:41 -07:00
jhendersonHDF
a3e37c587b Only include and run Subfiling example when Subfiling is enabled (#1991) 2022-08-10 14:20:28 -07:00
luzpaz
174505ca3c Fix typos found in the rest of the hdf5 code-base (#1985)
* Fix typos found in the rest of the hdf5 code-base

* Typo in tool.cmake

* Revert "Typo in tool.cmake"

This reverts commit 06b8403d99.

* Fix clang-format styling
2022-08-10 13:57:26 -07:00
Allen Byrne
e794dc5ec4 Fix CMake macro code commands (#1989) 2022-08-10 12:20:16 -07:00
luzpaz
f04312a792 Fix bug in cppcheck in tool.cmake (#1988)
Ref: https://github.com/HDFGroup/hdf5/pull/1985#issuecomment-1211046328
2022-08-10 12:19:26 -07:00
Gerd Heber
4c73c284e0 Fix4 readme (#1986)
* Update README.md

* Update README.md
2022-08-10 11:17:56 -07:00
Dana Robinson
9777d05848 Move VFD SWMR release version to 1.13.3 in README.md (#1987) 2022-08-10 11:17:43 -07:00
luzpaz
36ae75fb93 Fix typos found in src/ subdirectory (#1983)
Found via `codespell -q 3 -S ./release_docs,./bin/trace,./hl/tools/h5watch/h5watch.c,./tools/test/h5jam/tellub.c -L isnt,inout,nd,parms,parm,ba,offsetP,ser,ois,had,fiter,fo,clude,refere,minnum,offsetp,creat,ans:,eiter,lastr,ans,isn\'t,ifset,sur,trun,dne,tthe,hda,filname,te,htmp,minnum,ake,gord,numer,ro,oce`
2022-08-10 09:03:22 -07:00
jhendersonHDF
6c5b5e0759 Remove usage of libgen.h (#1984) 2022-08-10 09:00:45 -07:00
jhendersonHDF
95ceec786a Add documentation for parallel compression feature (#1981) 2022-08-09 16:08:13 -07:00
jhendersonHDF
ef33ac8bac Subfiling VFD - tidying up and fixing a few new testing failures (#1977)
* Rename Subfiling IOC "thread_pool_count" field to "thread_pool_size"

* Add simple HDF5 example for Subfiling VFD

* Subfiling VFD - never cache app topology as it may change

* Subfiling VFD - cleanup unused funtionality and tidy up some TODOs

* Subfiling VFD - tidy up subfiling error handling in H5subfiling_common.c

* Subfiling VFD - show number of failed I/O requests on close

* Subfiling VFD - Update file cmp callback after switching to MPI I/O VFD

* Amend RELEASE.txt with info about h5fuse.sh and Subfiling limitations

* Subfiling VFD - switch to using H5_basename and H5_dirname
2022-08-09 16:05:37 -07:00
Allen Byrne
b84241e57a Remove stray files, incorrect vds and mpi clean files (#1979) 2022-08-09 08:29:58 -07:00
Allen Byrne
60ba2356de Revert for older cmake versions (#1974)
* Revert for older cmake versions

* Remove unneeded clear/clean tests because of changes to test macro
2022-08-08 13:01:06 -07:00
Dana Robinson
5d63aca604 Turns off clang optimizations on a couple of onion VFD functions (#1976)
Ancient clang (7.7.3 on OS X) emits problematic code with -O2 or -O3
2022-08-08 12:04:01 -07:00
Dana Robinson
57926d549e Fixes some badness in the onion VFD revision comment code (#1975)
* Fixes some badness in the revision comment code

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-08-08 11:00:13 -07:00
Allen Byrne
956282fa6f Develop cmake clean (#1971) 2022-08-06 23:59:31 -05:00
Mark Kittisopikul
7127d8912c H5Dchunk_iter now passes offsets in units of dataset elements, fix #1419 (#1969)
* H5Dchunk_iter now passes chunk dimension scaled offsets, fix #1419

* Update docs for H5Dchunk_iter, H5Dget_chunk_info*

Modified description for `H5Dchunk_iter`, `H5Dget_chunk_info`, and `H5Dget_chunk_info_by_coord` to the following

 * offset          Logical position of the chunk’s first element in units of dataset elements
 * filter_mask  Bitmask indicating the filters used when the chunk was written
 * size             Chunk size in bytes, 0 if the chunk does not exist
2022-08-06 23:57:41 -05:00
jhendersonHDF
043f66afd2 Set entry pointer mem type for vector I/O in collective metadata writes (#1961) 2022-08-06 23:56:32 -05:00
jhendersonHDF
597ce3cdbf Fix issues in tools code after Subfiling VFD updates (#1973) 2022-08-05 22:17:16 -05:00
Dana Robinson
b22984600a Adds platform-independent basename and dirname (#1951)
* Adds platform-independent basename and dirname

* Tidy up H5_dirname and H5_basename implementations and add tests

* Committing clang-format changes

* Fix misspelling

* Several fixes for H5_dirname/H5_basename from review

* Committing clang-format changes

* Add reason to VERIFY_STR macros in th5_system.c

* Use H5MM_free instead of HDfree in H5_dirname/H5_basename tests

Co-authored-by: Jordan Henderson <jhenderson@hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-08-05 16:11:13 -07:00
raylu-hdf
54f116b42d Sanity checks for H5Xcreate_anon functions to make sure the IDs passed in are valid (#1967)
* Sanity checks for H5Xcreate_anon functions to make sure the IDs passed in are valid.

* Removed unnecessary sanity checks.

Co-authored-by: songyulu <songyulu@jelly.ad.hdfgroup.org>
2022-08-05 09:05:07 -07:00
jhendersonHDF
bf07e0f2c9 Subfiling updates for release (#1963)
* Remove generated file h5fuse.sh

* Link pthreads library when Subfiling VFD is built

* Switch to MPI I/O driver for Subfiling HDF5 stub file

* Rough first implementation for Subfiling file deletion

* Subfiling VFD - get file dirname for file deletion

* Subfiling VFD - set lock callback to NULL for now to avoid performance
issues

* Committing clang-format changes

* Minor tidying up of Subfiling testing

* Fixups for Subfiling VFD support in tools

* Tidy up Subfiling public interface and add Doxygen

* Respect Subfiling configuration settings from application

* Add release note for Subfiling VFD

* Committing clang-format changes

* Committing clang-format changes

* Shorten some Subfiling environment variable names

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-08-04 10:56:48 -07:00
Dana Robinson
a71534fcc2 Corrects some bad format strings in the tools (#1964) 2022-08-03 07:46:33 -05:00
Dana Robinson
fcf41b3cd6 Onion VFD (#1953)
* Onion VFD feature

* Fixes onion VFD errors with non-sec2 backing store VFDs

* Disables the onion VFD tests w/ ph5diff

* Disables non-sec2 VFDs as onion VFD backing stores

* Committing clang-format changes

* Formatted source

* Typo

* Adds onion VFD tools tests to CMake

* Fixes for v16 API compatibility

* Memset structs to avoid bad frees on errors

* H5Dwrite() calls now use H5T_NATIVE_INT as the memory type vs LE

* Properly decodes checksums on BE machines

* Be more careful about uint64_t to haddr_t/hsize_t conversions

* Another fix for BE data comparison

* Removed double underscores from onion constants

* Replace hard-coded onion header string w/ constant

* Fixes cleanup paths in H5FD__onion_ingest_history()

* Fixed use of size_t revision numbers

* Fix h5dump revision count format string

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-08-02 12:54:40 -07:00
Dana Robinson
ea13de1bb0 Removes the Win32 macro for strdup (#1960)
Windows has strdup() so there's no need for this. We already ignore all
the other "underscore" string functions.
2022-08-01 11:45:04 -07:00
Dana Robinson
ecab3d9c2a Removes Windows getlogin() equivalents (#1958)
This function is no longer used in the library
2022-08-01 09:55:54 -07:00
Dana Robinson
47fee5bfd7 Removes (void *) cast from memset on Windows (#1957)
The comments claim this fixes an MSVC warning. Let's see if that's
still true.
2022-08-01 09:55:36 -07:00
Dana Robinson
76bc0a77b1 Fixes clang warnings in H5Tconv.c (long) double macros (#1956) 2022-07-31 10:39:45 -07:00
Dana Robinson
9c21d38ab8 Fixes format string warnings in tselect w/ clang 14 (#1955) 2022-07-31 10:39:29 -07:00
Dana Robinson
0b3220636d Reverts the autogen.sh changes from e3800e1 (#1906) (#1954)
This is causing issues with the NAG Fortran compiler in our daily
testing due to -shared getting inappropriately added. Reverting while
we investigate.
2022-07-30 10:30:15 -07:00
Dana Robinson
8ed84bf4f0 Cleanup of mirror VFD test code (#1949) 2022-07-30 10:29:51 -07:00
Dana Robinson
ce4c41f89f Created a pull request template 2022-07-29 08:26:32 -07:00
Dana Robinson
ab948ff9f9 Update issue templates 2022-07-29 07:57:36 -07:00
Gerd Heber
33f6ade6e7 Fixes #1946 (#1947) 2022-07-29 05:20:55 -07:00
Dana Robinson
05077d3784 H5Tconv: fix H5T_CONV_INTERNAL_ checks (#1936)
Other locations check if it is defined and in cases where it is not
available, it is undefined, not 0.
2022-07-27 09:47:06 -07:00
Dana Robinson
21aca4f380 Misc clang production warning fixes (#1932)
* Misc clang production warning fixes

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-26 17:42:40 -07:00
Allen Byrne
ae414872f5 Develop clang 13 format (#1933)
* Update format source to clang 13

* More format changes
2022-07-26 14:45:46 -07:00
jhendersonHDF
213eac2588 Subfiling VFD misc fixes (#1938)
* Remove generated file h5fuse.sh

* Link pthreads library when Subfiling VFD is built
2022-07-26 14:20:16 -07:00
Dana Robinson
96edaa304c Removes padding-right from Doxygen href tags (#1934) 2022-07-26 09:23:11 -07:00
Allen Byrne
0636395851 Implement improved CMake fortran module folder from GH#1411 (#1922)
* Implement improved CMake fortran module folder from GH#1411

* Update docs

* Fix whitespace

* Fix name case
2022-07-26 09:02:40 -07:00
Dana Robinson
b0240030f5 Updates uthash and makes some tweaks (#1928)
* Updates uthash from 2.1.0 to 2.3.0

* Moves uthash include and related to H5private.h

* Defines HASH_NONFATAL_OOM so that out-of-memory doesn't abort()

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-25 10:38:57 -07:00
Dana Robinson
df1ce61f1b Switches the tagged metadata list from a skip list to a hash table (#1925) 2022-07-25 11:51:09 -05:00
jhendersonHDF
b49c284718 Avoid allocating chunk map for contiguous and compact dataset I/O (#1927) 2022-07-25 07:59:11 -07:00
Dana Robinson
a4a8e1d23f Removes the CodeQL action as it mainly outputs noise (#1926) 2022-07-23 09:06:40 -07:00
jhendersonHDF
7983a584a0 Switch to vector I/O for collective metadata writes (#1911)
* Switch to vector I/O for collective metadata writes

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-22 13:03:44 -07:00
jhendersonHDF
27bb358f7a Subfiling VFD (#1883)
* Added support for vector I/O calls to the VFD layer, and
associated test code.  Note that this includes the optimization
to allow shortened sizes and types arrays to allow more space
efficient representations of vectors in which all entries are
of the same size and/or type.  See the Selection I/o RFC for
further details.

Tested serial and parallel, debug and production on Charis.
       serial and parallel debug only on Jelly.

* ran code formatter

quick serial build and test on jelly

* Add H5FD_read_selection() and H5FD_write_selection().  Currently only
translate to scalar calls.  Fix const buf in H5FD_write_vector().

* Format source

* Fix comments

* Add selection I/O to chunk code, used when: not using chunk cache, no
datatype conversion, no I/O filters, no page buffer, not using collective
I/O.  Requires global variable H5_use_selection_io_g be set to TRUE.
Implemented selection to vector I/O transaltion at the file driver
layer.

* Fix formatting unrelated to previous change to stop github from
complaining.

* Add full API support for selection I/O.  Add tests for this.

* Implement selection I/O for contiguous datasets.  Fix bug in selection
I/O translation.  Add const qualifiers to some internal selection I/O
routines to maintain const-correctness while avoiding memcpys.

* Added vector read / write support to the MPIO VFD, with associated
test code (see testpar/t_vfd.c).

Note that this implementation does NOT support vector entries of
size greater than 2 GB.  This must be repaired before release,
but it should be good enough for correctness testing.

As MPIO requires vector I/O requests to be sorted in increasing
address order, also added a vector sort utility in H5FDint.c  This
function is tested in passing by the MPIO vector I/O extension.

In passing, repaired a bug in size / type vector extension management
in H5FD_read/write_vector()

Tested parallel debug and production on charis and Jelly.

* Ran source code formatter

* Add support for independent parallel I/O with selection I/O.  Add
HDF5_USE_SELECTION_IO env var to control selection I/O (default off).

* Implement parallel collective support for selection I/O.

* Fix comments and run formatter.

* Update selection IO branch with develop (#1215)

Merged branch 'develop' into selection_io

* Sync with develop (#1262)

Updated the branch with develop changes.

* Implement big I/O support for vector I/O requests in the MPIO file
driver.

* Free arrays in H5FD__mpio_read/write_vector() as soon as they're not
needed, to cut down on memory usage during I/O.

* Address comments from code review.  Fix const warnings with
H5S_SEL_ITER_INIT().

* Committing clang-format changes

* Feature/subfiling (#1464)

* Initial checkin of merged sub-filing VFD.

Passes regression tests (debug/shared/paralle) on Jelly.
However, bugs and many compiler warnings remain -- not suitable
for merge to develop.

* Minor mods to src/H5FDsubfile_mpi.c to address errors reported by autogen.sh

* Code formatting run -- no test

* Merged my subfiling code fixes into the new selection_io_branch

* Forgot to add the FindMERCURY.cmake file. This will probably disappear soon

* attempting to make a more reliable subfile file open which doesn't return errors. For some unknown reason, the regular posix open will occasionally fail to create a subfile.  Some better error handling for file close has been added.

* added NULL option for H5FD_subfiling_config_t in H5Pset_fapl_subfiling (#1034)

* NULL option automatically stacks IOC VFD for subfiling and returns a valid fapl.
* added doxygen subfiling APIs

* Various fixes which allow the IOR benchmark to run correctly

* Lots of updates including the packaging up of the mercury_util source files to enable easier builds for our Benchmarking

* Interim checkin of selection_io_with_subfiling_vfd branch

    Moddified testpar/t_vfd.c to test the subfiling vfd with default configuration.
    Must update this code to run with a variety of configurations -- most particularly
    multiple IO concentrators, and stripe depth small enough to test the other IO
    concentrators.

    testpar/t_vfd.c exposed a large number of race condidtions -- symtoms included:

      1) Crashes (usually seg faults)

      2) Heap corruption

      3) Stack corruption

      4) Double frees of heap space

      5) Hangs

      6) Out of order execution of I/O requests / violations of POSIX semantics

      7) Swapped write requests

        Items 1 - 4 turned out to be primarily caused by file close issues --
    specifically, the main I/O concentrator thread and its pool of worker threads
    were not being shut down properly on file close.  Addressing this issue in
    combination with some other minor fixes seems to have addressed these issues.

        Items 5 & 6 appear to have been caused by issue of I/O requests to the
    thread pool in an order that did not maintain POSIX semantics.  A rewrite of
    the I/O request dispatch code appears to have solved these issues.

        Item 7 seems to have been caused by multiple write requests from a given
    rank being read by the wrong worker thread.  Code to issue "unique" tags for
    each write request via the ACK message appears to have cleaned this up.

        Note that the code is still in poor condtition.  A partial list of known
    defects includes:

     a) Race condiditon on file close that allows superblock writes to arrive
        at the I/O concentrator after it has been shutdown.  This defect is
        most evident when testpar/t_subfiling_vfd is run with 8 ranks.

     b) No error reporting from I/O concentrators -- must design and implement
        this.  For now, mostly just asserts, which suggests that it should be
        run in debug mode.

     c) Much commented out and/or un-used code.

     d) Code orgnaization

     e) Build system with bits of Mercury is awkward -- think of shifting
        to pthreads with our own thread pool code.

     f) Need to add native support for vector and selection I/O to the subfiling
        VFD.

     g) Need to review, and posibly rework configuration code.

     h) Need to store subfile configuration data in a superblock extension message,
        and add code to use this data on file open.

     i) Test code is inadequate -- expect more issues as it is extended.

        In particular, there is no unit test code for the I/O request dispatch code.
        While I think it is correct at present, we need test code to verify this.

        Similarly, we need to test with multiple I/O concentrators and much smaller
        stripe depth.

    My actual code changes were limited to:

          src/H5FDioc.c
          src/H5FDioc_threads.c
          src/H5FDsubfile_int.c
          src/H5FDsubfile_mpi.c
          src/H5FDsubfiling.c
          src/H5FDsubfiling.h
          src/H5FDsubfiling_priv.h
          testpar/t_subfiling_vfd.c
          testpar/t_vfd.c

    I'm not sure what is going on with the deletions in src/mercury/src/util.

    Tested parallel/debug on Charis and Jelly

* subfiling with selection IO (#1219)

Merged branch 'selection_io' into subfiling branch.

* Subfile name fixes (#1250)

* fixed subfiling naming convention, and added leading zero to rank names.

* Merge branch 'selection_io' into selection_io_with_subfiling_vfd (#1265)

* Added script to join subfiles into a single HDF5 file (#1350)

* Modified  H5FD__subfiling_query() to report that the sub-filing VFD supports MPI
This exposed issues with truncate and get EOF in the sub-filing VFD.
I believe I have addressed these issues (get EOF not as fully tested as it should be), howeer,
it exposed race conditions resulting in hangs.  As of this writing, I have not been able
to chase these down.

Note that the tests that expose these race conditions are in testpar/t_subfiling_vfd.c, and
are currently skipped.  Unskip these tests to reproduce the race conditions.

tested (to the extent possible) debug/parallel on charis and jelly.

* Committing clang-format changes

* fixed H5MM_free

Co-authored-by: mainzer <mainzer#hdfgroup.org>
Co-authored-by: jrmainzer <72230804+jrmainzer@users.noreply.github.com>
Co-authored-by: Richard Warren <Richard.Warren@hdfgroup.org>
Co-authored-by: Richard.Warren <richard.warren@jelly.ad.hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Move Subfiling VFD components into H5FDsubfiling source directory

* Update Autotools build and add H5_HAVE_SUBFILING_VFD macro to H5pubconf.h

* Tidy up CMake build of subfiling sources

* Merge branch 'develop' into feature/subfiling (#1539)

Merge branch 'develop' into feature/subfiling

* Add VFD interface version field to Subfiling and IOC VFDs

* Merge branch 'develop' into feature/subfiling (#1557)

Merge branch 'develop' into feature/subfiling

* Merge branch 'develop' into feature/subfiling (#1563)

Merge branch 'develop' into feature/subfiling

* Tidy up merge artifacts after rebase on develop

* Fix incorrect variable in mirror VFD utils CMake

* Ensure VFD values are always defined

* Add subfiling to CMake VFD_LIST if built

* Mark MPI I/O driver self-initialization global as static

* Add Subfiling VFD to predefined VFDs for HDF5_DRIVER env. variable

* Initial progress towards separating private vs. public subfiling code

* include libgen.h in t_vfd tests for correct dirname/basename

* Committing clang-format changes

* removed mercury option, included subfiling header path (#1577)

Added subfiling status to configure output, installed h5fuse.sh to build directory for use in future tests.

* added check for stdatomic.h (#1578)

* added check for stdatomic.h with subfiling
* added H5_HAVE_SUBFILING_VFD for cmake

* fix old-style-definition warning (#1582)

* fix old-style-definition warning

* added test for enable parallel with subfiling VFD (#1586)

Fails if subfiling VFD is not used with parallel support.

* Subfiling/IOC VFD fixes and tidying (#1619)

* Rename CMake option for Subfiling VFD to be consistent with other VFDs

* Miscellaneous Subfiling fixes

Add error message for unset MPI communicator

Support dynamic loading of subfiling VFD with default configuration

* Temporary fix for subfile name issue

* Added subfile checks (#1634)

* added subfile checks

* Feature/subfiling (#1655)

* Subfiling/IOC VFD cleanup

Fix misuse of MPI_COMM_WORLD in IOC VFD

Propagate Subfiling FAPL MPI settings down to IOC FAPL in default
configuration case

Cleanup IOC VFD debugging code

Change sprintf to snprintf in a few places

* Major work on separating Subfiling and IOC VFDs from each other

* Re-write async_completion func to not overuse stack

* Replace usage of MPI_COMM_WORLD with file's actual MPI communicator

* Refactor H5FDsubfile_mpi.c

* Remove empty file H5FDsubfile_mpi.c

* Separate IOC VFD errors to its own error stack

* Committing clang-format changes

* Remove H5TRACE macros from H5FDioc.c

* Integrate H5FDioc_threads.c with IOC error stack

* Fix for subfile name generation

Use number of I/O concentrators from existing subfiling configuration file, if one exists

* Add temporary barrier in "Get EOF" operation to prevent races on EOF

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Fix for retrieval of machine Host ID

* Default to MPI_COMM_WORLD if no MPI params set

* added libs rt and pthreads (#1673)

* added libs rt and pthreads

* Feature/subfiling (#1689)

* More tidying of IOC VFD and subfiling debug code

* Remove old unused log file code

* Clear FID from active file map on failure

* Fix bug in generation of subfile names when truncating file

* Change subfile names to start from 1 instead of 0

* Use long long for user-specified stripe size from environment variable

* Skip 0-sized I/Os in low-level IOC I/O routines

* Don't update EOF on read

* Convert printed warning about data size mismatch to assertion

* Don't add base file address to I/O addresses twice

Base address should already be applied as part of H5FDwrite/read_vector calls

* Account for 0-sized I/O vector entries in subfile write/read functions

* Rewrite init_indep_io for clarity

* Correction for IOC wraparound calculations

* Some corrections to iovec calculations

* Remove temporary barrier on EOF retrieval

* Complete work request queue entry on error instead of skipping over

* Account for stripe size wraparound for sf_col_offset calculation

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Re-write and fix bugs in I/O vector filling routines (#1703)

* Rewrite I/O vector filling routines for clarity

* Fix bug with iovec_fill_last when last I/O size is 0

* added subfiling_dir line read (#1714)

* added subfiling_dir line read and use it
* shellcheck fixes

* I/O request dispatch logic update (#1731)

Short-circuit I/O request dispatch when head of I/O queue is an
in-progress get EOF or truncate operation. This prevents an issue where
a write operation can be dispatched alongside a get EOF/truncate
operation, whereas all I/O requests are supposed to be ineligible for
dispatch until the get EOF/truncate is completed

* h5fuse.sh.in clean-up (#1757)

* Added command-line options

* Committing clang-format changes

* Align with changes from develop

* Mimic MPI I/O VFD for EOF handling

* Initialize context_id field for work request objects

* Use logfile for some debugging information

* Use atomic store to set IOC ready flag

* Use separate communicator for sending file EOF data

Minor IOC cleanup

* Use H5_subfile_fid_to_context to get context ID for file in Subfiling
VFD

* IOVEC calculation fixes

* Updates for debugging code

* Minor fixes for threaded code

* Committing clang-format changes

* Use separate MPI communicator for barrier operations

* Committing clang-format changes

* Rewrite EOF routine to use nonblocking MPI communication

* Committing clang-format changes

* Always dispatch I/O work requests in IOC main loop

* Return distinct MPI communicator to library when requested

* Minor warning cleanup

* Committing clang-format changes

* Generate h5fuse.sh from h5fuse.sh.in in CMake

* Send truncate messages to correct IOC rank

* Committing clang-format changes

* Miscellaneous cleanup

Post some MPI receives before sends

Free some duplicated MPI communicator/Info objects

Remove unnecessary extra MPI_Barrier

* Warning cleanup

* Fix for leaked MPI communicator

* Retrieve file EOF on single rank and bcast it

* Fixes for a few failure paths

* Cleanup of IOC file opens

* Committing clang-format changes

* Use plan MPI_Send for send of EOF messages

* Always check MPI thread support level during Subfiling init

* Committing clang-format changes

* Handle a hang on failure when IOCs can't open subfiles

* Committing clang-format changes

* Refactor file open status consensus check

* Committing clang-format changes

* Fix for MPI_Comm_free being called after MPI_Finalize

* Fix VFD test by setting MPI params before setting subfiling on FAPL

* Update Subfiling VFD error handling and error stack usage

* Improvements for Subfiling logfiles

* Remove prototypes for currently unused routines

* Disable I/O queue stat collecting by default

* Remove unused serialization mutex variable

* Update VFD testing to take subfiling VFD into account

* Fix usage of global subfiling application layout object

* Minor fixes for failure pathways

* Keep track of the number of failures in an IOC I/O queue

* Make sure not to exceed MPI_TAG_UB value for data communication messages

* Committing clang-format changes

* Update for rename of some H5FD 'ctl' opcodes

* Always include Subfiling's public header files in hdf5.h

* Remove old unused code and comments

* Implement support for per-file I/O queues

Allows the subfiling VFD to have multiple HDF5 files open simultaneously

* Use simple MPI_Iprobe over unnecessary MPI_Improbe

* Committing clang-format changes

* Update HDF5 testing to query driver for H5FD_FEAT_DEFAULT_VFD_COMPATIBLE
flag

* Fix a few bugs related to file multi-opens

* Avoid calling MPI routines if subfiling gets reinitialized

* Fix issue when files are closed in a random order

* Update HDF5 testing to query VFD for "using MPI" feature flag

* Register atexit handler in subfiling VFD to call MPI_Finalize after HDF5
closes

* Fail for collective I/O requests until support is implemented

* Correct VOL test function prototypes

* Minor cleanup of old code and comments

* Update mercury dependency

* Cleanup of subfiling configuration structure

* Committing clang-format changes

* Build system updates for Subfiling VFD

* Fix possible hang on failure in t_vfd tests caused by mismatched
MPI_Barrier calls

* Copy subfiling IOC fapl in "fapl get" method

* Mirror subfiling superblock writes to stub file for legacy POSIX-y HDF5
applications

* Allow collective I/O for MPI_BYTE types and rank 0 bcast strategy

* Committing clang-format changes

* Use different scheme for subfiling write message MPI tag calculations

* Committing clang-format changes

* Avoid performing fstat calls on all MPI ranks

* Add MPI_Barrier before finalizing IOC threads

* Use try_lock in I/O queue dispatch to minimize contention from worker threads

* Use simple Waitall for nonblocking I/O waits

* Add configurable IOC main thread delay and try_lock option to I/O queue dispatch

* Fix bug that could cause serialization of non-overlapping I/O requests

* Temporarily treat collective subfiling vector I/O calls as independent

* Removed unused mercury bits

* Add stubs for subfiling and IOC file delete callback

* Update VFD testing for Subfiling VFD

* Work around HDF5 metadata cache bug for Subfiling VFD when MPI Comm size
= 1

* Committing clang-format changes

Co-authored-by: mainzer <mainzer#hdfgroup.org>
Co-authored-by: Neil Fortner <nfortne2@hdfgroup.org>
Co-authored-by: Scot Breitenfeld <brtnfld@hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: jrmainzer <72230804+jrmainzer@users.noreply.github.com>
Co-authored-by: Richard Warren <Richard.Warren@hdfgroup.org>
Co-authored-by: Richard.Warren <richard.warren@jelly.ad.hdfgroup.org>
2022-07-22 13:03:12 -07:00
Dana Robinson
32caa567a2 Updates the release schedule (#1923)
1.13.2 moved to August
Subfiling moved to 1.13.2 from 1.13.3
2022-07-22 05:12:27 -07:00
Dana Robinson
1cfca8fd93 Fixes an incorrect brief description for H5Dopen2() (#1920) 2022-07-21 06:39:05 -07:00
Dana Robinson
eb2cf08bac Fix for MSVC compile failures in h5dumpgentest due to VLA use (#1916) 2022-07-21 01:34:41 -07:00
Dana Robinson
363e3ecef8 Removes the ability to build h5perf and h5perf_serial in standalone mode (#1897)
This feature has been broken for some time and lacks a clear use case.
2022-07-20 05:38:07 -07:00
Neil Fortner
4461678fb5 Fix bug in swmr.c introduced with recent merge (double_t) (#1913)
* Fix HDFFV-11308 - dataset access properties disappear when using
H5Fstart_swmr_write.

* Committing clang-format changes

* Add test for H5Fstart_swmr_write() persisting DAPL settings.  Fix bugs
in the library exposed by this test.

* Committing clang-format changes

* Replace accidental use of double_t in swmr.c with double.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-19 14:19:02 -07:00
Dana Robinson
96c4e0521c Marks H5Fformat_convert as internal in Doxygen (#1910) 2022-07-19 09:30:07 -07:00
Dana Robinson
8225778fae Updates H5Zdevelop.h with better memory alloc/free advice (#1909)
* Updates H5Zdevelop.h with better memory alloc/free advice

Addresses GitHub #348

* Updates the filter memory calloc/free docs
2022-07-19 08:08:20 -07:00
Neil Fortner
5c39998224 Persist dataset access properties when using H5Fstart_swmr_write (#1862)
* Fix HDFFV-11308 - dataset access properties disappear when using
H5Fstart_swmr_write.

* Committing clang-format changes

* Add test for H5Fstart_swmr_write() persisting DAPL settings.  Fix bugs
in the library exposed by this test.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-18 18:15:48 -07:00
Dana Robinson
e3800e1235 Greatly simplifies Autotools bits of autogen.sh (#1906) 2022-07-18 14:23:29 -05:00
jhendersonHDF
43f052352c Remove perf tool from CMakeTests file (#1907) 2022-07-18 14:23:10 -05:00
jhendersonHDF
ba5385ffc2 Fix some hard-coded MPI I/O VFD checks (#1898) 2022-07-18 10:32:09 -07:00
Scot Breitenfeld
69ddcd02f1 Removed compiler messages for language elements that are not standard in Fortran 2003, #1344 (#1905) 2022-07-18 09:42:56 -07:00
Dana Robinson
f4c9931abd Updates comments in the gcc 4.8 warnings files (#1902) 2022-07-18 07:36:26 -07:00
Dana Robinson
079df9d216 Fixes a stack overflow in the h5dump test generator (#1903)
* Fixes a stack overflow in the h5dump test generator

The opaque type size was declared to be the size of the "buffer of two
opaque elements", causing a memcpy call in H5Dwrite to read outside
the buffer, dumping garbage into the file.

Note that this only affected the test generator and not h5dump itself.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-18 07:35:43 -07:00
Dana Robinson
08b797de73 Fixes warnings when the deflate filter is not built (#1904)
* Fixes warnings when the deflate filter is not built

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-18 07:35:21 -07:00
Dana Robinson
9b53b2f6ca Adds Autotools dependencies to CodeQL workflow 2022-07-17 04:14:59 -07:00
Dana Robinson
c9785cc5fc Set up CodeQL analysis for develop
This will be reverted if it causes problems
2022-07-17 04:09:40 -07:00
Dana Robinson
8c6a3ce1d7 Removes the small perf tool (#1896)
* Removes the small perf tool

This tool doesn't really do anything special and installed, which
conflicts with gnu's perf tool.

* Adds suggestions from code review
2022-07-16 22:21:18 -07:00
Dana Robinson
65a24325af Updates branches-explained.md (#1900) 2022-07-15 16:26:50 -07:00
Dana Robinson
7c9e725134 Create CODE_OF_CONDUCT.md
Taken directly from GitHub's suggested CoC document. We may modify this in the future.
2022-07-14 09:37:57 -07:00
Allen Byrne
6be07134de Add utility JNI function for 1.10 style references (#1888)
* Add utility JNI function for 1.10 style references

* Clarify text

* Correct signature

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-14 09:33:35 -07:00
vchoi-hdfgroup
c835d16a59 Remove superfluous dot from hl doxygen. (#1891)
Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
2022-07-14 09:32:09 -07:00
Allen Byrne
385ae6ac5b Eliminate unnecessary error output (#1879)
* Eliminate unnecessary error output

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-14 09:07:53 -07:00
Dana Robinson
b0d9fa058a Fix for a tools compile issue when deprecated symbols are disabled (#1884)
* Fix for a tools compile issue when deprecated symbols are disabled

Introduced in #1811, also uses VOL token instead of addr

* Fix for similar compile issues in java when deprecated symbols are
disabled.

* Committing clang-format changes

Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-13 08:53:00 -07:00
raylu-hdf
b2363a8195 H5Oflush fails for parallel (#1876)
* H5Oflush causes H5Fclose to trigger an assertion failure in metadata cache for parallel.  This commit makes sure H5Oflush fails for parallel until this problem is solved in the future.

* Committing clang-format changes

* Changed the use of H5F_get_driver_id to H5F_HAS_FEATURE.

Co-authored-by: songyulu <songyulu@jelly.ad.hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-12 08:55:34 -07:00
Dana Robinson
c62b026609 Moves -Wstrict-overflow=5 to the developer warnings (#1878) 2022-07-11 18:25:57 -07:00
Dana Robinson
af014fa61c Removes definitions for (U)LLONG_MAX and (S)SIZET_MAX (#1877)
LLONG_MAX and ULLONG_MAX are defined in limits.h in C99. SIZET_MAX
should be SIZE_MAX, which has been around forever. SSIZET_MAX should
be SSIZE_MAX, which is defined whereever ssize_t is found. I've kept
the definition for SSIZE_MAX (renamed from SSIZET_MAX) for platforms
where ssize_T is not present (e.g., Windows).
2022-07-11 15:48:51 -07:00
Dana Robinson
dae6064103 Fixes for warnings identified by MSVC (#1874)
* Fixes for warnings identified by MSVC

* Committing clang-format changes

* Moves the casts in H5Binternal.c to the other side

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-11 14:01:08 -07:00
Allen Byrne
b9d1d66ab8 Develop HDFFV-11310 (#1811)
* Rework java vl check in read/write. Handle old refs in h5dump

* Committing clang-format changes

* Java changes allow default VL reads

* Fix JNI utility for old refs

* HDFFV-11310 - implement vlen read/write for atomic types.

* format check fix

* Mostly format fixes

* More format issues

* Two format changes

* Use JNI names for sizeof

* format change

* fix size typo

* Change to older method to initialize list

* remove unused var

* format fix

* switch writeVL functions to use datatype instead of java type

* Add VL option to generic read/write check

* Correct function name typo

* Add JIRA issue

* Correct note to match change in code.

* HDFFV-11318 add VL references as byte arrays

* Add release note and format changes

* Another format update

* Fix unreleased allocations

* Fix format

* format correction

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-11 13:59:52 -07:00
Houjun Tang
663321087a Support for UnifyFS with MPI_File_sync (#1801)
* Initial implementation for supporting UnifyFS in HDF5 with MPI_File_sync after write

* Committing clang-format changes

* Fix format

* Fix env variable and return value check

* Fix flag retrieve

* Fix issues with getting/setting the flag

* Fix merge conflicts

* Update

* Committing clang-format changes

* Update based on suggestions

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-11 13:59:19 -07:00
Dana Robinson
63ce6839b5 Reduce -Wstrict-overflow= from 5 to 2 (#1872)
The signal-to-noise ratio of the higher warning level is very low
and the noise obscures things we should fix
2022-07-11 11:26:03 -07:00
Dana Robinson
e269f90364 Misc clang warning fixes (#1869)
* Minor clang warning fixes in src and test

* Fixes clang warnings in high-level library

* clang format-nonliteral warning fixes

* Committing clang-format changes

* Fixed int -> float clang cast warnings in h5diff

* Quiets some warnings in flex/bison generated code

* Suppress overlong string warning for libinfo on clang

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-11 10:28:02 -07:00
Dana Robinson
0412d3f292 Fixes for production mode gcc warnings (#1873)
* Fixes for production mode gcc warnings

With the strict-overflow changes, this brings the number of warnings
in the C library w/ gcc 12 to zero.

* Fix typo

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-11 10:27:40 -07:00
Dana Robinson
fa7caf8435 Fixes C++ sign-conversion warnings w/ clang 14 (#1870)
* Fixes sign-conversion warnings w/ clang 14

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-11 08:59:51 -07:00
Dana Robinson
f599e2ac7f Adds the mirror and direct VFDs to GitHub actions (#1871)
Both Autotools and CMake
2022-07-11 08:58:06 -07:00
Dana Robinson
460ee73a64 Adds format attribute to additional functions (#1868)
* Adds format attribute to (s|p)io_perf code

* Added gcc format attribute to additional test code

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-11 08:29:46 -07:00
Dana Robinson
be5de999a9 Adds explicit indices to drivernames array in the tools (#1867)
Will help keep the drivernames and driver_idx enum in sync.
2022-07-11 08:28:52 -07:00
Dana Robinson
e0c2443179 Adds the malloc attribute to appropriate functions (gcc only) (#1866) 2022-07-11 08:27:44 -07:00
Dana Robinson
ff089f8e36 Updates the patterns in Doxyfile.in to accept H5*develop.h (#1865) 2022-07-11 06:35:46 -07:00
Gerd Heber
7e1bde1cb5 Merge pull request #1857 from gheber/fix-doxy-warnings-and-cosmetics
Fix Doxygen warnings and cosmetics
2022-07-08 20:16:43 -05:00
jhendersonHDF
aa688c68c4 Convert assertion on (possibly corrupt) file contents to normal error check (#1861)
* Convert assertion on (possibly corrupt) file contents to normal error check

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-08 13:04:23 -07:00
Dana Robinson
a45ade8886 Fix for memory allocation issue identified by fuzzer (#732) (#1860)
GitHub issue #732 claims that mallocing a particular buffer can
trigger undefined behavior and that callocing the buffer fixes
the issue.

I did not confirm this, but the fix is trivial and better anyway.
2022-07-08 12:10:29 -07:00
Gerd Heber
6bc41fb11c Resolve Allen's comment. 2022-07-08 12:48:48 -05:00
Gerd Heber
74c80f0bbf Fix Doxygen warnings 2022-07-08 06:08:46 -05:00
Gerd Heber
52d6626254 Cosmetic changes to RM front 2022-07-08 06:06:47 -05:00
Gerd Heber
fe413fc6ff Updated vfd swmr and reference rfcs (#1854)
* Add PDFs

* Update RFC table
2022-07-07 15:59:48 -07:00
Dana Robinson
0c0d3bf736 Suppress the "obsolete" category of Autoconf warnings (#1853)
Very new versions of Autoconf complain about our Java macros. Since they
work well enough and there is no obvious upgrade path, we're suppressing
the category for now.
2022-07-06 14:56:26 -07:00
Dana Robinson
424bbd16eb Updates messages in cache sanity checks (#1851)
Previous messages were numbered, which was a maintenance burden,
and non-specific
2022-07-06 13:12:48 -07:00
Dana Robinson
a7f1ce46b9 Minor tweaks to the test_swmr.sh script (#1852) 2022-07-06 13:12:31 -07:00
Gerd Heber
9cd0d042f0 Update Doxyfile.in (#1849) 2022-07-06 13:01:36 -07:00
jhendersonHDF
8937054d5a Fix invalid comment about character encoding in H5Fint.c (#1845) 2022-07-01 15:49:23 -07:00
Dana Robinson
6aa7aeeda4 Fixes a bug in h5perf that causes file open failures (#1846)
When the filename was flipped from being a static array to being
dynamically allocated, the sizeof(filename) call wasn't updated.
This always returns the size of the pointer, truncating the
filename.
2022-07-01 12:53:45 -07:00
vchoi-hdfgroup
9e97a22b8b Add doxygen for high level modules. (#1839)
* Add doxygen for high level modules.

* Committing clang-format changes

* Correct spelling based on spelling check.

* Committing clang-format changes

Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-07-01 08:42:15 -07:00
Dana Robinson
3ed53d17fb Removes workaround for platforms that lack C99 stuct initializers (#1842) 2022-06-30 19:50:42 -07:00
Dana Robinson
41526f68f3 Adds a -Werror debug Autotools test and updates build modes (#1833)
* Adds a -Werror debug Autotools test and updates build modes

* Un-excludes the workflow files

* Strip out -Werror from CFLAGS when running configure tests

This causes spurious test failures when the configuration test
programs raise warnings.
2022-06-30 15:38:18 -07:00
Dana Robinson
015240e790 Minor configure comment tweaks from autoupdate (#1838)
The autoupdate script identified some cruft that was unchanged
in the comments
2022-06-30 12:26:38 -07:00
Dana Robinson
552833213c Fix for gcc-12 warning about string truncation in H5Opline.c (#1835) 2022-06-30 07:47:34 -07:00
Dana Robinson
72db748dd9 Updates the MinGW configure.ac check for the Winsock library (#1837)
Ubuntu 22.04 complains about AC_HAVE_LIBRARY being deprecated. This
updates the macro to AC_CHECK_LIB, where the specified function
is GetUserName().

This change only affects MinGW
2022-06-30 06:59:03 -07:00
Dana Robinson
89fae896ae Set Autotools build mode correctly in GitHub workflows (#1834)
* Sets build mode correctly w/ Autotools

* Updates GitHub gcc and installs OpenMPI
2022-06-30 06:58:37 -07:00
Dana Robinson
03ed7a3d79 Fixes a few minor parallel warnings (#1832) 2022-06-29 13:02:38 -07:00
Scot Breitenfeld
cb424b2c45 remove superfluous initialization, fixes issue with Cray compiler (Crusher) (#1828) 2022-06-29 10:44:21 -05:00
Dana Robinson
9e00089307 Quiets const warnings (#1831) 2022-06-29 10:41:45 -05:00
Dana Robinson
57a850f897 Documents Windows badness in H5Pget_external() (#1821)
* Documents Windows badness in H5Pget_external()

* Added a cast for off_t
2022-06-24 06:14:29 -07:00
Dana Robinson
249008d4ce Converts testhdf5 macros to h5test macros in chunk_info.c (#1820)
The two macro schemes were not designed to work together. Also
quiets some MSVC warnings about comparing pointers and integers.
2022-06-23 09:11:30 -07:00
Dana Robinson
1a0fe1f6b7 Set a constant in H5SM to UINT_MAX instead of -1U (#1822) 2022-06-23 09:11:04 -07:00
Dana Robinson
896304d2da Work-around for MSVC issue with flex-generated code (#1823) 2022-06-23 07:45:47 -07:00
Dana Robinson
9e55d51f46 Quiets size warning in large library setting string (#1818)
* Quiets size warning in large library setting string

If the size of this string is a problem, people can build with the
embedded library info disabled.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-06-22 12:44:34 -07:00
Sean McBride
c064d3481b sprintf to snprintf (#1815)
* Straightforward conversion of sprintf to the safer snprintf

* Trickier conversion of sprintf to safer snprintf

This involved minor changes to private function signatures to take the size of the buffer.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-06-22 11:58:18 -07:00
Dana Robinson
d6f05069c1 Adds notes about off_t sizes on Windows (#1817) 2022-06-22 11:21:51 -07:00
Dana Robinson
8bfed8aae4 Converts __int64 to LONGLONG in LARGE_INTEGER union members on Windows (#1816) 2022-06-22 10:48:51 -07:00
Sean McBride
8b2e7b32b5 Various warning fixes (#1812)
* Fixed -Wreserved-id-macro warnings from header include guards

* Removed all __int64 and LL suffix stuff now that C99 is minimum requirement

* Rename `H5FD_CTL__` to `H5FD_CTL_` to fix -Wreserved-id-macro warnings

Double underscore is reserved in C++ and this public header should be C++ compatible.

* Never define __STDC_FORMAT_MACROS anymore

Defining it causes a -Wreserved-id-macro.

Happily, according to the C++11 standard:

"The macros defined by <stdint> are provided unconditionally. In particular, the symbols __STDC_LIMIT_MACROS and __STDC_CONSTANT_MACROS (mentioned in C99 footnotes 219, 220, and 222) play no role in C++."

https://cplusplus.github.io/LWG/issue984

So looks like it's not necessary to define it with reasonably new toolchains.

* Fixed some -Wunused-macros warnings, removed dead code

* Fixed all -Wdouble-promotion warnings in C++ files

* Fixed remaining -Wsuggest-destructor-override warnings

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-06-22 09:16:30 -07:00
Neil Fortner
55f4cc0caa Fix bugs in parallel selection I/O (#1803) 2022-06-09 16:01:47 -05:00
Allen Byrne
8b54201509 Develop sync (#1793)
* Synch update

* Branch sync changes
2022-06-09 08:35:33 -05:00
Dana Robinson
acb6e2da30 Updates the release schedule (#1796) 2022-06-06 09:31:41 -05:00
Allen Byrne
a124f02f2c Update windows VS2019 and fix comments (#1781) 2022-05-20 16:41:39 -07:00
Larry Knox
ca9bbc1908 Remove extra ')'. (#1779)
Add configure help text for TestExpress.
2022-05-19 09:49:08 -05:00
Allen Byrne
081b3902bf Set default level to minimum time (#1722) 2022-05-17 11:54:05 -05:00
Allen Byrne
e7b184e79e Replace obsolete docver arg with arg to supply branch name (#1771) 2022-05-13 18:00:46 -05:00
Dana Robinson
a30342e1d7 Allows parallel + thread-safe combination in Autotools and CMake (#1760) 2022-05-12 21:53:45 -05:00
Allen Byrne
b9dc83934d Use windows ENV var for default install dir setting (#1763)
* Use windows ENV var for default install dir setting

* Change variable name
2022-05-12 09:03:32 -05:00
Dana Robinson
d0ea8a1d8c Revert "Cleanup and shellcheck fixes in the SWMR test script (#1741)" (#1758)
This reverts commit cea0dc4565.
2022-05-10 09:42:22 -07:00
David Young
110f6a1c7d Add a workaround for buggy PGI C Compiler version 19.10, which quits (#1705)
with the unhelpful and misleading error message `PGC-S-0053-Illegal use
of void type` when I put a `static` qualifier on my tables.
2022-05-07 19:04:33 -07:00
Allen Byrne
01b7165df5 Use git command to get branch name (#1748) 2022-05-07 18:19:21 -05:00
Dana Robinson
cea0dc4565 Cleanup and shellcheck fixes in the SWMR test script (#1741) 2022-05-07 12:52:41 -07:00
David Young
779ab44aa0 Re-promote some GNU warnings to errors (#1508)
* Promote some GCC warnings back to errors and deal with build breakage.
I tested configuring and building with GCC 8.3.0 with

`.../configure --prefix=$HOME/wrk/install/manydsets --enable-build-mode=production --enable-symbols=-gdwarf-4 --enable-warnings-as-errors --enable-java --enable-hl --enable-fortran --enable-cxx --enable-mirror-vfd --enable-ros3-vfd --enable-direct-vfd`

and with the same options but `--enable-build-mode=debug`.

* Promote -Wrestrict to -Werror=restrict to help catch overlapping
arguments to strcpy, memcpy, et cetera, at compile time.

* NFCI: sort lines.

* Committing clang-format changes

* NFCI: remove whitespace at EOL.

* Re-use temporary variable `shared`.

* Remove merge oopsie.

* Mention that no op codes are understood per review comment.

* Change this back to the develop branch's way, since this change isn't
integral to the PR.

* Committing clang-format changes

* NFCI: reduce differences with `develop` branch in the hope that that
also kicks off CI.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
2022-05-06 16:26:48 -07:00
Allen Byrne
ca737ece9a Change order of naming commands (#1742) 2022-05-06 13:42:05 -07:00
Dana Robinson
1b413fe61a Minor things noticed while merging to 1.10 (#1739)
* Minor things noticed while merging to 1.10

* Set HDF5_DRIVER back in string
2022-05-06 11:06:52 -07:00
jhendersonHDF
13985a7f52 Fixes for S3 and HDFS VFDs with tools (#1740) 2022-05-06 11:06:07 -07:00
jhendersonHDF
cdf837d0a1 Fix a few Clang sanitizer warnings (#1727) 2022-05-05 13:32:49 -07:00
Scot Breitenfeld
c4cd250408 changes parallel tests to .F90 ext. and fixes the CALL check strings (#1725) 2022-05-04 16:29:49 -07:00
Dana Robinson
838d050a63 h5test.h testing macros get enclosed in do..while loops (#1721)
* h5test.h testing macros get enclosed in do..while loops

* Adds missed macro in hl C++ code

* Fixes macro in Windows code
2022-05-04 08:49:01 -07:00
Dana Robinson
19a59c29b9 Fixes a bug where t_cache fails due to a string size being too small (#1720)
* Fixes a bug where t_cache fails due to a string size being too small

Recent warning reductions led to an incorrect string size being passed
to h5_fileaccess, causing the test to silently fail. In addition to
fixing the bug, the test will now fail noisily on setup failures.

* Updates the t_cache test to fail noisily on setup errors
2022-05-04 07:53:13 -07:00
Dana Robinson
08861aa903 Fixes unused/unset variable warnings from aocc (#1712)
* Fixes unused/unset variable warnings from aocc

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-05-02 19:25:57 -05:00
Dana Robinson
dad94a100e Updates warnhist script to deal with AMD Optimizing Compiler messages (#1713) 2022-05-02 19:22:07 -05:00
Ben Boeckel
b3ef8044cf test/dt_arith: use isnan instead of self comparison (#1709)
It is more obvious what is being asked this way.

Also include a `clang-format` fix.
2022-05-02 19:20:51 -05:00
jhendersonHDF
2ba7a01bff Fix some const cast and stack/static object size warnings (#1700)
* Fix various warnings

* Move HDfree_const to H5private.h for wider use

* Print output from all ranks in parallel tests on allocation failure

* Move const pointer freeing macro to h5test.h for now
2022-05-01 15:54:41 -05:00
jhendersonHDF
5eba258d90 Fix object size warnings in cache.c test (#1701) 2022-05-01 15:54:22 -05:00
jhendersonHDF
7e586e0e44 Fix SWMR/refresh bug hidden by library free lists (#1702) 2022-04-30 08:35:45 -05:00
David Young
c4d70e3571 Be a bit safer with signed arithmetic, thus quieting some signed-overflow warnings from GCC (#1706)
* Avoid a signed overflow: check the range of `entry_ptr->age` before
increasing it instead of increasing it and then checking the range.
This quiets a GCC warning.

* Avoid the potential for signed overflow by rewriting expressions
`MAX(0, fwidth - n)` as `MAX(n, fwidth) - n` for various `n`.
This change quiets some GCC warnings.

* Change some local variables that cannot take sensible negative values
from signed to unsigned.  This quiets GCC warnings about potential
signed overflow.

* In a handful of instances, check the range of a signed integer before
increasing/decreasing it, just in case the increase/decrease overflows.
This quiets a handful of GCC signed-overflow warnings.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-04-29 22:29:20 -05:00
Dana Robinson
ffd311cf36 Fixes stack size warnings in set_extent test (#1698) 2022-04-28 16:04:23 -07:00
Dana Robinson
180a7f9c71 Fixes stack size warnings in dtransform (#1696) 2022-04-26 21:52:00 -05:00
Dana Robinson
0ea3be4dc1 Fixes stack size warnings in ntypes (#1695) 2022-04-26 13:18:52 -05:00
jhendersonHDF
d7565f316d Address some warnings from casting away of const (#1684) 2022-04-26 13:18:18 -05:00
Quincey Koziol
417ee1393b Correct concurrency bugs when running tests, along with a bugfix & small warning cleanup (#1683)
* Correct concurrency bugs when running tests, along with a bugfix & small
warning cleanup.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-04-26 11:07:15 -05:00
Dana Robinson
0fffb26c01 Minor things noticed while bringing VFD SWMR in line with develop (#1691) 2022-04-25 16:29:00 -05:00
Dana Robinson
0fba6cbcda Fixes stack size warnings in tcoords.c (#1688) 2022-04-25 12:17:48 -05:00
Dana Robinson
336ead720e Clean stack size warnings in sio_engine (#1687) 2022-04-25 12:17:33 -05:00
jhendersonHDF
2b323159a7 Warnings fixes (#1680) 2022-04-25 12:16:18 -05:00
Quincey Koziol
c8b87fa320 Improve handling of copying of dynamic libraries and clean them up after (#1681)
test finishes.
2022-04-24 09:59:12 -05:00
David Young
a6876a9745 Replace H5detect's build-time detection of C99 integer properties with a (#1400)
* Replace H5detect's build-time detection of C99 integer properties with a
table-driven routine, `H5T__init_native_int()`, that is run at library
initialization time.
2022-04-22 08:52:01 -05:00
Dana Robinson
ac9c21dd6f Changes Github action hdf5 dev CI to PR hdf5 dev CI (#1666)
So the PR action name is not the same as the one in main.yml
2022-04-20 18:43:07 -05:00
Dana Robinson
2fcfc42d67 Adds paths-ignore to the Github pull request workflow (#1663) 2022-04-20 18:42:25 -05:00
Dana Robinson
822fa43b12 Adds the -q flag to all swmr test programs, quieting noisy output (#1665) 2022-04-20 14:51:52 -07:00
Scot Breitenfeld
823bc7381c added HDFFV-11306 entry (#1662) 2022-04-19 16:31:09 -07:00
Allen Byrne
03c3a54695 Develop clang format java (#1653) 2022-04-19 13:08:09 -05:00
Scot Breitenfeld
32ef796e47 HDFFV-11306 Fixed (#1657)
* HDFFV-11306,
 * Fixed it so both h5open_f and h5close_f can be called multiple times.
 * Fixed an issue with open objects remaining after h5close_f was called.
 * Added additional tests.

* comments clean-up
2022-04-19 13:05:54 -05:00
Dana Robinson
14da68ebff Removes HDF Group paths, adds shellcheck fixes (#1656)
For more information:
  https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ...
  https://www.shellcheck.net/wiki/SC2230 -- which is non-standard. Use builti...
  https://www.shellcheck.net/wiki/SC2006 -- Use $(...) notation instead of le...
2022-04-19 07:44:41 -05:00
Quincey Koziol
f875a73711 Mirror vfd test fixes (#1629)
* Use the FAPL that was created earlier in the test (and delete an unused
variable).   This allows 'make check-vfd' to pass with --enable-mirror-vfd.

* Check for testing directory before creating, to avoid warning from bash.
Clean out .libs directory before re-using it (after a failed test), to
remove any files generated by libtool.

* Committing clang-format changes

* Increment error count on failed file open and skip tests for VFDs that need
modified filenames.

* Skip the mirror VFD for 'make check-vfd' - the mirror VFD requires networking
configuration parameters and can't be provided for an automated test that
is configured with an environment variable.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-04-18 15:28:20 -05:00
Dana Robinson
17be6c2574 Cleans up some HL library code that inappropriately returns htri_t values cast to herr_t (#1651)
* Cleans up some HL library code that inappropriately returns
htri_t values cast to herr_t

* Committing clang-format changes

* Formatted source

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-04-18 07:29:00 -05:00
David Young
a80897ee49 Remove H5_NO_ALIGNMENT_RESTRICTIONS (#1426)
* Do not conditionally compile code that uses a pointer dereference
and assignment to copy a potentially unaligned variable to aligned
automatic storage, or vice versa.  Instead, always use naked `memcpy(3)`s.
Disassembling the generated code reveals that the `memcpy(3)`s optimize
(`-O3`) to a single `mov` instruction for x86_64, which is not strict
about alignment.

This change reduces the size of code and scripts by 143 lines, eases
our way to cross-compilation, and avoids invoking undefined behavior.

* Committing clang-format changes

* Per discussion, use HD and add comments.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-04-16 10:21:18 -05:00
H. Joe Lee
463ef39bc7 OESS-168: Remove clang warnings. (#1376) 2022-04-15 13:19:21 -07:00
H. Joe Lee
9131d5289c OESS-168: Remove clang warnings. (#1309)
* OESS-168: Remove clang warnings.

* OESS-168: Address @lrknox review.
2022-04-15 13:16:37 -07:00
Wei-keng Liao
aaa6614c48 Retrieve MPI-IO hints used by MPI library after file open (#1636)
H5Pget_fapl_mpio() should return an MPI info object containing all the
MPI-IO hints used by the MPI library underneath, after the file is
opened. Some hints, such as cb_nodes (number of I/O aggregators), are
useful for HDF5 applications and I/O libraries built on top of HDF5.
2022-04-15 08:24:46 -07:00
Wei-keng Liao
b793e5005c free MPI datatypes previously created (#1637) 2022-04-15 08:23:12 -07:00
Wei-keng Liao
ea26955d18 not depend on doIO to free an MPI_Comm object (#1642) 2022-04-15 08:22:01 -07:00
Dana Robinson
852ed69f4f Adds build and license shields to README.md (#1641)
* First stab at a Github status bar

* Adds a .tokeignore file for counting lines of code accurately

* Yanks lines of code calculation since it wildly overcounts
2022-04-15 08:20:59 -07:00
Wei-keng Liao
f2a8ac16d7 free MPI_Group/MPI_Comm/MPI_Datatype objects (#1638)
* free MPI_Group/MPI_Comm/MPI_Datatype objects

* fix clang-format style
2022-04-15 08:20:23 -07:00
Dana Robinson
6a4d37f189 Fixes a typo in H5.c (#1639) 2022-04-14 10:35:54 -07:00
Allen Byrne
ab69f5df77 Fix these Doxygen warnings #1581 (#1589) 2022-04-13 14:16:44 -05:00
Dana Robinson
29de83ab8e Removes unused definitions from module headers (#1624) 2022-04-11 13:06:37 -05:00
Dana Robinson
d433f0f7a6 Removes the STATIC flavor of FUNC_ENTER macros (#1622)
* Removes the STATIC flavor of FUNC_ENTER macros
2022-04-08 14:32:40 -05:00
Dana Robinson
95909be9c3 Minor things discovered while preparing 1.12.2 (#1620) 2022-04-08 09:50:20 -05:00
Dana Robinson
41936183d7 Updates bin/restore.sh to clean up src/H5config.h.in (#1605) 2022-04-07 09:14:47 -05:00
Dana Robinson
fe6b2b8e55 Minor C++ changes noticed while normalizing with 1.12 (#1606) 2022-04-07 06:03:15 -07:00
Dana Robinson
ead0f95877 Removes bin/README (#1602)
This README file refers to scripts that were removed when the MANIFEST
went away.
2022-04-07 05:36:43 -07:00
Dana Robinson
a0aeb114e9 Adds additional files/paths to ignore for github actions (#1585) 2022-04-05 17:26:35 -05:00
Dana Robinson
a2b2148e73 Adds release timeline to README.md (#1584) 2022-04-05 17:26:00 -05:00
Allen Byrne
da562164ce Add missing release note for h5repack timing (#1572) 2022-04-04 22:51:17 -05:00
Allen Byrne
b6398dd60e Move error-stack text to top of usage display (#1564)
* Move error-stack text to top of usage display

* Add optional tag ti list

* format changes

* Revert incorrect change
2022-04-01 15:55:22 -05:00
Dana Robinson
f15afef6b3 Fix for unexported global variable on Windows (#1568) 2022-04-01 15:42:22 -05:00
Scot Breitenfeld
e8ea850004 spelling fixes (#1561) 2022-03-31 10:42:30 -05:00
Allen Byrne
870ee8feee Update command format, avoid policy warning (#1550) 2022-03-30 18:01:22 -05:00
jhendersonHDF
ab48ed6df0 Add H5_CHECK_MPI_VERSION macro to check for at least MPI version of (version, subversion) (#1556)
* Fix MPI version check for MPI_Aint_diff usage

* Add H5_CHECK_MPI_VERSION macro

Adds new H5_CHECK_MPI_VERSION macro to check for MPI library version/subversion pair of at least the specified pair
2022-03-30 17:58:55 -05:00
Quincey Koziol
82bf2e97f7 Update MacOS configure, bringing it closer to Linux configure (#1554) 2022-03-30 17:02:18 -05:00
jhendersonHDF
9ed2613789 Add missing H5FL_TRACK debug parameters to free list array routines (#1555) 2022-03-30 16:42:11 -05:00
Allen Byrne
3a906eb3e5 Minor comment corrections (#1544) 2022-03-29 20:57:32 -05:00
Dana Robinson
23ab623955 Removes unused H5HP (private heap) package (#1548) 2022-03-29 20:56:37 -05:00
Dana Robinson
b86be5df35 Removes unused memory pool (H5MP) package (#1547) 2022-03-29 20:54:02 -05:00
Dana Robinson
71af37d38f Removes cruft from a few header files (#1534)
* Unnecessary extern C guards in cache headers
* Non-existent H5Xpublic.h includes hidden behind NOT_YET symbols
2022-03-29 07:11:56 -05:00
Neil Fortner
0d3182c7da Fix bugs in test/vfd.c (#1538)
* Added support for vector I/O calls to the VFD layer, and
associated test code.  Note that this includes the optimization
to allow shortened sizes and types arrays to allow more space
efficient representations of vectors in which all entries are
of the same size and/or type.  See the Selection I/o RFC for
further details.

Tested serial and parallel, debug and production on Charis.
       serial and parallel debug only on Jelly.

* ran code formatter

quick serial build and test on jelly

* Add H5FD_read_selection() and H5FD_write_selection().  Currently only
translate to scalar calls.  Fix const buf in H5FD_write_vector().

* Format source

* Fix comments

* Added vector read / write support to the MPIO VFD, with associated
test code (see testpar/t_vfd.c).

Note that this implementation does NOT support vector entries of
size greater than 2 GB.  This must be repaired before release,
but it should be good enough for correctness testing.

As MPIO requires vector I/O requests to be sorted in increasing
address order, also added a vector sort utility in H5FDint.c  This
function is tested in passing by the MPIO vector I/O extension.

In passing, repaired a bug in size / type vector extension management
in H5FD_read/write_vector()

Tested parallel debug and production on charis and Jelly.

* Ran source code formatter

* Minor optimization to vector I/O request sort routine.

Tested parallel / debug on Jelly.

* Add selection I/O to chunk code, used when: not using chunk cache, no
datatype conversion, no I/O filters, no page buffer, not using collective
I/O.  Requires global variable H5_use_selection_io_g be set to TRUE.
Implemented selection to vector I/O transaltion at the file driver
layer.

* Fix formatting unrelated to previous change to stop github from
complaining.

* Add full API support for selection I/O.  Add tests for this.

* Added "ctl" callback to the VFD interfance, and the associated
H5FDctl() and H5FD_ctl() calls.

Modified the MPIO VFD accordingly -- specifically:

Added ctl() call with op-code support to expose rank, size,
and communicator.

Modified H5FD_mpi_get_rank(), H5FD_mpi_get_size(),
and H5FD_mpi_get_comm() to use the new ctl() callback.  In passing
removed the const qualifier from the file parameter of these
functions, as the file parameter of the ctl callback is not
const.

Deleted the old  H5FD__mpio_mpi_rank(), H5FD__mpio_mpi_size(), and
H5FD__mpio_communicator() calls from the MPIO VFD.

Deleted H5FD_class_mpi_t from H5FDprivate.h, and modified the
MPIO VFD accordingly.  Note that all VFDs now use H5FD_class_t,
with no special class for VFDs that that support MPI.

Some minor touch ups to the Neil's selection I/O mods in passing.

Tested serial and parallel, debug and production on charis and
jelly.

* Implement selection I/O for contiguous datasets.  Fix bug in selection
I/O translation.  Add const qualifiers to some internal selection I/O
routines to maintain const-correctness while avoiding memcpys.

* Add support for independent parallel I/O with selection I/O.  Add
HDF5_USE_SELECTION_IO env var to control selection I/O (default off).

* Implement parallel collective support for selection I/O.

* Fix comments and run formatter.

* Committing clang-format changes

* Run formatter

* Update selection IO branch with develop (#1215)

Merged branch 'develop' into selection_io

* Sync with develop (#1262)

Updated the branch with develop changes.

* Committing clang-format changes

* fixed spelling (#1353)

* Implement big I/O support for vector I/O requests in the MPIO file
driver.

* Committing clang-format changes

* Update dummy file driver in h5test.c to match new struct.

* Fix another outdated H5FD_class_t struct.

* Free arrays in H5FD__mpio_read/write_vector() as soon as they're not
needed, to cut down on memory usage during I/O.

* Committing clang-format changes

* Address comments from code review.  Fix const warnings with
H5S_SEL_ITER_INIT().

* Committing clang-format changes

* Fix warnings in selection I/O code

* Refactor H5FD__mpio_read/write_vector() to bring common code for
constructing MPI Datatype into a shared function.

* Committing clang-format changes

* Fix bugs in selection I/O code.

* Fix bugs in selection I/O code

* Address comments from review.  Add "version" field to H5FD_class_t
struct and accompanying H5FD_CLASS_VERSION macro.

* Committing clang-format changes

* Committing clang-format changes

* Address comments from selection I/O PR

* Committing clang-format changes

* Fix spelling

* Fix bugs int test/vfd.c

Co-authored-by: mainzer <mainzer#hdfgroup.org>
Co-authored-by: jrmainzer <72230804+jrmainzer@users.noreply.github.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Scot Breitenfeld <brtnfld@hdfgroup.org>
2022-03-28 22:34:18 -05:00
Wei-keng Liao
80f9af144c Use AC_MSG_RESULT/AC_MSG_ERROR/AC_MSG_WARN instead of echo (#1532) 2022-03-28 07:32:17 -05:00
Neil Fortner
42b767fc67 Merge initial version of selection I/O feature into develop (#1367) 2022-03-26 14:30:53 -05:00
Allen Byrne
25ef608e2f Change ENV to CMake var (#1520)
* Change ENV to CMake var

* Fix typo deletion

* Fix typo
2022-03-24 16:56:06 -05:00
Allen Byrne
7d9b850574 Windows and *nix use different forms for warnings (#1515) 2022-03-24 08:27:29 -05:00
Orion Poplawski
be36ac1d33 Mark minusone as a PARAMETER in tH5A_1_8.F90. Resolves #1379. (#1412) 2022-03-18 11:09:29 -05:00
Dana Robinson
abad1f320a Removes HD prefix from pthreads calls (#1507) 2022-03-17 20:24:53 -05:00
Dana Robinson
fb29cd77ad Fixes a bug in the FSM section merge code exposed by VFD SWMR (#1505)
From Vailin:

    Fix the FSM bug when setting the FSM threshold to a non-default value.
    Check for smaller or larger section size after merging and shrinking a
    section, for this case is the section that is smaller than threshold
    (see H5MF_xfree() in H5MF.c). It is possible for the section to be
    smaller after merging/shrinking (see H5MF__sect_large_shrink() in
    H5MFsection.c).
2022-03-17 10:39:55 -05:00
Dana Robinson
96398e47af Misc fixes highlighted by the checkposix script (#1499) 2022-03-17 09:53:19 -05:00
Wei-keng Liao
34442f3740 free an MPI communicator (#1503) 2022-03-17 09:52:20 -05:00
Dana Robinson
b9846d1749 Removes the MANIFEST file and unused release files (#1497)
* Removes the MANIFEST file and unused release files

* Updated tar command
2022-03-14 22:47:32 -05:00
Allen Byrne
872b87d5d6 Update release note (#1485)
* Flip inits to correct ifdef section

* rework ifdef to be simpler

* format issue

* Reformat ifdef inits

* remove static attribute

* format compliance

* Update names

* Revert because logic relies on float not being int

* Changes noticed from creating merge of #412

* Double underscore change

* Correct compiler version variable used

* Remove header guard underscores

* Whitespace cleanup

* Split format source and commit changes on repo push

* remove pre-split setting

* Change windows TS to use older VS.

* correct window os name

* HDFFV-11212 JNI export util and Javadoc

* Suggested review changes

* Another change found

* Committing clang-format changes

* HDFFV-11113 long double in tools

* HDFFV-11113 add note

* Disable long double tests for now

* HDFFV-11228 remove arbitrary CMake warning groups.

* Make each flag a string

* Some Javadoc warning fixes

* Updated javadoc fixes

* # WARNING: head commit changed in the meantime

HDFFV-11229 Fix long double usage in tools and java

Changed h5dump and h5ls to just print 128-bit for long double type.
Added test and file for dataset and attributes with all three float
types.

* Committing clang-format changes

* HDFFV-11229 Add release note

* HDFFV-11229 windows testfile needed

* fix typo

* Remove non supported message text

* HDFFV-11229 - change ldouble test to check both native and general

* HDFFV-11229 add second file compare

* HDFFV-11229 fix reference file

* HDFFV-11229 autotools check two refs

* HDFFV-11229 revert back to removal of NATIVE_LDOUBLE in tools output

* Committing clang-format changes

* Update release note

* Update attribute type of ref file

* Change source of ninja for macs

* try port instead of brew

* Recommended is to use brew.

* Undo non long double changes

* remove unneeded file

* Fix perl and doxygen CMake code

* Add "option" command for clang options

* Rework CMake add_custom to use the BYPRODUCTS argument

* Add stamp files to BYPRODUCTS

* Only one copy of file exists

* Fix custom cmmand depends targets

* Fix fortran custom command DEPENDS

* Add LD_LIBRARY_PATH to tests

* Add custom target for DEPENDS

* Add h5detect conditionaly to generated target DEPENDS

* Correct DEPENDS targets

* Parallel builds need the mpi compiler for pkgconfig scripts.

* install only if MPI build

* Fortran target depends

* Remove incorrect source attribute

* doxygen adjustments

* doxygen build updates

* Correct version

* Correct function version - function has been merged to 1.12

* Correct version string for map functions

* Cleanup warnings for tools debug builds

* TRILAB-227 - fix indexing for h5diff selections

* Correct location of pos to index function call

* TRILAB-227 Enable test

* Quote subset args

* Use MATCHES because of AppleClang

* if blocks needed for build modes

* Update list of DT platforms

* VS2019 correctly displays float values

* revert VS2019 change

* Issue #669 remove version from pkgcfg filename

* remove version from h5cc script

* Java reference functions updated

enabled fortran in cmake gcc action yaml file
java reference test changed to correctly test refs
jni reference functions that create ids changed to use jni id logging

* Correct BYPRODUCTS argument

* Correct more genereated files BYPRODUCTS

* BYPRODUCTS must have unique locations

* Fix typo

* Fix fortran configure checks

* Rework H5_PAC_C_MAX_REAL_PRECISION setting logic

* Add note about fortran configure change

* Adds a quick for for some egregious chunk_info badness (#722)

* Fixes issue with ccmake that prevents building Fortran (#723)

ccmake runs iteratively, and the check_fortran_source_runs
macros were clobbering a single output file that did not
get updated on further configure iterations

* Fix conflicts with merge

* Move MAX_PRECISION back to HDF5UseFortran.cmake

* Use STREQUAL to test macro argument

* Move C language test to ConfigureChecks from HDF5UseFortran

* MAX_PRECISION defines must be defined

* Organize flags and align autotools and cmake

* Fix comment in no-error-general warnings files.

* Flag cleanup and fix typos

* Add comment

* Correct VAR used to find configure time file

* Set the path correctly

* Update missing release note info.

* Update code owners

* Correct JIRA note

* add known problem.

* Use only core library for testing dynamic plugins.

* Committing clang-format changes

* Update main.yml

* Update HISTORY-1_13.txt

Missing release note added for changed location of CMake configuration files.

* Update HISTORY-1_13.txt

Fix typo

* Update main.yml

revert change

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
2022-03-11 13:17:26 -06:00
Dana Robinson
1f9228316c Fixes for format string warnings raised by -Wformat=2/-Wformat-security (#1489) 2022-03-11 13:10:20 -06:00
Dana Robinson
01209d2777 Removes remaining register keywords (#1481) 2022-03-10 16:14:05 -06:00
David Young
1fb3743f04 Fix JNI bug, fix warnings noticed in MSVC CI, be modern & portable (#1480) 2022-03-10 13:54:20 -06:00
Dana Robinson
b9470be379 Plugin test cleanup (#1479)
* Autotools plugin test cleanup

* Combines filter, VFD, and VOL plugin tests
* Adds VFD plugin tests to the Autotools
* Implements a uniform shell script naming scheme in test/

* codespell fix

* Changes after code review
2022-03-10 12:22:16 -06:00
Dana Robinson
d5ddf61803 Minor tweaks from VFD SWMR normalization (#1474) 2022-03-09 14:07:54 -06:00
Allen Byrne
16f482de8f Remove addition of non-standalone use of H5_ATTR_FORMAT (#1470) 2022-03-07 08:32:45 -06:00
Larry Knox
44d9926840 Update HISTORY-1_13.txt and clean RELEASE.txt (#1471)
* Committing clang-format changes

* Spelling of preceed was corrected to proceed, but should have been
corrected to precede.

* Correct spelling correction of 'preceed' incorrectly to 'proceed'.  It should be 'precede'.

* Update version to 1.13.2-1 after 1.13.1 release.
Add Makefile.in to MANIFEST for addition of utils/tools and h5dwalk.

* Update VERS_RELEASE_EXCEPTIONS with new incompatible release version.

* Add HDF5 1.13.1 RELEASE.txt to HISTORY-1_13.txt contents to
HISTORY-1_13.txt.
Clean entries from RELEASE.txt.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-03-07 08:31:58 -06:00
jhendersonHDF
ba032bb28b Disable memory alloc sanity checks by default for Autotools debug builds (#1468) 2022-03-03 09:46:18 -06:00
Larry Knox
7c18723404 Create HISTORY-1_13.txt and update release.txt (#1466) 2022-03-01 14:48:12 -06:00
Larry Knox
94d01f5377 Update version to 1.13.2-1 after 1.13.1 release; add new Makefile.in files to MANIFEST (#1460) 2022-03-01 14:20:34 -06:00
Allen Byrne
ebf89bb231 Fix install config and java test name conflict (#1467) 2022-03-01 14:19:43 -06:00
Scot Breitenfeld
c72904c6b3 fixed buf definition in h5dfill_f (#1465)
* H5Lexists docs: Removed reference to 1.8.16 since the change is the 1.8.x releases, HDFFV-11289

* H5Oget_info_by_name, name can be any object, not just a group

* fixed buf definition in h5dfill_f
2022-03-01 14:19:15 -06:00
Allen Byrne
b7d8177404 Remove targets PERMISSIONS setting (#1462) 2022-03-01 13:33:18 -06:00
jhendersonHDF
8e0b427bf2 Fix metadata cache bug when resizing a pinned/protected entry (v2) (#1463) 2022-03-01 12:30:49 -06:00
jhendersonHDF
80954e8272 Fix for parallel compression examples on Windows (#1459) 2022-02-27 23:15:38 -06:00
jhendersonHDF
f46e5db184 Amend RELEASE.txt with note about parallel compression testing assertion failure (#1458) 2022-02-25 14:36:34 -06:00
Allen Byrne
e19e7e3faa Add release note for #1438 (#1457)
* Add release note for #1438

* fix typos.
2022-02-25 14:31:55 -06:00
Allen Byrne
2db4c770f8 Fix release check version logic (#1438) 2022-02-24 10:59:15 -06:00
jhendersonHDF
758e97c1e5 Parallel Compression improvements (#1302) 2022-02-24 10:04:59 -06:00
Allen Byrne
5a5a770402 Enable creation of RPM/DEB binaries GH#1394 (#1398)
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
2022-02-22 08:36:28 -06:00
Larry Knox
3107fafa2c Oess 168 utils mirror vfd (#1444)
* Committing clang-format changes

* Spelling of preceed was corrected to proceed, but should have been
corrected to precede.

* Correct spelling correction of 'preceed' incorrectly to 'proceed'.  It should be 'precede'.

* OESS-168: Remove clang warnings.

* OESS-168: Address @lrknox and @gnuoyd reviews.

* Eliminate clang warnings listed in PR #1310 without adding new ssize_t
variables.

* Committing clang-format changes

* Add H5_ATTR_UNUSED to wait_for_child call.
Remove unneeded casts in mirror_log calls.

* Keep ssize_t in mirror_server.c line 479.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hyo-Kyung Lee <hyoklee@hdfgroup.org>
2022-02-21 15:32:29 -06:00
David Young
c302773438 Sprinkle H5_ATTR_FORMAT over printf(3)-like functions in tools and fix issues (#1423)
* Correct some conversion specifications.

* Replace many "%lld" occurrences with "%" PRIuHSIZE except for a few
instances where PRIdHSIZE was appropriate.  Remove a couple of casts and
use correct format strings, instead.

* Copy values from a possibly unaligned buffer to aligned local variables
instead of casting and dereferencing pointers into the buffer.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-02-21 15:31:57 -06:00
jhendersonHDF
705ba09e76 Fix cache sanity checking code by moving functions to wider scope (#1435) 2022-02-21 12:36:29 -06:00
Neil Fortner
42c419e266 Fix issue with copying null new references (#1440)
* Fix issue with copying null new references. Fix assertion failure when
reference copying fails.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-02-18 09:28:21 -06:00
Neil Fortner
feda52b084 Implement support for big I/O for independent reads (#1446)
* Implement big I/O support for independent reads. Add test for this.

* Committing clang-format changes

* Update big I/O code in H5FD__mpio_write() to match that in
H5FD__mpio_read().

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-02-18 09:27:05 -06:00
Gerd Heber
cfe4198b6c Updated reference for Multi Dataset RFC. (#1442) 2022-02-18 08:02:10 -06:00
Allen Byrne
08ec5b45a5 Add SZ filter and adjust compression file names to avoid conflicts (#1434)
* Updated plugins project required adjustments

* Updated plugins need to distinguish external libs

* Fix missed rename

* Use same name for windows as linux - avoid name clash
2022-02-17 10:05:08 -06:00
myd7349
bbb0acf353 Fixed AbstractDs::getVarLenType documentation (#1441) 2022-02-16 14:58:35 -06:00
loricooperhdf
d25d05d740 Creating FUNDING.yml (#1427)
* Creating FUNDING.yml

Will add sponsor this project widget to the repo's page.

* Correct file name

Add sponsor widget; filename typo fixed.

* Update MANIFEST
2022-02-16 07:46:19 -06:00
Larry Knox
97b8fad8ed Revert "Fix metadata cache bug when resizing a pinned/protected entry (#1358)" (#1437)
This reverts commit 01092658a3.
2022-02-11 22:34:16 -06:00
Allen Byrne
96cf19499b Add -showconfig to h5cc scripts to dump libhdf5.settings file (#1391)
* Add -showconfig to h5cc scripts to dump libhdf5.settings file

* Added note
2022-02-11 07:46:09 -06:00
David Young
4ce6373a0b Make a buffer bigger: GCC 8.3.0 warned that a snprintf may be (#1403)
truncated because the recipient buffer was too small, and the warning is
one that we promote to an error.  Now there is a warning that the buffer
is too big (-Wlarger-than=), but we don't promote those to errors, yet.
2022-02-11 07:39:52 -06:00
Scot Breitenfeld
7bfa873983 Fixed H5Dget_storage_size_f for zero sized datasets (#1430)
* H5Lexists docs: Removed reference to 1.8.16 since the change is the 1.8.x releases, HDFFV-11289

* H5Oget_info_by_name, name can be any object, not just a group

* fixed handling size 0 datasets

* fixed handling size 0 datasets

* format
2022-02-11 06:56:33 -06:00
Neil Fortner
0f847f30bc Fix issue with H5_now_usec() with 32 bit systems (#1416)
* Implement H5ESget_requests() to retrieve request pointers and
corresponding connector IDs from an event set.  Add tests for this
function.

* Add "order" parameter to H5ESget_requests, to allow the user to specify
the order in which requests are returned.

* Fix bugs with H5ESinsert_request()

* Change H5ESget_requests() API to have separate input and output
parameters for the allocated array length(s) (in) and the number of events in
the event set (out).

* Fix issue with H5_now_usec() with 32 bit systems.  Also improve event
set test.

* Committing clang-format changes

* Add comments to H5_now_usec() explaining rationale for uint64_t casts.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-02-10 17:23:26 -06:00
jhendersonHDF
01092658a3 Fix metadata cache bug when resizing a pinned/protected entry (#1358)
When resizing a pinned/protected cache entry, the metadata
cache code previously would wait until after resizing the
entry to attempt to log the newly-dirtied entry. This would
cause H5C_resize_entry to mark the entry as dirty and make
H5AC_resize_entry think that it doesn't need to add the
newly-dirtied entry to the dirty entries skiplist.

Thus, a subsequent H5AC__log_moved_entry would think it
needs to allocate a new entry for insertion into the dirty
entry skip list, since the entry doesn't exist on that list.
This causes an assertion failure, as the code to allocate a
new entry assumes that the entry is not dirty.
2022-02-07 22:04:41 -06:00
Allen Byrne
c9347450c1 Implemented suggestion in #1381 for dtransform test (#1415)
* Implemented suggestion in #1380

* fix typo

* Use differently named files in tests

* format fix

* Revise comments

* fic typos

* Change test display and file name
2022-02-04 16:50:45 -06:00
2182 changed files with 195446 additions and 116210 deletions

View File

@@ -1,5 +1,4 @@
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -1,7 +1,20 @@
---
Language: Cpp
BasedOnStyle: LLVM
AlignConsecutiveAssignments: true
BraceWrapping:
AfterFunction: true
#llvm10-11: AfterControlStatement: false - Never
BeforeCatch: true
BeforeElse: true
#llvm11: BeforeLambdaBody: false
#llvm11: BeforeWhile: false
BreakBeforeBraces: Stroustrup
BreakAfterJavaFieldAnnotations: true
BreakStringLiterals: true
ColumnLimit: 110 # Update $max_trace_macro_line_len in bin/trace also
IndentWidth: 4
---
Language: Cpp
#llvm11: AlignConsecutiveBitFields: false
AlignConsecutiveDeclarations: true
AlignConsecutiveMacros: true
@@ -22,17 +35,6 @@ AlwaysBreakAfterReturnType: AllDefinitions
# - H5_ATTR_CONST
# - H5_ATTR_PURE
# - H5_ATTR_FALLTHROUGH
BraceWrapping:
AfterFunction: true
#llvm10-11: AfterControlStatement: false - Never
BeforeCatch: true
BeforeElse: true
#llvm11: BeforeLambdaBody: false
#llvm11: BeforeWhile: false
BreakBeforeBraces: Stroustrup
BreakAfterJavaFieldAnnotations: true
BreakStringLiterals: true
ColumnLimit: 110 # Update $max_trace_macro_line_len in bin/trace also
ForEachMacros: ['ALL_MEMBERS', 'UNIQUE_MEMBERS']
IncludeCategories:
- Regex: '^"(llvm|llvm-c|clang|clang-c)/'
@@ -55,7 +57,6 @@ IndentCaseLabels: true
#llvm11: IndentCaseBlocks: false
IndentGotoLabels: false
#llvm11: IndentExternBlock: AfterExternBlock
IndentWidth: 4
#llvm11: InsertTrailingCommas: None
MacroBlockBegin: "^BEGIN_FUNC"
MacroBlockEnd: "^END_FUNC"
@@ -92,5 +93,8 @@ StatementMacros:
#llvm11: WhitespaceSensitiveMacros:
#llvm11: - STRINGIZE
#llvm11: - PP_STRINGIZE
---
Language: Java
BreakAfterJavaFieldAnnotations: true
JavaImportGroups: ['java', 'hdf', 'hdf.hdf5lib', 'org']
...

37
.github/CODEOWNERS vendored
View File

@@ -2,41 +2,10 @@
# Each line is a file pattern followed by one or more owners.
# These owners will be the default owners for everything in the repo.
* @lrknox
* @lrknox @derobins @byrnHDF @fortnern @jhendersonHDF @qkoziol @vchoi-hdfgroup @bmribler @raylu-hdf
# Order is important. The last matching pattern has the most precedence.
# So if a pull request only touches javascript files, only these owners
# will be requested to review.
*.cmake @byrnHDF @ChristopherHogan @gnuoyd @derobins
CMakeLists.txt @byrnHDF @ChristopherHogan @gnuoyd @derobins
CMakeTests.* @byrnHDF @ChristopherHogan @gnuoyd @derobins
/bin/ @lrknox @ChristopherHogan @gnuoyd @derobins @qkoziol
/c++/ @bmribler @byrnHDF @ChristopherHogan @gnuoyd @derobins
/config/ @lrknox @byrnHDF @ChristopherHogan @gnuoyd @derobins @qkoziol
/doc/ @ChristopherHogan @gnuoyd @jrmainzer
/examples/ @lrknox @bmribler @ChristopherHogan @gnuoyd @derobins
/fortran/ @brtnfld @epourmal
/hl/ @bmribler @byrnHDF @ChristopherHogan @gnuoyd @derobins
/java/ @jhendersonHDF @byrnHDF
/m4/ @lrknox @ChristopherHogan @gnuoyd @derobins
/release_docs/ @lrknox @bmribler @byrnHDF
/src/ @jhendersonHDF @fortnern @soumagne @vchoi-hdfgroup @ChristopherHogan @gnuoyd @derobins @jrmainzer @qkoziol
/test/ @jhendersonHDF @fortnern @soumagne @vchoi-hdfgroup @ChristopherHogan @gnuoyd @derobins @jrmainzer @qkoziol
/testpar/ @jhendersonHDF @ChristopherHogan @gnuoyd @jrmainzer @qkoziol
/tools/ @byrnHDF @bmribler @ChristopherHogan @gnuoyd @derobins
/utils/ @lrknox @byrnHDF @ChristopherHogan @gnuoyd @derobins
/fortran/ @brtnfld
/java/ @jhendersonHDF @byrnHDF @derobins

3
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,3 @@
# These are supported funding model platforms
custom: "https://hdfgroup.org/about-us/donate-to-the-hdf-group/"

25
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,25 @@
---
name: Bug report
about: Report a problem with HDF5
title: "[BUG]"
labels: bug
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**Expected behavior**
A clear and concise description of what you expected to happen.
**Platform (please complete the following information)**
- HDF5 version (if building from a maintenance branch, please include the commit hash)
- OS and version
- Compiler and version
- Build system (e.g. CMake, Autotools) and version
- Any configure options you specified
- MPI library and version (parallel HDF5)
**Additional context**
Add any other context about the problem here.

View File

@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an improvement to HDF5
title: "[Feature Request]"
labels: enhancement
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

View File

@@ -0,0 +1,8 @@
## Describe your changes
## Issue ticket number (GitHub or JIRA)
## Checklist before requesting a review
- [ ] My code conforms to the guidelines in CONTRIBUTING.md
- [ ] I made an entry in release_docs/RELEASE.txt (bug fixes, new features)
- [ ] I added a test (bug fixes, new features)

View File

@@ -7,12 +7,12 @@ jobs:
runs-on: ubuntu-latest
if: "!contains(github.event.head_commit.message, 'skip-ci')"
steps:
- uses: actions/checkout@v2
- name: Run clang-format style check for C programs.
uses: DoozyX/clang-format-lint-action@v0.11
- uses: actions/checkout@v3
- name: Run clang-format style check for C and Java programs.
uses: DoozyX/clang-format-lint-action@v0.13
with:
source: '.'
extensions: 'c,h,cpp,hpp'
clangFormatVersion: 10
extensions: 'c,h,cpp,hpp,java'
clangFormatVersion: 13
style: file
exclude: './config ./hl/src/H5LTanalyze.c ./hl/src/H5LTparse.c ./hl/src/H5LTparse.h ./src/H5Epubgen.h ./src/H5Einit.h ./src/H5Eterm.h ./src/H5Edefin.h ./src/H5version.h ./src/H5overflow.h'

View File

@@ -8,13 +8,13 @@ jobs:
runs-on: ubuntu-latest
if: "!contains(github.event.head_commit.message, 'skip-ci')"
steps:
- uses: actions/checkout@v2
- name: Run clang-format style check for C programs.
uses: DoozyX/clang-format-lint-action@v0.11
- uses: actions/checkout@v3
- name: Run clang-format style check for C and Java programs.
uses: DoozyX/clang-format-lint-action@v0.13
with:
source: '.'
extensions: 'c,h,cpp,hpp'
clangFormatVersion: 10
extensions: 'c,h,cpp,hpp,java'
clangFormatVersion: 13
inplace: True
style: file
exclude: './config ./hl/src/H5LTanalyze.c ./hl/src/H5LTparse.c ./hl/src/H5LTparse.h ./src/H5Epubgen.h ./src/H5Einit.h ./src/H5Eterm.h ./src/H5Edefin.h ./src/H5version.h ./src/H5overflow.h'

View File

@@ -8,8 +8,8 @@ jobs:
name: Check for spelling errors
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- uses: codespell-project/actions-codespell@master
with:
skip: ./bin/trace,./hl/tools/h5watch/h5watch.c,./tools/test/h5jam/tellub.c
ignore_words_list: isnt,inout,nd,parms,parm,ba,offsetP,ser,ois,had,fiter,fo,clude,refere,minnum,offsetp,creat,ans:,eiter,lastr,ans,isn't,ifset,sur,trun,dne,tthe,hda,filname,te,htmp,minnum
skip: ./bin/trace,./hl/tools/h5watch/h5watch.c,./tools/test/h5jam/tellub.c,./config/sanitizer/LICENSE,./tools/test/h5repack/testfiles/*.dat
ignore_words_list: isnt,inout,nd,parms,parm,ba,offsetP,ser,ois,had,fiter,fo,clude,refere,minnum,offsetp,creat,ans:,eiter,lastr,ans,isn't,ifset,sur,trun,dne,tthe,hda,filname,te,htmp,minnum,ake,gord,numer,ro,oce

View File

@@ -4,216 +4,481 @@ name: hdf5 dev CI
on:
workflow_dispatch:
push:
branches: [ develop, hdf5_1_12, hdf5_1_10, hdf5_1_8 ]
pull_request:
branches: [ develop, hdf5_1_14, hdf5_1_12, hdf5_1_10, hdf5_1_8 ]
paths-ignore:
- '.github/**'
- 'doc/**'
- 'release_docs/**'
- '.github/CODEOWNERS'
- '.github/FUNDING.yml'
- 'doc/**'
- 'release_docs/**'
- 'ACKNOWLEDGEMENTS'
- 'COPYING**'
- '**.md'
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
# A workflow run is made up of one or more jobs that can run sequentially or
# in parallel. We just have one job, but the matrix items defined below will
# run in parallel.
jobs:
# This workflow contains a single job called "build"
build:
# A workflow that builds the library and runs all the tests
build_and_test:
strategy:
# fail-fast: false
# The current matrix has three dimensions:
#
# * config name
# * thread-safety on/off
# * release vs. debug build
#
# Most configuration information is added via the 'include' mechanism,
# which will append the key-value pairs in the configuration where the
# names match.
matrix:
name: ["Windows Latest MSVC", "Ubuntu Latest GCC", "Ubuntu Debug GCC", "macOS Latest Clang", "Ubuntu Autotools GCC", "Windows TS MSVC", "Ubuntu TS GCC", "TS Debug GCC", "macOS TS Clang", "TS Autotools GCC"]
name:
- "Windows MSVC CMake"
- "Ubuntu gcc CMake"
- "Ubuntu gcc Autotools"
- "Ubuntu gcc Autotools parallel (build only)"
- "MacOS Clang CMake"
thread_safety:
- enabled: true
text: " TS"
- enabled: false
text: ""
build_mode:
- text: " REL"
cmake: "Release"
autotools: "production"
- text: " DBG"
cmake: "Debug"
autotools: "debug"
# This is where we list the bulk of the options for each configuration.
# The key-value pair values are usually appropriate for being CMake or
# Autotools configure values, so be aware of that.
include:
- name: "Windows Latest MSVC"
artifact: "Windows-MSVC.tar.xz"
# Windows w/ MSVC + CMake
#
# No Fortran, parallel, or VFDs that rely on POSIX things
- name: "Windows MSVC CMake"
os: windows-2022
build_type: "Release"
toolchain: ""
cpp: ON
fortran: OFF
java: ON
ts: OFF
hl: ON
parallel: OFF
mirror_vfd: OFF
direct_vfd: OFF
generator: "-G \"Visual Studio 17 2022\" -A x64"
- name: "Ubuntu Latest GCC"
artifact: "Linux.tar.xz"
run_tests: true
# Linux (Ubuntu) w/ gcc + CMake
#
# We might think about adding Clang, but MacOS already tests that
# so it's not critical
- name: "Ubuntu gcc CMake"
os: ubuntu-latest
build_type: "Release"
cpp: ON
fortran: ON
java: ON
ts: OFF
hl: ON
parallel: OFF
mirror_vfd: ON
direct_vfd: ON
toolchain: "config/toolchain/GCC.cmake"
generator: "-G Ninja"
- name: "macOS Latest Clang"
artifact: "macOS.tar.xz"
os: macos-latest
build_type: "Release"
cpp: ON
fortran: OFF
java: ON
ts: OFF
hl: ON
parallel: OFF
toolchain: "config/toolchain/clang.cmake"
generator: "-G Ninja"
- name: "Ubuntu Debug GCC"
artifact: "LinuxDBG.tar.xz"
run_tests: true
# Linux (Ubuntu) w/ gcc + Autotools
#
# Keep this identical to the CMake configs. Note the difference in
# the values.
- name: "Ubuntu gcc Autotools"
os: ubuntu-latest
build_type: "Debug"
cpp: ON
fortran: OFF
java: OFF
ts: OFF
hl: ON
parallel: OFF
toolchain: "config/toolchain/GCC.cmake"
generator: "-G Ninja"
- name: "Ubuntu Autotools GCC"
artifact: "LinuxA.tar.xz"
os: ubuntu-latest
build_type: "Release"
cpp: enable
fortran: enable
java: enable
ts: disable
hl: enable
parallel: disable
mirror_vfd: enable
direct_vfd: enable
deprec_sym: enable
default_api: v114
toolchain: ""
generator: "autogen"
# Threadsafe runs
- name: "Windows TS MSVC"
artifact: "Windows-MSVCTS.tar.xz"
os: windows-2019
build_type: "Release"
toolchain: ""
cpp: OFF
fortran: OFF
java: OFF
ts: ON
hl: OFF
parallel: OFF
generator: "-G \"Visual Studio 16 2019\" -A x64"
- name: "Ubuntu TS GCC"
artifact: "LinuxTS.tar.xz"
flags: ""
run_tests: true
# Parallel Linux (Ubuntu) w/ gcc + Autotools
#
# The GitHub runners are inadequate for running parallel HDF5 tests,
# so we catch most issues in daily testing. What we have here is just
# a compile check to make sure nothing obvious is broken.
- name: "Ubuntu gcc Autotools parallel (build only)"
os: ubuntu-latest
build_type: "Release"
cpp: OFF
cpp: disable
fortran: enable
java: disable
parallel: enable
mirror_vfd: disable
direct_vfd: disable
deprec_sym: enable
default_api: v114
toolchain: ""
generator: "autogen"
flags: "CC=mpicc"
run_tests: false
# MacOS w/ Clang + CMake
#
# We could also build with the Autotools via brew installing them,
# but that seems unnecessary
- name: "MacOS Clang CMake"
os: macos-11
cpp: ON
fortran: OFF
java: OFF
ts: ON
hl: OFF
parallel: OFF
toolchain: "config/toolchain/GCC.cmake"
generator: "-G Ninja"
- name: "macOS TS Clang"
artifact: "macOSTS.tar.xz"
os: macos-latest
build_type: "Release"
cpp: OFF
fortran: OFF
java: OFF
ts: ON
hl: OFF
java: ON
parallel: OFF
mirror_vfd: ON
direct_vfd: OFF
toolchain: "config/toolchain/clang.cmake"
generator: "-G Ninja"
- name: "TS Debug GCC"
artifact: "LinuxTSDBG.tar.xz"
run_tests: true
#
# SPECIAL AUTOTOOLS BUILDS
#
# These do not run tests and are not built into the matrix and instead
# become NEW configs as their name would clobber one of the matrix
# names (so make sure the names are UNIQUE).
#
- name: "Ubuntu gcc Autotools v1.6 default API (build only)"
os: ubuntu-latest
build_type: "Debug"
cpp: OFF
fortran: OFF
java: OFF
ts: ON
hl: OFF
parallel: OFF
toolchain: "config/toolchain/GCC.cmake"
generator: "-G Ninja"
- name: "TS Autotools GCC"
artifact: "LinuxATS.tar.xz"
os: ubuntu-latest
build_type: "Release"
cpp: disable
fortran: disable
java: disable
ts: enable
hl: disable
cpp: enable
fortran: enable
java: enable
parallel: disable
mirror_vfd: enable
direct_vfd: enable
deprec_sym: enable
default_api: v16
toolchain: ""
generator: "autogen"
# - name: "Ubuntu Parallel GCC"
# artifact: "LinuxPar.tar.xz"
# os: ubuntu-latest
# build_type: "Release"
# cpp: OFF
# fortran: OFF
# parallel: ON
# toolchain: "config/toolchain/GCC.cmake"
# generator: "-G Ninja"
flags: ""
run_tests: false
thread_safety:
enabled: false
text: ""
build_mode:
text: " DBG"
cmake: "Debug"
autotools: "debug"
- name: "Ubuntu gcc Autotools v1.8 default API (build only)"
os: ubuntu-latest
cpp: enable
fortran: enable
java: enable
parallel: disable
mirror_vfd: enable
direct_vfd: enable
deprec_sym: enable
default_api: v18
toolchain: ""
generator: "autogen"
flags: ""
run_tests: false
thread_safety:
enabled: false
text: ""
build_mode:
text: " DBG"
cmake: "Debug"
autotools: "debug"
- name: "Ubuntu gcc Autotools v1.10 default API (build only)"
os: ubuntu-latest
cpp: enable
fortran: enable
java: enable
parallel: disable
mirror_vfd: enable
direct_vfd: enable
deprec_sym: enable
default_api: v110
toolchain: ""
generator: "autogen"
flags: ""
run_tests: false
thread_safety:
enabled: false
text: ""
build_mode:
text: " DBG"
cmake: "Debug"
autotools: "debug"
- name: "Ubuntu gcc Autotools v1.12 default API (build only)"
os: ubuntu-latest
cpp: enable
fortran: enable
java: enable
parallel: disable
mirror_vfd: enable
direct_vfd: enable
deprec_sym: enable
default_api: v112
toolchain: ""
generator: "autogen"
flags: ""
run_tests: false
thread_safety:
enabled: false
text: ""
build_mode:
text: " DBG"
cmake: "Debug"
autotools: "debug"
- name: "Ubuntu gcc Autotools v1.14 default API (build only)"
os: ubuntu-latest
cpp: enable
fortran: enable
java: enable
parallel: disable
mirror_vfd: enable
direct_vfd: enable
deprec_sym: enable
default_api: v114
toolchain: ""
generator: "autogen"
flags: ""
run_tests: false
thread_safety:
enabled: false
text: ""
build_mode:
text: " DBG"
cmake: "Debug"
autotools: "debug"
- name: "Ubuntu gcc Autotools no deprecated symbols (build only)"
os: ubuntu-latest
cpp: enable
fortran: enable
java: enable
parallel: disable
mirror_vfd: enable
direct_vfd: enable
deprec_sym: disable
default_api: default
toolchain: ""
generator: "autogen"
flags: ""
run_tests: false
thread_safety:
enabled: false
text: ""
build_mode:
text: " DBG"
cmake: "Debug"
autotools: "debug"
- name: "Ubuntu gcc Autotools -Werror (build only)"
os: ubuntu-latest
cpp: enable
fortran: disable
java: disable
parallel: disable
mirror_vfd: disable
direct_vfd: enable
deprec_sym: enable
default_api: v114
toolchain: ""
generator: "autogen"
flags: "CFLAGS=-Werror"
run_tests: false
thread_safety:
enabled: false
text: ""
build_mode:
text: " DBG"
cmake: "Debug"
autotools: "debug"
- name: "Ubuntu gcc Autotools -Werror (build only)"
os: ubuntu-latest
cpp: enable
fortran: disable
java: disable
parallel: disable
mirror_vfd: disable
direct_vfd: enable
deprec_sym: enable
default_api: v114
toolchain: ""
generator: "autogen"
flags: "CFLAGS=-Werror"
run_tests: false
thread_safety:
enabled: false
text: ""
build_mode:
text: " REL"
cmake: "Release"
autotools: "production"
# Sets the job's name from the properties
name: "${{ matrix.name }}${{ matrix.build_mode.text }}${{ matrix.thread_safety.text }}"
# Don't run the action if the commit message says to skip CI
if: "!contains(github.event.head_commit.message, 'skip-ci')"
name: ${{ matrix.name }}
# The type of runner that the job will run on
runs-on: ${{ matrix.os }}
if: "!contains(github.event.head_commit.message, 'skip-ci')"
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- name: Install Dependencies (Linux)
run: sudo apt-get install ninja-build
if: matrix.os == 'ubuntu-latest'
- name: Install Autotools Dependencies (Linux)
run: sudo apt-get install automake autoconf libtool libtool-bin
if: matrix.generator == 'autogen'
- name: Install Dependencies (Windows)
run: choco install ninja
if: matrix.os == 'windows-latest'
- name: Install Dependencies (macOS)
run: brew install ninja
if: matrix.os == 'macos-latest'
- name: Set environment for MSVC (Windows)
if: matrix.os == 'windows-latest'
run: |
# Set these env vars so cmake picks the correct compiler
echo "CXX=cl.exe" >> $GITHUB_ENV
echo "CC=cl.exe" >> $GITHUB_ENV
#
# SETUP
#
#Useful for debugging
- name: Dump matrix context
run: echo '${{ toJSON(matrix) }}'
- name: Install CMake Dependencies (Linux)
run: sudo apt-get install ninja-build
if: matrix.os == 'ubuntu-latest'
- name: Install Autotools Dependencies (Linux, serial)
run: |
sudo apt update
sudo apt install automake autoconf libtool libtool-bin
sudo apt install gcc-11 g++-11 gfortran-11
echo "CC=gcc-11" >> $GITHUB_ENV
echo "CXX=g++-11" >> $GITHUB_ENV
echo "FC=gfortran-11" >> $GITHUB_ENV
if: (matrix.generator == 'autogen') && (matrix.parallel != 'enable')
- name: Install Autotools Dependencies (Linux, parallel)
run: |
sudo apt update
sudo apt install automake autoconf libtool libtool-bin
sudo apt install openmpi-bin openmpi-common mpi-default-dev
echo "CC=mpicc" >> $GITHUB_ENV
echo "FC=mpif90" >> $GITHUB_ENV
if: (matrix.generator == 'autogen') && (matrix.parallel == 'enable')
- name: Install Dependencies (Windows)
run: choco install ninja
if: matrix.os == 'windows-latest'
- name: Install Dependencies (macOS)
run: brew install ninja
if: matrix.os == 'macos-11'
- name: Set environment for MSVC (Windows)
run: |
# Set these environment variables so CMake picks the correct compiler
echo "CXX=cl.exe" >> $GITHUB_ENV
echo "CC=cl.exe" >> $GITHUB_ENV
if: matrix.os == 'windows-latest'
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- name: Get Sources
uses: actions/checkout@v2
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- name: Get Sources
uses: actions/checkout@v3
- name: Autotools Configure
if: matrix.generator == 'autogen'
run: |
sh ./autogen.sh
sh ./bin/chkmanifest
mkdir "${{ runner.workspace }}/build"
cd "${{ runner.workspace }}/build"
$GITHUB_WORKSPACE/configure --enable-shared --${{ matrix.ts }}-threadsafe --${{ matrix.hl }}-hl --${{ matrix.parallel }}-parallel --${{ matrix.cpp }}-cxx --${{ matrix.fortran }}-fortran --${{ matrix.java }}-java
shell: bash
#
# AUTOTOOLS CONFIGURE
#
- name: Configure
if: matrix.generator != 'autogen'
run: |
mkdir "${{ runner.workspace }}/build"
cd "${{ runner.workspace }}/build"
cmake ${{ matrix.generator }} -DCMAKE_BUILD_TYPE=${{ matrix.build_type }} -DCMAKE_TOOLCHAIN_FILE=${{ matrix.toolchain }} -DBUILD_SHARED_LIBS=ON -DHDF5_ENABLE_ALL_WARNINGS=ON -DHDF5_ENABLE_THREADSAFE:BOOL=${{ matrix.ts }} -DHDF5_BUILD_HL_LIB:BOOL=${{ matrix.hl }} -DHDF5_ENABLE_PARALLEL:BOOL=${{ matrix.parallel }} -DHDF5_BUILD_CPP_LIB:BOOL=${{ matrix.cpp }} -DHDF5_BUILD_FORTRAN=${{ matrix.fortran }} -DHDF5_BUILD_JAVA=${{ matrix.java }} $GITHUB_WORKSPACE
shell: bash
- name: Autotools Configure
run: |
sh ./autogen.sh
mkdir "${{ runner.workspace }}/build"
cd "${{ runner.workspace }}/build"
${{ matrix.flags }} $GITHUB_WORKSPACE/configure --enable-build-mode=${{ matrix.build_mode.autotools }} --${{ matrix.deprec_sym }}-deprecated-symbols --with-default-api-version=${{ matrix.default_api }} --enable-shared --${{ matrix.parallel }}-parallel --${{ matrix.cpp }}-cxx --${{ matrix.fortran }}-fortran --${{ matrix.java }}-java --${{ matrix.mirror_vfd }}-mirror-vfd --${{ matrix.direct_vfd }}-direct-vfd
shell: bash
if: (matrix.generator == 'autogen') && (! matrix.thread_safe.enabled)
- name: Autotools Build
if: matrix.generator == 'autogen'
run: make
working-directory: ${{ runner.workspace }}/build
- name: Autotools Configure (Thread-Safe)
run: |
sh ./autogen.sh
mkdir "${{ runner.workspace }}/build"
cd "${{ runner.workspace }}/build"
${{ matrix.flags }} $GITHUB_WORKSPACE/configure --enable-build-mode=${{ matrix.build_mode.autotools }} --enable-shared --enable-threadsafe --disable-hl --${{ matrix.parallel }}-parallel --${{ matrix.mirror_vfd }}-mirror-vfd --${{ matrix.direct_vfd }}-direct-vfd
shell: bash
if: (matrix.generator == 'autogen') && (matrix.thread_safe.enabled)
- name: Build
if: matrix.generator != 'autogen'
run: cmake --build . --config ${{ matrix.build_type }}
working-directory: ${{ runner.workspace }}/build
#
# CMAKE CONFIGURE
#
- name: Autotools Test
if: matrix.generator == 'autogen'
run: make check
working-directory: ${{ runner.workspace }}/build
- name: CMake Configure
run: |
mkdir "${{ runner.workspace }}/build"
cd "${{ runner.workspace }}/build"
cmake ${{ matrix.generator }} -DCMAKE_BUILD_TYPE=${{ matrix.build_mode.cmake }} -DCMAKE_TOOLCHAIN_FILE=${{ matrix.toolchain }} -DBUILD_SHARED_LIBS=ON -DHDF5_ENABLE_ALL_WARNINGS=ON -DHDF5_ENABLE_PARALLEL:BOOL=${{ matrix.parallel }} -DHDF5_BUILD_CPP_LIB:BOOL=${{ matrix.cpp }} -DHDF5_BUILD_FORTRAN=${{ matrix.fortran }} -DHDF5_BUILD_JAVA=${{ matrix.java }} -DHDF5_ENABLE_MIRROR_VFD:BOOL=${{ matrix.mirror_vfd }} -DHDF5_ENABLE_DIRECT_VFD:BOOL=${{ matrix.direct_vfd }} $GITHUB_WORKSPACE
shell: bash
if: (matrix.generator != 'autogen') && (! matrix.thread_safe.enabled)
- name: Test
if: matrix.generator != 'autogen'
run: ctest --build . -C ${{ matrix.build_type }} -V
working-directory: ${{ runner.workspace }}/build
- name: CMake Configure (Thread-Safe)
run: |
mkdir "${{ runner.workspace }}/build"
cd "${{ runner.workspace }}/build"
cmake ${{ matrix.generator }} -DCMAKE_BUILD_TYPE=${{ matrix.build_mode.cmake }} -DCMAKE_TOOLCHAIN_FILE=${{ matrix.toolchain }} -DBUILD_SHARED_LIBS=ON -DHDF5_ENABLE_ALL_WARNINGS=ON -DHDF5_ENABLE_THREADSAFE:BOOL=ON -DHDF5_ENABLE_PARALLEL:BOOL=${{ matrix.parallel }} -DHDF5_BUILD_HL_LIB:BOOL=OFF -DHDF5_ENABLE_MIRROR_VFD:BOOL=${{ matrix.mirror_vfd }} -DHDF5_ENABLE_DIRECT_VFD:BOOL=${{ matrix.direct_vfd }} $GITHUB_WORKSPACE
shell: bash
if: (matrix.generator != 'autogen') && (matrix.thread_safe.enabled)
#
# BUILD
#
- name: Autotools Build
run: make
working-directory: ${{ runner.workspace }}/build
if: matrix.generator == 'autogen'
- name: CMake Build
run: cmake --build . --config ${{ matrix.build_mode.cmake }}
working-directory: ${{ runner.workspace }}/build
if: matrix.generator != 'autogen'
#
# RUN TESTS
#
- name: Autotools Run Tests
run: make check
working-directory: ${{ runner.workspace }}/build
if: (matrix.generator == 'autogen') && (matrix.run_tests)
- name: CMake Run Tests
run: ctest --build . -C ${{ matrix.build_mode.cmake }} -V
working-directory: ${{ runner.workspace }}/build
# Skip Debug MSVC while we investigate H5L Java test timeouts
if: (matrix.generator != 'autogen') && (matrix.run_tests) && ! ((matrix.name == 'Windows MSVC CMake') && (matrix.build_mode.cmake == 'Debug'))
#
# INSTALL (note that this runs even when we don't run the tests)
#
- name: Autotools Install
run: make install
working-directory: ${{ runner.workspace }}/build
if: (matrix.generator == 'autogen')
- name: Autotools Verify Install
run: make check-install
working-directory: ${{ runner.workspace }}/build
if: (matrix.generator == 'autogen')

View File

@@ -1,214 +0,0 @@
name: hdf5 dev CI
# Controls when the action will run. Triggers the workflow on push or pull request
on:
pull_request:
branches: [ develop, hdf5_1_12, hdf5_1_10, hdf5_1_8 ]
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
strategy:
# fail-fast: false
matrix:
name: ["Windows Latest MSVC", "Ubuntu Latest GCC", "Ubuntu Debug GCC", "macOS Latest Clang", "Ubuntu Autotools GCC", "Windows TS MSVC", "Ubuntu TS GCC", "TS Debug GCC", "macOS TS Clang", "TS Autotools GCC"]
include:
- name: "Windows Latest MSVC"
artifact: "Windows-MSVC.tar.xz"
os: windows-2022
build_type: "Release"
toolchain: ""
cpp: ON
fortran: OFF
java: ON
ts: OFF
hl: ON
parallel: OFF
generator: "-G \"Visual Studio 17 2022\" -A x64"
- name: "Ubuntu Latest GCC"
artifact: "Linux.tar.xz"
os: ubuntu-latest
build_type: "Release"
cpp: ON
fortran: ON
java: ON
ts: OFF
hl: ON
parallel: OFF
toolchain: "config/toolchain/GCC.cmake"
generator: "-G Ninja"
- name: "macOS Latest Clang"
artifact: "macOS.tar.xz"
os: macos-latest
build_type: "Release"
cpp: ON
fortran: OFF
java: ON
ts: OFF
hl: ON
parallel: OFF
toolchain: "config/toolchain/clang.cmake"
generator: "-G Ninja"
- name: "Ubuntu Debug GCC"
artifact: "LinuxDBG.tar.xz"
os: ubuntu-latest
build_type: "Debug"
cpp: ON
fortran: OFF
java: OFF
ts: OFF
hl: ON
parallel: OFF
toolchain: "config/toolchain/GCC.cmake"
generator: "-G Ninja"
- name: "Ubuntu Autotools GCC"
artifact: "LinuxA.tar.xz"
os: ubuntu-latest
build_type: "Release"
cpp: enable
fortran: enable
java: enable
ts: disable
hl: enable
parallel: disable
toolchain: ""
generator: "autogen"
# Threadsafe runs
- name: "Windows TS MSVC"
artifact: "Windows-MSVCTS.tar.xz"
os: windows-2019
build_type: "Release"
toolchain: ""
cpp: OFF
fortran: OFF
java: OFF
ts: ON
hl: OFF
parallel: OFF
generator: "-G \"Visual Studio 16 2019\" -A x64"
- name: "Ubuntu TS GCC"
artifact: "LinuxTS.tar.xz"
os: ubuntu-latest
build_type: "Release"
cpp: OFF
fortran: OFF
java: OFF
ts: ON
hl: OFF
parallel: OFF
toolchain: "config/toolchain/GCC.cmake"
generator: "-G Ninja"
- name: "macOS TS Clang"
artifact: "macOSTS.tar.xz"
os: macos-latest
build_type: "Release"
cpp: OFF
fortran: OFF
java: OFF
ts: ON
hl: OFF
parallel: OFF
toolchain: "config/toolchain/clang.cmake"
generator: "-G Ninja"
- name: "TS Debug GCC"
artifact: "LinuxTSDBG.tar.xz"
os: ubuntu-latest
build_type: "Debug"
cpp: OFF
fortran: OFF
java: OFF
ts: ON
hl: OFF
parallel: OFF
toolchain: "config/toolchain/GCC.cmake"
generator: "-G Ninja"
- name: "TS Autotools GCC"
artifact: "LinuxATS.tar.xz"
os: ubuntu-latest
build_type: "Release"
cpp: disable
fortran: disable
java: disable
ts: enable
hl: disable
parallel: disable
toolchain: ""
generator: "autogen"
# - name: "Ubuntu Parallel GCC"
# artifact: "LinuxPar.tar.xz"
# os: ubuntu-latest
# build_type: "Release"
# cpp: OFF
# fortran: OFF
# parallel: ON
# toolchain: "config/toolchain/GCC.cmake"
# generator: "-G Ninja"
name: ${{ matrix.name }}
# The type of runner that the job will run on
runs-on: ${{ matrix.os }}
if: "!contains(github.event.head_commit.message, 'skip-ci')"
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- name: Install Dependencies (Linux)
run: sudo apt-get install ninja-build
if: matrix.os == 'ubuntu-latest'
- name: Install Autotools Dependencies (Linux)
run: sudo apt-get install automake autoconf libtool libtool-bin
if: matrix.generator == 'autogen'
- name: Install Dependencies (Windows)
run: choco install ninja
if: matrix.os == 'windows-latest'
- name: Install Dependencies (macOS)
run: brew install ninja
if: matrix.os == 'macos-latest'
- name: Set environment for MSVC (Windows)
if: matrix.os == 'windows-latest'
run: |
# Set these env vars so cmake picks the correct compiler
echo "CXX=cl.exe" >> $GITHUB_ENV
echo "CC=cl.exe" >> $GITHUB_ENV
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- name: Get Sources
uses: actions/checkout@v2
- name: Autotools Configure
if: matrix.generator == 'autogen'
run: |
sh ./autogen.sh
sh ./bin/chkmanifest
mkdir "${{ runner.workspace }}/build"
cd "${{ runner.workspace }}/build"
$GITHUB_WORKSPACE/configure --enable-shared --${{ matrix.ts }}-threadsafe --${{ matrix.hl }}-hl --${{ matrix.parallel }}-parallel --${{ matrix.cpp }}-cxx --${{ matrix.fortran }}-fortran --${{ matrix.java }}-java
shell: bash
- name: Configure
if: matrix.generator != 'autogen'
run: |
mkdir "${{ runner.workspace }}/build"
cd "${{ runner.workspace }}/build"
cmake ${{ matrix.generator }} -DCMAKE_BUILD_TYPE=${{ matrix.build_type }} -DCMAKE_TOOLCHAIN_FILE=${{ matrix.toolchain }} -DBUILD_SHARED_LIBS=ON -DHDF5_ENABLE_ALL_WARNINGS=ON -DHDF5_ENABLE_THREADSAFE:BOOL=${{ matrix.ts }} -DHDF5_BUILD_HL_LIB:BOOL=${{ matrix.hl }} -DHDF5_ENABLE_PARALLEL:BOOL=${{ matrix.parallel }} -DHDF5_BUILD_CPP_LIB:BOOL=${{ matrix.cpp }} -DHDF5_BUILD_FORTRAN=${{ matrix.fortran }} -DHDF5_BUILD_JAVA=${{ matrix.java }} $GITHUB_WORKSPACE
shell: bash
- name: Autotools Build
if: matrix.generator == 'autogen'
run: make
working-directory: ${{ runner.workspace }}/build
- name: Build
if: matrix.generator != 'autogen'
run: cmake --build . --config ${{ matrix.build_type }}
working-directory: ${{ runner.workspace }}/build
- name: Autotools Test
if: matrix.generator == 'autogen'
run: make check
working-directory: ${{ runner.workspace }}/build
- name: Test
if: matrix.generator != 'autogen'
run: ctest --build . -C ${{ matrix.build_type }} -V
working-directory: ${{ runner.workspace }}/build

View File

@@ -1,5 +1,4 @@
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

9
BRANCH.md Normal file
View File

@@ -0,0 +1,9 @@
# Architecture Documentation Branch
The purpose of this branch is to store work-in-progress architecture
documentation while it is being written. This documentation will live in
`doc/arch` (at least temporarily) and will probably NOT be in Doxygen format.
This branch can be deleted when the architecture documentation has been
merged back to develop

View File

@@ -19,7 +19,7 @@ include (FetchContent)
set (HDF5_ALLOW_EXTERNAL_SUPPORT "NO" CACHE STRING "Allow External Library Building (NO GIT TGZ)")
set_property (CACHE HDF5_ALLOW_EXTERNAL_SUPPORT PROPERTY STRINGS NO GIT TGZ)
if (HDF5_ALLOW_EXTERNAL_SUPPORT MATCHES "GIT" OR HDF5_ALLOW_EXTERNAL_SUPPORT MATCHES "TGZ")
option (ZLIB_USE_EXTERNAL "Use External Library Building for ZLIB" 1)
option (ZLIB_USE_EXTERNAL "Use External Library Building for HDF5_ZLIB" 1)
option (SZIP_USE_EXTERNAL "Use External Library Building for SZIP" 1)
if (HDF5_ALLOW_EXTERNAL_SUPPORT MATCHES "GIT")
set (ZLIB_URL ${ZLIB_GIT_URL} CACHE STRING "Path to zlib git repository")
@@ -33,9 +33,7 @@ if (HDF5_ALLOW_EXTERNAL_SUPPORT MATCHES "GIT" OR HDF5_ALLOW_EXTERNAL_SUPPORT MAT
set (ZLIB_URL ${TGZPATH}/${ZLIB_TGZ_NAME})
if (NOT EXISTS "${ZLIB_URL}")
set (HDF5_ENABLE_Z_LIB_SUPPORT OFF CACHE BOOL "" FORCE)
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "Filter ZLIB file ${ZLIB_URL} not found")
endif ()
message (VERBOSE "Filter ZLIB file ${ZLIB_URL} not found")
endif ()
set (SZIP_URL ${TGZPATH}/${SZIP_TGZ_NAME})
if (USE_LIBAEC)
@@ -43,9 +41,7 @@ if (HDF5_ALLOW_EXTERNAL_SUPPORT MATCHES "GIT" OR HDF5_ALLOW_EXTERNAL_SUPPORT MAT
endif ()
if (NOT EXISTS "${SZIP_URL}")
set (HDF5_ENABLE_SZIP_SUPPORT OFF CACHE BOOL "" FORCE)
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "Filter SZIP file ${SZIP_URL} not found")
endif ()
message (VERBOSE "Filter SZIP file ${SZIP_URL} not found")
endif ()
else ()
set (ZLIB_USE_EXTERNAL 0)
@@ -81,9 +77,7 @@ if (HDF5_ENABLE_Z_LIB_SUPPORT)
set (H5_HAVE_FILTER_DEFLATE 1)
set (H5_HAVE_ZLIB_H 1)
set (H5_HAVE_LIBZ 1)
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "Filter ZLIB is built")
endif ()
message (VERBOSE "Filter HDF5_ZLIB is built")
else ()
message (FATAL_ERROR " ZLib is Required for ZLib support in HDF5")
endif ()
@@ -99,9 +93,7 @@ if (HDF5_ENABLE_Z_LIB_SUPPORT)
endif ()
set (LINK_COMP_LIBS ${LINK_COMP_LIBS} ${ZLIB_STATIC_LIBRARY})
INCLUDE_DIRECTORIES (${ZLIB_INCLUDE_DIRS})
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "Filter ZLIB is ON")
endif ()
message (VERBOSE "Filter HDF5_ZLIB is ON")
endif ()
#-----------------------------------------------------------------------------
@@ -142,16 +134,12 @@ if (HDF5_ENABLE_SZIP_SUPPORT)
set (H5_HAVE_FILTER_SZIP 1)
set (H5_HAVE_SZLIB_H 1)
set (H5_HAVE_LIBSZ 1)
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "Filter SZIP is built")
endif ()
message (VERBOSE "Filter SZIP is built")
if (USE_LIBAEC)
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "... with library AEC")
endif ()
set (SZ_PACKAGE_NAME ${LIBAEC_PACKAGE_NAME})
message (VERBOSE "... with library AEC")
set (SZIP_PACKAGE_NAME ${LIBAEC_PACKAGE_NAME})
else ()
set (SZ_PACKAGE_NAME ${SZIP_PACKAGE_NAME})
set (SZIP_PACKAGE_NAME ${SZIP_PACKAGE_NAME})
endif ()
else ()
message (FATAL_ERROR "SZIP is Required for SZIP support in HDF5")
@@ -159,9 +147,7 @@ if (HDF5_ENABLE_SZIP_SUPPORT)
endif ()
set (LINK_COMP_LIBS ${LINK_COMP_LIBS} ${SZIP_STATIC_LIBRARY})
INCLUDE_DIRECTORIES (${SZIP_INCLUDE_DIRS})
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "Filter SZIP is ON")
endif ()
message (VERBOSE "Filter SZIP is ON")
if (H5_HAVE_FILTER_SZIP)
set (EXTERNAL_FILTERS "${EXTERNAL_FILTERS} DECODE")
endif ()

View File

@@ -52,7 +52,7 @@ endif ()
# Set includes needed for build
#-----------------------------------------------------------------------------
set (HDF5_INCLUDES_BUILD_TIME
${HDF5_SRC_DIR} ${HDF5_CPP_SRC_DIR} ${HDF5_HL_SRC_DIR}
${HDF5_SRC_INCLUDE_DIRS} ${HDF5_CPP_SRC_DIR} ${HDF5_HL_SRC_DIR}
${HDF5_TOOLS_SRC_DIR} ${HDF5_SRC_BINARY_DIR}
)
@@ -242,7 +242,7 @@ if (NOT HDF5_EXTERNALLY_CONFIGURED)
endif ()
install (
FILES ${release_files}
DESTINATION ${HDF5_INSTALL_DATA_DIR}
DESTINATION ${HDF5_INSTALL_DOC_DIR}
COMPONENT hdfdocuments
)
endif ()
@@ -273,7 +273,7 @@ if (NOT HDF5_EXTERNALLY_CONFIGURED AND NOT HDF5_NO_PACKAGES)
else ()
set (CPACK_PACKAGE_INSTALL_DIRECTORY "${CPACK_PACKAGE_VENDOR}/${CPACK_PACKAGE_NAME}/${CPACK_PACKAGE_VERSION}")
endif ()
set (CPACK_PACKAGE_ICON "${HDF_RESOURCES_EXT_DIR}/hdf.bmp")
set (CPACK_PACKAGE_ICON "${HDF_RESOURCES_DIR}/hdf.bmp")
set (CPACK_GENERATOR "TGZ")
if (WIN32)
@@ -296,10 +296,10 @@ if (NOT HDF5_EXTERNALLY_CONFIGURED AND NOT HDF5_NO_PACKAGES)
endif ()
# set the install/unistall icon used for the installer itself
# There is a bug in NSI that does not handle full unix paths properly.
set (CPACK_NSIS_MUI_ICON "${HDF_RESOURCES_EXT_DIR}\\\\hdf.ico")
set (CPACK_NSIS_MUI_UNIICON "${HDF_RESOURCES_EXT_DIR}\\\\hdf.ico")
set (CPACK_NSIS_MUI_ICON "${HDF_RESOURCES_DIR}\\\\hdf.ico")
set (CPACK_NSIS_MUI_UNIICON "${HDF_RESOURCES_DIR}\\\\hdf.ico")
# set the package header icon for MUI
set (CPACK_PACKAGE_ICON "${HDF_RESOURCES_EXT_DIR}\\\\hdf.bmp")
set (CPACK_PACKAGE_ICON "${HDF_RESOURCES_DIR}\\\\hdf.bmp")
set (CPACK_NSIS_DISPLAY_NAME "${CPACK_NSIS_PACKAGE_NAME}")
if (OVERRIDE_INSTALL_VERSION)
set (CPACK_PACKAGE_INSTALL_DIRECTORY "${CPACK_PACKAGE_VENDOR}\\\\${CPACK_PACKAGE_NAME}\\\\${OVERRIDE_INSTALL_VERSION}")
@@ -329,7 +329,7 @@ if (NOT HDF5_EXTERNALLY_CONFIGURED AND NOT HDF5_NO_PACKAGES)
set (CPACK_RESOURCE_FILE_LICENSE "${HDF5_BINARY_DIR}/COPYING.txt")
# .. variable:: CPACK_WIX_PRODUCT_ICON
# The Icon shown next to the program name in Add/Remove programs.
set(CPACK_WIX_PRODUCT_ICON "${HDF_RESOURCES_EXT_DIR}\\\\hdf.ico")
set(CPACK_WIX_PRODUCT_ICON "${HDF_RESOURCES_DIR}\\\\hdf.ico")
#
# .. variable:: CPACK_WIX_UI_BANNER
#
@@ -360,14 +360,14 @@ if (NOT HDF5_EXTERNALLY_CONFIGURED AND NOT HDF5_NO_PACKAGES)
list (APPEND CPACK_GENERATOR "DragNDrop")
set (CPACK_COMPONENTS_ALL_IN_ONE_PACKAGE ON)
set (CPACK_PACKAGING_INSTALL_PREFIX "/${CPACK_PACKAGE_INSTALL_DIRECTORY}")
set (CPACK_PACKAGE_ICON "${HDF_RESOURCES_EXT_DIR}/hdf.icns")
set (CPACK_PACKAGE_ICON "${HDF_RESOURCES_DIR}/hdf.icns")
option (HDF5_PACK_MACOSX_FRAMEWORK "Package the HDF5 Library in a Frameworks" OFF)
if (HDF5_PACK_MACOSX_FRAMEWORK AND HDF5_BUILD_FRAMEWORKS)
set (CPACK_BUNDLE_NAME "${HDF5_PACKAGE_STRING}")
set (CPACK_BUNDLE_LOCATION "/") # make sure CMAKE_INSTALL_PREFIX ends in /
set (CMAKE_INSTALL_PREFIX "/${CPACK_BUNDLE_NAME}.framework/Versions/${CPACK_PACKAGE_VERSION}/${CPACK_PACKAGE_NAME}/")
set (CPACK_BUNDLE_ICON "${HDF_RESOURCES_EXT_DIR}/hdf.icns")
set (CPACK_BUNDLE_ICON "${HDF_RESOURCES_DIR}/hdf.icns")
set (CPACK_BUNDLE_PLIST "${HDF5_BINARY_DIR}/CMakeFiles/Info.plist")
set (CPACK_SHORT_VERSION_STRING "${CPACK_PACKAGE_VERSION}")
#-----------------------------------------------------------------------------
@@ -382,7 +382,7 @@ if (NOT HDF5_EXTERNALLY_CONFIGURED AND NOT HDF5_NO_PACKAGES)
${HDF5_BINARY_DIR}/CMakeFiles/PkgInfo @ONLY
)
configure_file (
${HDF_RESOURCES_EXT_DIR}/version.plist.in
${HDF_RESOURCES_DIR}/version.plist.in
${HDF5_BINARY_DIR}/CMakeFiles/version.plist @ONLY
)
install (
@@ -395,13 +395,19 @@ if (NOT HDF5_EXTERNALLY_CONFIGURED AND NOT HDF5_NO_PACKAGES)
set (CPACK_PACKAGING_INSTALL_PREFIX "/${CPACK_PACKAGE_INSTALL_DIRECTORY}")
set (CPACK_COMPONENTS_ALL_IN_ONE_PACKAGE ON)
list (APPEND CPACK_GENERATOR "DEB")
set (CPACK_DEBIAN_PACKAGE_SECTION "Libraries")
set (CPACK_DEBIAN_PACKAGE_MAINTAINER "${HDF5_PACKAGE_BUGREPORT}")
# list (APPEND CPACK_GENERATOR "RPM")
list (APPEND CPACK_GENERATOR "RPM")
set (CPACK_RPM_PACKAGE_RELEASE "1")
set (CPACK_RPM_PACKAGE_RELEASE_DIST ON)
set (CPACK_RPM_COMPONENT_INSTALL ON)
set (CPACK_RPM_PACKAGE_RELOCATABLE ON)
set (CPACK_RPM_FILE_NAME "RPM-DEFAULT")
set (CPACK_RPM_PACKAGE_NAME "${CPACK_PACKAGE_NAME}")
set (CPACK_RPM_PACKAGE_VERSION "${CPACK_PACKAGE_VERSION}")
set (CPACK_RPM_PACKAGE_VENDOR "${CPACK_PACKAGE_VENDOR}")
set (CPACK_RPM_PACKAGE_LICENSE "BSD-style")
set (CPACK_RPM_PACKAGE_GROUP "Development/Libraries")
set (CPACK_RPM_PACKAGE_URL "${HDF5_PACKAGE_URL}")
@@ -442,11 +448,11 @@ The HDF5 data model, file format, API, library, and tools are open and distribut
if (HDF5_ALLOW_EXTERNAL_SUPPORT MATCHES "GIT" OR HDF5_ALLOW_EXTERNAL_SUPPORT MATCHES "TGZ")
if (ZLIB_FOUND AND ZLIB_USE_EXTERNAL)
if (WIN32)
set (CPACK_INSTALL_CMAKE_PROJECTS "${CPACK_INSTALL_CMAKE_PROJECTS};${ZLIB_INCLUDE_DIR_GEN};ZLIB;ALL;/")
set (CPACK_INSTALL_CMAKE_PROJECTS "${CPACK_INSTALL_CMAKE_PROJECTS};${ZLIB_INCLUDE_DIR_GEN};HDF5_ZLIB;ALL;/")
else ()
set (CPACK_INSTALL_CMAKE_PROJECTS "${CPACK_INSTALL_CMAKE_PROJECTS};${ZLIB_INCLUDE_DIR_GEN};ZLIB;libraries;/")
set (CPACK_INSTALL_CMAKE_PROJECTS "${CPACK_INSTALL_CMAKE_PROJECTS};${ZLIB_INCLUDE_DIR_GEN};ZLIB;headers;/")
set (CPACK_INSTALL_CMAKE_PROJECTS "${CPACK_INSTALL_CMAKE_PROJECTS};${ZLIB_INCLUDE_DIR_GEN};ZLIB;configinstall;/")
set (CPACK_INSTALL_CMAKE_PROJECTS "${CPACK_INSTALL_CMAKE_PROJECTS};${ZLIB_INCLUDE_DIR_GEN};HDF5_ZLIB;libraries;/")
set (CPACK_INSTALL_CMAKE_PROJECTS "${CPACK_INSTALL_CMAKE_PROJECTS};${ZLIB_INCLUDE_DIR_GEN};HDF5_ZLIB;headers;/")
set (CPACK_INSTALL_CMAKE_PROJECTS "${CPACK_INSTALL_CMAKE_PROJECTS};${ZLIB_INCLUDE_DIR_GEN};HDF5_ZLIB;configinstall;/")
endif ()
endif ()
if (SZIP_FOUND AND SZIP_USE_EXTERNAL)

View File

@@ -1,14 +1,22 @@
cmake_minimum_required (VERSION 3.12)
cmake_minimum_required (VERSION 3.18)
project (HDF5 C)
if (POLICY CMP0074)
# find_package() uses <PackageName>_ROOT variables.
cmake_policy (SET CMP0074 NEW)
endif ()
if (POLICY CMP0083)
# To control generation of Position Independent Executable (PIE) or not,
# some flags are required at link time.
cmake_policy (SET CMP0083 NEW)
endif ()
# Avoid warning about DOWNLOAD_EXTRACT_TIMESTAMP in CMake 3.24:
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.24.0")
cmake_policy(SET CMP0135 NEW)
endif()
#-----------------------------------------------------------------------------
# Instructions for use : Normal Build
#
@@ -36,16 +44,6 @@ if (CMAKE_CURRENT_SOURCE_DIR STREQUAL CMAKE_CURRENT_BINARY_DIR)
)
endif ()
# CMake version 3.14 added option --ignore-eol to compare files
# cmake -E compare_files --ignore-eol file1 file2
set (CMAKE_IGNORE_EOL "--ignore-eol")
if (CMAKE_VERSION VERSION_LESS "3.14.0")
set (CMAKE_IGNORE_EOL "")
if (WIN32)
message (FATAL_ERROR "Windows builds require a minimum of CMake 3.14")
endif()
endif ()
#-----------------------------------------------------------------------------
# Instructions for use : Sub-Project Build
#
@@ -212,7 +210,6 @@ set (HDF5_HL_F90_C_LIBSH_TARGET "${HDF5_HL_F90_C_LIB_CORENAME}-shared")
#-----------------------------------------------------------------------------
set (HDF_CONFIG_DIR ${HDF5_SOURCE_DIR}/config)
set (HDF_RESOURCES_DIR ${HDF5_SOURCE_DIR}/config/cmake)
set (HDF_RESOURCES_EXT_DIR ${HDF5_SOURCE_DIR}/config/cmake_ext_mod)
set (HDF5_SRC_DIR ${HDF5_SOURCE_DIR}/src)
set (HDF5_TEST_SRC_DIR ${HDF5_SOURCE_DIR}/test)
set (HDF5_CPP_SRC_DIR ${HDF5_SOURCE_DIR}/c++)
@@ -234,6 +231,8 @@ set (HDF5_JAVA_LOGGING_NOP_JAR ${HDF5_SOURCE_DIR}/java/lib/ext/slf4j-nop-1.7
set (HDF5_JAVA_LOGGING_SIMPLE_JAR ${HDF5_SOURCE_DIR}/java/lib/ext/slf4j-simple-1.7.33.jar)
set (HDF5_DOXYGEN_DIR ${HDF5_SOURCE_DIR}/doxygen)
set (HDF5_SRC_INCLUDE_DIRS ${HDF5_SRC_DIR})
#-----------------------------------------------------------------------------
# parse the full version number from H5public.h and include in H5_VERS_INFO
#-----------------------------------------------------------------------------
@@ -246,9 +245,7 @@ string (REGEX REPLACE ".*#define[ \t]+H5_VERS_RELEASE[ \t]+([0-9]*).*$"
"\\1" H5_VERS_RELEASE ${_h5public_h_contents})
string (REGEX REPLACE ".*#define[ \t]+H5_VERS_SUBRELEASE[ \t]+\"([0-9A-Za-z._-]*)\".*$"
"\\1" H5_VERS_SUBRELEASE ${_h5public_h_contents})
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (TRACE "VERSION: ${H5_VERS_MAJOR}.${H5_VERS_MINOR}.${H5_VERS_RELEASE}-${H5_VERS_SUBRELEASE}")
endif ()
message (TRACE "VERSION: ${H5_VERS_MAJOR}.${H5_VERS_MINOR}.${H5_VERS_RELEASE}-${H5_VERS_SUBRELEASE}")
#-----------------------------------------------------------------------------
# parse the full soversion number from config/lt_vers.am and include in H5_SOVERS_INFO
@@ -261,9 +258,7 @@ string (REGEX REPLACE ".*LT_VERS_REVISION[ \t]+=[ \t]+([0-9]*).*$"
string (REGEX REPLACE ".*LT_VERS_AGE[ \t]+=[ \t]+([0-9]*).*$"
"\\1" H5_LIB_SOVERS_RELEASE ${_lt_vers_am_contents})
math (EXPR H5_LIB_SOVERS_MAJOR ${H5_LIB_SOVERS_INTERFACE}-${H5_LIB_SOVERS_RELEASE})
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "SOVERSION: ${H5_LIB_SOVERS_MAJOR}.${H5_LIB_SOVERS_RELEASE}.${H5_LIB_SOVERS_MINOR}")
endif ()
message (VERBOSE "SOVERSION: ${H5_LIB_SOVERS_MAJOR}.${H5_LIB_SOVERS_RELEASE}.${H5_LIB_SOVERS_MINOR}")
string (REGEX MATCH ".*LT_TOOLS_VERS_INTERFACE[ \t]+=[ \t]+([0-9]*).*$" H5_TOOLS_SOVERS_EXISTS ${_lt_vers_am_contents})
if (H5_TOOLS_SOVERS_EXISTS)
string (REGEX REPLACE ".*LT_TOOLS_VERS_INTERFACE[ \t]+=[ \t]+([0-9]*).*$"
@@ -273,9 +268,7 @@ if (H5_TOOLS_SOVERS_EXISTS)
string (REGEX REPLACE ".*LT_TOOLS_VERS_AGE[ \t]+=[ \t]+([0-9]*).*$"
"\\1" H5_TOOLS_SOVERS_RELEASE ${_lt_vers_am_contents})
math (EXPR H5_TOOLS_SOVERS_MAJOR ${H5_TOOLS_SOVERS_INTERFACE}-${H5_TOOLS_SOVERS_RELEASE})
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "SOVERSION_TOOLS: ${H5_TOOLS_SOVERS_MAJOR}.${H5_TOOLS_SOVERS_RELEASE}.${H5_TOOLS_SOVERS_MINOR}")
endif ()
message (VERBOSE "SOVERSION_TOOLS: ${H5_TOOLS_SOVERS_MAJOR}.${H5_TOOLS_SOVERS_RELEASE}.${H5_TOOLS_SOVERS_MINOR}")
endif ()
string (REGEX MATCH ".*LT_CXX_VERS_INTERFACE[ \t]+=[ \t]+([0-9]*).*$" H5_CXX_SOVERS_EXISTS ${_lt_vers_am_contents})
if (H5_CXX_SOVERS_EXISTS)
@@ -286,9 +279,7 @@ if (H5_CXX_SOVERS_EXISTS)
string (REGEX REPLACE ".*LT_CXX_VERS_AGE[ \t]+=[ \t]+([0-9]*).*$"
"\\1" H5_CXX_SOVERS_RELEASE ${_lt_vers_am_contents})
math (EXPR H5_CXX_SOVERS_MAJOR ${H5_CXX_SOVERS_INTERFACE}-${H5_CXX_SOVERS_RELEASE})
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "SOVERSION_CXX: ${H5_CXX_SOVERS_MAJOR}.${H5_CXX_SOVERS_RELEASE}.${H5_CXX_SOVERS_MINOR}")
endif ()
message (VERBOSE "SOVERSION_CXX: ${H5_CXX_SOVERS_MAJOR}.${H5_CXX_SOVERS_RELEASE}.${H5_CXX_SOVERS_MINOR}")
endif ()
string (REGEX MATCH ".*LT_F_VERS_INTERFACE[ \t]+=[ \t]+([0-9]*).*$" H5_F_SOVERS_EXISTS ${_lt_vers_am_contents})
if (H5_F_SOVERS_EXISTS)
@@ -299,9 +290,7 @@ if (H5_F_SOVERS_EXISTS)
string (REGEX REPLACE ".*LT_F_VERS_AGE[ \t]+=[ \t]+([0-9]*).*$"
"\\1" H5_F_SOVERS_RELEASE ${_lt_vers_am_contents})
math (EXPR H5_F_SOVERS_MAJOR ${H5_F_SOVERS_INTERFACE}-${H5_F_SOVERS_RELEASE})
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "SOVERSION_F: ${H5_F_SOVERS_MAJOR}.${H5_F_SOVERS_RELEASE}.${H5_F_SOVERS_MINOR}")
endif ()
message (VERBOSE "SOVERSION_F: ${H5_F_SOVERS_MAJOR}.${H5_F_SOVERS_RELEASE}.${H5_F_SOVERS_MINOR}")
endif ()
string (REGEX MATCH ".*LT_HL_VERS_INTERFACE[ \t]+=[ \t]+([0-9]*).*$" H5_HL_SOVERS_EXISTS ${_lt_vers_am_contents})
if (H5_HL_SOVERS_EXISTS)
@@ -312,9 +301,7 @@ if (H5_HL_SOVERS_EXISTS)
string (REGEX REPLACE ".*LT_HL_VERS_AGE[ \t]+=[ \t]+([0-9]*).*$"
"\\1" H5_HL_SOVERS_RELEASE ${_lt_vers_am_contents})
math (EXPR H5_HL_SOVERS_MAJOR ${H5_HL_SOVERS_INTERFACE}-${H5_HL_SOVERS_RELEASE})
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "SOVERSION_HL: ${H5_HL_SOVERS_MAJOR}.${H5_HL_SOVERS_RELEASE}.${H5_HL_SOVERS_MINOR}")
endif ()
message (VERBOSE "SOVERSION_HL: ${H5_HL_SOVERS_MAJOR}.${H5_HL_SOVERS_RELEASE}.${H5_HL_SOVERS_MINOR}")
endif ()
string (REGEX MATCH ".*LT_HL_CXX_VERS_INTERFACE[ \t]+=[ \t]+([0-9]*).*$" H5_HL_CXX_SOVERS_EXISTS ${_lt_vers_am_contents})
if (H5_HL_CXX_SOVERS_EXISTS)
@@ -325,9 +312,7 @@ if (H5_HL_CXX_SOVERS_EXISTS)
string (REGEX REPLACE ".*LT_HL_CXX_VERS_AGE[ \t]+=[ \t]+([0-9]*).*$"
"\\1" H5_HL_CXX_SOVERS_RELEASE ${_lt_vers_am_contents})
math (EXPR H5_HL_CXX_SOVERS_MAJOR ${H5_HL_CXX_SOVERS_INTERFACE}-${H5_HL_CXX_SOVERS_RELEASE})
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "SOVERSION_HL_CXX: ${H5_HL_CXX_SOVERS_MAJOR}.${H5_HL_CXX_SOVERS_RELEASE}.${H5_HL_CXX_SOVERS_MINOR}")
endif ()
message (VERBOSE "SOVERSION_HL_CXX: ${H5_HL_CXX_SOVERS_MAJOR}.${H5_HL_CXX_SOVERS_RELEASE}.${H5_HL_CXX_SOVERS_MINOR}")
endif ()
string (REGEX MATCH ".*LT_HL_F_VERS_INTERFACE[ \t]+=[ \t]+([0-9]*).*$" H5_HL_F_SOVERS_EXISTS ${_lt_vers_am_contents})
if (H5_HL_F_SOVERS_EXISTS)
@@ -338,9 +323,7 @@ if (H5_HL_F_SOVERS_EXISTS)
string (REGEX REPLACE ".*LT_HL_F_VERS_AGE[ \t]+=[ \t]+([0-9]*).*$"
"\\1" H5_HL_F_SOVERS_RELEASE ${_lt_vers_am_contents})
math (EXPR H5_HL_F_SOVERS_MAJOR ${H5_HL_F_SOVERS_INTERFACE}-${H5_HL_F_SOVERS_RELEASE})
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "SOVERSION_HL_F: ${H5_HL_F_SOVERS_MAJOR}.${H5_HL_F_SOVERS_RELEASE}.${H5_HL_F_SOVERS_MINOR}")
endif ()
message (VERBOSE "SOVERSION_HL_F: ${H5_HL_F_SOVERS_MAJOR}.${H5_HL_F_SOVERS_RELEASE}.${H5_HL_F_SOVERS_MINOR}")
endif ()
string (REGEX MATCH ".*LT_JAVA_VERS_INTERFACE[ \t]+=[ \t]+([0-9]*).*$" H5_JAVA_SOVERS_EXISTS ${_lt_vers_am_contents})
if(H5_JAVA_SOVERS_EXISTS)
@@ -351,9 +334,7 @@ if(H5_JAVA_SOVERS_EXISTS)
string (REGEX REPLACE ".*LT_JAVA_VERS_AGE[ \t]+=[ \t]+([0-9]*).*$"
"\\1" H5_JAVA_SOVERS_RELEASE ${_lt_vers_am_contents})
math (EXPR H5_JAVA_SOVERS_MAJOR ${H5_JAVA_SOVERS_INTERFACE}-${H5_JAVA_SOVERS_RELEASE})
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "SOVERSION_JAVA: ${H5_JAVA_SOVERS_MAJOR}.${H5_JAVA_SOVERS_RELEASE}.${H5_JAVA_SOVERS_MINOR}")
endif ()
message (VERBOSE "SOVERSION_JAVA: ${H5_JAVA_SOVERS_MAJOR}.${H5_JAVA_SOVERS_RELEASE}.${H5_JAVA_SOVERS_MINOR}")
endif ()
#-----------------------------------------------------------------------------
@@ -430,11 +411,11 @@ set (HDF5_PACKAGE_BUGREPORT "help@hdfgroup.org")
#-----------------------------------------------------------------------------
# Include some macros for reusable code
#-----------------------------------------------------------------------------
include (${HDF_RESOURCES_EXT_DIR}/HDFMacros.cmake)
include (${HDF_RESOURCES_DIR}/HDFMacros.cmake)
HDF_DIR_PATHS(${HDF5_PACKAGE_NAME})
include (${HDF_RESOURCES_EXT_DIR}/HDFLibMacros.cmake)
include (${HDF_RESOURCES_DIR}/HDFLibMacros.cmake)
include (${HDF_RESOURCES_DIR}/HDF5PluginMacros.cmake)
include (${HDF_RESOURCES_DIR}/HDF5Macros.cmake)
@@ -562,15 +543,6 @@ if (HDF5_ENABLE_COVERAGE)
endif ()
endif ()
#-----------------------------------------------------------------------------
# Option to indicate using dmalloc
#-----------------------------------------------------------------------------
# option (HDF5_ENABLE_USING_DMALLOC "Indicate that dmalloc is used" OFF)
# if (HDF5_ENABLE_USING_DMALLOC)
# find_package (DMALLOC)
# set (H5_HAVE_DMALLOC DMALLOC_FOUND)
# endif ()
#-----------------------------------------------------------------------------
# Option to indicate using a memory checker
#-----------------------------------------------------------------------------
@@ -579,14 +551,6 @@ if (HDF5_ENABLE_USING_MEMCHECKER)
set (H5_USING_MEMCHECKER 1)
endif ()
#-----------------------------------------------------------------------------
# Option to indicate internal memory allocation sanity checks are enabled
#-----------------------------------------------------------------------------
option (HDF5_MEMORY_ALLOC_SANITY_CHECK "Indicate that internal memory allocation sanity checks are enabled" OFF)
if (HDF5_MEMORY_ALLOC_SANITY_CHECK)
set (H5_MEMORY_ALLOC_SANITY_CHECK 1)
endif ()
#-----------------------------------------------------------------------------
# Option to enable/disable using pread/pwrite for VFDs
#-----------------------------------------------------------------------------
@@ -654,6 +618,13 @@ if (${HDF_CFG_NAME} MATCHES "Debug")
mark_as_advanced (HDF5_ENABLE_INSTRUMENT)
endif ()
#-----------------------------------------------------------------------------
# Add some definitions for Developer Builds
#-----------------------------------------------------------------------------
if (${HDF_CFG_NAME} MATCHES "Developer")
include (${HDF_RESOURCES_DIR}/HDF5DeveloperBuild.cmake)
endif ()
#-----------------------------------------------------------------------------
# Option to embed library info into executables
#-----------------------------------------------------------------------------
@@ -667,7 +638,7 @@ else ()
endif ()
include (${HDF_RESOURCES_DIR}/HDFCompilerFlags.cmake)
set (CMAKE_MODULE_PATH ${HDF_RESOURCES_DIR} ${HDF_RESOURCES_EXT_DIR} ${CMAKE_MODULE_PATH})
set (CMAKE_MODULE_PATH ${HDF_RESOURCES_DIR} ${HDF_RESOURCES_DIR} ${CMAKE_MODULE_PATH})
#-----------------------------------------------------------------------------
# Option to Enable HDFS
@@ -708,10 +679,14 @@ if (HDF5_ENABLE_PARALLEL)
# Used by Parallel Compression feature
set (PARALLEL_FILTERED_WRITES ON)
CHECK_SYMBOL_EXISTS (MPI_Mprobe "mpi.h" H5_HAVE_MPI_Mprobe)
CHECK_SYMBOL_EXISTS (MPI_Imrecv "mpi.h" H5_HAVE_MPI_Imrecv)
if (NOT H5_HAVE_MPI_Mprobe OR NOT H5_HAVE_MPI_Imrecv)
message (WARNING "The MPI_Mprobe and/or MPI_Imrecv functions could not be located.
CHECK_SYMBOL_EXISTS (MPI_Ibarrier "mpi.h" H5_HAVE_MPI_Ibarrier)
CHECK_SYMBOL_EXISTS (MPI_Issend "mpi.h" H5_HAVE_MPI_Issend)
CHECK_SYMBOL_EXISTS (MPI_Iprobe "mpi.h" H5_HAVE_MPI_Iprobe)
CHECK_SYMBOL_EXISTS (MPI_Irecv "mpi.h" H5_HAVE_MPI_Irecv)
if (H5_HAVE_MPI_Ibarrier AND H5_HAVE_MPI_Issend AND H5_HAVE_MPI_Iprobe AND H5_HAVE_MPI_Irecv)
set (H5_HAVE_PARALLEL_FILTERED_WRITES 1)
else ()
message (WARNING "The MPI_Ibarrier/MPI_Issend/MPI_Iprobe/MPI_Irecv functions could not be located.
Parallel writes of filtered data will be disabled.")
set (PARALLEL_FILTERED_WRITES OFF)
endif ()
@@ -738,6 +713,56 @@ if (H5_HAVE_PARALLEL)
endif ()
endif ()
# Determine whether to build the HDF5 Subfiling VFD
set (H5FD_SUBFILING_DIR ${HDF5_SRC_DIR}/H5FDsubfiling)
set (HDF5_SRC_INCLUDE_DIRS
${HDF5_SRC_INCLUDE_DIRS}
${H5FD_SUBFILING_DIR}
)
option (HDF5_ENABLE_SUBFILING_VFD "Build Parallel HDF5 Subfiling VFD" OFF)
if (HDF5_ENABLE_SUBFILING_VFD)
if (NOT HDF5_ENABLE_PARALLEL)
message (FATAL_ERROR "Subfiling VFD requires a parallel HDF5 build")
else ()
# Check for MPI_Comm_split_type
CHECK_SYMBOL_EXISTS (MPI_Comm_split_type "mpi.h" H5_HAVE_MPI_Comm_split_type)
if (NOT H5_HAVE_MPI_Comm_split_type)
message (FATAL_ERROR "Subfiling VFD requires MPI-3 support for MPI_Comm_split_type")
endif ()
endif()
if (NOT DEFINED Threads_FOUND)
set (THREADS_PREFER_PTHREAD_FLAG ON)
find_package (Threads REQUIRED)
endif ()
# For now, make sure we're using pthreads. Once Subfiling can be
# supported on Windows, we should allow Win32 threads as well
if (NOT ${Threads_FOUND} OR NOT ${CMAKE_USE_PTHREADS_INIT})
message (FATAL_ERROR "Subfiling requires pthreads for system thread library")
endif ()
CHECK_INCLUDE_FILE("stdatomic.h" HAVE_STDATOMIC_H)
if (NOT HAVE_STDATOMIC_H)
message (FATAL_ERROR "Subfiling VFD requires atomic operations support. C11 stdatomic.h NOT available.")
else()
set (H5_HAVE_STDATOMIC_H 1)
endif()
set (H5_HAVE_SUBFILING_VFD 1)
# IOC VFD is currently only built when subfiling is enabled
set (H5_HAVE_IOC_VFD 1)
message (STATUS "Setting up to use Mercury components")
set (H5FD_SUBFILING_MERCURY_DIR ${H5FD_SUBFILING_DIR}/mercury/src/util)
set (HDF5_SRC_INCLUDE_DIRS
${HDF5_SRC_INCLUDE_DIRS}
${H5FD_SUBFILING_MERCURY_DIR}
)
set (H5_HAVE_MERCURY_H 1)
set (CMAKE_REQUIRED_INCLUDES "${H5FD_SUBFILING_MERCURY_DIR}")
endif()
#option (DEFAULT_API_VERSION "Enable v1.14 API (v16, v18, v110, v112, v114)" "v114")
set (DEFAULT_API_VERSION "v114" CACHE STRING "Enable v1.14 API (v16, v18, v110, v112, v114)")
set_property (CACHE DEFAULT_API_VERSION PROPERTY STRINGS v16 v18 v110 v112 v114)
@@ -787,6 +812,17 @@ if (DEFAULT_API_VERSION MATCHES "v114")
set (H5_USE_114_API_DEFAULT 1)
endif ()
#-----------------------------------------------------------------------------
# Option to use 1.16.x API
#-----------------------------------------------------------------------------
if (NOT DEFAULT_API_VERSION)
set (DEFAULT_API_VERSION "v116")
endif ()
set (H5_USE_116_API_DEFAULT 0)
if (DEFAULT_API_VERSION MATCHES "v116")
set (H5_USE_116_API_DEFAULT 1)
endif ()
#-----------------------------------------------------------------------------
# Include user macros
#-----------------------------------------------------------------------------
@@ -821,51 +857,32 @@ option (HDF5_ENABLE_THREADSAFE "Enable thread-safety" OFF)
if (HDF5_ENABLE_THREADSAFE)
# check for unsupported options
if (WIN32)
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE " **** thread-safety option not supported with static library **** ")
message (VERBOSE " **** thread-safety option will not be used building static library **** ")
endif ()
endif ()
if (HDF5_ENABLE_PARALLEL)
if (NOT ALLOW_UNSUPPORTED)
message (FATAL_ERROR " **** parallel and thread-safety options are not supported, override with ALLOW_UNSUPPORTED option **** ")
else ()
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE " **** Allowing unsupported parallel and thread-safety options **** ")
endif ()
endif ()
message (VERBOSE " **** thread-safety option not supported with static library **** ")
message (VERBOSE " **** thread-safety option will not be used building static library **** ")
endif ()
if (HDF5_BUILD_FORTRAN)
if (NOT ALLOW_UNSUPPORTED)
message (FATAL_ERROR " **** Fortran and thread-safety options are not supported, override with ALLOW_UNSUPPORTED option **** ")
else ()
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE " **** Allowing unsupported Fortran and thread-safety options **** ")
endif ()
message (VERBOSE " **** Allowing unsupported Fortran and thread-safety options **** ")
endif ()
endif ()
if (HDF5_BUILD_CPP_LIB)
if (NOT ALLOW_UNSUPPORTED)
message (FATAL_ERROR " **** C++ and thread-safety options are not supported, override with ALLOW_UNSUPPORTED option **** ")
else ()
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE " **** Allowing unsupported C++ and thread-safety options **** ")
endif ()
message (VERBOSE " **** Allowing unsupported C++ and thread-safety options **** ")
endif ()
endif ()
if (HDF5_BUILD_HL_LIB)
if (NOT ALLOW_UNSUPPORTED)
message (FATAL_ERROR " **** HL and thread-safety options are not supported, override with ALLOW_UNSUPPORTED option **** ")
else ()
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE " **** Allowing unsupported HL and thread-safety options **** ")
endif ()
message (VERBOSE " **** Allowing unsupported HL and thread-safety options **** ")
endif ()
endif ()
if (H5_HAVE_IOEO)
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE " **** Win32 threads requires WINVER>=0x600 (Windows Vista/7/8) **** ")
endif ()
message (VERBOSE " **** Win32 threads requires WINVER>=0x600 (Windows Vista/7/8) **** ")
set (H5_HAVE_WIN_THREADS 1)
else ()
if (NOT H5_HAVE_PTHREAD_H)
@@ -895,10 +912,10 @@ add_subdirectory (src)
if (HDF5_ALLOW_EXTERNAL_SUPPORT MATCHES "GIT" OR HDF5_ALLOW_EXTERNAL_SUPPORT MATCHES "TGZ")
if (ZLIB_FOUND AND ZLIB_USE_EXTERNAL)
if (NOT ONLY_SHARED_LIBS)
add_dependencies (${HDF5_LIB_TARGET} ZLIB)
add_dependencies (${HDF5_LIB_TARGET} HDF5_ZLIB)
endif ()
if (BUILD_SHARED_LIBS)
add_dependencies (${HDF5_LIBSH_TARGET} ZLIB)
add_dependencies (${HDF5_LIBSH_TARGET} HDF5_ZLIB)
endif ()
endif ()
if (SZIP_FOUND AND SZIP_USE_EXTERNAL)
@@ -958,8 +975,13 @@ if (BUILD_TESTING)
mark_as_advanced (HDF5_TEST_FHEAP_PASSTHROUGH VOL)
endif ()
option (HDF_TEST_EXPRESS "Control testing framework (0-3)" "0")
set (H5_TEST_EXPRESS_LEVEL_DEFAULT "3")
set (HDF_TEST_EXPRESS "${H5_TEST_EXPRESS_LEVEL_DEFAULT}"
CACHE STRING "Control testing framework (0-3) (0 = exhaustive testing; 3 = quicker testing)")
mark_as_advanced (HDF_TEST_EXPRESS)
if (NOT "${HDF_TEST_EXPRESS}" STREQUAL "")
set (H5_TEST_EXPRESS_LEVEL_DEFAULT "${HDF_TEST_EXPRESS}")
endif ()
enable_testing ()
include (CTest)
@@ -1092,11 +1114,9 @@ set (H5_FC_FUNC_ "H5_FC_FUNC_(name,NAME) name ## _")
if (EXISTS "${HDF5_SOURCE_DIR}/fortran" AND IS_DIRECTORY "${HDF5_SOURCE_DIR}/fortran")
option (HDF5_BUILD_FORTRAN "Build FORTRAN support" OFF)
if (HDF5_BUILD_FORTRAN)
include (${HDF_RESOURCES_EXT_DIR}/HDFUseFortran.cmake)
include (${HDF_RESOURCES_DIR}/HDFUseFortran.cmake)
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE "Fortran compiler ID is ${CMAKE_Fortran_COMPILER_ID}")
endif ()
message (VERBOSE "Fortran compiler ID is ${CMAKE_Fortran_COMPILER_ID}")
include (${HDF_RESOURCES_DIR}/HDFFortranCompilerFlags.cmake)
include (${HDF_RESOURCES_DIR}/HDF5UseFortran.cmake)
set (LINK_Fortran_LIBS ${LINK_LIBS})
@@ -1107,6 +1127,7 @@ if (EXISTS "${HDF5_SOURCE_DIR}/fortran" AND IS_DIRECTORY "${HDF5_SOURCE_DIR}/for
# Parallel IO usage requires MPI to be Linked and Included
if (H5_HAVE_PARALLEL)
find_package(MPI REQUIRED COMPONENTS Fortran)
set (LINK_Fortran_LIBS ${LINK_Fortran_LIBS} ${MPI_Fortran_LIBRARIES})
if (MPI_Fortran_LINK_FLAGS)
set (CMAKE_Fortran_EXE_LINKER_FLAGS "${MPI_Fortran_LINK_FLAGS} ${CMAKE_EXE_LINKER_FLAGS}")
@@ -1145,9 +1166,7 @@ if (EXISTS "${HDF5_SOURCE_DIR}/c++" AND IS_DIRECTORY "${HDF5_SOURCE_DIR}/c++")
if (NOT ALLOW_UNSUPPORTED)
message (FATAL_ERROR " **** Parallel and C++ options are mutually exclusive, override with ALLOW_UNSUPPORTED option **** ")
else ()
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.15.0")
message (VERBOSE " **** Allowing unsupported Parallel and C++ options **** ")
endif ()
message (VERBOSE " **** Allowing unsupported Parallel and C++ options **** ")
endif ()
endif ()

128
CODE_OF_CONDUCT.md Normal file
View File

@@ -0,0 +1,128 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
help@hdfgroup.org.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series
of actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within
the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder](https://github.com/mozilla/diversity).
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
https://www.contributor-covenant.org/translations.

142
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,142 @@
# How to contribute to HDF5
The HDF Group encourages community members to contribute to the HDF5 project. We accept and are very grateful for any contributions,
from minor typos and bug fixes to new features. The HDF Group is committed to work with the code contributors and make contribution
process enjoyable and straightforward.
This document describes guiding principles for the HDF5 code contributors and does not pretend to address any possible
contribution. If in doubt, please do not hesitate to ask us for guidance.
***Note that no contribution may be accepted unless the donor agrees with the HDF Group software license terms
found in the COPYING file in every branch's top source directory.***
> We will assume that you are familiar with `git` and `GitHub`. If not, you may go through the GitHub tutorial found at
[https://guides.github.com/activities/hello-world/](https://guides.github.com/activities/hello-world/). This tutorial should only take
around 10 minutes.
## Table of Contents
* [Workflow](#workflow)
* [Acceptance criteria for a pull request](#criteria)
* [Release Note](#releasenote)
* [Check List](#checklist)
# Workflow <A NAME="workflow"></A>
The process for contributing code to HDF5 is as follows:
* Open an issue on [HDF5 GitHub](https://github.com/HDFGroup/hdf5/issues).
> This step is ***required*** unless the change is minor (e.g., typo fix).
* Fork the [HDF5](https://github.com/HDFGroup/hdf5) repository.
* Make the desired changes to the HDF5 software.
* New features should always go to _develop_ branch first and later should be merged to the appropriate maintenance branches.
* Bug fixes should go to all appropriate branches (_develop_ and maintenance).
* Build and test your changes. Detailed instructions on building and testing HDF5 can be found in the `INSTALL*` files in the `release_docs` directory.
* Push your changes to GitHub.
* Issue a pull request and address any code formatting and testing issues reported.
Once a pull request is correctly formatted and passes **ALL** CI tests, it will be reviewed and evaluated by The HDF Group developers and HDF5
community members who can approve pull requests. The HDF Group developers will work with you to ensure that the pull request satisfies the acceptance
criteria described in the next section.
# Acceptance criteria for a pull request <A NAME="criteria"></A>
We appreciate every contribution we receive, but we may not accept them all. Those that we *do* satisfy the following criteria:
* **The pull request has a clear purpose** - What does the pull request address? How does it benefit the HDF5 community?
If the pull request does not have a clear purpose and benefits, it will not be accepted.
* **The pull request is documented** - The HDF5 developers must understand not only *what* a change is doing, but *how* it is doing it.
Documenting the code makes it easier for us to understand your patch and maintain the code in the future.
* **The pull request passes HDF5 regression testing** - Any issue fixed or functionality added should be accompanied by the corresponding
tests and pass HDF5 regression testing run by The HDF Group. We do not expect you to perform comprehensive testing across multiple platforms
before we accept the pull request. If the pull request does not pass regression testing after the merge, The HDF Group developers will work
with you on the fixes.
* **The pull request does not compromise the principles behind HDF5** - HDF5 has a 100% commitment to backward compatibility.
* Any file ever created with HDF5 must be readable by any future version of HDF5.
If your patch's purpose is to modify the HDF5 data model or file format,
**please** discuss this with us first. File format changes and features required by those changes can be introduced only in a new major release.
* HDF5 has a commitment to remaining *machine-independent*; data created on one platform/environment/architecture **must** remain readable by HDF5 on any other.
* For binary compatibility, no changes are allowed to public APIs and data structures in the maintenance releases; new APIs can be added.
* **New features are documented** - Any new features should have proper documentation; talk to us if you have any questions.
* **When to Write a Release Note** - Generally, a release note must be written for every change that is made to the code for which
users might see a change in the way the software works. In other words, if a user might see a difference in the way the software works,
a note should be written. By code we mean the text that will be compiled into one of the company's software products. The code includes
configuration changes and changes to tools users might work with to configure and build our software.
* Notes should be added for known problems. Known problems are issues that we know about and have not yet been able to fix.
* Any change made to address a user-reported problem should be described in a release note.
* A release note does not need to be written for changes to the code that users will not see. Here are some examples. If you add a
comment, you do not need to write a release note describing the comment you added. If you rewrite some code to make it read more
clearly and if there is no change in functionality or performance, then you do not need to write a release note. If you change the
process by which user software is made, you may not need to write a release note since the change was not made to the code.
* Users. We have different kinds of users. A release note may be written to be helpful to
application developers and not system administrators. Users who may find the RELEASE.txt file helpful include the following:
application developers, library developers, and system administrators.
# Release Note <A NAME="releasenote"></A>
* **Entry Syntax**
The release note entry syntax is shown below.
```
- Title/Problem
Problem/Solution
Signature
```
* **Entry Elements** - The elements of the entry - title, problem, solution, and signature - are described in more detail in the table
below. Descriptions of the problem and the solution should be clear without any ambiguities and should be short without losing clarity or specifics.
* **Title** - The title or tag should identify one or more categories that will help readers decide if the entry is something they need to study. Can be combined with the `Problem` element
* **Problem** - Describe the problem and how users might see the problem in a paragraph.
You might also consider the following as you describe the problem:
* Under what specific conditions does this issue arise?
* Under what specific conditions are we sure this issue will not arise?
* For a performance issue, instead of saying something is a performance issue, describe what the performance impact of issue is?
* **Solution** - Describe the solution in another paragraph.
You might also consider the following as you describe the solution:
* What was done to resolve the issue?
* What is the functional impact?
* Is there a workaround a way for users design their software so as not to encounter the issue? If so, what is the workaround?
* For a performance fix, how has the performance improved? Links to published documentation would be good.
* **Signature** - Each entry must be signed with the initials of the author, the date in YYYY/MM/DD format, and the JIRA ticket number. The
following is an example entry written by developer Xavier Zolo on April 16, 2014 about JIRA ticket HDFFV-5555: (XYZ - 2014/04/16, HDFFV-5555). The
signature is enclosed in parentheses. JIRA or Github numbers should not be used in the description of the problem or the solution. They are like
abbreviations that customers and external users will not be able to interpret.
# Checklist <A NAME="checklist"></A>
Please make sure that you check the items applicable to your pull request:
* Code
* [ ] Does the pull request have a corresponding GitHub issue and clear purpose?
* [ ] Does the pull request follow HDF5 best practices (naming conventions, code portability, code structure, etc.)? <<TODO: link to the document>>
* [ ] If changes were done to Autotools build, were they added to CMake and vice versa?
* [ ] Is the pull request applicable to any other branches? If yes, which ones? Please document it in the GitHub issue.
* [ ] Is the new code sufficiently documented for future maintenance?
* [ ] Does the new feature require a change to an existing API? See "API Compatibility Macros" document (https://portal.hdfgroup.org/display/HDF5/API+Compatibility+Macros)
* Documentation
* [ ] Was the change described in the release_docs/RELEASE.txt file?
* [ ] Was MANIFEST updated if new files had been added to the source?
* [ ] Was the new function documented in the corresponding public header file using [Doxygen](https://docs.hdfgroup.org/hdf5/develop/_r_m_t.html)?
* [ ] Was new functionality documented for the HDF5 community (the level of documentation depends on the feature; ask us what would be appropriate)
* Testing
* [ ] Does the pull request have tests?
* [ ] Does the pull request affect HDF5 library performance?
We want as many contributions as we can get, and we are here to help. Feel free to reach out to us if you have any questions
Thank you for your contribution!

View File

@@ -12,7 +12,7 @@
## This file should be placed in the root directory of your project.
## Then modify the CMakeLists.txt file in the root directory of your
## project to incorporate the testing dashboard.
## # The following are required to uses Dart and the Cdash dashboard
## # The following are required to use Dart and the CDash dashboard
## ENABLE_TESTING()
## INCLUDE(CTest)
set (CTEST_PROJECT_NAME "HDF5")

3890
MANIFEST

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,5 @@
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -1,7 +1,6 @@
# Top-level distributed Makefile -*- makefile -*-
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -1,18 +1,24 @@
HDF5 version 1.13.1-1 currently under development
HDF5 version 1.15.0 currently under development
![HDF5 Logo](doxygen/img/HDF5.png)
[![develop build status](https://img.shields.io/github/actions/workflow/status/HDFGroup/hdf5/main.yml?branch=develop&label=develop)](https://github.com/HDFGroup/hdf5/actions?query=branch%3Adevelop)
[![1.14 build status](https://img.shields.io/github/actions/workflow/status/HDFGroup/hdf5/main.yml?branch=hdf5_1_14&label=1.14)](https://github.com/HDFGroup/hdf5/actions?query=branch%3Ahdf5_1_14)
[![1.12 build status](https://img.shields.io/github/actions/workflow/status/HDFGroup/hdf5/main.yml?branch=hdf5_1_12&label=1.12)](https://github.com/HDFGroup/hdf5/actions?query=branch%3Ahdf5_1_12)
[![1.10 build status](https://img.shields.io/github/actions/workflow/status/HDFGroup/hdf5/main.yml?branch=hdf5_1_10&label=1.10)](https://github.com/HDFGroup/hdf5/actions?query=branch%3Ahdf5_1_10)
[![1.8 build status](https://img.shields.io/github/actions/workflow/status/HDFGroup/hdf5/main.yml?branch=hdf5_1_8&label=1.8)](https://github.com/HDFGroup/hdf5/actions?query=branch%3Ahdf5_1_8)
[![BSD](https://img.shields.io/badge/License-BSD-blue.svg)](https://github.com/HDFGroup/hdf5/blob/develop/COPYING)
*Please refer to the release_docs/INSTALL file for installation instructions.*
THE HDF GROUP
---------------
This repository contains a high-performance library's source code and a file format
specification that implement the HDF5® data model. The model has been adopted across
many industries and this implementation has become a de facto data management standard
in science, engineering, and research communities worldwide.
The HDF Group is the developer of HDF5®, a high-performance software library and
data format that has been adopted across multiple industries and has become a
de facto standard in scientific and research communities.
More information about The HDF Group, the HDF5 Community and the HDF5 software
project, tools and services can be found at the Group's website.
The HDF Group is the developer, maintainer, and steward of HDF5 software. Find more
information about The HDF Group, the HDF5 Community, and other HDF5 software projects,
tools, and services at The HDF Group's website.
https://www.hdfgroup.org/
@@ -69,6 +75,32 @@ use and configuration
https://forum.hdfgroup.org/t/quickstart-guide-welcome-to-the-new-hdf-forum
RELEASE SCHEDULE
----------------
![HDF5 release schedule](doc/img/release-schedule.png)
HDF5 does not release on a regular schedule. Instead, releases are driven by
new features and bug fixes, though we try to have at least one release of each
maintenance branch per year. Future HDF5 releases indicated on this schedule
are tentative.
**NOTE**: HDF5 1.12 is being retired early due to its incomplete and incompatible VOL
layer.
| Release | New Features |
| ------- | ------------ |
| 1.8.23 | last HDF5 1.8 release |
| 1.10.10 | CVE fixes, performance improvements, H5Dchunk\_iter() |
| 1.12.3 | CVE fixes, performance improvements, H5Dchunk\_iter(), last HDF5 1.12 release |
| 1.14.1 | selection I/O with datatype conversion |
| 2.0.0 | TBD |
| TBD | VFD SWMR |
This list of feature release versions is also tentative, and the specific release
in which a feature is introduced may change.
SNAPSHOTS, PREVIOUS RELEASES AND SOURCE CODE
--------------------------------------------
Periodically development code snapshots are provided at the following URL:

View File

@@ -2,7 +2,6 @@ dnl -------------------------------------------------------------------------
dnl -------------------------------------------------------------------------
dnl
dnl Copyright by The HDF Group.
dnl Copyright by the Board of Trustees of the University of Illinois.
dnl All rights reserved.
dnl
dnl This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -1,10 +1,10 @@
#!/bin/sh
#
# Copyright by The HDF Group.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# Copyright by The HDF Group.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
@@ -50,17 +50,7 @@
# Note that aclocal will attempt to include libtool's share/aclocal
# directory.
#
# This script takes two potential options:
#
# -p
#
# When this is selected, the autotools versions are set to the paths
# and versions used by The HDF Group to produce the released versions
# of the library.
#
# NOTE: This is probably temporary. Once we update our dev machines
# to have recent versions of the autotools this option will probably
# be removed.
# Aside from -h for help, this script takes one potential option:
#
# -v
#
@@ -72,9 +62,6 @@ echo "* HDF5 autogen.sh script *"
echo "**************************"
echo
# Default is not production
production=false
# Default is not verbose output
verbose=false
@@ -86,10 +73,6 @@ while getopts "$optspec" optchar; do
echo
echo " -h Print this help message."
echo
echo " -p Used by THG to use hard-codes autotools"
echo " paths on THG machines. Not for non-HDF-Group"
echo " users!"
echo
echo " -v Show more verbose output."
echo
echo " NOTE: Each tool can be set via an environment variable."
@@ -97,11 +80,6 @@ while getopts "$optspec" optchar; do
echo
exit 0
;;
p)
echo "Setting THG production mode..."
echo
production=true
;;
v)
echo "Setting verbosity: high"
echo
@@ -117,84 +95,52 @@ while getopts "$optspec" optchar; do
esac
done
if [ "$production" = true ] ; then
# Production mode
#
# Hard-code canonical HDF Group tool locations.
# If paths to tools are not specified, assume they are
# located in /usr/hdf/bin/AUTOTOOLS and set paths accordingly.
if test -z ${HDF5_AUTOCONF}; then
HDF5_AUTOCONF=/usr/hdf/bin/AUTOTOOLS/autoconf
fi
if test -z ${HDF5_AUTOMAKE}; then
HDF5_AUTOMAKE=/usr/hdf/bin/AUTOTOOLS/automake
fi
if test -z ${HDF5_AUTOHEADER}; then
HDF5_AUTOHEADER=/usr/hdf/bin/AUTOTOOLS/autoheader
fi
if test -z ${HDF5_ACLOCAL}; then
HDF5_ACLOCAL=/usr/hdf/bin/AUTOTOOLS/aclocal
fi
if test -z ${HDF5_LIBTOOL}; then
HDF5_LIBTOOL=/usr/hdf/bin/AUTOTOOLS/libtool
fi
if test -z ${HDF5_M4}; then
HDF5_M4=/usr/hdf/bin/AUTOTOOLS/m4
fi
else
# Not in production mode
#
# If paths to autotools are not specified, use whatever the system
# has installed as the default. We use 'which <tool>' to
# show exactly what's being used.
if test -z ${HDF5_AUTOCONF}; then
HDF5_AUTOCONF=$(which autoconf)
fi
if test -z ${HDF5_AUTOMAKE}; then
HDF5_AUTOMAKE=$(which automake)
fi
if test -z ${HDF5_AUTOHEADER}; then
HDF5_AUTOHEADER=$(which autoheader)
fi
if test -z ${HDF5_ACLOCAL}; then
HDF5_ACLOCAL=$(which aclocal)
fi
if test -z ${HDF5_LIBTOOL}; then
case "`uname`" in
Darwin*)
# libtool on OS-X is non-gnu
HDF5_LIBTOOL=$(which glibtool)
;;
*)
HDF5_LIBTOOL=$(which libtool)
;;
esac
fi
if test -z ${HDF5_M4}; then
HDF5_M4=$(which m4)
fi
fi # production
# If paths to autotools are not specified, use whatever the system
# has installed as the default. We use 'command -v <tool>' to
# show exactly what's being used (shellcheck complains that 'which'
# is non-standard and deprecated).
if test -z "${HDF5_AUTOCONF}"; then
HDF5_AUTOCONF="$(command -v autoconf)"
fi
if test -z "${HDF5_AUTOMAKE}"; then
HDF5_AUTOMAKE="$(command -v automake)"
fi
if test -z "${HDF5_AUTOHEADER}"; then
HDF5_AUTOHEADER="$(command -v autoheader)"
fi
if test -z "${HDF5_ACLOCAL}"; then
HDF5_ACLOCAL="$(command -v aclocal)"
fi
if test -z "${HDF5_LIBTOOL}"; then
case "$(uname)" in
Darwin*)
# libtool on OS-X is non-gnu
HDF5_LIBTOOL="$(command -v glibtool)"
;;
*)
HDF5_LIBTOOL="$(command -v libtool)"
;;
esac
fi
if test -z "${HDF5_M4}"; then
HDF5_M4="$(command -v m4)"
fi
# Make sure that these versions of the autotools are in the path
AUTOCONF_DIR=`dirname ${HDF5_AUTOCONF}`
LIBTOOL_DIR=`dirname ${HDF5_LIBTOOL}`
M4_DIR=`dirname ${HDF5_M4}`
AUTOCONF_DIR=$(dirname "${HDF5_AUTOCONF}")
LIBTOOL_DIR=$(dirname "${HDF5_LIBTOOL}")
M4_DIR=$(dirname "${HDF5_M4}")
PATH=${AUTOCONF_DIR}:${LIBTOOL_DIR}:${M4_DIR}:$PATH
# Make libtoolize match the specified libtool
case "`uname`" in
case "$(uname)" in
Darwin*)
# On OS X, libtoolize could be named glibtoolize or
# libtoolize. Try the former first, then fall back
# to the latter if it's not found.
HDF5_LIBTOOLIZE="${LIBTOOL_DIR}/glibtoolize"
if [ ! -f $HDF5_LIBTOOLIZE ] ; then
if [ ! -f "$HDF5_LIBTOOLIZE" ] ; then
HDF5_LIBTOOLIZE="${LIBTOOL_DIR}/libtoolize"
fi
;;
@@ -249,7 +195,7 @@ echo
# LIBTOOLIZE
libtoolize_cmd="${HDF5_LIBTOOLIZE} --copy --force"
echo ${libtoolize_cmd}
echo "${libtoolize_cmd}"
if [ "$verbose" = true ] ; then
${HDF5_LIBTOOLIZE} --version
fi
@@ -264,7 +210,7 @@ if test -e "${LIBTOOL_DIR}/../share/aclocal" ; then
aclocal_include="-I ${LIBTOOL_DIR}/../share/aclocal"
fi
aclocal_cmd="${HDF5_ACLOCAL} --force -I m4 ${aclocal_include}"
echo ${aclocal_cmd}
echo "${aclocal_cmd}"
if [ "$verbose" = true ] ; then
${HDF5_ACLOCAL} --version
fi
@@ -273,7 +219,7 @@ echo
# AUTOHEADER
autoheader_cmd="${HDF5_AUTOHEADER} --force"
echo ${autoheader_cmd}
echo "${autoheader_cmd}"
if [ "$verbose" = true ] ; then
${HDF5_AUTOHEADER} --version
fi
@@ -282,7 +228,7 @@ echo
# AUTOMAKE
automake_cmd="${HDF5_AUTOMAKE} --copy --add-missing --force-missing"
echo ${automake_cmd}
echo "${automake_cmd}"
if [ "$verbose" = true ] ; then
${HDF5_AUTOMAKE} --version
fi
@@ -290,8 +236,11 @@ ${automake_cmd} || exit 1
echo
# AUTOCONF
autoconf_cmd="${HDF5_AUTOCONF} --force"
echo ${autoconf_cmd}
# The "obsolete" warnings category flags our Java macros as obsolete.
# Since there is no clear way to upgrade them (Java support in the Autotools
# is not great) and they work well enough for now, we suppress those warnings.
autoconf_cmd="${HDF5_AUTOCONF} --force --warnings=no-obsolete"
echo "${autoconf_cmd}"
if [ "$verbose" = true ] ; then
${HDF5_AUTOCONF} --version
fi

View File

@@ -1,6 +1,5 @@
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -1,2 +0,0 @@
The daily tests run copies of some of the scripts in this directory from another repository, notably snapshot and runtest. The copies in this directory should work, but are not used in daily tests, though they should be tested occasionally.

31
bin/README.md Normal file
View File

@@ -0,0 +1,31 @@
# Scripts in `bin` and their purpose
Programs run via `autogen.sh` (or the equivalent in CMake) are indicated.
|Program|Purpose|
|-------|-------|
|`buildhdf5`|Convenience script to build HDF5 using the Autotools|
|`checkapi`|Checks if public API calls are used in internal functions|
|`checkposix`|Checks if C/POSIX calls are prefixed with `HD`|
|`chkcopyright`|Checks if files have appropriate copyright statements|
|`cmakehdf5`|Convenience script to build HDF5 using CMake|
|`debug-ohdr`|Examines debug output from `H5O_open/close` to look for open objects|
|`format_source`|Runs `clang-format` over the source files, applying our rules|
|`genparser`|Creates the flex/bison-based parser files in the high-level library|
|`h5cc.in`|Input file from which h5cc is created|
|`h5redeploy.in`|Input file from which h5redeploy is created|
|`h5vers`|Updates the library version number|
|`make_err`|Generates the H5E header files (called in `autogen.sh`)|
|`make_vers`|Generates H5version.h (called in `autogen.sh`)|
|`make_overflow`|Generates H5overflow.h (called in `autogen.sh`)|
|`output_filter`|Used in the tools test code to strip extraneous output before we diff files|
|`restore.sh`|Removes files generated by `autogen.sh`|
|`runbkprog`|Used by CMake to run test programs in the background|
|`switch_maint_mode`|Switches maintainer mode on/off in `configure.ac`|
|`trace`|Adds `TRACE` macros to HDF5 C library source files (run by `autogen.sh`)|
|`warnhist`|Generates compiler warning statistics for gcc/clang when fed output of make|
## TODO
* chkcopyright is currently semi-broken as it doesn't handle the full variety of copyright headers we need. We're leaving it in place, though, in the hopes that someone will update it in the future.
* Extending warnhist to better understand the output of additional compilers/languages would be nice.

View File

@@ -11,11 +11,13 @@ CTEST_CMD=`which ctest`
cd @HDF5_BINARY_DIR@
if [[ $SUMMARY_FILE == *"ctestS"* ]]; then
CMD="${CTEST_CMD} . -E MPI_TEST_ -C Release -j 32 -T test"
CMD="${CTEST_CMD} -S ctest_serial.cmake"
qsub -t 60 -n 1 -q debug-flat-quad -A ${ACCOUNT_ID} ${CMD} >& ${SUMMARY_FILE}
echo "Done running ctest serial command."
touch ctestS.done
else
CMD="${CTEST_CMD} . -R MPI_TEST_ ${SKIP_TESTS} -C Release -T test"
CMD="${CTEST_CMD} -S ctest_parallel.cmake"
qsub -t 60 -n 1 -q debug-flat-quad -A ${ACCOUNT_ID} ${CMD} >& ${SUMMARY_FILE}
echo "Done running ctest parallel command."
touch ctestP.done
fi
qsub -t 60 -n 1 -q debug-flat-quad -A ${ACCOUNT_ID} ${CMD} >& ${SUMMARY_FILE}
echo "Done running ctest parallel command."

View File

@@ -14,6 +14,7 @@
cd @HDF5_BINARY_DIR@
echo "Run parallel test command. Test output will be in build/ctestP.out"
ctest . -R MPI_TEST_ -C Release -T test >& ctestP.out
ctest -S ctest_parallel.cmake >& ctestP.out
echo "Done running ctest parallel command."
touch ctestP.done

View File

@@ -8,7 +8,8 @@
#SBATCH --job-name=h5_ctestP
cd @HDF5_BINARY_DIR@
ctest . -R MPI_TEST_ -C Release -T test >& ctestP.out
echo "Done running ctestP.sl"
echo "Run parallel test command. Test output will be in build/ctestP.out"
ctest -S ctest_parallel.cmake >& ctestP.out
echo "Done running ctest parallel command."
touch ctestP.done

View File

@@ -11,8 +11,7 @@
cd @HDF5_BINARY_DIR@
echo "Run command. Test output will be in build/ctestS.out"
ctest . -E MPI_TEST_ -C Release -j 32 -T test >& ctestS.out
ctest -S ctest_serial.cmake >& ctestS.out
##$CMD >& ctestS.out
echo "Done running command."
touch ctestS.done

View File

@@ -8,8 +8,8 @@
#SBATCH --job-name=h5_ctestS
cd @HDF5_BINARY_DIR@
CMD="ctest . -E MPI_TEST_ -C Release -j 32 -T test"
echo "Run command. Test output will be in build/ctestS.out"
ctest -S ctest_serial.cmake >& ctestS.out
echo "Run $CMD. Test output will be in build/ctestS.out"
$CMD >& ctestS.out
echo "Done running $CMD"
echo "Done running command."
touch ctestS.done

View File

@@ -0,0 +1,12 @@
if(NOT "$ENV{CI_SITE_NAME}" STREQUAL "")
set(CTEST_SITE "$ENV{CI_SITE_NAME}")
endif()
if(NOT "$ENV{CI_BUILD_NAME}" STREQUAL "")
set(CTEST_BUILD_NAME "$ENV{CI_BUILD_NAME}")
endif()
ctest_start ("$ENV{CI_MODEL}" "@HDF5_SOURCE_DIR@" "@HDF5_BINARY_DIR@" APPEND)
ctest_test (BUILD "@HDF5_BINARY_DIR@" APPEND INCLUDE MPI_TEST_ RETURN_VALUE res)
if (${res} LESS 0 OR ${res} GREATER 0)
file (APPEND ${CTEST_SCRIPT_DIRECTORY}/FailedCTest.txt "Failed Tests: ${res}\n")
endif ()

View File

@@ -0,0 +1,12 @@
if(NOT "$ENV{CI_SITE_NAME}" STREQUAL "")
set(CTEST_SITE "$ENV{CI_SITE_NAME}")
endif()
if(NOT "$ENV{CI_BUILD_NAME}" STREQUAL "")
set(CTEST_BUILD_NAME "$ENV{CI_BUILD_NAME}")
endif()
ctest_start ("$ENV{CI_MODEL}" "@HDF5_SOURCE_DIR@" "@HDF5_BINARY_DIR@" APPEND)
ctest_test (BUILD "@HDF5_BINARY_DIR@" APPEND EXCLUDE MPI_TEST_ PARALLEL_LEVEL 32 RETURN_VALUE res)
if (${res} LESS 0 OR ${res} GREATER 0)
file (APPEND ${CTEST_SCRIPT_DIRECTORY}/FailedCTest.txt "Failed Tests: ${res}\n")
endif ()

View File

@@ -9,8 +9,8 @@
#SBATCH --job-name=h5_ctestP
cd @HDF5_BINARY_DIR@
#run parallel tests except t_cache_image test
ctest . -R MPI_TEST_ -C Release -T test >& ctestP.out
echo "Done running $CMD"
echo "Run parallel test command. Test output will be in build/ctestP.out"
ctest -S ctest_parallel.cmake >& ctestP.out
echo "Done running ctest parallel command."
touch ctestP.done

View File

@@ -9,9 +9,8 @@
#SBATCH --job-name=h5_ctestS
cd @HDF5_BINARY_DIR@
CMD="ctest . -E MPI_TEST_ -C Release -j 32 -T test"
echo "Run $CMD. Test output will be in build/ctestS.out"
$CMD >& ctestS.out
echo "Done running $CMD"
echo "Run command. Test output will be in build/ctestS.out"
ctest -S ctest_serial.cmake >& ctestS.out
echo "Done running command."
touch ctestS.done

View File

@@ -15,6 +15,8 @@
cd @HDF5_BINARY_DIR@
echo "Run parallel test command. Test output will be in build/ctestP.out"
ctest . -R 'MPI_TEST_' -C Release -T test >& ctestP.out
ctest -S ctest_parallel.cmake >& ctestP.out
echo "Done running ctest parallel command."
touch ctestP.done
~

View File

@@ -11,8 +11,7 @@
cd @HDF5_BINARY_DIR@
echo "Run command. Test output will be in build/ctestS.out"
ctest . -E 'MPI_TEST_' -C Release -j 32 -T test >& ctestS.out
ctest -S ctest_serial.cmake >& ctestS.out
##$CMD >& ctestS.out
echo "Done running command."
touch ctestS.done

View File

@@ -1,7 +1,6 @@
#!/bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
@@ -14,32 +13,13 @@
# Make a release of hdf5.
#
# Programmer: Robb Matzke
# Creation date: on or before 1998-01-29.
# NOTE:
# This script differs from bin/release in that this has an added
# --revision option to create private releases with the code revision
# hash in the version strings.
#
# Modifications
# Robb Matzke, 1999-07-16
# The SunOS 5.6 sed *must* have slashes as delimiters. I changed things like
# `sed s+/CVS++' to `sed 's/\/CVS//'
#
# Albert Cheng, 1999-10-26
# Moved the MANIFEST checking to a separate command file so that
# it can be invoked individually.
#
# Albert Cheng, 2004-08-14
# Added the --private option.
#
# James Laird, 2005-09-07
# Added the md5 method.
#
# Larry Knox, 2016-08-30
# Added the --revision option to create private releases with the
# code revision hash in the version strings. Currently the version
# of this script with the --revision option is named bbrelease. It
# can probably be merged into the original release script in the
# future.
# Commands to get the revision hash have now been converted to git
# to match the source repository change.
# This script can probably be merged into the original release script in
# the future.
# Function definitions
#
@@ -47,27 +27,27 @@
USAGE()
{
cat << EOF
Usage: $0 -d <dir> [--docver BRANCHNAME] [-h] [--nocheck] [--private] <methods> ...
-d DIR The name of the directory where the release(s) should be
Usage: $0 -d <dir> [-h] [--private] [--revision [--branch BRANCHNAME]] <methods> ...
-d DIR The name of the directory where the release(s) should be
placed.
--docver BRANCHNAME This is added for 1.8 and beyond to get the correct
version of documentation files from the hdf5docs
--branch BRANCHNAME This is to get the correct version of the branch name from the
repository. BRANCHNAME for v1.8 should be hdf5_1_8.
-h print the help page.
--nocheck Ignore errors in MANIFEST file.
--private Make a private release with today's date in version information.
--revision Make a private release with the code revision number in version information.
--private Make a private release with today's date in version information.
--revision Make a private release with the code revision number in version information.
This allows --branch to be used for the file name.
--branch BRANCHNAME This is to get the correct version of the branch name from the
repository. BRANCHNAME for v1.8 should be hdf5_1_8.
This must be run at the top level of the source directory.
The other command-line options are the names of the programs to use
for compressing the resulting tar archive (if none are given then
"tar" is assumed):
tar -- use tar and don't do any compressing.
gzip -- use gzip with "-9" and append ".gz" to the output name.
tar -- use tar and don't do any compressing.
gzip -- use gzip with "-9" and append ".gz" to the output name.
bzip2 -- use bzip2 with "-9" and append ".bz2" to the output name.
zip -- convert all text files to DOS style and form a zip file for Windows use.
doc -- produce the latest doc tree in addition to the archive.
zip -- convert all text files to DOS style and form a zip file for Windows use.
An md5 checksum is produced for each archive created and stored in the md5 file.
@@ -97,15 +77,10 @@ EOF
# Function name: tar2zip
# Convert the release tarball to a Windows zipball.
#
# Programmer: Albert Cheng
# Creation date: 2014-04-23
#
# Modifications
#
# Steps:
# 1. untar the tarball in a temporary directory;
# Note: do this in a temporary directory to avoid changing
# the original source directory which maybe around.
# the original source directory which may be around.
# 2. convert all its text files to DOS (LF-CR) style;
# 3. form a zip file which is usable by Windows users.
#
@@ -119,8 +94,8 @@ EOF
tar2zip()
{
if [ $# -ne 3 ]; then
echo "usage: tar2zip <tarfilename> <zipfilename>"
return 1
echo "usage: tar2zip <tarfilename> <zipfilename>"
return 1
fi
ztmpdir=/tmp/tmpdir$$
mkdir -p $ztmpdir
@@ -132,23 +107,23 @@ tar2zip()
(cd $ztmpdir; tar xf -) < $tarfile
# sanity check
if [ ! -d $ztmpdir/$version ]; then
echo "untar did not create $ztmpdir/$version source dir"
# cleanup
rm -rf $ztmpdir
return 1
echo "untar did not create $ztmpdir/$version source dir"
# cleanup
rm -rf $ztmpdir
return 1
fi
# step 2: convert text files
# There maybe a simpler way to do this.
# options used in unix2dos:
# -k Keep the date stamp
# -k Keep the date stamp
# -q quiet mode
# grep redirect output to /dev/null because -q or -s are not portable.
find $ztmpdir/$version | \
while read inf; do \
if file $inf | grep "$inf\: .*text" > /dev/null 2>&1 ; then \
unix2dos -q -k $inf; \
fi\
done
while read inf; do \
if file $inf | grep "$inf\: .*text" > /dev/null 2>&1 ; then \
unix2dos -q -k $inf; \
fi\
done
# step 3: make zipball
# -9 maximum compression
# -y Store symbolic links as such in the zip archive
@@ -163,14 +138,6 @@ tar2zip()
# This command must be run at the top level of the hdf5 source directory.
# Verify this requirement.
# Since we are running bbrelease to create an HDF5 source tarfile for buildbot
# testing with source that is not for release, there is not a file named
# "configure" but there will be one named "configure.ac". The "configure"
# file will be created when autogen.sh runs. There probably will always
# be a bin/release file, but just in case it is removed, we can check for
# this script, bbrelease, in the bin directory. The bin/release script should
# continue to check for "configure" because it should be present in release
# source.
if [ ! \( -f configure.ac -a -f bin/bbrelease \) ]; then
echo "$0 must be run at the top level of the hdf5 source directory"
exit 1
@@ -182,24 +149,22 @@ VERS=`perl bin/h5vers`
VERS_OLD=
test "$VERS" || exit 1
verbose=yes
check=yes
release_date=`date +%F`
today=`date +%Y%m%d`
pmode='no'
revmode='no'
tmpdir="../#release_tmp.$$" # tmp work directory
DOC_URL=https://git@bitbucket.hdfgroup.org/scm/hdffv/hdf5doc.git
tmpdir="../#release_tmp.$$" # tmp work directory
CPPLUS_RM_NAME=cpplus_RM
# Restore previous Version information
RESTORE_VERSION()
{
if [ X-${VERS_OLD} != X- ]; then
echo restoring version information back to $VERS_OLD
rm -f config/lt_vers.am
cp $tmpdir/lt_vers.am config/lt_vers.am
bin/h5vers -s $VERS_OLD
VERS_OLD=
echo restoring version information back to $VERS_OLD
rm -f config/lt_vers.am
cp $tmpdir/lt_vers.am config/lt_vers.am
bin/h5vers -s $VERS_OLD
VERS_OLD=
fi
}
@@ -209,35 +174,32 @@ while [ -n "$1" ]; do
arg=$1
shift
case "$arg" in
-d)
DEST=$1
shift
;;
--nocheck)
check=no
;;
-h)
USAGE
exit 0
;;
--private)
pmode=yes
;;
-d)
DEST=$1
shift
;;
-h)
USAGE
exit 0
;;
--private)
pmode=yes
;;
--revision)
revmode=yes
;;
--docver)
DOCVERSION=$1
--branch)
BRANCHNAME=$1
shift
;;
-*)
echo "Unknown switch: $arg" 1>&2
USAGE
exit 1
;;
*)
methods="$methods $arg"
;;
-*)
echo "Unknown switch: $arg" 1>&2
USAGE
exit 1
;;
*)
methods="$methods $arg"
;;
esac
done
@@ -276,14 +238,17 @@ if [ X$revmode = Xyes ]; then
# Copy old version of config/lt_vers.am, since it's hard to
# "undo" changes to it.
cp config/lt_vers.am $tmpdir
branch=`git branch | grep '*' | awk '{print $NF}'`
if [ "${BRANCHNAME}" = "" ]; then
BRANCHNAME=`git symbolic-ref -q --short HEAD`
fi
revision=`git rev-parse --short HEAD`
# Set version information to m.n.r-r$revision.
# Set version information to m.n.r-r$revision.
# (h5vers does not correctly handle just m.n.r-$today.)
VERS=`echo $VERS | sed -e s/-.*//`-$revision
echo Private release of $VERS
HDF5_VERS=hdf5-$BRANCHNAME-$revision
echo file base of $HDF5_VERS
bin/h5vers -s $VERS
HDF5_VERS=hdf5-$branch-$revision
# use a generic directory name for revision releases
HDF5_IN_VERS=hdfsrc
else
@@ -299,23 +264,10 @@ if [ ! -d $DEST ]; then
exit 1
fi
# Check the validity of the MANIFEST file.
bin/chkmanifest || fail=yes
if [ "X$fail" = "Xyes" ]; then
if [ $check = yes ]; then
exit 1
else
echo "Continuing anyway..."
fi
fi
# Create a manifest that contains only files for distribution.
MANIFEST=$tmpdir/H5_MANIFEST
grep '^\.' MANIFEST | grep -v _DO_NOT_DISTRIBUTE_ >$MANIFEST
# Prepare the source tree for a release.
#ln -s `pwd` $tmpdir/$HDF5_VERS || exit 1
# Create a symlink to the source so files in the tarball have the prefix
# we want (gnu's --transform isn't portable)
ln -s `pwd` $tmpdir/$HDF5_IN_VERS || exit 1
# Save a backup copy of Makefile if exists.
test -f Makefile && mv Makefile $tmpdir/Makefile.x
cp -p Makefile.dist Makefile
@@ -330,64 +282,38 @@ for f in README.md release_docs/RELEASE.txt; do
chmod 644 $f
done
# trunk is different than branches.
if [ "${DOCVERSION}" ]; then
DOC_URL=https://git@bitbucket.hdfgroup.org/scm/hdffv/hdf5doc.git -b ${DOCVERSION}
fi
# Create the tar file
test "$verbose" && echo " Running tar..." 1>&2
( \
cd $tmpdir; \
tar cf $HDF5_VERS.tar $HDF5_IN_VERS/Makefile \
`sed 's/^\.\//'$HDF5_IN_VERS'\//' $MANIFEST` || exit 1 \
)
(cd "$tmpdir" && exec tar -ch --exclude-vcs -f "$HDF5_VERS.tar" "./$HDF5_IN_VERS" || exit 1 )
# Compress
MD5file=$HDF5_VERS.md5
cp /dev/null $DEST/$MD5file
for comp in $methods; do
case $comp in
tar)
cp -p $tmpdir/$HDF5_VERS.tar $DEST/$HDF5_VERS.tar
(cd $DEST; md5sum $HDF5_VERS.tar >> $MD5file)
;;
gzip)
test "$verbose" && echo " Running gzip..." 1>&2
gzip -9 <$tmpdir/$HDF5_VERS.tar >$DEST/$HDF5_VERS.tar.gz
(cd $DEST; md5sum $HDF5_VERS.tar.gz >> $MD5file)
;;
bzip2)
test "$verbose" && echo " Running bzip2..." 1>&2
bzip2 -9 <$tmpdir/$HDF5_VERS.tar >$DEST/$HDF5_VERS.tar.bz2
(cd $DEST; md5sum $HDF5_VERS.tar.bz2 >> $MD5file)
;;
zip)
test "$verbose" && echo " Creating zip ball..." 1>&2
tar2zip $HDF5_IN_VERS $tmpdir/$HDF5_VERS.tar $DEST/$HDF5_VERS.zip 1>&2
(cd $DEST; md5sum $HDF5_VERS.zip >> $MD5file)
;;
doc)
if [ "${DOCVERSION}" = "" ]; then
DOCVERSION=master
fi
test "$verbose" && echo " Creating docs..." 1>&2
# Check out docs from git repo
(cd $tmpdir; git clone $DOC_URL > /dev/null) || exit 1
# Create doxygen C++ RM
(cd c++/src && doxygen cpp_doc_config > /dev/null ) || exit 1
# Replace version of C++ RM with just-created version
rm -rf $tmpdir/${DOCVERSION}/html/$CPPLUS_RM_NAME
mv c++/src/$CPPLUS_RM_NAME $tmpdir/${DOCVERSION}/html/$CPPLUS_RM_NAME
# Compress the docs and move them to the release area
mv $tmpdir/$DOCVERSION $tmpdir/${HDF5_VERS}_docs
(cd $tmpdir && tar cf ${HDF5_VERS}_docs.tar ${HDF5_VERS}_docs)
mv $tmpdir/${HDF5_VERS}_docs.tar $DEST
;;
*)
echo "***Error*** Unknown method $comp"
exit 1
;;
tar)
cp -p $tmpdir/$HDF5_VERS.tar $DEST/$HDF5_VERS.tar
(cd $DEST; md5sum $HDF5_VERS.tar >> $MD5file)
;;
gzip)
test "$verbose" && echo " Running gzip..." 1>&2
gzip -9 <$tmpdir/$HDF5_VERS.tar >$DEST/$HDF5_VERS.tar.gz
(cd $DEST; md5sum $HDF5_VERS.tar.gz >> $MD5file)
;;
bzip2)
test "$verbose" && echo " Running bzip2..." 1>&2
bzip2 -9 <$tmpdir/$HDF5_VERS.tar >$DEST/$HDF5_VERS.tar.bz2
(cd $DEST; md5sum $HDF5_VERS.tar.bz2 >> $MD5file)
;;
zip)
test "$verbose" && echo " Creating zip ball..." 1>&2
tar2zip $HDF5_IN_VERS $tmpdir/$HDF5_VERS.tar $DEST/$HDF5_VERS.zip 1>&2
(cd $DEST; md5sum $HDF5_VERS.zip >> $MD5file)
;;
*)
echo "***Error*** Unknown method $comp"
exit 1
;;
esac
done
@@ -408,4 +334,6 @@ fi
# Remove temporary things
rm -rf $tmpdir
echo "DONE"
exit 0

View File

@@ -1,7 +1,6 @@
#!/bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -1,7 +1,6 @@
#!/usr/bin/env perl
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -4,7 +4,6 @@ use warnings;
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
@@ -115,7 +114,7 @@ foreach $arg (@ARGV) {
# These are really HDF5 functions/macros even though they don't
# start with `h' or `H'.
next if $name =~ /^FUNC_(ENTER|LEAVE)(_(NO)?API|_PACKAGE|_STATIC)?(_NOFS|_NOCLEAR|_NOINIT|_NOPUSH)?(_NOFUNC|_TAG)?$/;
next if $name =~ /^FUNC_(ENTER|LEAVE)(_(NO)?API|_PACKAGE|_STATIC)?(_NAMECHECK_ONLY|_NOFS|_NOCLEAR|_NOINIT|_NOPUSH)?(_NOFUNC|_TAG)?$/;
next if $name =~ /^(BEGIN|END)_FUNC$/;
next if $name =~ /^U?INT(8|16|32|64)(ENCODE|DECODE)(_VAR)?$/;
next if $name =~ /^CI_(PRINT_STATS|INC_SRC|INC_DST)$/;
@@ -123,17 +122,20 @@ foreach $arg (@ARGV) {
next if $name =~ /^(MIN3?|MAX3?|NELMTS|POWER_OF_TWO|REGION_OVERFLOW)$/;
next if $name =~ /^(SIZE_OVERFLOW|UNIQUE_MEMBERS|S_ISDIR)$/;
next if $name =~ /^addr_defined$/;
next if $name =~ /^TERMINATOR$/;
# These functions/macros are exempt.
# op, cb, and OP are often spuriously flagged so ignore them.
next if $name =~ /^(main|op|cb|OP)$/;
# Ignore callback invocation
next if $name =~ /^(op|cb|OP|iter_op|func)$/;
# Ignore main
next if $name =~ /^main$/;
# This often appears in preprocessor lines that span multiple lines
next if $name =~ /^(defined)$/;
# These are Windows system calls. Ignore them.
next if $name =~ /^(_get_osfhandle|GetFileInformationByHandle|SetFilePointer|GetLastError|SetEndOfFile)$/;
next if $name =~ /^(FindNextFile|FindClose|_tzset|Wgettimeofday|GetSystemTimeAsFileTime|Wgetlogin|GetUserName)$/;
next if $name =~ /^(FindNextFile|FindClose|_tzset|Wgettimeofday|GetSystemTimeAsFileTime|GetUserName)$/;
next if $name =~ /^(DeleteCriticalSection|TlsFree|TlsGetValue|CreateThread)$/;
next if $name =~ /^(ExpandEnvironmentStringsA|LockFileEx|UnlockFileEx)$/;
next if $name =~ /^(DllMain|LocalAlloc|LocalFree)$/;
@@ -141,7 +143,7 @@ foreach $arg (@ARGV) {
next if $name =~ /^(_beginthread|(Initialize|Enter|Leave)CriticalSection|TlsAlloc)$/;
# These are MPI function calls. Ignore them.
next if $name =~ /^(MPI_|MPE_)/;
next if $name =~ /^(MPI_)/;
# These are POSIX threads function calls. Ignore them.
next if $name =~ /^pthread_/;

View File

@@ -1,82 +0,0 @@
#!/bin/sh
##
## Copyright by the Board of Trustees of the University of Illinois.
## All rights reserved.
##
## This file is part of HDF5. The full HDF5 copyright notice, including
## terms governing use, modification, and redistribution, is contained in
## the COPYING file, which can be found at the root of the source code
## distribution tree, or in https://www.hdfgroup.org/licenses.
## If you do not have access to either file, you may request a copy from
## help@hdfgroup.org.
##
# Check that all the configure files are properly generated.
#
# Programmer: Albert Cheng
# Created Date: 2004/12/07
#
# Configure: should be generated by autoconf version 2.69.
# autoconf: should be of version 2.69.
# variable initialization
nerrors=0
AUTOCONFVERSION=2.69
AUTOCONFVERSIONLEAD='Generated by GNU Autoconf'
CONFIGUREFILES="configure"
# Function definitions
#
# PRINTMSG
# Print a one line message left justified in a field of 70 characters
# without newline. More output for this line later.
#
PRINTMSG() {
SPACES=" "
echo "$* $SPACES" | cut -c1-70 | tr -d '\012'
}
# print result passed.
PASSED() {
echo " PASSED"
}
# print result failed.
FAILED() {
echo "*FAILED*"
}
# Main body
# Check configure files
# The autoconf version should be among the first 5 lines.
echo "Check autoconf version. Should be version $AUTOCONFVERSION"
for xf in $CONFIGUREFILES; do
PRINTMSG $xf
if [ ! -f $xf ]; then
FAILED
echo File not found
nerrors=`expr $nerrors + 1`
continue
fi
autoconf_version=`head -5 $xf | grep "$AUTOCONFVERSIONLEAD"`
echo $autoconf_version | grep "$AUTOCONFVERSIONLEAD $AUTOCONFVERSION" > /dev/null 2>&1
if [ $? -eq 0 ]; then
PASSED
else
FAILED
echo "Expected: $AUTOCONFVERSIONLEAD $AUTOCONFVERSION"
echo "Got: $autoconf_version"
nerrors=`expr $nerrors + 1`
fi
done
# Summary
echo $0 found $nerrors errors
if [ $nerrors != 0 ]; then
exit 1
fi
exit 0

View File

@@ -1,7 +1,6 @@
#! /bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
@@ -10,7 +9,6 @@
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
#
# Check Copyright notice.
# Check that all the files have the proper copyright notice.
@@ -39,8 +37,8 @@ NFIXEDFILES=0 # Number of files fixed.
NFIXFAILEDFILES=0 # Number of files fix failed.
NUMBEGINLINES=60 # Copyright notice should be located within the
# this number of lines at the beginning of the file.
UICOPYRIGHTSTR="Copyright by the Board of Trustees of the University of Illinois"
THGCOPYRIGHTSTR="Copyright by The HDF Group."
UICOPYRIGHTSTR="Copyright by the Board of Trustees of the University of Illinois"
PASSEDLOG=/tmp/h5chkright_passed.$$
SKIPPEDLOG=/tmp/h5chkright_skipped.$$
@@ -111,113 +109,92 @@ BUILDCOPYRIGHT()
# C and C++ source Copyright notice
cat > ${C_COPYRIGHT} << \EOF
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *
* terms governing use, modification, and redistribution, is contained in *
* the files COPYING and Copyright.html. COPYING can be found at the root *
* of the source code distribution tree; Copyright.html can be found at the *
* root level of an installed copy of the electronic HDF5 document set and *
* is linked from the top-level documents page. It can also be found at *
* http://hdfgroup.org/HDF5/doc/Copyright.html. If you do not have *
* access to either file, you may request a copy from help@hdfgroup.org. *
* the COPYING file, which can be found at the root of the source code *
* distribution tree, or in https://www.hdfgroup.org/licenses. *
* If you do not have access to either file, you may request a copy from *
* help@hdfgroup.org. *
EOF
# Fortran9X source Copyright notice
cat > ${FTN_COPYRIGHT} << \EOF
! Copyright by The HDF Group. *
! Copyright by the Board of Trustees of the University of Illinois. *
! All rights reserved. *
! *
! This file is part of HDF5. The full HDF5 copyright notice, including *
! terms governing use, modification, and redistribution, is contained in *
! the files COPYING and Copyright.html. COPYING can be found at the root *
! of the source code distribution tree; Copyright.html can be found at the *
! root level of an installed copy of the electronic HDF5 document set and *
! is linked from the top-level documents page. It can also be found at *
! http://hdfgroup.org/HDF5/doc/Copyright.html. If you do not have *
! access to either file, you may request a copy from help@hdfgroup.org. *
! the COPYING file, which can be found at the root of the source code *
! distribution tree, or in https://www.hdfgroup.org/licenses. *
! If you do not have access to either file, you may request a copy from *
! help@hdfgroup.org. *
EOF
# HTML file Copyright notice
cat > ${HTM_COPYRIGHT} << \EOF
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *
* terms governing use, modification, and redistribution, is contained in *
* the files COPYING and Copyright.html. COPYING can be found at the root *
* of the source code distribution tree; Copyright.html can be found at the *
* root level of an installed copy of the electronic HDF5 document set and *
* is linked from the top-level documents page. It can also be found at *
* http://hdfgroup.org/HDF5/doc/Copyright.html. If you do not have *
* access to either file, you may request a copy from help@hdfgroup.org. *
* the COPYING file, which can be found at the root of the source code *
* distribution tree, or in https://www.hdfgroup.org/licenses. *
* If you do not have access to either file, you may request a copy from *
* help@hdfgroup.org. *
EOF
# Shell style Copyright notice
cat > ${SH_COPYRIGHT} << \EOF
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the files COPYING and Copyright.html. COPYING can be found at the root
# of the source code distribution tree; Copyright.html can be found at the
# root level of an installed copy of the electronic HDF5 document set and
# is linked from the top-level documents page. It can also be found at
# http://hdfgroup.org/HDF5/doc/Copyright.html. If you do not have
# access to either file, you may request a copy from help@hdfgroup.org.
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
EOF
# Shell style Copyright notice (2nd type)
cat > ${SH_COPYRIGHT2} << \EOF
## Copyright by The HDF Group.
## Copyright by the Board of Trustees of the University of Illinois.
## All rights reserved.
##
## This file is part of HDF5. The full HDF5 copyright notice, including
## terms governing use, modification, and redistribution, is contained in
## the files COPYING and Copyright.html. COPYING can be found at the root
## of the source code distribution tree; Copyright.html can be found at the
## root level of an installed copy of the electronic HDF5 document set and
## is linked from the top-level documents page. It can also be found at
## http://hdfgroup.org/HDF5/doc/Copyright.html. If you do not have
## access to either file, you may request a copy from help@hdfgroup.org.
## the COPYING file, which can be found at the root of the source code
## distribution tree, or in https://www.hdfgroup.org/licenses.
## If you do not have access to either file, you may request a copy from
## help@hdfgroup.org.
EOF
# Windows Batch file Copyright notice
cat > ${WINBAT_COPYRIGHT} << \EOF
@REM Copyright by The HDF Group.
@REM Copyright by the Board of Trustees of the University of Illinois.
@REM All rights reserved.
@REM
@REM This file is part of HDF5. The full HDF5 copyright notice, including
@REM terms governing use, modification, and redistribution, is contained in
@REM the files COPYING and Copyright.html. COPYING can be found at the root
@REM of the source code distribution tree; Copyright.html can be found at the
@REM root level of an installed copy of the electronic HDF5 document set and
@REM is linked from the top-level documents page. It can also be found at
@REM http://hdfgroup.org/HDF5/doc/Copyright.html. If you do not have
@REM access to either file, you may request a copy from help@hdfgroup.org.
@REM the COPYING file, which can be found at the root of the source code
@REM distribution tree, or in https://www.hdfgroup.org/licenses.
@REM If you do not have access to either file, you may request a copy from
@REM help@hdfgroup.org.
EOF
# configure.ac file Copyright notice
cat > ${CONFIGURE_AC_COPYRIGHT} << \EOF
dnl Copyright by The HDF Group.
dnl Copyright by the Board of Trustees of the University of Illinois.
dnl All rights reserved.
dnl
dnl This file is part of HDF5. The full HDF5 copyright notice, including
dnl terms governing use, modification, and redistribution, is contained in
dnl the files COPYING and Copyright.html. COPYING can be found at the root
dnl of the source code distribution tree; Copyright.html can be found at the
dnl root level of an installed copy of the electronic HDF5 document set and
dnl is linked from the top-level documents page. It can also be found at
dnl http://hdfgroup.org/HDF5/doc/Copyright.html. If you do not have
dnl access to either file, you may request a copy from help@hdfgroup.org.
dnl the COPYING file, which can be found at the root of the source code
dnl distribution tree, or in https://www.hdfgroup.org/licenses.
dnl If you do not have access to either file, you may request a copy from
dnl help@hdfgroup.org.
EOF
}
@@ -364,7 +341,7 @@ FindLineInFile()
# $1 file which contains the expected copyright notice.
# $2 file in which to look for the copyright notice.
# Copyright notice must be found within the beginning $NUMBEGINLINES of lines.
# Hunt for the particular string $UICOPYRIGHTSTR which signifies the beginning
# Hunt for the particular string $THGCOPYRIGHTSTR which signifies the beginning
# of the copyright notice.
#
MATCH_COPYRIGHT()
@@ -379,7 +356,7 @@ MATCH_COPYRIGHT()
nlines=`wc -l ${COPYRIGHTFILE} | cut -f1 -d' '`
# Find a line that contains the copyright string and its line number in
# the file.
begin=`FindLineInFile "${UICOPYRIGHTSTR}" $f`
begin=`FindLineInFile "${THGCOPYRIGHTSTR}" $f`
if [ "$begin" -le 0 ] ; then
# Not found, generate an empty dummy file
cp /dev/null ${EXTRACTEDFILE}
@@ -404,7 +381,7 @@ MATCH_COPYRIGHT()
# $1 file which contains the expected copyright notice.
# $2 file in which to look for the copyright notice.
# Copyright notice must be found within the beginning $NUMBEGINLINES of lines.
# Hunt for the particular string $UICOPYRIGHTSTR which signifies the beginning
# Hunt for the particular string $THGCOPYRIGHTSTR which signifies the beginning
# of the copyright notice.
#
FIX_COPYRIGHT()
@@ -427,7 +404,12 @@ FIX_COPYRIGHT()
# the file.
insertbegin=`FindLineInFile "${THGCOPYRIGHTSTR}" $f`
if [ $insertbegin -gt 0 ]; then
insertend=`expr $insertbegin + $nlines` # no need to -1. See below.
insertUIbegin=`FindLineInFile "${UICOPYRIGHTSTR}" $f`
if [ $insertUIbegin -gt 0 ]; then
insertend=`expr $insertbegin + $nlines + 1`
else
insertend=`expr $insertbegin + $nlines`
fi
else
insertbegin=`FindLineInFile "${UICOPYRIGHTSTR}" $f`
if [ $insertbegin -gt 0 ]; then

View File

@@ -1,154 +0,0 @@
#!/bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
#
# Check that all the files in MANIFEST exist and (if this is a
# GIT checkout) that all the GIT-managed files appear in the
# MANIFEST.
#
verbose=yes
MANIFEST=/tmp/HD_MANIFEST.$$
AUTOGEN=./autogen.sh
AUTOGEN_LOG=/tmp/autogen.log.$$
# Main
test "$verbose" && echo " Checking MANIFEST..." 1>&2
# clean up $MANIFEST file when exits
trap "rm -f $MANIFEST" 0
# Only split lines on newline, not whitespace
set -f
IFS='
'
# First make sure i am in the directory in which there is an MANIFEST file
# and then do the checking from there. Will try the following,
# current directory, parent directory, the directory this command resides.
if [ -f MANIFEST ]; then
continue
elif [ -f ../MANIFEST ]; then
cd ..
else
commanddir=`dirname $0`
if [ -d "$commanddir" -a -f $commanddir/MANIFEST ]; then
cd $commanddir
continue
else
echo MANIFEST file not found. Abort.
exit 1
fi
fi
# Do an autogen if generated files (e.g., configure) is not present
if [ ! -f configure ]; then
echo " running $AUTOGEN"
$AUTOGEN > $AUTOGEN_LOG 2>&1
if [ $? -ne 0 ]; then
echo $AUTOGEN encountered error. Abort.
echo output from $AUTOGEN:
cat $AUTOGEN_LOG
exit 1
fi
rm $AUTOGEN_LOG
fi
# Check for duplicate entries. This can be done at any time, but it may as
# well be sooner so that if something else fails the presence of duplicates
# will already be known.
errcode=0
DUPLICATES=`perl -ne 's/#.*//; next if /^\s*$/; if ($uniq{$_}++) { print $_; }' MANIFEST`
if [ "$DUPLICATES" ]; then
cat 1>&2 <<EOF
These entries appear more than once in the MANIFEST:
$DUPLICATES
Please remove the duplicate lines and try again.
EOF
errcode=1
fi
# Copy the manifest file to get a list of file names.
grep '^\.' MANIFEST | expand | cut -f1 -d' ' >$MANIFEST
for file in `cat $MANIFEST`; do
if [ ! -f $file ]; then
echo "- $file"
fail=yes
fi
done
# Get the list of files under version control and check that they are
# present.
#
# First get a list of all the pending files with git status and
# check those.
git_stat=`git status -s`
for file in $git_stat; do
# Newly added files are not listed by git ls-files, which
# we check later.
# The line listing new files starts with 'A'.
letter=`echo $file | head -c 1`
if [ "$letter" = "A" ]; then
# Convert the git status columns to './' so it matches
# the manifest file name.
#
# There is a space between the status columns and file name, hence
# the '3'.
path=`echo $file | sed 's/^.\{3\}/\.\//g'`
# Ignore directories
if [ ! -d $path ]; then
if (grep ^$path$ $MANIFEST >/dev/null); then
:
else
echo "- $path"
fail=yes
fi
fi
fi
done
# Next check git ls-files, which gets a list of all files that are
# checked in.
git_ls=`git ls-files`
for file in $git_ls; do
path="./${file}"
# Ignore directories
if [ ! -d $path ]; then
if (grep ^$path$ $MANIFEST >/dev/null); then
:
else
echo "+ $path"
fail=yes
fi
fi
done
# Finish up
if [ "X$fail" = "Xyes" ]; then
cat 1>&2 <<EOF
The MANIFEST is out of date. Files marked with a minus sign (-) no
longer exist; files marked with a plus sign (+) are GIT-managed but do
not appear in the MANIFEST. Please remedy the situation and try again.
EOF
exit 1
fi
if [ $errcode -ne 0 ]; then
exit 1
fi
test "$verbose" && echo " The MANIFEST is up to date." 1>&2
exit 0

View File

@@ -1,7 +1,6 @@
#!/usr/bin/env perl
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -1,51 +0,0 @@
#!/usr/bin/env perl
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
#
use warnings;
my $depend_file;
my $new_depend_file;
my $srcdir;
my $top_srcdir;
my $top_builddir;
while ($_ = shift @ARGV) {
if (/^--top_srcdir=([^ \t\n]*)/) {
$top_srcdir = $1;
$top_srcdir =~ s/\+/\\\+/g;
$top_srcdir =~ s/\./\\\./g;
} elsif (/^--top_builddir=([^ \t\n]*)/) {
$top_builddir = $1;
$top_builddir =~ s/\+/\\\+/g;
$top_builddir =~ s/\./\\\./g;
} else {
$depend_file = $_;
$new_depend_file = "$_.new";
last;
}
}
open(DEPEND, "<$depend_file") || die "cannot open file $depend_file: $!\n";
open(NEW, ">$new_depend_file") || die "cannot open file $new_depend_file: $!\n";
while (<DEPEND>) {
s/\.o(\b)/\.lo$1/g;
s/ $top_srcdir/ \$\(top_srcdir\)/g;
s/ $top_builddir/ \$\(top_builddir\)/g;
print NEW $_;
}
close(DEPEND);
close(NEW);
`mv $new_depend_file $depend_file`;

View File

@@ -1,58 +0,0 @@
#!/bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
#
# Deploy the HDF5 binary.
#
# Programmer: Albert Cheng
# Created Date: 2004/12/15
#
# Modifications
# Function definitions
#
# Print Usage page
USAGE()
{
cat << EOF
Usage: $0 <dir>
Install the binary to directory <dir>
Examples:
$ bin/deploy /usr/local/hdf5
....
EOF
}
# Variables
if [ $# != 1 ]; then
USAGE
exit 1
fi
installdir=$1
# create installdir if it does not exist yet.
if [ -d $installdir ] || mkdir $installdir ; then
${MAKE:-gmake} install prefix=$installdir && \
( cd $installdir/bin; ./h5redeploy -force)
exit $?
else
echo $installdir is not a valid directory
USAGE
exit 1
fi

View File

@@ -1,27 +0,0 @@
#!/bin/sh
#! -*-perl-*-
eval 'exec perl -p -x -S $0 ${1+"$@"}'
if 0;
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
#
# Usage: $0 [<].depend
# Takes dependency info and generates on stdout dependencies suitable for
# distribution by removing all the system include files from the list and
# removing all but the base name of other include files (since the Makefiles
# contain the logic for searching).
($h,$_)=/\s*\\/?$h.$`:("",$h.$_);
s|( +/\S*)*( *)|$2?" \\\n ":""|eg;
#s|(([-\w\.]+)/)+([-\w\.]+)|\3|g;

View File

@@ -1,139 +0,0 @@
#!/usr/bin/env perl
require 5.003;
use warnings;
use Text::Tabs;
# NOTE: THE FORMAT OF HRETURN_ERROR AND HGOTO_ERROR MACROS HAS
# CHANGED. THIS SCRIPT NO LONGER WORKS! --rpm
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
#
# Robb Matzke
# 30 Aug 1997
#
# Purpose: This script will read standard input which should be a
# function prologue followed by a C function and will emit
# on standard output the same source code with the function
# prologue containing documentation for the various errors
# that occur in the function.
#
# Errors are raised by calling HGOTO_ERROR() or
# HRETURN_ERROR(). The reason for the error message is a
# comment which appears immediately after the error macro
# call and is contained entirely on one line:
#
# HRETURN_ERROR (...); /*entry not found*/
#
# If such a comment doesn't exist, then the previous comment
# is used, subject to the constraint that raising an error
# clears the previous comment.
#
# /* Entry not found */
# HGOTO_ERROR (...);
#
# Emacs users can use this script interactively with the
# c-mark-function and shell-command-on-region functions which
# are normally bound to M-C-h and M-|.
# Split STDIN into the prolog and the function body. Preserve leading
# white space.
$_ = join "", <STDIN>;
my ($head, $prolog, $body) = (/^(\s*)(\/\*(.*?)\*\/)?(.*)/s)[0,2,3];
$prolog = "" unless $prolog;
# Find each error and the comment that goes with it.
for ($_=$body,$comment=""; /\/\*|H(RETURN|GOTO)_ERROR/s;) {
$_ = $&.$';
if (/^H(RETURN|GOTO)_ERROR\s*\(\s*H5E_(\w+)\s*,\s*H5E_(\w+)\s*,/s) {
($major, $minor, $_) = ($2, $3, $');
$comment=$1 if /^.*?\)\s*;\s*\/\*\s*(.*?)\s*\*\//;
$comment =~ s/^\s*\*+\s*/ /mg; # leading asterisks.
$comment =~ s/^\s+//s; # leading white space.
$comment =~ s/\s+$//s; # trailing white space.
$comment =~ s/(\w)$/$1./s; # punctuation.
$comment ||= "***NO COMMENT***";
$errors{"$major\000$minor\000\u$comment"} = 1;
$comment = "";
} else {
($comment) = /^\/\*\s*(.*?)\s*\*\//s;
$_ = $';
}
}
# Format an error so it isn't too wide.
sub fmt_error ($) {
local ($_) = @_;
my ($prefix,$space,$err) = /^((.*?)([A-Z_0-9]+\s+[A-Z_0-9]+\s+))/;
$_ = $';
tr/\n / /s;
my $w = 70 - length expand $prefix;
s/(.{$w}\S+)\s+(\S)/$1."\n".$space.' 'x(length $err).$2/eg;
return $prefix . $_."\n";
}
# Sort the errors by major, then minor, then comment. Duplicate
# triplets have already been removed.
sub by_triplet {
my ($a_maj, $a_min, $a_com) = split /\000/, $a;
my ($b_maj, $b_min, $b_com) = split /\000/, $b;
$a_maj cmp $b_maj || $a_min cmp $b_min || $a_com cmp $b_com;
}
@errors = map {sprintf "%-9s %-13s %s\n", split /\000/}
sort by_triplet keys %errors;
# Add the list of errors to the prologue depending on the type of
# prolog.
if (($front, $back) = $prolog=~/^(.*?Errors:\s*?(?=\n)).*?\n\s*\*\s*\n(.*)/s) {
#| * Errors: |#
#| * __list_of_error_messages__ (zero or more lines) |#
#| * |#
print $head, "/*", $front, "\n";
map {print fmt_error " *\t\t".$_} @errors;
print " *\n", $back, "*/", $body;
} elsif (($front,$back) = $prolog =~
/(.*?\n\s*ERRORS:?\s*?(?=\n)).*?\n\s*\n(.*)/s) {
#| ERRORS |#
#| __list_of_error_messages__ (zero or more lines) |#
#| |#
print $head, "/*", $front, "\n";
map {print fmt_error " ".$_} @errors;
print "\n", $back, "*/", $body;
} elsif ($prolog eq "") {
# No prolog present.
print $head;
print " \n/*", "-"x73, "\n * Function:\t\n *\n * Purpose:\t\n *\n";
print " * Errors:\n";
map {print fmt_error " *\t\t".$_} @errors;
print " *\n * Return:\tSuccess:\t\n *\n *\t\tFailure:\t\n *\n";
print " * Programmer:\t\n *\n * Modifications:\n *\n *", '-'x73, "\n";
print " */\n", $body;
} else {
# Prolog format not recognized.
print $head, "/*", $prolog, "*/\n\n";
print "/*\n * Errors returned by this function...\n";
map {print fmt_error " *\t".$_} @errors;
print " */\n", $body;
}

View File

@@ -20,7 +20,7 @@ find . \( -type d -path ./config -prune -and -not -path ./config \) \
-or -name H5version.h \
-or -name H5overflow.h \
\) \) \
-and \( -iname *.h -or -iname *.c -or -iname *.cpp -or -iname *.hpp \) \) \
-and \( -iname *.h -or -iname *.c -or -iname *.cpp -or -iname *.hpp -or -iname *.java \) \) \
| xargs clang-format -style=file -i -fallback-style=none
exit 0

View File

@@ -1,34 +0,0 @@
#!/bin/bash
#
# Recursively format all C & C++ sources and header files, except those in the
# 'config' directory and generated files, such as H5LTanalyze.c, etc.
#
# Note that any files or directories that are excluded here should also be
# added to the 'exclude' list in .github/workflows/clang-format-check.yml
#
# (Remember to update both bin/format_source and bin/format_source_patch)
find . \( -type d -path ./config -prune -and -not -path ./config \) \
-or \( \( \! \( \
-name H5LTanalyze.c \
-or -name H5LTparse.c \
-or -name H5LTparse.h \
-or -name H5Epubgen.h \
-or -name H5Einit.h \
-or -name H5Eterm.h \
-or -name H5Edefin.h \
-or -name H5version.h \
-or -name H5overflow.h \
\) \) \
-and \( -iname *.h -or -iname *.c -or -iname *.cpp -or -iname *.hpp \) \) \
| xargs clang-format -style=file -i -fallback-style=none
git diff > clang_format.patch
# Delete if 0 size
if [ ! -s clang_format.patch ]
then
rm clang_format.patch
fi
exit 0

View File

@@ -1,51 +0,0 @@
#! /bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
# A script to generate coverage files for HDF5 using gcov.
# Configures, builds, and runs tests in-place; the output files will be placed
# in a directory called gcov_logs.
# Must be invoked from the root hdf5 directory.
# This script has been tested on kagiso.
CFLAGS="$CFLAGS -ftest-coverage -fprofile-arcs"
export CFLAGS
LDFLAGS="$LDFLAGS -lgcov"
export LDFLAGS
CC=gcc
export CC
./configure
make
make check
mkdir gcov_logs
cd src
for j in *.h *.c
do
ln -s ../$j .libs/$j
done
cd .libs
for j in *.gcda
do
gcov -b $j >> gcov.log 2>&1
done
for j in *.gcov
do
mv $j ../../gcov_logs
done
mv gcov.log ../../gcov_logs
for j in *.c *.h
do
rm $j
done
cd ../..

View File

@@ -3,7 +3,7 @@
# Copyright by The HDF Group.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
@@ -21,7 +21,7 @@
#
# There is NO dependency in either the autotools or CMake to regenerate
# the parser code. If you modify H5LT analyze.l or H5LTparse.y, you
# will need to run this scrpit manually on a system with a suitable
# will need to run this script manually on a system with a suitable
# lexer and parser generator.
#
# IMPORTANT OS X NOTE
@@ -236,6 +236,7 @@ do
echo '#pragma GCC diagnostic ignored "-Wsign-conversion" ' >> tmp.out
echo '#pragma GCC diagnostic ignored "-Wstrict-overflow" ' >> tmp.out
echo '#pragma GCC diagnostic ignored "-Wstrict-prototypes" ' >> tmp.out
echo '#pragma GCC diagnostic ignored "-Wimplicit-fallthrough" ' >> tmp.out
echo '#if !defined (__clang__) ' >> tmp.out
echo '#pragma GCC diagnostic ignored "-Wlarger-than=" ' >> tmp.out
echo '#pragma GCC diagnostic ignored "-Wsuggest-attribute=const" ' >> tmp.out

View File

@@ -1,7 +1,6 @@
#! /bin/sh
##
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
@@ -40,7 +39,7 @@ HL="@HL@"
## $CLINKER $H5BLD_CPPFLAGS $CPPFLAGS $H5BLD_CFLAGS $CFLAGS $LDFLAGS ##
## $LIBS $clibpath $link_objs $link_args $shared_link ##
## ##
## These settings can be overridden by setting HDF5_CFLAGS, ##
## These settings can be overridden by setting HDF5_CFLAGS, ##
## HDF5_CPPFLAGS, HDF5_LDFLAGS, or HDF5_LIBS in the environment. ##
## ##
############################################################################

View File

@@ -1,7 +1,6 @@
#! /bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -7,7 +7,6 @@ require 5.003;
use strict;
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
@@ -377,7 +376,7 @@ if ($H5_JAVA) {
my $version_string2 = sprintf("%d, %d, %d", @newver[0,1,2]);
$data =~ s/\@version HDF5 .* <BR>/\@version HDF5 $version_string1 <BR>/;
$data =~ s/ public final static int LIB_VERSION\[\] = \{ \d*, \d*, \d* \};/ public final static int LIB_VERSION[] = \{ $version_string2 \};/;
$data =~ s/ public final static int LIB_VERSION\[\] = \{\d*,.\d*,.\d*\};/ public final static int LIB_VERSION[] = \{$version_string2\};/;
write_file($H5_JAVA, $data);
}
@@ -394,7 +393,7 @@ if ($TESTH5_JAVA) {
my $version_string1 = sprintf("%d, %d, %d", @newver[0,1,2]);
my $version_string2 = sprintf("int majnum = %d, minnum = %d, relnum = %d", @newver[0,1,2]);
$data =~ s/ int libversion\[\] = \{ .* \};/ int libversion\[\] = \{ $version_string1 \};/;
$data =~ s/ int libversion\[\] = \{.*\};/ int libversion\[\] = \{$version_string1\};/;
$data =~ s/ int majnum = \d*, minnum = \d*, relnum = \d*;/ $version_string2;/;
write_file($TESTH5_JAVA, $data);

View File

@@ -1,7 +1,6 @@
#!/usr/bin/env perl
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -1,238 +0,0 @@
#!/bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
#
# Try to locate the software as named in argument.
# This is a sequential search of all possible locations of the software.
# Usage: locate_sw <SW-Name>
# It prints a string showing the paths leading to the include, lib and bin
# directory of the software, separated by colons. E.g., if the software is
# located in /usr/sdt/*, it prints
# /usr/sdt/include:/usr/sdt/lib:/usr/sdt/bin
# Any component that is not found will be returned as an empty string. E.g.,
# if somehow the header files of the software are not found, it prints
# :/usr/sdt/lib;/usr/sdt/bin
# Function definitions
USAGE()
{
echo "Usage: locate_sw <SW-Name>"
echo " where <SW-Name> can be hdf4, hdf5, zlib"
echo " It prints the paths leading the header files (include),"
echo " library (lib), and tools (bin). E.g.,"
echo " /usr/sdt/include:/usr/sdt/lib:/usr/sdt/bin"
echo " Any component that is not found will be returned as an empty string. E.g.,"
echo " if somehow the header files of the software are not found, it prints"
echo " :/usr/sdt/lib;/usr/sdt/bin"
echo "Exit code: 0 if software located; otherwise non-zero"
}
# locate hdf4 software
locate_hdf4()
{
# this default is the best guess of locating hdf4 software
swpaths_defaults="/usr/ncsa /usr/sdt /usr/local"
swpaths=
case "$OSname" in
SunOS)
case "$OSrelease" in
5.7)
swpaths="/afs/ncsa/packages/hdf/SunOS_5.7"
;;
*)
# use default
;;
esac
;;
HP-UX)
case "$OSrelease" in
B.11.00)
swpaths="/afs/ncsa/packages/hdf/HPUX_11.00"
;;
*)
# use default
;;
esac
;;
Linux)
swpaths="/afs/ncsa/packages/hdf/Linux"
;;
OSF1)
swpaths="/afs/ncsa/packages/hdf/OSF1_V4.0"
;;
*)
# just use the defaults
;;
esac
# Check if the hdf4 software is actually available.
# Accept the directory only if needed .h, .a and tools are found
# in the same place. That way, they are more likely to be of the
# same version.
#
swpaths="$swpaths $swpaths_defaults"
for sw in $swpaths; do
if [ -r $sw/include/hdf.h -a -r $sw/lib/libdf.a -a -r $sw/bin/hdp ]; then
SW_inc=$sw/include
SW_lib=$sw/lib
SW_bin=$sw/bin
SW_Location=$sw
break
fi
done
}
# locate hdf5 software
locate_hdf5()
{
# this default is the best guess of locating hdf5 software
swpaths_defaults="/usr/ncsa /usr/sdt /usr/local"
swpaths=
case "$OSname" in
SunOS)
case "$OSrelease" in
5.7)
swpaths="/afs/ncsa/packages/hdf5/SunOS_5.7"
;;
*)
# use default
;;
esac
;;
HP-UX)
case "$OSrelease" in
B.11.00)
swpaths="/afs/ncsa/packages/hdf5/HPUX_11.00"
;;
*)
# use default
;;
esac
;;
Linux)
swpaths="/afs/ncsa/packages/hdf5/Linux"
;;
FreeBSD)
swpaths="/afs/ncsa/packages/hdf5/FreeBSD"
;;
OSF1)
swpaths="/afs/ncsa/packages/hdf5/OSF1_V4.0"
;;
*)
# just use the defaults
;;
esac
# Check if the hdf5 software is actually available.
# Accept the directory only if needed .h, .a and tools are found
# in the same place. That way, they are more likely to be of the
# same version.
#
swpaths="$swpaths $swpaths_defaults"
for sw in $swpaths; do
if [ -r $sw/include/hdf5.h -a -r $sw/lib/libhdf5.a -a -r $sw/bin/h5dump ]; then
SW_inc=$sw/include
SW_lib=$sw/lib
SW_bin=$sw/bin
SW_Location=$sw
break
fi
done
}
# locate zlib software
locate_zlib()
{
# this default is the best guess of locating zlib software
swpaths_defaults="/usr /usr/local /usr/ncsa /usr/sdt"
swpaths=
# Check if the zlib software is actually available.
# Accept the directory only if needed .h, .a and tools are found
# in the same place. That way, they are more likely to be of the
# same version.
# Don't know something specific to check the bin directory. Maybe gzip?
# Just make sure it exists.
#
swpaths="$swpaths $swpaths_defaults"
for sw in $swpaths; do
if [ -r $sw/include/zlib.h -a \
\( -r $sw/lib/libz.a -o -r $sw/lib/libz.so \) -a -d $cw/bin ]; then
SW_inc=$sw/include
SW_lib=$sw/lib
SW_bin=$sw/bin
SW_Location=$sw
break
fi
done
# if none found, try HDF4 software which contains a version of zlib.
if [ x-$SW_Location = x- ]; then
locate_hdf4
fi
}
# Main
#
# Options
#
if [ $# -lt 1 ]; then
USAGE
exit 1
fi
if [ "$1" = -h ]; then
USAGE
exit 0
fi
SW=$1
shift
# locations of the software seeked.
SW_inc= # include place
SW_lib= # library place
SW_bin= # binary place
SW_Location= # parent directory of all the above
OSname=`uname -s`
OSrelease=`uname -r`
case $SW in
hdf4|hdf)
locate_hdf4
;;
hdf5)
locate_hdf5
;;
zlib)
locate_zlib
;;
*)
echo "unknown software ($SW)"
USAGE
exit 1
;;
esac
# show the results located, separated by commas.
if [ -n "${SW_inc}" -a -n "${SW_lib}" -a -n "${SW_bin}" ]; then
echo ${SW_inc},${SW_lib},${SW_bin}
exit 0
else
exit 1
fi

View File

@@ -5,7 +5,6 @@ use warnings;
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
@@ -32,7 +31,6 @@ sub print_copyright ($) {
print $fh "/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\n";
print $fh " * Copyright by The HDF Group. *\n";
print $fh " * Copyright by the Board of Trustees of the University of Illinois. *\n";
print $fh " * All rights reserved. *\n";
print $fh " * *\n";
print $fh " * This file is part of HDF5. The full HDF5 copyright notice, including *\n";

View File

@@ -10,7 +10,6 @@ my @ctypes = ( () );
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
@@ -61,7 +60,6 @@ sub print_copyright ($) {
print $fh "/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\n";
print $fh " * Copyright by The HDF Group. *\n";
print $fh " * Copyright by the Board of Trustees of the University of Illinois. *\n";
print $fh " * All rights reserved. *\n";
print $fh " * *\n";
print $fh " * This file is part of HDF5. The full HDF5 copyright notice, including *\n";

View File

@@ -6,10 +6,10 @@ use warnings;
# (The max_idx parameter is the only thing that needs to be changed when adding
# support for a new major release. If support for a prior major release
# is added (like support for 1.4, etc), the min_sup_idx parameter will
# need to be decremented. - QAK)
# need to be decremented.)
# Max. library "index" (0 = v1.0, 1 = 1.2, 2 = 1.4, 3 = 1.6, 4 = 1.8, 5 = 1.10, 6 = 1.12, 7 = 1.14, etc)
$max_idx = 7;
# Max. library "index" (0 = v1.0, 1 = 1.2, 2 = 1.4, 3 = 1.6, 4 = 1.8, 5 = 1.10, 6 = 1.12, 7 = 1.14, 8 = 1.16, etc)
$max_idx = 8;
# Min. supported previous library version "index" (0 = v1.0, 1 = 1.2, etc)
$min_sup_idx = 3;
@@ -19,7 +19,6 @@ $indent = 2;
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
@@ -46,7 +45,6 @@ sub print_copyright ($) {
print $fh "/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\n";
print $fh " * Copyright by The HDF Group. *\n";
print $fh " * Copyright by the Board of Trustees of the University of Illinois. *\n";
print $fh " * All rights reserved. *\n";
print $fh " * *\n";
print $fh " * This file is part of HDF5. The full HDF5 copyright notice, including *\n";

View File

@@ -1,33 +0,0 @@
#! /bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
#
# This is a small program which will create directories n-levels deep.
# You just call it with something like:
#
# mkdirs /tmp/foo/bar/baz
#
# and it will create all the directories from /tmp down to baz which
# don't exist.
#
chmodprog="${CHMODPROG-chmod}"
mkdirprog="${MKDIRPROG-mkdir}"
make_dir () {
if test ! -d $1; then
make_dir `echo $1 | sed -e 's#/[^/]*$##'`
$mkdirprog $1
$chmodprog 755 $1
fi
}
make_dir `echo $1 | sed -e 's#/$##'`

View File

@@ -1,43 +0,0 @@
#!/bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
#
# Compare the modification time of file argument 1 against other file arguments.
# Return true (0) if argument 1 is newer than all others, otherwise return
# false (1). If any of the argument is not a file, return false (1).
#
# Programmer: Albert Cheng
# Created Date: 2005/07/06
# Modification:
# Albert Cheng 2005/8/30
# Changed from two arguments to multiple arguments.
if test $# -lt 2; then
exit 1
fi
if test ! -f $1; then
exit 1
fi
f1=$1
shift
for f in $*; do
if test ! -f $f; then
exit 1
fi
if test X = X`find $f1 -newer $f -print`; then
exit 1
fi
done
# passed all tests. Must be a file newer than all others.
exit 0

View File

@@ -61,26 +61,21 @@ STDOUT_FILTER() {
# Remove them from the stderr result file.
# $1 is the file name of the file to be filtered.
# Cases of filter needed:
# 1. MPE:
# In parallel mode and if MPE library is used, it prints the following
# two message lines whether the MPE tracing is used or not.
# Writing logfile.
# Finished writing logfile.
# 2. LANL MPI:
# * LANL MPI:
# The LANL MPI will print some messages like the following,
# LA-MPI: *** mpirun (1.5.10)
# LA-MPI: *** 3 process(es) on 2 host(s): 2*fln21 1*fln22
# LA-MPI: *** libmpi (1.5.10)
# LA-MPI: *** Copyright 2001-2004, ACL, Los Alamos National Laboratory
# 3. h5diff debug output:
# * h5diff debug output:
# Debug output all have prefix "h5diff debug: ".
# 4. AIX system prints messages like these when it is aborting:
# * AIX system prints messages like these when it is aborting:
# ERROR: 0031-300 Forcing all remote tasks to exit due to exit code 1 in task 0
# ERROR: 0031-250 task 4: Terminated
# ERROR: 0031-250 task 3: Terminated
# ERROR: 0031-250 task 2: Terminated
# ERROR: 0031-250 task 1: Terminated
# 5. LLNL Blue-Gene mpirun prints messages like there when it exit non-zero:
# * LLNL Blue-Gene mpirun prints messages like there when it exit non-zero:
# <Apr 12 15:01:49.075658> BE_MPI (ERROR): The error message in the job record is as follows:
# <Apr 12 15:01:49.075736> BE_MPI (ERROR): "killed by exit(1) on node 0"
STDERR_FILTER() {
@@ -91,12 +86,6 @@ STDERR_FILTER() {
cp $result_file $tmp_file
sed -e '/ BE_MPI (ERROR): /d' \
< $tmp_file > $result_file
# Filter MPE messages
if test -n "$pmode"; then
cp $result_file $tmp_file
sed -e '/^Writing logfile./d' -e '/^Finished writing logfile./d' \
< $tmp_file > $result_file
fi
# Filter LANL MPI messages
# and LLNL srun messages
# and AIX error messages

View File

@@ -1,7 +1,6 @@
#! /bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -2,7 +2,6 @@
# makeTarFiles.pl
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -1,215 +0,0 @@
#!/usr/bin/perl
# makeInternalREADME.pl
#
# Copyright by The HDF Group.
# All rights reserved.
#
# This file is part of HDF4. The full HDF4 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the files COPYING and Copyright.html. COPYING can be found at the root
# of the source code distribution tree; Copyright.html can be found at the
# root level of an installed copy of the electronic HDF4 document set and
# is linked from the top-level documents page. It can also be found at
# http://hdfgroup.org/HDF4/doc/Copyright.html. If you do not have
# access to either file, you may request a copy from help@hdfgroup.org.
#
use warnings;
use strict;
my $section2="For information on compilers and settings used to build these HDF5
libraries, please refer to:
./lib/libhdf5.settings
The contents of this directory are:
COPYING - Copyright notice
README - This file
RELEASE.txt - Detailed information regarding this release
bin/ - Directory containing HDF5 pre-compiled utilities
include/ - Directory containing HDF5 include files
lib/ - Directory containing HDF5 libraries and settings
share/ - Directory containing example code in C, C++, and
Fortran using HDF5 and HDF5 HL library APIs. The
shell scripts provided with these examples will
compile and run them, and will also test the
h5cc, h5c++, and h5fc compile scripts found
in the installed bin directory.
These binaries were built with the ZLIB and SZIP (version 2.1, Encoder
ENABLED) external libraries which are included in the lib directory for
convenience.
We also provide the ZLIB and SZIP source code on our ftp server at:
ftp://ftp.hdfgroup.org/lib-external/
The official ZLIB and SZIP pages are at:
ZLIB: http://www.zlib.net/
SZIP: http://hdfgroup.org/doc_resource/SZIP/
";
my $section3 = "If using the shared libraries, you must add the HDF5 library path
to the LD_LIBRARY_PATH variable.
";
my $section4 = "We provide scripts for compiling applications with the HDF5 libraries:
bin/h5cc - for C
bin/h5fc - for F90 (if Fortran 90 library is included with the binaries)
bin/h5c++ - for C++ (if C++ library is included with the binaries)
After you have installed the binaries to their final destination, you can use
these scripts (h5cc, h5fc, h5c++) to compile. However, you must first run
./h5redeploy in the bin directory to change site specific paths in the scripts.
You may also need to change other variables in the scripts, depending
on how things are set up on your system. Here are some of the variables
to check:
prefix - Path to the HDF5 top level installation directory
CCBASE - Name of the C compiler
CLINKERBASE - Name of the linker
LIBS - Libraries your application will link with
For further details refer to the INSTALL files in
ftp://ftp.hdfgroup.org/HDF5/current/src/unpacked/release_docs/
or in the ./release_docs/ directory of the HDF5 source code, which can be found
on the HDF Group ftp server at ftp://ftp.hdfgroup.org/HDF5/current/src/.
Please send questions, comments, and suggestions to the appropriate
contact address from http://www.hdfgroup.org/about/contact.html
";
my $indirectory = ".";
$indirectory = shift;
my $linktype = "shared";
if ($indirectory =~ /static/) {
$linktype = "static";
}
my $modestring="";
if ($indirectory =~ /32/) {
$modestring = "in 32 bit mode ";
}
my $version;
my $outfile = "$indirectory/README";
open OUTFILE, ">$outfile" or die "$!Couldn't open $outfile - check permissions for $indirectory\n";
my $hostname;
my $cmd = "grep \"HDF5 Version\" $indirectory/lib/libhdf5.settings";
$_ = `$cmd`;
#print OUTFILE $_, "\n";
s/HDF5 Version://;
s/^\s+//;
chomp;
$version = $_;
#print OUTFILE $_, "\n";
my $versionstring= "This directory contains the $linktype binary distribution of HDF5-".$version;
$cmd = "grep \"Uname information:\" $indirectory/lib/libhdf5.settings";
$_ = `$cmd`;
s/Uname information://;
s/^\s+//;
#print OUTFILE $_;
chomp;
#s/(^\w+)(\s)(\S+)/$1/;
#s/(^.*)(-)(.*)(200[7-8])(.*)(\s)(\S+)/$1 $5/;
#my $platformstring = "\nthat was compiled on:" . $_ . " ";
my $platformstring = "";
my $hostnamestring = $_;
my @hostnamestring = split / /, $hostnamestring;
#print OUTFILE "Size of hostnamestring is ", scalar @hostnamestring, "\n";
#print OUTFILE $hostnamestring[0] . "\t" . $hostnamestring[2]."\t".$hostnamestring[19]."\n";
$hostname = $hostnamestring[1];
#my $size = scalar @hostnamestring;
if ($hostname =~ /loyalty/) {
$platformstring = "\nthat was compiled " . $modestring . "on: " . $hostnamestring[0]." " . $hostnamestring[2]." " . $hostnamestring[-1] . " ";
}
elsif ($hostname =~ /freedom/) {
$platformstring = "\nthat was compiled " . $modestring . "on: " . $hostnamestring[0]." " . $hostnamestring[2]." " . $hostnamestring[-1] . " ";
} elsif ($hostname =~ /emu/) {
$platformstring = "\nthat was compiled " . $modestring . "on: " . $hostnamestring[0]." " . $hostnamestring[2] . " " . $hostnamestring[-2] . " ";
} elsif ($hostname =~ /fred/) {
$platformstring = "\nthat was compiled " . $modestring . "on: " . $hostnamestring[0]." " . $hostnamestring[2] . " " . $hostnamestring[-1] . " ";
} else {
$_ = $hostnamestring[2];
my $pos = index $_, '-';
my $os = substr $_, 0, $pos;
$platformstring = "\nthat was compiled " . $modestring . "on: " . $hostnamestring[0] . " " . $os . " " . $hostnamestring[-2] . " ";
}
my $mappingstring = "";
#no mappingstring for 1.6. Put it back for 1.8.
#$cmd = "grep \"Default API Mapping:\" $indirectory/lib/libhdf5.settings";
#$_ = `$cmd`;
#s/Default API Mapping://;
#s/^\s+//;
#chomp;
#if (/v\d+/) {
# s/v//;
# s/(\d)(\d)/$1\.$2/g;
# $mappingstring = "using the default\nAPI mapping for VERSION ".$_.".";
#
#}
print OUTFILE $versionstring;
print OUTFILE $platformstring."\n\n";
#print OUTFILE $mappingstring;
#if ($hostname eq "loyalty.hdfgroup.uiuc.edu" || $hostname eq "freedom.hdfgroup.uiuc.edu") {
# print OUTFILE " It includes the C APIs,\nbuilt using the following ";
# print OUTFILE "compilers:\n\n";
#}
#else {
if ($linktype eq "shared" && !($hostname =~ /32/)) {
print OUTFILE "\n\nIt includes the C, C++, F90 and Java APIs, built using the following\n";
} else {
print OUTFILE "\n\nIt includes the C, C++, and F90 APIs, built using the following\n";
}
print OUTFILE "compilers:\n\n";
#}
# Only the gcc compiler version is in libhdf5.settings, so for now I looked
# up the versions and hardcoded them here. We will put them in libhdf5.settings
# for the next release.
if ($indirectory =~ /gnu484/) {
print OUTFILE "\tgcc, g++, and gfortran 4.8.4\n\n";
} elsif ($hostname =~ /jam/ || $hostname =~ /koala/) {
print OUTFILE "\tgcc, g++, and gfortran 4.1.2\n\n";
} elsif ($hostname =~ /platypus/) {
print OUTFILE "\tgcc, g++, and gfortran 4.4.7\n\n";
if ($linktype eq "shared" && !($hostname =~ /32/)) {
print OUTFILE "\tjava 1.8.0_51\n\n";
}
} elsif ($hostname =~ /moohan/) {
print OUTFILE "\tgcc, g++, and gfortran 4.8.5\n\n";
if ($linktype eq "shared" && !($hostname =~ /32/)) {
print OUTFILE "\tjava 1.8.0_51\n\n";
}
} elsif ($hostname =~ /emu/) {
print OUTFILE "\tSun C and C++ 5.12, Sun Fortran 95 8.6\n\n";
} elsif ($hostname =~ /loyalty/ || $hostname =~ /freedom/) {
print OUTFILE "\tgcc, g++, and gfortran 4.6.1\n\n";
} elsif ($hostname =~ /duck/) {
print OUTFILE "\tApple clang/clang++ 3.0 from Xcode 4.6.1 and gfortran 4.8.2\n\n";
} elsif ($hostname =~ /kite/) {
print OUTFILE "\tApple clang/clang++ 5.1 from Xcode 5.0.2 and gfortran 4.8.2\n\n";
} elsif ($hostname =~ /quail/) {
print OUTFILE "\tgcc, g++ 5.1 from Xcode 5.1 and gfortran 4.8.2\n\n";
} elsif ($hostname =~ /osx1010test/) {
print OUTFILE "\tgcc, g++ 5.1 from Xcode 5.1 and gfortran 4.8.2\n\n";
}
print OUTFILE $section2;
print OUTFILE $section3;
print OUTFILE $section4;

View File

@@ -1,182 +0,0 @@
#!/usr/bin/perl
# makeOuterREADME.pl
# Copyright by The HDF Group.
# All rights reserved.
#
# This file is part of HDF4. The full HDF4 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the files COPYING and Copyright.html. COPYING can be found at the root
# of the source code distribution tree; Copyright.html can be found at the
# root level of an installed copy of the electronic HDF4 document set and
# is linked from the top-level documents page. It can also be found at
# http://hdfgroup.org/HDF4/doc/Copyright.html. If you do not have
# access to either file, you may request a copy from help@hdfgroup.org.
#
use warnings;
use strict;
my $indirectory = ".";
$indirectory = shift;
my $outdirectory = ".";
$outdirectory = shift;
my $outsubdir = shift;
my $version;
my $outfile = "$outdirectory/$outsubdir/README";
open OUTFILE, ">$outfile";
my $hostname="";
my $cmd = "grep \"HDF5 Version\" $indirectory-static/lib/libhdf5.settings";
$_ = `$cmd`;
print OUTFILE $_, "\n";
s/HDF5 Version://;
s/^\s+//;
chomp;
$version = $_;
#print OUTFILE $_, "\n";
my $versionstring= "This directory contains the precompiled HDF5 $version binary distribution\n(include files, libraries, utilities) for";
$cmd = "grep \"Uname information:\" $indirectory-static/lib/libhdf5.settings";
$_ = `$cmd`;
s/Uname information://;
s/^\s+//;
chomp;
print "String to work with is $_\n";
my $platformstring = "";
my $hostnamestring = $_;
my @hostnamestring = split / /, $hostnamestring;
$platformstring = "$hostnamestring[0] ";
if ($indirectory =~ /jam/ || $indirectory =~ /koala/) {
$hostnamestring = $hostnamestring[2];
my $pos = index $hostnamestring, "-";
if ($pos > 0) {
$platformstring .= substr $hostnamestring, 0, $pos;
} else {
$platformstring .= $hostnamestring[2];
}
$platformstring .= " ".$hostnamestring[-3];
} elsif ($indirectory =~ /linew/) {
$platformstring .= "$hostnamestring[2] $hostnamestring[-2]";
} else {
$platformstring .= "$hostnamestring[2] $hostnamestring[-1]";
}
print OUTFILE $versionstring." ".$platformstring.":\n\n";
my $line1;
my $line3;
my $line5;
my $compilerstring="";
my $compilerstring1="";
my $compilerstring2="";
print $indirectory, "\n";
if ($indirectory =~ /ostrich/) {
# $line1 = " hdf5-$version-$outsubdir-16API.tar.gz - Includes C, C++, F90 APIs (using\n";
$line3 = " hdf5-$version-$outsubdir-static.tar.gz - Includes C, C++, F90 APIs (using";
$line5 = " hdf5-$version-$outsubdir-shared.tar.gz - Includes C, C++, F90 APIs (using";
$compilerstring = " gcc, g++, and gfortran 4.4.7)";
}
elsif ($indirectory =~ /platypus/) {
# $line1 = " hdf5-$version-$outsubdir-16API.tar.gz - Includes C, C++, F90 APIs (using\n";
$line3 = " hdf5-$version-$outsubdir-static.tar.gz - Includes C, C++, F90 APIs (using";
$compilerstring1 = " gcc, g++, and gfortran 4.4.7)\n";
if ($indirectory =~ /32/) {
$line5 = " hdf5-$version-$outsubdir-shared.tar.gz - Includes C, C++, F90 and Java APIs (using";
$compilerstring2 = " gcc, g++, and gfortran 4.4.7)\n";
} else {
$line5 = " hdf5-$version-$outsubdir-shared.tar.gz - Includes C, C++, F90 and Java APIs (using";
$compilerstring2 = " gcc, g++, and gfortran 4.4.7 and java 1.8.0_51)\n";
}
}
elsif ($indirectory =~ /moohan/) {
# $line1 = " hdf5-$version-$outsubdir-16API.tar.gz - Includes C, C++, F90 APIs (using\n";
$line3 = " hdf5-$version-$outsubdir-static.tar.gz - Includes C, C++, F90 APIs (using";
$compilerstring1 = " gcc, g++, and gfortran 4.8.5)\n";
if ($indirectory =~ /32/) {
$line5 = " hdf5-$version-$outsubdir-shared.tar.gz - Includes C, C++, F90 and Java APIs (using";
$compilerstring2 = " gcc, g++, and gfortran 4.4.7)\n";
} else {
$line5 = " hdf5-$version-$outsubdir-shared.tar.gz - Includes C, C++, F90 and Java APIs (using";
$compilerstring2 = " gcc, g++, and gfortran 4.8.5 and java 1.8.0_51)\n";
}
}
elsif ($indirectory =~ /emu/) {
# $line1 = " hdf5-$version-$outsubdir-16API.tar.gz - Includes C, C++, F90 APIs (using\n";
$line3 = " hdf5-$version-$outsubdir-static.tar.gz - Includes C, C++, F90 APIs (using";
$line5 = " hdf5-$version-$outsubdir-shared.tar.gz - Includes C, C++, F90 APIs (using";
$compilerstring = " Sun C and C++ 5.12, Sun Fortran 95 8.6)\n";
}
elsif ($indirectory =~ /kite/) {
$line3 = " hfd5-$version-$outsubdir-static.tar.gz - Includes C, C++, F90 APIs (using";
$line5 = " hfd5-$version-$outsubdir-shared.tar.gz - Includes C, C++, F90 APIs (using";
$compilerstring = " Apple clang/clang++ 5.1 from Xcode 5.0.2,
gfortran 4.8.2)\n";
}
elsif ($indirectory =~ /quail/) {
$line3 = " hfd5-$version-$outsubdir-static.tar.gz - Includes C, C++, F90 APIs (using";
$line5 = " hfd5-$version-$outsubdir-shared.tar.gz - Includes C, C++, F90 APIs (using";
$compilerstring = " Apple clang/clang++ 6.0 from Xcode 5.1,
gfortran 4.9.2)\n";
}
elsif ($indirectory =~ /osx1010test/) {
$line3 = " hdf5-$version-$outsubdir-static.tar.gz - Includes C, C++, F90 APIs (using";
$line5 = " hdf5-$version-$outsubdir-shared.tar.gz - Includes C, C++, F90 APIs (using";
$compilerstring = " Apple clang/clang++ 6.1 from Xcode 6.1,
gfortran 4.9.2)\n";
}
elsif ($indirectory =~ /osx1011test/) {
$line3 = " hdf5-$version-$outsubdir-static.tar.gz - Includes C, C++, F90 APIs (using";
$line5 = " hdf5-$version-$outsubdir-shared.tar.gz - Includes C, C++, F90 APIs (using";
$compilerstring = " Apple clang/clang++ 7.0.2 from Xcode 7.0,
gfortran 5.2.0)\n";
}
print OUTFILE $line3;
print OUTFILE $compilerstring1."\n";
if ($line5 ne "") {
print OUTFILE $line5;
print OUTFILE $compilerstring2."\n";
}
#elsif ($indirectory =~ /-16API/) {
# print OUTFILE $line1;
# print OUTFILE $compilerstring;
# print OUTFILE " USES THE VERSION 1.6 DEFAULT API\n";
# print OUTFILE " MAPPING.\n\n";
#}
print OUTFILE " utilities/ - Directory containing the compiled HDF5 utilities.\n";
print OUTFILE " These utilities are STATICALLY linked and will run as is.\n\n";
#print OUTFILE " The tools can be downloaded separately, or ALL in one\n";
#print OUTFILE " compressed file (5-$version-$outsubdir-static-util.tar.gz).\n\n";
print OUTFILE "\n";
print OUTFILE "To obtain the HDF5 distribution, 'unzip' and 'untar' the distribution\n";
print OUTFILE "file:\n\n";
print OUTFILE " gzip -cd <gz file from above> | tar xvf -\n";
#$cmd = "grep \"Configured by:\" $indirectory/$key-static/lib/libhdf5.settings";
#$_ = `$cmd`;
#s/Configured by://;
#s/^\s+//;
#print OUTFILE $_;
#chomp;
#my $hostnamestring = $_;
#s/(^\w+)(\s)(\S+)/$1/;
#s/(^.*)(-)(.*)(200[7-8])(.*)(\s)(\S+)/$1 $5/;
#my $platformstring = $_ . ":\n\n";
#my @hostnamestring = split /@/, $hostnamestring;
#print "Size of hostnamestring is ", scalar @hostnamestring, "\n";
#print $hostnamestring[0] . "\t" . $hostnamestring[2]."\t".$hostnamestring[19]."\n";
#my $platformstring = $hostnamestring[1].":\n\n";
#$hostnamestring = $hostnamestring[1];
#my $pos = index $hostnamestring, ".";
#if ($pos > 0) {
# @hostnamestring = split /\./, $hostnamestring;
# $platformstring = $hostnamestring[0].":\n\n";
#}

View File

@@ -1,7 +1,6 @@
#!/bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
@@ -13,24 +12,6 @@
#
# Make a release of hdf5.
#
# Programmer: Robb Matzke
# Creation date: on or before 1998-01-29.
#
# Modifications
# Robb Matzke, 1999-07-16
# The SunOS 5.6 sed *must* have slashes as delimiters. I changed things like
# `sed s+/CVS++' to `sed 's/\/CVS//'
#
# Albert Cheng, 1999-10-26
# Moved the MANIFEST checking to a separate command file so that
# it can be invoked individually.
#
# Albert Cheng, 2004-08-14
# Added the --private option.
#
# James Laird, 2005-09-07
# Added the md5 method.
# Function definitions
#
@@ -38,14 +19,13 @@
USAGE()
{
cat << EOF
Usage: $0 -d <dir> [--docver BRANCHNAME] [-h] [--nocheck] [--private] <methods> ...
-d DIR The name of the directory where the release(es) should be
Usage: $0 -d <dir> [--docver BRANCHNAME] [-h] [--private] <methods> ...
-d DIR The name of the directory where the release(s) should be
placed.
--docver BRANCHNAME This is added for 1.8 and beyond to get the correct
version of documentation files from the hdf5docs
repository. BRANCHNAME for v1.8 should be hdf5_1_8.
-h print the help page.
--nocheck Ignore errors in MANIFEST file.
--private Make a private release with today's date in version information.
This must be run at the top level of the source directory.
@@ -72,23 +52,23 @@ for compressing the resulting tar archive (if none are given then
information is available in the README_HPC file.
doc -- produce the latest doc tree in addition to the archive.
An md5 checksum is produced for each archive created and stored in the md5 file.
A sha256 checksum is produced for each archive created and stored in the sha256 file.
Examples:
$ bin/release -d /tmp
/tmp/hdf5-1.8.13-RELEASE.txt
/tmp/hdf5-1.8.13.md5
/tmp/hdf5-1.8.13.sha256
/tmp/hdf5-1.8.13.tar
$ bin/release -d /tmp gzip
/tmp/hdf5-1.8.13-RELEASE.txt
/tmp/hdf5-1.8.13.md5
/tmp/hdf5-1.8.13.sha256
/tmp/hdf5-1.8.13.tar.gz
$ bin/release -d /tmp tar gzip zip
/tmp/hdf5-1.8.13-RELEASE.txt
/tmp/hdf5-1.8.13.md5
/tmp/hdf5-1.8.13.sha256
/tmp/hdf5-1.8.13.tar
/tmp/hdf5-1.8.13.tar.gz
/tmp/hdf5-1.8.13.tar.zip
@@ -100,11 +80,6 @@ EOF
# Function name: tar2zip
# Convert the release tarball to a Windows zipball.
#
# Programmer: Albert Cheng
# Creation date: 2014-04-23
#
# Modifications
#
# Steps:
# 1. untar the tarball in a temporary directory;
# Note: do this in a temporary directory to avoid changing
@@ -167,11 +142,6 @@ tar2zip()
# Function name: tar2cmakezip
# Convert the release tarball to a Windows zipball with files to run CMake build.
#
# Programmer: Larry Knox
# Creation date: 2017-02-20
#
# Modifications
#
# Steps:
# 1. untar the tarball in a temporary directory;
# Note: do this in a temporary directory to avoid changing
@@ -235,7 +205,7 @@ tar2cmakezip()
# step 3: add LIBAEC.tar.gz, ZLib.tar.gz and cmake files
cp /mnt/scr1/pre-release/hdf5/CMake/LIBAEC.tar.gz $cmziptmpsubdir
cp /mnt/scr1/pre-release/hdf5/CMake/ZLib.tar.gz $cmziptmpsubdir
cp /mnt/scr1/pre-release/hdf5/CMake/HDF5Examples-1.14.4-Source.zip $cmziptmpsubdir
cp /mnt/scr1/pre-release/hdf5/CMake/HDF5Examples-2.0.2-Source.zip $cmziptmpsubdir
cp /mnt/scr1/pre-release/hdf5/CMake/hdf5_plugins-master.zip $cmziptmpsubdir
cp $cmziptmpsubdir/$version/config/cmake/scripts/CTestScript.cmake $cmziptmpsubdir
cp $cmziptmpsubdir/$version/config/cmake/scripts/HDF5config.cmake $cmziptmpsubdir
@@ -269,10 +239,6 @@ tar2cmakezip()
# Function name: tar2cmaketgz
# Convert the release tarball to a gzipped tar file with files to run CMake build.
#
# Programmer: Larry Knox
# Creation date: 2017-02-20
#
# Modifications
#
# Steps:
# 1. untar the tarball in a temporary directory;
@@ -331,7 +297,7 @@ tar2cmaketgz()
# step 3: add LIBAEC.tar.gz, ZLib.tar.gz and cmake files
cp /mnt/scr1/pre-release/hdf5/CMake/LIBAEC.tar.gz $cmgztmpsubdir
cp /mnt/scr1/pre-release/hdf5/CMake/ZLib.tar.gz $cmgztmpsubdir
cp /mnt/scr1/pre-release/hdf5/CMake/HDF5Examples-1.14.4-Source.tar.gz $cmgztmpsubdir
cp /mnt/scr1/pre-release/hdf5/CMake/HDF5Examples-2.0.2-Source.tar.gz $cmgztmpsubdir
cp /mnt/scr1/pre-release/hdf5/CMake/hdf5_plugins-master.tar.gz $cmgztmpsubdir
cp $cmgztmpsubdir/$version/config/cmake/scripts/CTestScript.cmake $cmgztmpsubdir
cp $cmgztmpsubdir/$version/config/cmake/scripts/HDF5config.cmake $cmgztmpsubdir
@@ -347,11 +313,6 @@ tar2cmaketgz()
# and HDF5options.cmake files for parallel or serial only builds where build
# tests are run on compute nodes using batch scripts.
#
# Programmer: Larry Knox
# Creation date: 2019-01-28
#
# Modifications
#
# Steps:
# 1. untar the tarball in a temporary directory;
# Note: do this in a temporary directory to avoid changing
@@ -415,7 +376,7 @@ tar2hpccmaketgz()
# step 3: add LIBAEC.tar.gz, ZLib.tar.gz and cmake files
cp /mnt/scr1/pre-release/hdf5/CMake/LIBAEC.tar.gz $cmgztmpsubdir
cp /mnt/scr1/pre-release/hdf5/CMake/ZLib.tar.gz $cmgztmpsubdir
cp /mnt/scr1/pre-release/hdf5/CMake/HDF5Examples-1.14.4-Source.tar.gz $cmgztmpsubdir
cp /mnt/scr1/pre-release/hdf5/CMake/HDF5Examples-2.0.2-Source.tar.gz $cmgztmpsubdir
cp /mnt/scr1/pre-release/hdf5/CMake/hdf5_plugins-master.tar.gz $cmgztmpsubdir
cp $cmgztmpsubdir/$version/config/cmake/scripts/CTestScript.cmake $cmgztmpsubdir
cp $cmgztmpsubdir/$version/config/cmake/scripts/HDF5config.cmake $cmgztmpsubdir
@@ -443,7 +404,6 @@ VERS=`perl bin/h5vers`
VERS_OLD=
test "$VERS" || exit 1
verbose=yes
check=yes
release_date=`date +%F`
today=`date +%Y%m%d`
pmode='no'
@@ -482,9 +442,6 @@ while [ -n "$1" ]; do
DEST=$1
shift
;;
--nocheck)
check=no
;;
-h)
USAGE
exit 0
@@ -546,28 +503,10 @@ if [ ! -d $DEST ]; then
exit 1
fi
# Check the validity of the MANIFEST file.
bin/chkmanifest || fail=yes
if [ "X$fail" = "Xyes" ]; then
if [ $check = yes ]; then
echo ""
echo "Note! If you are running bin/release in a development branch"
echo "later than v 1.8 the MANIFEST check is expected to fail when"
echo "autogen.sh has not been run successfully. Either run autogen.sh "
echo "with /usr/hdf/bin/AUTOTOOLS at the beginning of PATH or add the"
echo "--nocheck argument to the bin/release command."
exit 1
else
echo "Continuing anyway..."
fi
fi
# Create a manifest that contains only files for distribution.
MANIFEST=$tmpdir/H5_MANIFEST
grep '^\.' MANIFEST | grep -v _DO_NOT_DISTRIBUTE_ >$MANIFEST
# Prepare the source tree for a release.
# Create a symlink to the source so files in the tarball have the prefix
# we want (gnu's --transform isn't portable)
ln -s `pwd` $tmpdir/$HDF5_VERS || exit 1
# Save a backup copy of Makefile if exists.
test -f Makefile && mv Makefile $tmpdir/Makefile.x
cp -p Makefile.dist Makefile
@@ -582,57 +521,53 @@ for f in README.md release_docs/RELEASE.txt; do
chmod 644 $f
done
# trunk is different than branches.
# develop is different than branches.
if [ "${DOCVERSION}" ]; then
DOC_URL="$DOC_URL -b ${DOCVERSION}"
fi
# Create the tar file
test "$verbose" && echo " Running tar..." 1>&2
( \
cd $tmpdir; \
tar cf $HDF5_VERS.tar $HDF5_VERS/Makefile \
`sed 's/^\.\//hdf5-'$VERS'\//' $MANIFEST` || exit 1 \
)
(cd "$tmpdir" && exec tar -ch --exclude-vcs -f "$HDF5_VERS.tar" "./$HDF5_VERS" || exit 1 )
# Compress
MD5file=$HDF5_VERS.md5
cp /dev/null $DEST/$MD5file
SHA256=$HDF5_VERS.sha256
cp /dev/null $DEST/$SHA256
for comp in $methods; do
case $comp in
tar)
cp -p $tmpdir/$HDF5_VERS.tar $DEST/$HDF5_VERS.tar
(cd $DEST; md5sum $HDF5_VERS.tar >> $MD5file)
(cd $DEST; sha256sum $HDF5_VERS.tar >> $SHA256)
;;
gzip)
test "$verbose" && echo " Running gzip..." 1>&2
gzip -9 <$tmpdir/$HDF5_VERS.tar >$DEST/$HDF5_VERS.tar.gz
(cd $DEST; md5sum $HDF5_VERS.tar.gz >> $MD5file)
(cd $DEST; sha256sum $HDF5_VERS.tar.gz >> $SHA256)
;;
cmake-tgz)
test "$verbose" && echo " Creating CMake tar.gz file..." 1>&2
tar2cmaketgz $HDF5_VERS $tmpdir/$HDF5_VERS.tar $DEST/CMake-$HDF5_VERS.tar.gz 1>&2
(cd $DEST; md5sum CMake-$HDF5_VERS.tar.gz >> $MD5file)
(cd $DEST; sha256sum CMake-$HDF5_VERS.tar.gz >> $SHA256)
;;
hpc-cmake-tgz)
test "$verbose" && echo " Creating HPC-CMake tar.gz file..." 1>&2
tar2hpccmaketgz $HDF5_VERS $tmpdir/$HDF5_VERS.tar $DEST/HPC-CMake-$HDF5_VERS.tar.gz 1>&2
(cd $DEST; md5sum HPC-CMake-$HDF5_VERS.tar.gz >> $MD5file)
(cd $DEST; sha256sum HPC-CMake-$HDF5_VERS.tar.gz >> $SHA256)
;;
bzip2)
test "$verbose" && echo " Running bzip2..." 1>&2
bzip2 -9 <$tmpdir/$HDF5_VERS.tar >$DEST/$HDF5_VERS.tar.bz2
(cd $DEST; md5sum $HDF5_VERS.tar.bz2 >> $MD5file)
(cd $DEST; sha256sum $HDF5_VERS.tar.bz2 >> $SHA256)
;;
zip)
test "$verbose" && echo " Creating zip ball..." 1>&2
tar2zip $HDF5_VERS $tmpdir/$HDF5_VERS.tar $DEST/$HDF5_VERS.zip 1>&2
(cd $DEST; md5sum $HDF5_VERS.zip >> $MD5file)
(cd $DEST; sha256sum $HDF5_VERS.zip >> $SHA256)
;;
cmake-zip)
test "$verbose" && echo " Creating CMake-zip ball..." 1>&2
tar2cmakezip $HDF5_VERS $tmpdir/$HDF5_VERS.tar $DEST/CMake-$HDF5_VERS.zip 1>&2
(cd $DEST; md5sum CMake-$HDF5_VERS.zip >> $MD5file)
(cd $DEST; sha256sum CMake-$HDF5_VERS.zip >> $SHA256)
;;
doc)
if [ "${DOCVERSION}" = "" ]; then
@@ -680,4 +615,6 @@ fi
# Remove temporary things
rm -rf $tmpdir
echo "DONE"
exit 0

View File

@@ -1,10 +1,10 @@
#!/bin/sh
#
# Copyright by The HDF Group.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# Copyright by The HDF Group.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
@@ -48,6 +48,9 @@ rm -f bin/missing
rm -f bin/test-driver
rm -f bin/depcomp
echo "Remove files generated by autoheader"
rm -f src/H5config.h.in
echo "Remove files generated by bin/make_err"
rm -f src/H5Epubgen.h
rm -f src/H5Einit.h

View File

@@ -5,7 +5,6 @@ $indent=4;
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -1,966 +0,0 @@
#! /bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
#
# run the hdf5/bin/snapshot
# Usage:
# runtest run the test for the local host
# runtest <hostname> run the test for <hostname>
# runtest -all run the test for all predefined hosts
#
# Assumptions in knowing where to find the right scripts to execute.
# 1. assume we are at the top level of the hdf5 source. So, bin/* are
# where the script files are.
# 2. after the cvs update is completed, we can go to the snapshot area
# hdf5 source tree and use the bin/* there.
# 3. Cannot use the snapshot area scripts from the beginning because
# for one, the current directory is renamed as previous right after
# a snapshot release; and for another, some scripts may be changed
# by the cvs update while it is being used.
# local setup
DEBUGMODE=""
test -n "$DEBUGMODE" && echo "******** DEBUGMODE is $DEBUGMODE ************"
WHEREAMI='pwd'
CMD=
# the name of this program
PROGNAME="bin/runtest $DEBUGMODE"
# Setup
HOSTNAME=`hostname | cut -f1 -d.` # no domain part
TODAY=`date +%m%d%a`
WEEKDAY=`date +%a`
H5VER= # default to current CVS version
H5VERSION= # default to current CVS version
n_test=0 # Number of tests ran
n_pass=0 # Number of tests passed
n_fail=0 # Number of tests failed
n_skip=0 # Number of tests skipped
# Default to do checkout (only once) and test, no release.
# Will run test only if there is significant differences from previous version.
# If srcdir is not used, don't launched multiple tests
SNAPSHOT="${DEBUGMODE:+echo }bin/snapshot"
SRCDIR="srcdir"
# Default standard Snaptest commands
SNAPCMD="$SRCDIR test clean"
# Default Standard snaptest command options
STANDARD_OPT=""
ENABLE_PARALLEL="--enable-parallel"
CONFIGNAME=$HOSTNAME # Name used in the SNAPTESTCFG file
# test host default as local host.
TESTHOST=""
#################################
# Function definitions
#################################
# Print messages to stdout
# Use this to show output heading to stdout
PRINT()
{
echo "$*"
}
# Show seconds since midnight.
# This is used to calculate seconds elapsed
SecOfDay()
{
set `date '+%H %M %S'`
t_sec=`expr $1 \* 3600 + $2 \* 60 + $3`
echo $t_sec
}
# Calculated the elapsed time (in seconds) between the first
# and second time. If second time is smaller than the first,
# we assume the clock has passed midnight and calculate appropriately.
ElapsedTime()
{
if [ $2 -lt $1 ]; then
t_sec=`expr 3600 \* 24 - $1 + $2`
else
t_sec=`expr $2 - $1`
fi
echo `expr $t_sec / 60`m `expr $t_sec % 60`s
}
# Report errors
# $1--an error message to be printed
REPORT_ERR()
{
ERRMSG=$1
# print it with a banner shifted right a bit
PRINT " *************************************"
PRINT " `date`"
PRINT " $ERRMSG"
PRINT " *************************************"
# report it in the FAILED-LOG file too
PRINT "$ERRMSG" >> $FAILEDLOG
}
#
# Report results of the last test done
REPORT_RESULT()
{
if [ $retcode -eq 0 ]; then
if [ $skiptest = yes ]; then
n_skip=`expr $n_skip + 1`
PRINT "SKIPPED ${HOSTNAME}: $TEST_TYPE" | tee -a $SKIPPEDLOG
else
n_pass=`expr $n_pass + 1`
PRINT "PASSED ${HOSTNAME}: $TEST_TYPE" | tee -a $PASSEDLOG
fi
else
# test failed.
n_fail=`expr $n_fail + 1`
REPORT_ERR "****FAILED ${HOSTNAME}: $TEST_TYPE****"
fi
}
# Print a blank line
PRINT_BLANK()
{
PRINT
}
# Print test trailer
PRINT_TEST_TRAILER()
{
PRINT "*** finished $TEST_TYPE tests for $HOSTNAME ***"
date; EndTime=`SecOfDay`
PRINT Total time = `ElapsedTime $StartTime $EndTime`
PRINT_BLANK
}
# Print trailer summary
PRINT_TRAILER()
{
PRINT "*** finished tests in $HOSTNAME ***"
date; TotalEndTime=`SecOfDay`
PRINT "${HOSTNAME}: Ran $n_test($n_pass/$n_fail/$n_skip) $runtest_type, Grand total test time = " \
"`ElapsedTime $TotalStartTime $TotalEndTime`" | tee -a $TIMELOG
PRINT_BLANK
}
# Figure out which remote command to use to reach a host.
# Try ssh first, then rsh since fewer machines support rsh exec.
# $1--hostname to reach.
CHECK_RSH()
{
# Figure out how to use ping command in this host.
# Some hosts use "ping host count", some use "ping -c count host".
# Test "ping -c 3 -w 5" since it has timeout feature.
# Test "ping -c ..." style before "ping host 3" because some machines
# that recognize -c treat 'ping localhost 3' as to ping host '3'.
if [ -z "$PING" ]; then
if ping -c 3 -w 5 localhost >/dev/null 2>&1; then
PING='ping -c 3 -w 5'
PINGCOUNT=
elif ping -c 3 localhost >/dev/null 2>&1; then
PING='ping -c 3'
PINGCOUNT=
elif ping localhost 3 >/dev/null 2>&1; then
PING=ping
PINGCOUNT=3
else # don't know how to use ping.
PING=no_ping
PINGCOUNT=
fi
fi
#
host=$1
# Try remote command with host if it responds to ping.
# Still try it if we don't know how to do ping.
if [ no_ping = "$PING" ] || $PING $host $PINGCOUNT >/dev/null 2>&1; then
if ssh $host -n hostname >/dev/null 2>&1; then
RSH=ssh
elif rsh $host -n hostname >/dev/null 2>&1; then
RSH=rsh
else
PRINT cannot remote command with $host
RSH="NoRemoteCommand"
fi
else
RSH="NotReachable"
fi
}
# Wait for a file for at most number of minutes
# $1--the file
# $2--number of minutes
# WAIT_STATUS set to:
# -1 if errors encountered
# 0 if file found within time limit
# 1 if file not found within time limit
WAITFOR()
{
wait_file=$1
nminutes=$2
if [ -z "$wait_file" -o ! "$nminutes" -ge 0 ]
then
PRINT "errors in argument of WAITFOR(): wait_file($1) or nminutes($2)"
WAIT_STATUS=-1
return
fi
while [ ! -f $wait_file ]; do
if [ $nminutes -gt 0 ]; then
PRINT "Wait For $wait_file to appear"
sleep 60 #sleep 1 minute
else
WAIT_STATUS=1
return
fi
nminutes=`expr $nminutes - 1`
done
WAIT_STATUS=0
return
}
# Wait till a file disappears for at most number of minutes.
# Useful to wait till a lock is removed by another process.
# $1--the file
# $2--number of minutes
# WAIT_STATUS set to:
# -1 if errors encountered
# 0 if file disappears within time limit
# 1 if file has not disappeared within time limit
WAITTILL()
{
wait_file=$1
nminutes=$2
if [ -z "$wait_file" -o ! "$nminutes" -ge 0 ]
then
PRINT "errors in argument of WAITTILL(): wait_file($1) or nminutes($2)"
WAIT_STATUS=-1
return
fi
while [ -f $wait_file ]; do
if [ $nminutes -gt 0 ]; then
PRINT "Wait till $wait_file has disappeared"
sleep 60 #sleep 1 minute
else
WAIT_STATUS=1
return
fi
nminutes=`expr $nminutes - 1`
done
WAIT_STATUS=0
return
}
# Run one snapshot test
# $*--Types of test being run
RUNSNAPTEST()
{
SNAPCMD_OPT="$STANDARD_OPT" # snapshot test option
SRCDIRNAME=${HOSTNAME}
# restore CC, PATH in case they were changed in the last test.
CC="$CC_SAVED"
PATH=$PATH_SAVED
export PATH # DEC OSF1 needs to export PATH explicitly
TEST_TYPE=$*
retcode=0
skiptest=no
date
PRINT "*** starting $TEST_TYPE tests in $HOSTNAME ***"
PRINT "Uname -a: `uname -a`"
# Parse the test type and set options accordingly.
# See comments of SNAPTEST_CONFIG_PARSE().
while [ $# -gt 0 ]; do
case $1 in
-n32) # want -n32 option
SRCDIRNAME=${SRCDIRNAME}-n32
CC="cc -n32"
export CC
;;
-64) # want -64 option
SRCDIRNAME=${SRCDIRNAME}-64
CC="cc -64"
export CC
;;
parallel) # want parallel test
SNAPCMD_OPT="$SNAPCMD_OPT $ENABLE_PARALLEL"
SRCDIRNAME=${SRCDIRNAME}-pp
;;
standard) # standard test
;;
--*)
# option for configure
SNAPCMD_OPT="$SNAPCMD_OPT $1"
;;
op-configure)
# option for configure
SNAPCMD_OPT="$SNAPCMD_OPT $1 $2"
shift
;;
op-snapshot)
# option for snapshot
shift
SNAPCMD_OPT="$SNAPCMD_OPT $1"
;;
setenv)
# pass them along to snapshot set environment variable
shift
SNAPCMD_OPT="$SNAPCMD_OPT setenv $1 $2"
shift
;;
setenvN)
# set environment variable with $1 values
# e.g., setenvN 3 x a b c is same as setenv x="a b c".
# pass them along to snapshot set environment variable
shift
envN=$1
shift
envname=$1
SNAPCMD_OPT="$SNAPCMD_OPT setenvN $envN $envname"
envalue=
while test $envN -gt 0; do
shift
envalue="$envalue $1"
envN=`expr $envN - 1`
done
SNAPCMD_OPT="$SNAPCMD_OPT $envalue"
;;
skip)
# skip this test
skiptest=yes
;;
srcdirname)
# Use this before using parallel and -n32 since this overrides
# the others.
shift
SRCDIRNAME=$1
;;
deploy)
# deploy the built binary.
shift
SNAPCMD_OPT="$SNAPCMD_OPT deploy $1"
;;
deploydir)
# default directory for deployment.
shift
SNAPCMD_OPT="$SNAPCMD_OPT deploydir $1"
;;
*) # unknown test
PRINT "$0: unknown type of test ($1)"
retcode=1
;;
esac
shift
done
if [ $retcode -ne 0 -o $skiptest = yes ]; then
errcode=$retcode
return $retcode
fi
# Track down the zlib software
ans=`$SNAPYARD/current/bin/locate_sw zlib`
if [ $? = 0 ]; then
Z_INC=`echo $ans | cut -f1 -d,`
Z_LIB=`echo $ans | cut -f2 -d,`
SNAPCMD_OPT="$SNAPCMD_OPT zlib $Z_INC,$Z_LIB"
else
# cannot locate zlib software.
# continue the test, maybe configure can find it.
:
fi
if [ -n "${SRCDIRNAME}" ]; then
SNAPCMD_OPT="$SNAPCMD_OPT srcdirname ${SRCDIRNAME}"
fi
# Setup log file name to save test output
THIS_MINUTE=`date +%H%M`
LOGFILE=${LOGBASENAME}/${SRCDIRNAME}_${TODAY}_${THIS_MINUTE}
PRINT "Running snapshot with output saved in"
PRINT " $LOGFILE"
(date; PRINT Hostname=$HOSTNAME) >> $LOGFILE
(
cd $SNAPYARD/current
$SNAPSHOT $SNAPCMD $SNAPCMD_OPT
) >> $LOGFILE 2>&1
retcode=$?
[ $retcode -ne 0 ] && errcode=$retcode
date >> $LOGFILE
if [ $retcode -ne 0 ]; then
# Dump the first 10 lines and the last 30 lines of the LOGFILE.
( ntail=30
echo =========================
echo "Dumping logfile of ${HOSTNAME}: $TEST_TYPE"
echo "Last $ntail lines of $LOGFILE"
echo =========================
tail -$ntail $LOGFILE
echo =========================
echo Dumping done
echo =========================
echo ""
) >> $FAILEDDETAIL
fi
}
TIMELIMIT_PARSE()
{
# Function returns timeparam for timekeeper via standard out -
# any debug statements should be 'echo "Debug string" >&2' or timekeeper
# will declare timeparam to be non-numeric and ignore it.
while read x y ; do
# Scan for entry for this weekday.
xd=`echo $x | cut -f1 -d/`
if [ "$xd" = ${WEEKDAY} ]; then
# strip away the weekday/ part.
timeparam=`echo $x | cut -f2 -d/`
break
fi
case "$x" in
'' | '#'*)
# blank or comment lines. Continue.
;;
???/*)
# Ignore any entry not of this weekday.
;;
*)
timeparam="$x"
;;
esac
done
echo $timeparam
return
}
# configuration parsing.
# Taking configuration from input.
# This should be invoke with configure file as stdin.
# Syntax of the configure file:
# All lines started with the # are comment lines and are ignored.
# Blank lines are ignored too.
# Each config line starts with a "Scope" followed by test types.
#
# Scope can be:
# standard ... # what the standard test types are.
# <host>: <test> Do <test> for <host>
# all: <test> Do <test> for all hosts.
# <weekday>/... Use this scope if the <weekday> matches.
# <weekday> can be {Mon,Tue,Wed,Thu,Fri,Sat,Sun}
# If no <host>: input for a <host>, the standard test is used.
#
# Test types:
# standard tests defined in standard scope.
# -n32 -n32 mode. Apply to 64/32 bit OS such as IRIX64.
# parallel parallel mode.
# op-configure <option> configure option
# op-snapshot <option> snapshot option
# --* configure option
# setenv <name> <value> set environment variable <name> to <value>
# Pass along to snapshot
# setenvN <N> <name> <value> ...
# set environment variable with <N> values
# e.g., setenvN 3 x a b c is same as setenv x="a b c".
# Pass along to snapshot.
# skip skip this test
# srcdirname <name> use <name> as the build-directory.
# deploy <name> deploy the built binary at directory <name>.
# deploydir <name> use <name> as the default directory for deployment.
SNAPTEST_CONFIG_PARSE()
{
while read x y ; do
# Scan for entry for this weekday.
xd=`echo $x | cut -f1 -d/`
if [ "$xd" = ${WEEKDAY} ]; then
# strip away the weekday/ part.
x=`echo $x | cut -f2 -d/`
fi
case "$x" in
'' | '#'*)
# blank or comment lines. Continue.
;;
???/*)
# Ignore any entry not of this weekday.
;;
standard)
#standard configuration
STANDARD_OPT="$y"
;;
all: | ${CONFIGNAME}:)
# types of test for all hosts or this host
if [ -n "$TEST_TYPES" ]; then
TEST_TYPES="$TEST_TYPES ; $y"
else
TEST_TYPES="$y"
fi
;;
*:) # ignore types of test for other hosts
;;
*) # unknown configuration option
PRINT $x $y
PRINT "***Unknown configuration option. Ignored.***"
;;
esac
done
}
# Snap Test configuration parsing.
# If TEST_TYPES is not set, set it to do the "standard" test.
SNAPTEST_CONFIG()
{
TEST_TYPES=
STANDARD_OPT=
if [ -f $SNAPTESTCFG ]; then
SNAPTEST_CONFIG_PARSE < $SNAPTESTCFG
fi
TEST_TYPES=${TEST_TYPES:-'standard'}
}
# Show usage page
USAGE()
{
cat <<EOF
Usage: runtest [-h] [-debug] [-r<version>] [-all] [-nocvs] [-nodiff] [<host> ...]
-h
print this help page
-debug
turn on debug mode
-r<version>
do runtest for <version>
-all
launch tests for all pre-defined testing hosts
-nocvs
do not do cvs commands
-nodiff
do not do diff commands
-setup
setup the directory structure for snapshot test
-configname <name>
use <name> as hostname in the parsing of the snaptest configure file
<host>
launch tests for <host>
-all and <host> are contradictory and whichever is specified last, is
the one to take effect. If neither are given, do the test for the
local host.
EOF
}
# Verify if directory ($1) exists. If not, create it.
CHECK_DIR()
{
dir=$1
if test ! -e $1; then
echo mkdir $1
mkdir $1
errcode=$?
elif test ! -d $1; then
echo $1 is not a directory
errcode=1
fi
}
#################################
# Main
#################################
#################################
# Set up global variables
#################################
retcode=0 # error code of individula task
errcode=0 # error code of the whole test
skiptest=no # if test is skipped
CC_SAVED="$CC" # CC & PATH maybe changed within a test.
PATH_SAVED=$PATH # These save the original values.
timelimit=300 # default time limit (minutes) for the timekeeper
#################################
# Parse options
#################################
while [ $# -gt 0 ]; do
case "$1" in
-h) # help--show usage
USAGE
exit 0
;;
-debug*)
# set debug mode
DEBUGMODE="$1"
SNAPSHOT="echo bin/snapshot"
PROGNAME="$PROGNAME $DEBUGMODE"
PRINT "******** DEBUGMODE is $DEBUGMODE ************"
;;
-r*)
# version string
H5VER="$1"
;;
-all)
# Test all hosts.
TESTHOST=-all
;;
-nocvs)
# do not do cvs commands.
NOCVS=nocvs
;;
-nodiff)
# do not do diff commands.
NODIFF=nodiff
;;
-configname)
# use <name> as hostname in the parsing of the snaptest configure file.
shift
CONFIGNAME=$1
;;
-setup)
# setup the directory structure for snapshot test.
CMD=setup
;;
-*) # Unknown option
PRINT "Unknown option ($1)"
USAGE
exit 1
;;
*)
TESTHOST=$*
break
;;
esac
shift
done
# setup H5VER if not set yet
if [ -z "$H5VER" -a -f bin/snapshot_version ]
then
. bin/snapshot_version
fi
if [ -n "$H5VER" ]
then
H5VERSION=hdf5_`echo $H5VER | sed -e s/-r// -e s/\\\./_/g`
PROGNAME="$PROGNAME $H5VER"
else
H5VERSION=hdf5
fi
#################################
# Setup snapshot test directories
#################################
BASEDIR=${HOME}/snapshots-${H5VERSION}
# initial processing of setup option if requested
if test x-$CMD = x-setup; then
CHECK_DIR $BASEDIR
test $errcode -ne 0 && exit 1
elif [ ! -d ${BASEDIR} ]; then
echo "BASEDIR ($BASEDIR) does not exist"
exit 1
fi
# Show the real physical path rather than the symbolic path
SNAPYARD=`cd $BASEDIR && /bin/pwd`
# Log file basename
LOGDIR=${SNAPYARD}/log
LOGBASENAME=${LOGDIR}
PASSEDLOG=${LOGDIR}/PASSED_LOG_${TODAY}
FAILEDLOG=${LOGDIR}/FAILED_LOG_${TODAY}
FAILEDDETAIL=${LOGDIR}/FAILED_DETAIL_${TODAY}
SKIPPEDLOG=${LOGDIR}/SKIPPED_LOG_${TODAY}
TIMELOG=${LOGDIR}/TIME_LOG_${TODAY}
TIMEKEEPERLOG=${LOGDIR}/TIMEKEEPER_LOG_${TODAY}
CVSLOG=${LOGDIR}/CVS_LOG_${TODAY}
CVSLOG_LOCK=${LOGDIR}/CVS_LOG_LOCK_${TODAY}
DIFFLOG=${LOGDIR}/DIFF_LOG_${TODAY}
COPYRIGHT_ERR=${LOGDIR}/COPYRIGHT_ERR_${TODAY}
# Snap Test hosts and Configuration files
ALLHOSTSFILE=${SNAPYARD}/allhostfile
SNAPTESTCFG=${SNAPYARD}/snaptest.cfg
TIMELIMIT=${SNAPYARD}/timelimit
TMPFILE="${LOGDIR}/#runtest.${TODAY}.$$"
# more processing of setup option if requested
if test x-$CMD = x-setup; then
CHECK_DIR $LOGDIR
test $errcode -ne 0 && exit 1
CHECK_DIR $LOGDIR/OLD
test $errcode -ne 0 && exit 1
CHECK_DIR $SNAPYARD/TestDir
test $errcode -ne 0 && exit 1
# create empty test hosts or configure files if non-existing
for f in $ALLHOSTSFILE $SNAPTESTCFG; do
if test ! -f $f; then
echo Creating $f
touch $f
fi
done
# create or update the current source.
echo update current source
$SNAPSHOT checkout
# setup completed. Exit.
exit 0
fi
#################################
# Show some host status numbers
#################################
# df sometimes hangs due to file system problems. Invoke it as background
# process and give it 10 seconds to finish. If it hangs, just continue.
uptime
df &
sleep 10
#################################
# Setup test host(s)
#################################
if [ "$TESTHOST" = -all ]; then
if [ -f $ALLHOSTSFILE ]; then
TESTHOST=`sed -e '/^#/d;/^ *$/d' $ALLHOSTSFILE`
else
PRINT "could not access the all-hosts-file ($ALLHOSTSFILE)"
USAGE
exit 1
fi
fi
#################################
# Setup to print a trailer summary when exiting not via
# the normal end of the script.
#################################
trap PRINT_TRAILER 0
#
TotalStartTime=`SecOfDay`
# Process the configuration
SNAPTEST_CONFIG
PRINT STANDARD_OPT=$STANDARD_OPT
PRINT TEST_TYPES=$TEST_TYPES
PRINT_BLANK
# Do a checkout if one has not been done today.
# Then check MANIFEST file and copyrights noitces.
if [ -z "$NOCVS" ]; then
PRINT "Running CVS checkout with output saved in"
PRINT " $CVSLOG"
# Set CVS lock first
touch $CVSLOG_LOCK
($SNAPSHOT checkout ) >> $CVSLOG 2>&1
# Save error code and remove the lock
errcode=$?
rm -f $CVSLOG_LOCK
if [ $errcode -ne 0 ]; then
# test failed.
REPORT_ERR "****FAILED ${HOSTNAME}: CVS checkout****"
exit $errcode
fi
# ===================
# Check MANIFEST file
# ===================
PRINT Checking MAINFEST file ...
(cd $SNAPYARD/current; bin/chkmanifest) > $TMPFILE 2>&1
errcode=$?
if [ $errcode -eq 0 ]; then
# test passed.
cat $TMPFILE
else
# test failed.
REPORT_ERR "****FAILED ${HOSTNAME}: MANIFEST check****"
( echo =========================
echo "MANIFEST checking failed output"
echo =========================
cat $TMPFILE
echo =========================
echo "MANIFEST checking failed output done"
echo =========================
echo ""
) >> $FAILEDDETAIL
fi
rm $TMPFILE
PRINT_BLANK
# No copyright checking until what need copyright is decided. 2006/4/7.
if false; then
# ======================
# Check Copyright notice
# ======================
PRINT Checking Copyrights notices ...
if (cd $SNAPYARD/current; bin/chkcopyright) > $TMPFILE 2>&1 ; then
echo Passed.
else
# Save the output and report some of it.
# Do not report it as failed for runtest yet.
# Send a separate report mail via hardcoding.
# Need fixes/cleanup later.
echo "Failed. See detail in another report mail"
cp $TMPFILE $COPYRIGHT_ERR
nheadlines=300
ntaillines=5 # Number of lines in report summary.
(
echo =========================
echo "Copyright checking failed. Showing first $nheadlines lines of output."
echo "Complete output is in file $COPYRIGHT_ERR"
echo =========================
nreportlines=`wc -l < $COPYRIGHT_ERR`
if [ $nreportlines -le `expr $nheadlines + $ntaillines` ]; then
# Just print the whole file.
cat $COPYRIGHT_ERR
else
# Show the first $nheadlines plus report summary
head -$nheadlines $COPYRIGHT_ERR
echo ...
tail -$ntaillines $COPYRIGHT_ERR
fi
) | Mail -s "${H5VERSION} Copyrights check Failed" hdf5lib
fi
rm $TMPFILE
PRINT_BLANK
fi
else
# make sure the cvs update, if done by another host, has completed.
# First wait for the presence of $CVSLOG which signals some host
# has started the cvs update. Then wait for the absence of $CVSLOG_LOCK
# which signals the host has completed the cvs update.
WAITFOR $CVSLOG 90
if [ $WAIT_STATUS -ne 0 ]; then
errcode=$WAIT_STATUS
REPORT_ERR "****FAILED ${HOSTNAME}: Time expired waiting CVS update to start****"
exit $errcode
fi
WAITTILL $CVSLOG_LOCK 10
if [ $WAIT_STATUS -ne 0 ]; then
errcode=$WAIT_STATUS
REPORT_ERR "****FAILED ${HOSTNAME}: Time expired waiting CVS update to finish****"
exit $errcode
fi
fi
# run a snapshot diff to see if any significant differences between
# the current and previous versions
if [ -z "$NODIFF" ]; then
$SNAPSHOT diff >> $DIFFLOG 2>&1
errcode=$?
# check the errcode only if NOT in DEBUG MODE
if [ -z "$DEBUGMODE" -a $errcode -eq 0 ]; then
# no need to run test
PRINT "NO TEST: no significant differences between current and previous versions" |
tee -a $PASSEDLOG
exit 0
fi
fi
# we can use the version of script in SNAPYARD/current now.
# Don't do the diff or cvs update any more.
PROGNAME="$SNAPYARD/current/$PROGNAME -nodiff -nocvs"
# Decide to do test for the local host or for remote hosts
if [ -n "$TESTHOST" -a $HOSTNAME != "$TESTHOST" ]; then
date
PRINT "*** launching tests from $HOSTNAME ***"
PRINT_BLANK
TEST_TYPE="launching"
cd ${SNAPYARD}/log
# Fork off timekeeper if concurrent tests will be used.
if [ -n "$SRCDIR" ]; then
timelimit=`TIMELIMIT_PARSE < $TIMELIMIT`
($SNAPYARD/current/bin/timekeeper $timelimit > $TIMEKEEPERLOG 2>&1 &)
PRINT " Fork off timekeeper $timelimit"
fi
runtest_type="hosts"
for h in $TESTHOST; do
# Must do CONFIGNAME before $h got changed by the second cut.
# cut returns the whole string if there is no / in the string
# at all. But that works okay for the CONFIGNAME too.
CONFIGNAME=`echo $h | cut -f2 -d/`
h=`echo $h | cut -f1 -d/`
n_test=`expr $n_test + 1`
TMP_OUTPUT="#${h}_${CONFIGNAME}.out"
(PRINT "=============="
PRINT "Testing $h"
PRINT "==============") > $TMP_OUTPUT
CHECK_RSH $h
# run the remote shell command with output to $TMP_OUTPUT
case "$RSH" in
rsh|ssh)
CMD="$RSH $h -n $PROGNAME -configname $CONFIGNAME"
PRINT $CMD
# launch concurrent tests only if srcdir is used
if [ -n "$SRCDIR" ]; then
$CMD || REPORT_ERR "****FAILED ${h}: Abnormal exit from runtest****" && PRINT_BLANK &
echo $! > PID.${h}_${CONFIGNAME}
else
$CMD || REPORT_ERR "****FAILED ${h}: Abnormal exit from runtest****" && PRINT_BLANK
fi
;;
NoRemoteCommand)
PRINT $h does not accept Remote Command "(`date`)"
;;
NotReachable)
PRINT $h is not reachable "(`date`)"
;;
*)
PRINT "CHECK_RSH for $h returned unknown result ($RSH)"
;;
esac >> $TMP_OUTPUT 2>&1
done
# wait for all launched tests to finish, then cat them back out.
wait
# Pause a moment in case the timekeeper is terminating processes.
wait 30
for h in $TESTHOST; do
CONFIGNAME=`echo $h | cut -f2 -d/`
h=`echo $h | cut -f1 -d/`
TMP_OUTPUT="#${h}_${CONFIGNAME}.out"
cat $TMP_OUTPUT
# Verify test script did complete by checking the last lines
(tail -5 $TMP_OUTPUT | grep -s 'Grand total' > /dev/null 2>&1) ||
(REPORT_ERR "****FAILED ${h}: snaptest did not complete****" &&
PRINT_BLANK)
rm -f $TMP_OUTPUT PID.${h}_${CONFIGNAME}
done
exit 0
fi
# run the test(s)
# Note that first field is cut without -s but all subsequent cut
# must use -s. If -s is not used at all, a $TEST_TYPES that has
# no ';' (only 1 test), will pass through intact in all cut. That
# results in infinite looping.
# If -s is used with the first field, it will suppress completely
# a $TYPE_TYPES that has no ';' (only 1 tst ). That results in no
# test at all.
# Note that n_test must start as 1.
#
n_test=1
runtest_type="tests"
TEST="`echo $TEST_TYPES | cut -f$n_test -d';'`"
while [ -n "$TEST" ]; do
StartTime=`SecOfDay`
RUNSNAPTEST $TEST
REPORT_RESULT
PRINT_TEST_TRAILER
n_test=`expr $n_test + 1`
TEST="`echo $TEST_TYPES | cut -f$n_test -s -d';'`"
done
# dec n_test to show the actual number of tests ran.
n_test=`expr $n_test - 1`
PRINT_TRAILER
# disable trailer summary printing since all trailers have been
# printed and we are exiting normally.
trap 0
exit $errcode

View File

@@ -1,837 +0,0 @@
#!/bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
# This script should be run nightly from cron. It checks out the source
# from the source repository and compares it against the previous
# snapshot. If anything significant changed then a new snapshot is
# created, the minor version number is incremented, and the change is
# checked back into the source repository.
#
# function definitions
TIMESTAMP()
{
echo "=====" "$1": "`date`" "====="
}
EXIT_BANNER()
{
TIMESTAMP "Exit $PROGNAME with status=$?"
}
# Show current total disk usage.
DISKUSAGE()
{
du -ks | \
( read x y; echo "Disk Usage=$x KB" )
}
# function provided for testing software downloaded as tar files. A version of
# this function that properly extracts the downloaded files can be provided in
# the snapshots-${sw}-overrides file.
EXTRACT()
{
echo "Error: ${SWVERSION} is in source repository - does not need extraction."
}
# Standard procedure for checking out or updating source code from an hdfgroup
# git repository. Override the function for other repositories or procedures.
SOURCE_CHECKOUT()
{
if test -n $GIT_URL; then
if [ -n "$AUTOGEN" ]; then
echo "Creating fresh clone of $GIT_URL in $BASEDIR/current_src"
# Check out the current version from source repository.
(cd $BASEDIR; rm -rf current_src
if test -z $GIT_BRANCH; then
echo "Testing empty branch $GIT_BRANCH."
git clone $GIT_URL current_src
else
echo "Testing branch $GIT_BRANCH."
git clone $GIT_URL -b $GIT_BRANCH current_src
fi
) || exit 1
else
echo "Creating fresh clone of $GIT_URL in $BASEDIR/current"
# Check out the current version from source repository.
(cd $BASEDIR; rm -rf current
if test -n $GIT_BRANCH; then
git clone $GIT_URL -b $GIT_BRANCH current
else
git clone $GIT_URL current
fi ) || exit 1
fi
else
echo "Warning! Source directory ("current") is not checked out from git."
fi
}
# Standard procedure for running the configure command in a build (test)
# directory
RUNCONFIGURE()
{
if [ "${CURRENT}" != "${TESTDIR}" -a "$CPSRC" = "yes" ]; then
echo "Copying source files to ${TESTDIR}."
cp -pr ${CURRENT}/* ${TESTDIR}
cd ${TESTDIR}
./${CONFIGURE}
elif [ -n "${AUTOGEN}" ]; then
${CURRENTSRC}/${CONFIGURE}
else
${CURRENT}/${CONFIGURE}
fi
}
# Sometimes "make distclean" doesn't adequately remove files from the previous
# build. If a build (test) directory was used, its contents can be entirely
# deleted to provide a clean start. If the test is building in the source
# directory, the contents can't be deleted, so run "make distclean".
DISTCLEAN()
{
if [ "${srcdir}" = "yes" -a -n "${SRCDIRNAME}" -a -d ${BASEDIR}/TestDir/${SRCDIRNAME} ]; then
echo "Remove contents of $SRCDIRNAME.\n"
rm -rf ${BASEDIR}/TestDir/${SRCDIRNAME}/*
else
echo "$MAKE distclean"
(cd ${TESTDIR} && ${MAKE} distclean)
fi
}
# Several of the software packages tested do not support make check-install.
# Those that support it should have a version of this function in their
# override with the following lines:
# TIMESTAMP "check-install $1"
# ${MAKE} check-install $1
CHECKINSTALL()
{
echo "check-install is not supported for ${SWVERSION}"
}
# Function for hdf4 and hdf5 to override to check in changes after snapshot.
# Safety measure to avoid unintended checkins to other repositories.
COMMITSNAPSHOT()
{
echo "original hdf5 script committed code changes back into git."
}
DISPLAYUSAGE()
{
set -
cat <<EOF
Usage: $PROGNAME [all] [checkout] [ftp <URL> [diff] [test] [srcdir] [release] [help]
[clean] [distclean] [echo] [deploy <dir>] [deploydir <dir>]
[zlib <zlib_path>] [releasedir <dir>] [srcdirname <dir>] [check-vfd]
[check-passthrough-vol]
[exec <command>] [module-load <module-list>] [op-configure <option>]
[--<option>]
all: Run all commands (checkout, test & release)
[Default is all]
checkout: Run source checkout
diff: Run diff on current and previous versions. Exit 0 if
no significant differences are found. Otherwise, non-zero.
deploy: deploy binary to directory <dir>
deploydir: use <dir> as the default directory for deployment
test: Run test
release: Run release
clean: Run make clean
distclean:Run make distclean
echo: Turn on echo mode (set -x)
setenv <name> <value>:
Set environment variable <name> to <value>.
setenvN <N> <name> <value> ...:
Set environment variable with <N> values.
E.g., setenvN 3 x a b c is same as setenv x="a b c".
srcdir: Use srcdir option (does not imply other commands)
"snapshot srcdir" is equivalent to "snapshot srcdir all"
"snapshot srcdir checkout" is equivalent to "snapshot checkout"
srcdirname <dir>:
Use <dir> as the srcdir testing directory if srcdir is chosen.
If <dir> starts with '-', it is append to the default name
E.g., "snapshot srcdir srcdirname -xx" uses hostname-xx
[Default is hostname]
help: Print this message
echo: Turn on shell echo
zlib <zlib_path>:
Use <zlib_path> as the ZLIB locations
[Default is $ZLIB_default]
releasedir <dir>:
Use <dir> as the release directory
[Default is $ReleaseDir_default]
check-vfd:
Run make check-vfd instead of just make check.
check-passthrough-vol:
Run make check-passthrough-vol instead of just make check.
NOTE: Will only succeed with passthrough VOL connectors
that use the native VOL connector as the terminal
connector.
exttest <testscript>;
Run testscript;
exec <command>:
Run <command>;
module-load <module-list>:
Load modules in comma-separated <module-list>;
op-configure <option>:
Pass <option> to the configure command
E.g., "snapshot op-configure --enable-parallel"
configures for parallel mode
--<option>:
Pass --<option> to the configure command
E.g., "snapshot --enable-parallel"
configures for parallel mode
EOF
exit $errcode
}
# MAIN
# SGI /bin/sh replaces $0 as function name if used in a function.
# Set the name here to avoid that ambiguity and better style too.
PROGNAME=$0
SNAPSHOTNAME=
HDFREPOS=
DOCVERSION=""
MODULELIST=""
if [ -f bin/snapshot_params ]; then
. bin/snapshot_params
echo "Added snapshot_params."
fi
if [ -z "$SWVER" -a -f bin/snapshot_version ]
then
. bin/snapshot_version
echo "Added snapshot_version."
fi
if [ -n ${HDFREPOS} -a -f bin/snapshot-${HDFREPOS}-overrides ]; then
. bin/snapshot-${HDFREPOS}-overrides
echo "Added snapshot-${HDFREPOS}-overrides."
fi
echo "====================================="
echo "$PROGNAME $*"
echo "====================================="
TIMESTAMP MAIN
uname -a
# setup exit banner message
trap EXIT_BANNER 0 1 2 9 15
# Dump environment variables before option parsing
echo ===Dumping environment variables before option parsing ===
printenv | sort
echo ===Done Dumping environment variables before option parsing ===
# snapshots release directory. Default relative to $BASEDIR.
ReleaseDir_default=release_dir
# Where is the zlib library?
# At NCSA, half of the machines have it in /usr/lib, the other half at
# /usr/ncsa/lib. Leave it unset.
ZLIB_default=
ZLIB=$ZLIB_default
# What compression methods to use? (md5 does checksum). Doc was apparently
# added as a compression method to create a separate tarfile containing the
# documentation files for v 1.8 and above.
if [ "${SWVERSION}" = "hdf5_1_6" ]; then
METHODS="gzip bzip2 md5"
else
METHODS="gzip bzip2 doc"
fi
# Use User's MAKE if set. Else use generic make.
MAKE=${MAKE:-make}
# Default check action.
CHECKVAL=check
#
# Command options
cmd="all"
test_opt=""
errcode=0
AUTOGEN=""
EXTTEST=""
EXEC_CMD_ARG=""
while [ $# -gt 0 ] ; do
case "$1" in
all)
cmd="all"
;;
checkout-autogen)
cmdcheckout="checkout"
AUTOGEN="autogen"
cmd=""
;;
checkout)
cmdcheckout="checkout"
cmd=""
;;
ftp)
echo "Setting ftp flags in snapshot script"
cmdcheckout="checkout"
cmdftp="ftp"
cmd=""
shift
if [ $# -lt 1 ]; then
echo "URL missing"
errcode=1
cmd="help"
break
fi
ftp_url="$1"
echo "ftp_url is $ftp_url"
;;
diff)
cmddiff="diff"
cmd=""
;;
deploy)
# deploy the built binary.
shift
if [ $# -lt 1 ]; then
echo "deploy <dir> missing"
errcode=1
cmd="help"
break
fi
cmddeploy="deploy"
DEPLOYDIRNAME="$1"
;;
deploydir)
# default directory for deployment.
shift
if [ $# -lt 1 ]; then
echo "deploydir <dir> missing"
errcode=1
cmd="help"
break
fi
deploydir="$1"
;;
test)
cmdtest="test"
cmd=""
;;
setenv)
# set environment variable
shift
eval $1="$2"
export $1
shift
;;
setenvN)
# set environment variable with $1 values
# e.g., setenvN 3 x a b c is same as setenv x="a b c".
# a kludge now--the extra single quotes are needed
# else eval complains.
shift
envN=$1
shift
envname=$1
envalue=
while test $envN -gt 0; do
shift
envalue="$envalue $1"
envN=`expr $envN - 1`
done
eval $envname="'$envalue'"
export $envname
;;
srcdir)
#use srcdir option for test
srcdir="yes"
;;
srcdirname)
shift
if [ $# -lt 1 ]; then
echo "srcdirname <dir> missing"
errcode=1
cmd="help"
break
fi
SRCDIRNAME="$1"
;;
release)
cmdrel="release"
cmd=""
;;
autogen-release)
cmdrel="autogen-release"
cmd=""
;;
clean | distclean)
cmdclean="$1"
cmd=""
;;
help)
cmd="help"
break
;;
echo)
set -x
break
;;
zlib)
shift
if [ $# -lt 1 ]; then
echo "ZLIB information missing"
errcode=1
cmd="help"
break
fi
ZLIB="$1"
;;
releasedir)
shift
if [ $# -lt 1 ]; then
echo "Release directory name missing"
errcode=1
cmd="help"
break
fi
ReleaseDir="$1"
;;
exttest)
shift
if [ $# -lt 1 ]; then
echo "exttest script name missing"
errcode=1
cmd="help"
break
fi
cmd=""
EXTTEST="$1"
;;
exec)
shift
if [ $# -lt 1 ]; then
echo "exec command name missing"
errcode=1
cmd="help"
break
fi
cmd=""
EXEC_CMD_ARG="$@"
# exit the parsing while loop since all arguments have been consumed.
break
;;
check-vfd)
CHECKVAL=check-vfd
;;
check-passthrough-vol)
CHECKVAL=check-passthrough-vol
;;
module-load)
shift
if [ $# -lt 1 ]; then
echo "missing module list to load"
errcode=1
cmd="help"
break
fi
MODULELIST="$1"
;;
--*)
OP_CONFIGURE="$OP_CONFIGURE $1"
;;
op-configure)
shift
if [ $# -lt 1 ]; then
echo "op-configure option missing"
errcode=1
cmd="help"
break
fi
OP_CONFIGURE="$OP_CONFIGURE $1"
;;
*)
echo "Unknown option $1"
errcode=1
cmd="help"
break
;;
esac
shift
done
if [ -n "$MODULELIST" ]; then
. ~/.bashrc
module use /opt/pkgs/modules/all
# load module command will take a space separated list of modules.
# If we have a comma separated list, convert ',' to ' '.
MODULELIST="$( echo -e "$MODULELIST" | tr ',' ' ' )"
module load $MODULELIST
fi
# Dump environment variables after option parsing
echo ===Dumping environment variables after option parsing ===
printenv | sort
echo ===Done Dumping environment variables after option parsing ===
if [ "$cmd" = help ]; then
DISPLAYUSAGE
fi
# Setup the proper configure option (--with-zlib) to use zlib library
# provide ZLIB is non-empty.
ZLIB=${ZLIB:+"--with-zlib="$ZLIB}
# Adding --prefix as a configure option will put the path to the deploy
# directory in the initial libhdf5*.la files
if [ -n "$DEPLOYDIRNAME" ]; then
OP_CONFIGURE="$OP_CONFIGURE --prefix=${deploydir}/${DEPLOYDIRNAME}"
fi
CONFIGURE="configure $OP_CONFIGURE"
# echo "Configure command is $CONFIGURE"
# Execute the requests
snapshot=yes
BASEDIR=${HOME}/snapshots-${SNAPSHOTNAME}
if [ ! -d ${BASEDIR} ]; then
echo "BASEDIR ($BASEDIR) does not exist"
exit 1
fi
CURRENT=${BASEDIR}/current
PREVIOUS=${BASEDIR}/previous
ReleaseDir=${ReleaseDir:=${BASEDIR}/${ReleaseDir_default}}
HOSTNAME=`hostname | cut -f1 -d.` # no domain part
# Try finding a version of diff that supports the -I option too.
DIFF=diff
for d in `echo $PATH | sed -e 's/:/ /g'` ; do
test -x $d/diff && $d/diff -I XYZ /dev/null /dev/null > /dev/null 2>&1 &&
DIFF=$d/diff && break
done
#=============================
# Run source checkout
#=============================
if [ "$cmd" = "all" -o -n "$cmdcheckout" ]; then
TIMESTAMP "checkout"
# ${BASEDIR}/bin is now updated from git by EveningMaint or DailyMaint
# to avoid updating the scripts in ${BASEDIR}/bin while they are running.
if [ -z "$AUTOGEN" ]; then
# If there is a Makefile in ${CURRENT}, the last test done in it
# has not been distclean'ed. They would interfere with other
# --srcdir build since make considers the files in ${CURRENT}
# take precedence over files in its own build-directory. Run
# a "make distclean" to clean them all out. This is not really
# part of the "checkout" functions but this is the most convenient
# spot to do the distclean. We will also continue the checkout process
# regardless of the return code of distclean.
( cd ${CURRENT}; test -f Makefile && ${MAKE} distclean)
fi
# echo "cmdftp is $cmdftp; ftp_url is $ftp_url"
if [ -n "$cmdftp" ]; then
echo "Get the NetCDF4 source from their ftp server."
echo "Command executed is: 2>&1 wget -N $ftp_url"
cd ${BASEDIR};
WGET_OUTPUT="`2>&1 wget -N $ftp_url`"
errcode=$?
if [[ $errcode -ne 0 ]]; then
exit $errcode
fi
if [ $? -ne 0 ];then
echo $0: "$WGET_OUTPUT" Exiting.
exit 1
fi
# echo "Wget output was $WGET_OUTPUT"
if echo "$WGET_OUTPUT" | fgrep 'not retrieving' &> /dev/null
then
echo "Snapshot unchanged"
else
echo "New snapshot downloaded"
EXTRACT
fi
else
SOURCE_CHECKOUT
fi
fi # Do source checkout
#=============================
# Run Test the HDF5 library
#=============================
if [ "$cmd" = "all" -o -n "$cmdtest" -o -n "$cmddiff" ]; then
TIMESTAMP "Run Tests"
# setup if srcdir is used.
if [ -z "$srcdir" ]; then
TESTDIR=${CURRENT}
else
#create TESTDIR if not exist yet
case "$SRCDIRNAME" in
"")
SRCDIRNAME=$HOSTNAME
;;
-*)
SRCDIRNAME="$HOSTNAME$SRCDIRNAME"
;;
esac
TESTDIR=${BASEDIR}/TestDir/${SRCDIRNAME}
test -d ${TESTDIR} || mkdir ${TESTDIR}
# set TESTDIR to use the direct path to the local test directory
# rather than the path through ${BASEDIR}.
cd ${TESTDIR}
TESTDIR=`pwd -P`
cd ${CURRENT}
fi
# Make sure current version exists and is clean
if [ -d ${TESTDIR} ]; then
DISTCLEAN
else
errcode=$?
snapshot=no
exit $errcode
fi
# Compare it with the previous version. Compare only files listed in
# the MANIFEST plus the MANIFEST itself.
if [ -d ${PREVIOUS} ]; then
if [ -z "${AUTOGEN}" ]; then
CURRENTSRC=${CURRENT}
else
CURRENTSRC=${BASEDIR}/current_src
fi
if (${DIFF} -c ${PREVIOUS}/MANIFEST ${CURRENTSRC}/MANIFEST); then
snapshot=no
for src in `grep '^\.' ${CURRENTSRC}/MANIFEST|expand|cut -f1 -d' '`; do
if ${DIFF} -I H5_VERS_RELEASE -I " released on " \
-I " currently under development" \
${PREVIOUS}/$src ${CURRENTSRC}/$src
then
: #continue
else
snapshot=yes
break
fi
done
fi
fi
# if diff is chosen, exit 0 if no significant differences are found.
# otherwise, exit 1. This includes cases of other failures.
if [ -n "$cmddiff" ]; then
if [ $snapshot = no ]; then
exit 0
else
exit 1
fi
fi
#=============================
# Execute command if defined
#=============================
#echo BEFORE EXEC command
#echo EXEC_CMD_ARG=${EXEC_CMD_ARG}
if [ -n "$EXEC_CMD_ARG" ]; then
TIMESTAMP ${EXEC_CMD_ARG}
TESTDIR=${BASEDIR}/TestDir/${SRCDIRNAME}
test -d ${TESTDIR} || mkdir ${TESTDIR}
if cd ${TESTDIR}; then
# clean up the directory before executing the command
# Do we need to clean first?
# rm -rf *
#
# If EXEC_CMD_ARG starts with a '/', it has an absolute path, else it is
# relative to the BASEDIR.
case "$EXEC_CMD_ARG" in
/*)
${EXEC_CMD_ARG}
;;
*)
${BASEDIR}/${EXEC_CMD_ARG}
;;
esac
errcode=$?
else
echo "${TESTDIR} not accessible"
errcode=1
fi
# exit snapshot since nothing else to do, for now.
exit $errcode
fi
# Build, run tests and install procedures
if [ "$snapshot" = "yes" ] && [ "$NOMAKE" != "yes" ]; then
FAIL_SECTION=""
if [ -f ${TESTDIR}/failsection ]; then
rm ${TESTDIR}/failsection
fi
if (cd ${TESTDIR} && \
TIMESTAMP "configure" && echo "configure" > ${TESTDIR}/failsection && \
RUNCONFIGURE && \
sleep 2 && \
TIMESTAMP "make" && echo "make" > ${TESTDIR}/failsection && \
${MAKE} && DISKUSAGE \
TIMESTAMP ${CHECKVAL} && echo "make check" > ${TESTDIR}/failsection && \
${MAKE} ${CHECKVAL} && DISKUSAGE \
TIMESTAMP "install" && echo "make install" > ${TESTDIR}/failsection && \
${MAKE} install && DISKUSAGE \
TIMESTAMP "check-install" && echo "make check-install" > ${TESTDIR}/failsection && \
CHECKINSTALL && DISKUSAGE \
TIMESTAMP "uninstall" && echo "make uninstall" > ${TESTDIR}/failsection && \
${MAKE} uninstall && DISKUSAGE); then
:
else
errcode=$?
FAIL_SECTION=`cat ${TESTDIR}/failsection`
echo "Failed running ${FAIL_SECTION}"
snapshot=no
exit $errcode
fi
elif [ $CPSRC ]; then
cp -pr ${CURRENT}/* ${TESTDIR}
else
cmdclean=""
fi
fi # Test the HDF5 library
# Run external test if configured
#=============================
#=============================
#if [ -d "$CURRENT" ]; then
if [ "$EXTTEST" != "" ]; then
TIMESTAMP ${EXTTEST}
TESTDIR=${BASEDIR}/TestDir/${SRCDIRNAME}
test -d ${TESTDIR} || mkdir ${TESTDIR}
cd ${TESTDIR}
sleep 1
TIMESTAMP $pwd
ls
${BASEDIR}/${EXTTEST}
errcode=$?
exit $errcode
fi
#=============================
# Run deployment if requested.
#=============================
if [ -n "$DEPLOYDIRNAME" ]; then
# The daily tests deploy to .../hdf5/... or .../hdf4/... except on cobalt where the
# deploy directory is in .../HDF5/... lc will take care of this. If hdf4 or hdf5
# either upper or lower case isn't in the path, RELEASE.txt won't be found unless
# it is in $CURRENT.
POS4=`perl -e "print index(lc(\"${deploydir}/${DEPLOYDIRNAME}\"), 'hdf4')"`
POS5=`perl -e "print index(lc(\"${deploydir}/${DEPLOYDIRNAME}\"), 'hdf5')"`
if [ "${POS4}" -ge "0" ]; then
RELEASE_TXT_LOC="release_notes"
elif [ "${POS5}" -ge "0" ]; then
RELEASE_TXT_LOC="release_docs"
else
RELEASE_TXT_LOC=""
fi
if [ "$snapshot" = "yes" ]; then
TIMESTAMP "deploy"
if (cd ${TESTDIR} &&
${BASEDIR}/bin/deploy ${deploydir}/${DEPLOYDIRNAME} && \
TIMESTAMP "clean" && \
${MAKE} clean && \
TIMESTAMP "check-install prefix=${deploydir}/${DEPLOYDIRNAME}" && \
CHECKINSTALL prefix=${deploydir}/${DEPLOYDIRNAME}); then
cd ${CURRENT}
cp ${RELEASE_TXT_LOC}/RELEASE.txt ${deploydir}/${DEPLOYDIRNAME}
cp COPYING ${deploydir}/${DEPLOYDIRNAME}
#: #continue
else
errcode=$?
exit $errcode
fi
fi
fi # Deploy
#=============================
# Run Release snapshot, update version, and commit to source repository
#=============================
if [ "$cmd" = "all" -o -n "$cmdrel" ]; then
if [ "$snapshot" = "yes" ]; then
TIMESTAMP "release"
DISTCLEAN
(
# Turn on exit on error in the sub-shell so that it does not
# commit source if errors encounter here.
set -e
if [ "$cmdrel" = "autogen-release" ]; then
cd ${BASEDIR}/current_src
else
cd ${CURRENT}
fi
if [ "$HDFREPOS" = "hdf4" ]; then
RELEASE_VERSION="`perl bin/h4vers -v`"
echo "Making snapshot release ($RELEASE_VERSION) to ${ReleaseDir}..."
bin/release -d $ReleaseDir $METHODS
perl bin/h4vers -i
elif [ "$HDFREPOS" = "hdf5" ]; then
RELEASE_VERSION="`perl bin/h5vers -v`"
echo "Making snapshot release ($RELEASE_VERSION) to ${ReleaseDir}..."
if [ "${DOCVERSION}" ]; then
bin/release -d $ReleaseDir --docver ${DOCVERSION} $METHODS
else
bin/release -d $ReleaseDir $METHODS
fi
perl bin/h5vers -i
else
echo "need real release steps. For now, only move current version to previous"
fi
COMMITSNAPSHOT
)
errcode=$?
fi
# Replace the previous version with the current version.
# Should check if the errcode of the release process but there
# are other failures after release was done (e.g. h5vers or git failures)
# that should allow the replacement to occur.
rm -rf ${PREVIOUS}
mv ${CURRENT} ${PREVIOUS}
fi #Release snapshot
#=============================
# Clean the test area. Default is no clean.
#=============================
if [ -n "$cmdclean" ] && [ "$NOMAKE" != "yes" ]; then
TIMESTAMP "clean"
# setup if srcdir is used.
if [ -z "$srcdir" ]; then
TESTDIR=${CURRENT}
else
case "$SRCDIRNAME" in
"")
SRCDIRNAME=$HOSTNAME
;;
-*)
SRCDIRNAME="$HOSTNAME$SRCDIRNAME"
;;
esac
TESTDIR=${BASEDIR}/TestDir/${SRCDIRNAME}
fi
# clean it
if (cd ${TESTDIR} && ${MAKE} $cmdclean ) then
:
else
errcode=$?
snapshot=no
exit $errcode
fi
fi # Clean the Test directory
exit $errcode

View File

@@ -1,19 +0,0 @@
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
#
# default version for snapshot test
# H5VERSION matches with a source version symbolic name. Will test use the
# latest revision of that branch. If set to "hdf5", it uses the main
# version.
# H5VER tells runtest which version to run
H5VERSION=hdf5

View File

@@ -1,7 +1,6 @@
#!/bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -1,129 +0,0 @@
#!/bin/sh
##
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
##
# As a time keeper of the remote daily test process launched by runtest.
# It sleeps for a certain time and then wakes up to hangup those processes
# that are still around, assuming they have run too long.
#
# Programmer: Albert Cheng
# Created Date: 2004/12/23
# variable initialization
waitminutes=300 # default to 5 hours == 300 minutes
debugtimelimit=
debugflag= # no debug
# Function definitions
#
# PRINTMSG
# Print a one line message left justified in a field of 70 characters
# without newline. More output for this line later.
#
PRINTMSG() {
SPACES=" "
echo "$* $SPACES" | cut -c1-70 | tr -d '\012'
}
USAGE()
{
echo "Usage: %0 [-h] [-debug] [<time-limit>]"
echo " Run timekeeper with <time-limit> minutes, default is $waitminutes."
echo " If <time-limit> is in the form of HH:MM, it means wait till then."
echo " -h print this help page"
echo " -debug run debug mode"
}
ParseOption()
{
if [ $# -gt 0 -a "$1" = -h ]; then
shift
USAGE
exit 0
fi
if [ $# -gt 0 -a "$1" = -debug ]; then
shift
debugflag=yes
waitminutes=1 # use shorter time for debug
fi
if [ $# -gt 0 ]; then
targettime=$1
shift
# find out it is minutes to wait or HH:MM to wake up
case $targettime in
*:*) # HH:MM
currenttime=`date +%H:%M`
currenthour=`echo $currenttime | cut -f1 -d:`
currentminute=`echo $currenttime | cut -f2 -d:`
targethour=`echo $targettime | cut -f1 -d:`
targetminute=`echo $targettime | cut -f2 -d:`
waitminutes=`expr \( $targethour - $currenthour \) \* 60 + $targetminute - $currentminute`
if test $waitminutes -le 0; then
# target time is in tomorrow, add 1 day of minutes
waitminutes=`expr 24 \* 60 + $waitminutes`
fi
;;
*)
waitminutes=$targettime
;;
esac
fi
}
# Main body
echo "Timekeeper started at `date`"
ParseOption $*
waitperiod=`expr $waitminutes \* 60` # convert to seconds
if [ -z "$debugflag" ]; then
# normal time keeping mode
# sleep first
echo Timekeeper sleeping for $waitperiod seconds
sleep $waitperiod
# Look for any processes still around
echo "Timekeeper woke up at `date`, looking for processes to terminate..."
for x in PID.* ; do
if [ -f $x ]; then
pid=`cat $x`
# check if process is still around
if test X$pid \!= X && ps -p $pid > /dev/null; then
echo "terminating process $x ($pid)"
kill -HUP $pid
echo "Remote shell command ended. But some processes might still be"
echo "running in the remote machine. Login there to verify."
fi
fi
done
else
# Debug mode. Launch two rsh process, one ends before, the other after
# waitperiod. Must launch timekeeper from a subshell, else the debug
# will wait for it too.
myhostname=`hostname`
( $0 $waitminutes &)
debugtimelimit=`expr $waitperiod - 10`
echo rsh $myhostname sleep $debugtimelimit
rsh $myhostname sleep $debugtimelimit &
echo $! > PID.before
debugtimelimit=`expr $waitperiod + 10`
echo rsh $myhostname sleep $debugtimelimit
rsh $myhostname sleep $debugtimelimit &
echo $! > PID.after
wait
rm PID.before PID.after
fi
echo "Timekeeper ended at `date`"

View File

@@ -1,7 +1,6 @@
#!/usr/bin/env perl
##
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
@@ -77,7 +76,6 @@ $Source = "";
"H5G_obj_t" => "Go",
"H5G_stat_t" => "Gs",
"hsize_t" => "h",
"H5_alloc_stats_t" => "Ha",
"H5_atclose_func_t" => "Hc",
"hssize_t" => "Hs",
"H5E_major_t" => "i", # H5E_major_t is typedef'd to hid_t
@@ -88,6 +86,7 @@ $Source = "";
"H5_index_t" => "Ii",
"H5I_iterate_func_t" => "II",
"H5_iter_order_t" => "Io",
"H5FD_subfiling_ioc_select_t" => "IO",
"H5I_future_realize_func_t" => "IR",
"int" => "Is",
"int32_t" => "Is",
@@ -188,6 +187,7 @@ $Source = "";
"H5Z_filter_t" => "Zf",
"H5Z_filter_func_t" => "ZF",
"ssize_t" => "Zs",
# Types below must be defined here, as they appear in function arguments,
# but they are not yet supported in the H5_trace_args() routine yet. If
# they are used as an actual parameter type (and not just as a pointer to
@@ -198,6 +198,7 @@ $Source = "";
"H5FD_t" => "#",
"H5FD_hdfs_fapl_t" => "#",
"H5FD_mirror_fapl_t" => "#",
"H5FD_onion_fapl_t" => "#",
"H5FD_ros3_fapl_t" => "#",
"H5FD_splitter_vfd_config_t" => "#",
"H5L_class_t" => "#",

View File

@@ -8,12 +8,10 @@ use warnings;
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the files COPYING and Copyright.html. COPYING can be found at the root
# of the source code distribution tree; Copyright.html can be found at the
# root level of an installed copy of the electronic HDF5 document set and
# is linked from the top-level documents page. It can also be found at
# http://hdfgroup.org/HDF5/doc/Copyright.html. If you do not have
# access to either file, you may request a copy from help@hdfgroup.org.
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
#
# Quincey Koziol
# 9 Aug 2013
@@ -229,6 +227,9 @@ while (<>) {
# Skip variables with the word 'warning' in them
next if $_ =~ /_warning_/;
# Skip AMD Optimizing Compiler (aocc) lines "<#> warning(s) generated."
next if $_ =~ / warnings? generated\./;
# "Hide" the C++ '::' symbol until we've parsed out the parts of the line
while($_ =~ /\:\:/) {
$_ =~ s/\:\:/@@@@/g;
@@ -244,6 +245,18 @@ while (<>) {
} elsif($_ =~ /^\s*[Ww]arning:.*/) {
$name = $last_c_name;
($toss, $warning, $extra, $extra2) = split /\:/, $_;
# Check for file-scope gcc Fortran warning output
} elsif($_ =~ /f\d\d\d: Warning:/) {
# These are interspersed with the "compiling a file" output
# when compiling with `make -j` and thus difficult to tie to
# any particular file. They are due to things like inappropriate
# build options and don't have a line number.
#
# They start with f, as in f951
$name = "(generic)";
$line = int(rand(1000000)); # Hack to avoid counting as duplictates
($warning) = $_ =~ /\[(.*)\]/x;
# Check for FORTRAN warning output
} elsif($_ =~ /^Warning:.*/) {
$name = $last_fort_name;
@@ -382,7 +395,7 @@ while (<>) {
$warning =~ s/"[A-Za-z_0-9]*"/"-"/g;
}
# Genericize [GCC?] C/C++ warning text about suggessted attribute
# Genericize [GCC?] C/C++ warning text about suggested attribute
if($warning =~ /attribute=[A-Za-z_0-9]*\]/) {
$warning =~ s/=[A-Za-z_0-9]*\]/=-\]/g;
}

View File

@@ -1,76 +0,0 @@
#!/bin/sh
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# help@hdfgroup.org.
#
# Fix configure file so that it can launch configure testing executable
# via the proper launching command, e.g., yod. (Thus the name yodconfigure
# is used.)
#
# Author: Albert Cheng
if [ "$1" = -f ]; then
FORCEMODE=yes
echo turn FORCEMODE to $FORCEMODE
shift
fi
if [ $# -ne 1 ]; then
echo "Usage: $0 [-f] <configure file>"
echo " -f apply the change even if it has been applied already."
exit 1
fi
confile=$1
if [ ! -w $confile ]; then
echo "$0: $confile is not writable"
exit 1
fi
ACTRY_NAME="ACTRY()"
if grep ^"$ACTRY_NAME"$ $confile >/dev/null 2>&1 && [ "$FORCEMODE" != yes ]; then
echo "$0: $confile is already yodconfigure ready. Use -f to force yodconfigure again."
exit 1
fi
# Insert the ACTRY function after the 1st line which is the #!/bin/sh.
# Change all "eval $ac_try" commands to call ACTRY.
# auto-configure have changed the ac_try syntax from 'eval $ac_try' to
# 'eval "$ac_try"'. Thus requiring two very similar global-substitute.
# The single quotes around EOF tell shell NOT to expand or do substitution in
# the body of ed input.
#
ed - $confile <<'EOF'
1a
# ===inserted by yodconfigure ====
# ACTRY will figure out when it is approprirate to run the command by the
# $RUNSERIAL launcher (e.g., yod -sz 1) and when to just run it as is.
# So far, ./a.out and ./conftest are names of real executable that should
# be run by $RUNSERIAL.
#
# (uncomment the echo line if you want to see what is going on.)
ACTRY()
{
#echo ACTRY: args are: $* > /dev/tty
if [ "$1" = ./a.out -o "$1" = ./conftest ]; then
# echo $RUNSERIAL $* > /dev/tty
$RUNSERIAL $*
else
$*
fi
}
# === end of ACTRY inserted by yodconfigure ====
.
g/eval $ac_try/s/eval/eval ACTRY/
g/eval "$ac_try"/s/eval/eval ACTRY/
w
q
EOF

View File

@@ -1,4 +1,4 @@
cmake_minimum_required (VERSION 3.12)
cmake_minimum_required (VERSION 3.18)
project (HDF5_CPP CXX)
add_subdirectory (src)

View File

@@ -1,6 +1,5 @@
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -1,4 +1,4 @@
cmake_minimum_required (VERSION 3.12)
cmake_minimum_required (VERSION 3.18)
project (HDF5_CPP_EXAMPLES CXX)
# --------------------------------------------------------------------
@@ -34,7 +34,7 @@ set (tutr_examples
foreach (example ${examples})
add_executable (cpp_ex_${example} ${HDF5_CPP_EXAMPLES_SOURCE_DIR}/${example}.cpp)
target_include_directories (cpp_ex_${example} PRIVATE "${HDF5_SRC_DIR};${HDF5_SRC_BINARY_DIR};$<$<BOOL:${HDF5_ENABLE_PARALLEL}>:${MPI_C_INCLUDE_DIRS}>")
target_include_directories (cpp_ex_${example} PRIVATE "${HDF5_SRC_INCLUDE_DIRS};${HDF5_SRC_BINARY_DIR};$<$<BOOL:${HDF5_ENABLE_PARALLEL}>:${MPI_C_INCLUDE_DIRS}>")
if (NOT BUILD_SHARED_LIBS)
TARGET_C_PROPERTIES (cpp_ex_${example} STATIC)
target_link_libraries (cpp_ex_${example} PRIVATE ${HDF5_CPP_LIB_TARGET} ${HDF5_LIB_TARGET})
@@ -59,7 +59,7 @@ endforeach ()
foreach (example ${tutr_examples})
add_executable (cpp_ex_${example} ${HDF5_CPP_EXAMPLES_SOURCE_DIR}/${example}.cpp)
target_include_directories (cpp_ex_${example} PRIVATE "${HDF5_SRC_DIR};${HDF5_SRC_BINARY_DIR};$<$<BOOL:${HDF5_ENABLE_PARALLEL}>:${MPI_C_INCLUDE_DIRS}>")
target_include_directories (cpp_ex_${example} PRIVATE "${HDF5_SRC_INCLUDE_DIRS};${HDF5_SRC_BINARY_DIR};$<$<BOOL:${HDF5_ENABLE_PARALLEL}>:${MPI_C_INCLUDE_DIRS}>")
if (NOT BUILD_SHARED_LIBS)
TARGET_C_PROPERTIES (cpp_ex_${example} STATIC)
target_link_libraries (cpp_ex_${example} PRIVATE ${HDF5_CPP_LIB_TARGET} ${HDF5_LIB_TARGET})

View File

@@ -16,17 +16,31 @@
##############################################################################
##############################################################################
# Remove any output file left over from previous test run
set (CPP_EX_CLEANFILES
Group.h5
SDS.h5
SDScompound.h5
SDSextendible.h5
Select.h5
)
add_test (
NAME CPP_ex-clear-objects
COMMAND ${CMAKE_COMMAND}
-E remove
Group.h5
SDS.h5
SDScompound.h5
SDSextendible.h5
Select.h5
-E remove ${CPP_EX_CLEANFILES}
)
set_tests_properties (CPP_ex-clear-objects PROPERTIES
FIXTURES_SETUP clear_cppex
WORKING_DIRECTORY ${PROJECT_BINARY_DIR}
)
add_test (
NAME CPP_ex-clean-objects
COMMAND ${CMAKE_COMMAND}
-E remove ${CPP_EX_CLEANFILES}
)
set_tests_properties (CPP_ex-clean-objects PROPERTIES
FIXTURES_CLEANUP clear_cppex
WORKING_DIRECTORY ${PROJECT_BINARY_DIR}
)
set_tests_properties (CPP_ex-clear-objects PROPERTIES FIXTURES_SETUP clear_cppex)
foreach (example ${examples})
if (HDF5_ENABLE_USING_MEMCHECKER)
@@ -41,7 +55,7 @@ foreach (example ${examples})
-D "TEST_OUTPUT=cpp_ex_${example}.txt"
#-D "TEST_REFERENCE=cpp_ex_${example}.out"
-D "TEST_FOLDER=${PROJECT_BINARY_DIR}"
-P "${HDF_RESOURCES_EXT_DIR}/runTest.cmake"
-P "${HDF_RESOURCES_DIR}/runTest.cmake"
)
endif ()
set_tests_properties (CPP_ex_${example} PROPERTIES FIXTURES_REQUIRED clear_cppex)
@@ -53,19 +67,32 @@ endforeach ()
#the following dependencies are handled by the order of the files
# SET_TESTS_PROPERTIES(CPP_ex_readdata PROPERTIES DEPENDS CPP_ex_create)
# SET_TESTS_PROPERTIES(CPP_ex_chunks PROPERTIES DEPENDS CPP_ex_extend_ds)
set (CPP_EX_TUTR_CLEANFILES
h5tutr_cmprss.h5
h5tutr_dset.h5
h5tutr_extend.h5
h5tutr_group.h5
h5tutr_groups.h5
h5tutr_subset.h5
)
add_test (
NAME CPP_ex_tutr-clear-objects
COMMAND ${CMAKE_COMMAND}
-E remove
h5tutr_cmprss.h5
h5tutr_dset.h5
h5tutr_extend.h5
h5tutr_group.h5
h5tutr_groups.h5
h5tutr_subset.h5
-E remove ${CPP_EX_TUTR_CLEANFILES}
)
set_tests_properties (CPP_ex_tutr-clear-objects PROPERTIES
FIXTURES_SETUP clear_cppex_tutr
WORKING_DIRECTORY ${PROJECT_BINARY_DIR}
)
add_test (
NAME CPP_ex_tutr-clean-objects
COMMAND ${CMAKE_COMMAND}
-E remove ${CPP_EX_TUTR_CLEANFILES}
)
set_tests_properties (CPP_ex_tutr-clean-objects PROPERTIES
FIXTURES_CLEANUP clear_cppex_tutr
WORKING_DIRECTORY ${PROJECT_BINARY_DIR}
)
set_tests_properties (CPP_ex_tutr-clear-objects PROPERTIES FIXTURES_SETUP clear_cppex_tutr)
foreach (example ${tutr_examples})
if (HDF5_ENABLE_USING_MEMCHECKER)
@@ -77,10 +104,10 @@ foreach (example ${tutr_examples})
-D "TEST_ARGS:STRING="
-D "TEST_EXPECT=0"
-D "TEST_SKIP_COMPARE=TRUE"
-D "TEST_OUTPUT=cpp_ex_${example}.txt"
#-D "TEST_REFERENCE=cpp_ex_${example}.out"
-D "TEST_OUTPUT=tutr_cpp_ex_${example}.txt"
#-D "TEST_REFERENCE=cpp_ex_tutr_${example}.out"
-D "TEST_FOLDER=${PROJECT_BINARY_DIR}"
-P "${HDF_RESOURCES_EXT_DIR}/runTest.cmake"
-P "${HDF_RESOURCES_DIR}/runTest.cmake"
)
endif ()
set_tests_properties (CPP_ex_${example} PROPERTIES FIXTURES_REQUIRED clear_cppex_tutr)

View File

@@ -1,6 +1,5 @@
#
# Copyright by The HDF Group.
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including

View File

@@ -1,6 +1,5 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *

View File

@@ -1,6 +1,5 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *

View File

@@ -1,6 +1,5 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *

View File

@@ -1,6 +1,5 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *

View File

@@ -1,6 +1,5 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *
@@ -70,7 +69,7 @@ main(void)
dims[1] = 20;
cdims[0] = 20;
cdims[1] = 20;
DataSpace * dataspace = new DataSpace(RANK, dims); // create new dspace
DataSpace *dataspace = new DataSpace(RANK, dims); // create new dspace
DSetCreatPropList ds_creatplist; // create dataset creation prop list
ds_creatplist.setChunk(2, cdims); // then modify it for compression
ds_creatplist.setDeflate(6);

View File

@@ -1,6 +1,5 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *

View File

@@ -1,6 +1,5 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *

View File

@@ -1,6 +1,5 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *

View File

@@ -1,6 +1,5 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *

View File

@@ -1,6 +1,5 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *

View File

@@ -1,6 +1,5 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *

View File

@@ -1,6 +1,5 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *

View File

@@ -1,6 +1,5 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *

Some files were not shown because too many files have changed in this diff Show More