Commit Graph

6681 Commits

Author SHA1 Message Date
Allen Byrne
b95f05638c TRILAB-111 fixes for parallel testing 2019-02-13 15:53:59 -06:00
Quincey Koziol
b5305b25a6 Align develop with incoming hyperslab_updates branch changes. 2019-02-12 21:51:15 -06:00
Jordan Henderson
34508f0620 Merge in latest from develop 2019-02-11 19:13:39 -06:00
Jordan Henderson
65a820ae89 Fix some collective metadata read issues 2019-02-11 18:31:23 -06:00
Jordan Henderson
cc6a61215e Re-write of Java JNI error handling 2019-02-11 10:58:58 -06:00
Quincey Koziol
163dcebec5 Add ID type (of eventual hid_t) to the VOL "wrap" callback. 2019-02-07 14:31:26 -06:00
Vailin Choi
7d124cb796 Modification based on feedback from pull request. 2019-02-05 18:13:11 -06:00
Vailin Choi
e272e64e74 Modification based on feedback from pull request. 2019-02-05 17:04:04 -06:00
Vailin Choi
7f718e9ea2 There is performance issue when closing an object. The slow down is due to the search of
the "tag_list" to find out the "corked" status of an object.
The fix:
(1)  Add a counter "num_objs_corked" in the cache structure to track the number
of "corked" objects.
(2) Skip the search of "tag_list" if the counter is zero i.e. no "corked" objects.
2019-02-04 22:30:48 -06:00
Dana Robinson
2d4f58adf0 Merge pull request #1515 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:pread_vfd_squash to develop
* commit '055208b71c573be3d4b1047b840d4d8b255bc25e':
  Minor tweak to CMake preadwrite entry.
  Updated sec2, log, and core VFDs to use pread/pwrite when available (can be controlled via a configure/CMake option)
2019-02-01 16:01:22 -06:00
Dana Robinson
f90e5bdc62 Merge pull request #1516 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:valgrind_fixes to develop
* commit '5f1f231310bcb3278f11a51e1ea47782f6ee3235':
  Updated H5Dio.c comment.
  Fixed a valgrind issue with the unprotect log message. The logging call was placed after the point where the cache entry was freed. The fix was to just use the entry's address directly in the log call instead of the entry itself.
  Fixed a bug revealed by the dtransform test. A datatype transform buffer needed to be calloc'd.
2019-02-01 15:14:15 -06:00
Dana Robinson
5f1f231310 Updated H5Dio.c comment. 2019-02-01 13:12:39 -08:00
Larry Knox
bae744199b Add more specific batch scripts.
Add script raybsub to handle submitting .lsf files on ray with bsub <
script.lsf syntax that we couldn't handle with CMake.
Add hpc-cmake-tgz option for bin/release.
2019-01-31 14:22:52 -06:00
Dana Robinson
99d70c494d Fixed a valgrind issue with the unprotect log message. The logging
call was placed after the point where the cache entry was freed.
The fix was to just use the entry's address directly in the log
call instead of the entry itself.
2019-01-31 00:14:11 -08:00
Dana Robinson
90bf59edc5 Fixed a bug revealed by the dtransform test. A datatype transform
buffer needed to be calloc'd.
2019-01-30 23:45:32 -08:00
Dana Robinson
f833001e39 Updated sec2, log, and core VFDs to use pread/pwrite when
available (can be controlled via a configure/CMake option)
2019-01-30 17:32:40 -08:00
Binh-Minh Ribler
25cd1ab02b Added test for HDFFV-10588
Platforms tested:
    Linux/64 (jelly)
    Linux/64 (platypus)
    Darwin (osx1010test)
2019-01-24 09:42:19 -06:00
Binh-Minh Ribler
d584b233a5 Fixed HDFFV-10684
Description:
    The test for HDFFV-10588 has revealed a bug in H5Ewalk.
    H5Ewalk did not stop midway even when the call back function returns
    H5_ITER_STOP. This is because a condition is missing from the for
    loops in H5E__walk causing the callback functions unable to stop until
    all the errors in the stack are iterated. Quincey advised on the final
    fix.  In this fix, "status" is switched to "ret_value" and HGOTO_ERROR
    to HERROR, and the for loops won't continue when "ret_value" is not 0.
Platforms tested:
    Linux/64 (jelly)
    Linux/64 (platypus)
    Darwin (osx1010test)
2019-01-20 17:46:28 -06:00
Binh-Minh Ribler
43bd93c4f5 Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/~bmribler/hdf5_bmr_fixbug into develop 2019-01-15 11:49:07 -06:00
Binh-Minh Ribler
bc3d878add Fixed HDFFV-10586 and HDFFV-10588
Description:
    HDFFV-10586 CVE-2018-17434  Divide by zero inh5repack_filters
        Added a check for zero value
    HDFFV-10588 CVE-2018-17437  Memory leak in H5O_dtype_decode_helper
        This is actually an Invalid read issue.  It was found that the
        attribute name length in an attribute message was corrupted,
        which caused the buffer pointer to be advanced too far and later
        caused an invalid read.
        Added a check to detect attribute name and its length mismatch.  The
        fix is not perfect, but it'll reduce the chance of this issue when a
        name length is corrupted or the attribute name is corrupted.
Platforms tested:
    Linux/64 (jelly)
    Linux/64 (platypus)
    Darwin (osx1010test)
2019-01-15 11:48:31 -06:00
Quincey Koziol
4729d04d91 Delta reduction against hyperslab_updates branch. 2019-01-10 23:08:51 -06:00
Dana Robinson
536a32c59b C and POSIX call cleanup 2019-01-10 17:51:42 -08:00
Quincey Koziol
6d897f3c10 Add API routines to wrap and retrieve objects 2019-01-10 18:00:13 -06:00
Jordan Henderson
294d9c2847 Merge pull request #1455 in HDFFV/hdf5 from ~JHENDERSON/hdf5:develop to develop
* commit '2d7eb9a4c3a841f7612b88f6b559792cb6143930':
  Align behavior of H5Aget_name_by_idx with H5Aget_name when a NULL buffer is supplied
2019-01-08 20:32:46 -06:00
Binh-Minh Ribler
56b32278c4 Merge pull request #1441 in HDFFV/hdf5 from ~BMRIBLER/hdf5_bmr_fixbug:develop to develop
* commit '90d13bef33f9e2e80b23996a0c39f16f7c34ecf8':
  Fixed typo Platforms tested:     Darwin (osx1010test)
  Refixed HDFFV-10578 Description:     Applied Neil's fix for this issue after removing previous     attempt.  The resources are now released in init_objs() when     failure occurs there.     Neil will fix HDFFV-10676 separately. Platforms tested:     Linux/64 (jelly)     Linux/64 (platypus)     Darwin (osx1010test)
  Removed previous change in H5O__chunk_deserialize().
  Removed the previous change in H5O__chunk_deserialize()
  Removed previous change in table_list_add().
  Removed the previous change in table_list_add()
  Updated per review Description:     HDFFV-10676 - CVE-2018-13873         Changed the new assert to if statement, per Dana's comment. Platforms tested:     Linux/64 (jelly)
  HDFFV-10578 and HDFFV-10676 Description:     HDFFV-10578 - CVE-2018-17234         The file has some issue, however, there was a bug in h5dump that caused         memory leaks after the problem in the file was encountered. The bug         was that an if statement was missing in the function table_list_add()         resulting in the memory not being freed at a later time.         After the fix had been applied, there were no more leaks after h5dump         detected the issue in the file and reported the error.
2019-01-08 18:59:22 -06:00
Jordan Henderson
2d7eb9a4c3 Align behavior of H5Aget_name_by_idx with H5Aget_name when a NULL buffer is supplied 2019-01-08 15:52:30 -06:00
Jordan Henderson
206da2919d Merge pull request #1452 in HDFFV/hdf5 from ~JHENDERSON/hdf5:develop to develop
* commit '805b64685371982e9c7fd693497865197990cbd6':
  Fix for parallel filters tests with new read-proc0-broadcast feature
2019-01-08 11:32:42 -06:00
Neil Fortner
0dd2c29f28 Merge branch 'develop' of ssh://bitbucket.hdfgroup.org:7999/~nfortne2/hdf5_naf into develop 2019-01-08 10:37:28 -06:00
Neil Fortner
214abbfd59 Remove H5O_BOGUS_INVALID_ID from H5O_msg_class_g initialization, since
space for it was removed.
2019-01-08 10:36:34 -06:00
Dana Robinson
c8c0cf9fa2 Merge pull request #1451 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:fortran_squash to develop
* commit '905766fa3e51a470c02328a63d92182d4a8481bf':
  Fortran wrappers for dataset obj header minimization API calls.
2019-01-08 10:14:26 -06:00
Jordan Henderson
805b646853 Fix for parallel filters tests with new read-proc0-broadcast feature 2019-01-08 09:45:35 -06:00
Dana Robinson
905766fa3e Fortran wrappers for dataset obj header minimization API calls. 2019-01-08 07:34:58 -08:00
Larry Knox
74a41f92a4 Merge pull request #1443 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:develop to develop
* commit 'e27df5a5fe365480b0809ce523b0e9cd1096a43d':
  Add variables to set up module switch between configure and build for cross compiling on CrayXC40.
  Remove install-exec-hook from hl/fortran/Makefile.am. Add check for existing links in hl/fortran/src/Makefile.am.
  Configure build directory in batch scripts.
  Add libhdf5_hl_fortran symlinks for libhdf5hl_fortran files. Set ctest*.lsf.in.cmake scripts for sierra. Correct src/CMakelists.txt syntax error.
  Correct syntax error in CMakeLists.txt.
  Add .lsf batch scripts and insert the build directory into the scripts during configuration.
2019-01-07 21:01:06 -06:00
Binh-Minh Ribler
78d0564c2a Refixed HDFFV-10578
Description:
    Applied Neil's fix for this issue after removing previous
    attempt.  The resources are now released in init_objs() when
    failure occurs there.
    Neil will fix HDFFV-10676 separately.
Platforms tested:
    Linux/64 (jelly)
    Linux/64 (platypus)
    Darwin (osx1010test)
2019-01-07 20:36:34 -06:00
Neil Fortner
9c12b625fd Merge pull request #1448 in HDFFV/hdf5 from ~NFORTNE2/hdf5_naf:develop to develop
Avoid potential invalid read when decoding unknown object header message.

* commit '685288beabc4f4383d69ec07be05c067fab78544':
  Delay checking if decoded message's "shareable" flag is appropriate for the message type until we've verified we understand the message type. Reduce size of H5O_msg_class_g to *not* include space for H5O_BOGUS_INVALID.  Make bogus messages shareable.  Add new bogus message test with shareable messages to cover the formerly problematic code.  Re-run gen_bogus.c to add this test case and also to fix the bogus_invalid messages that were no longer H5O_BOGUS_INVLAID due to a new message class being added in a previous commit.  Added comment to remind developers to run gen_bogus.c when adding a new message class.
2019-01-07 19:30:26 -06:00
Quincey Koziol
fed17ed383 HDFFV-10625 -- Implemented a process-0 read and then broadcast for collective read of full (HS_ALL), contiguous, atomic datasets by all the processes in the file communicator. 2019-01-07 16:55:59 -06:00
Neil Fortner
b043126e5e Delay checking if decoded message's "shareable" flag is appropriate for
the message type until we've verified we understand the message type.
Reduce size of H5O_msg_class_g to *not* include space for
H5O_BOGUS_INVALID.  Make bogus messages shareable.  Add new bogus
message test with shareable messages to cover the formerly problematic
code.  Re-run gen_bogus.c to add this test case and also to fix the
bogus_invalid messages that were no longer H5O_BOGUS_INVLAID due to a
new message class being added in a previous commit.  Added comment to
remind developers to run gen_bogus.c when adding a new message class.
2019-01-07 15:40:44 -06:00
Binh-Minh Ribler
6c7462b1a9 Removed previous change in H5O__chunk_deserialize(). 2019-01-07 14:09:30 -06:00
Binh-Minh Ribler
c092f9167c Removed the previous change in H5O__chunk_deserialize() 2019-01-07 14:07:25 -06:00
Binh-Minh Ribler
820d8e34c5 Updated per review
Description:
    HDFFV-10676 - CVE-2018-13873
        Changed the new assert to if statement, per Dana's comment.
Platforms tested:
    Linux/64 (jelly)
2019-01-06 21:42:16 -06:00
Larry Knox
619b0abfd6 Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit '7bf6d847ae36940f1b1147820356795c7764dbeb':
  Remove incorrect comment.
  Updated error stacks for h5dump testing.
  Corrected comment in src/H5VLint.c, fixed pass-through info size in src/H5VLpassthru.c, switched to stashing VOL connector ID & info in API context (in src/H5CX.c, src/H5CXprivate.h, src/H5F.c, src/H5Fint.c, and src/H5Fefc.c), patched up all sorts of issues in the tests, to make them work with 'check-vfd' (and 'check-vol' again).
  Clean up style issues, make function callbacks static, correct debug statements, etc.  No functionality changes at all.
2019-01-06 17:31:51 -06:00
Quincey Koziol
7bf6d847ae Merge pull request #1440 in HDFFV/hdf5 from stackable_vol_fix_14 to develop
* commit 'f0656a57dc3baf02709e1ad5e496cf63cf820877':
  Remove incorrect comment.
  Updated error stacks for h5dump testing.
  Corrected comment in src/H5VLint.c, fixed pass-through info size in src/H5VLpassthru.c, switched to stashing VOL connector ID & info in API context (in src/H5CX.c, src/H5CXprivate.h, src/H5F.c, src/H5Fint.c, and src/H5Fefc.c), patched up all sorts of issues in the tests, to make them work with 'check-vfd' (and 'check-vol' again).
2019-01-06 16:28:06 -06:00
Binh-Minh Ribler
aa62951a5f Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/~bmribler/hdf5_bmr_fixbug into develop 2019-01-06 01:45:15 -06:00
Binh-Minh Ribler
e1b59919bb HDFFV-10578 and HDFFV-10676
Description:
    HDFFV-10578 - CVE-2018-17234
        The file has some issue, however, there was a bug in h5dump that caused
        memory leaks after the problem in the file was encountered. The bug
        was that an if statement was missing in the function table_list_add()
        resulting in the memory not being freed at a later time.
        After the fix had been applied, there were no more leaks after h5dump
        detected the issue in the file and reported the error.

        In H5O__chunk_deserialize, replaced an assert with an if statement
        and reporting error, per Neil's recommendation

    HDFFV-10676 - CVE-2018-13873
        Also in H5O__chunk_deserialize, added an assertion to detect
        out of bound ids
2019-01-06 01:44:40 -06:00
Quincey Koziol
92300f954f Corrected comment in src/H5VLint.c, fixed pass-through info size in
src/H5VLpassthru.c, switched to stashing VOL connector ID & info in
API context (in src/H5CX.c, src/H5CXprivate.h, src/H5F.c, src/H5Fint.c, and
src/H5Fefc.c), patched up all sorts of issues in the tests, to make them work
with 'check-vfd' (and 'check-vol' again).
2019-01-05 23:06:45 -06:00
Quincey Koziol
8a5d2c130a Clean up style issues, make function callbacks static, correct debug statements,
etc.  No functionality changes at all.
2019-01-05 17:13:40 -06:00
Larry Knox
3be0fbd817 Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/~lrknox/hdf5_lrk into develop 2019-01-05 07:50:15 -06:00
Larry Knox
25fe692c5c Configure build directory in batch scripts. 2019-01-05 07:49:19 -06:00
Larry Knox
cceb9f06b8 Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit '0e34f0feaaeb6d2286f87f695f25fae45c070a42': (26 commits)
  HDFFV-10664 update reference file
  HDFFV-10664 update reference files
  HDFFV-10664 add missing function and check for restriction
  Remove "bad" statements at end of TEST_ERROR.     Formerly arguments to FAIL_PUTS_ERROR
  Add RELEASE.txt entry for HDFFV-10596.
  Add semicolon at end of TEST_ERROR() invocation for consistency.
  Uninstall should remove the libhdf5_hl_fortran links.
  Add symlinks named libhdf5_hl_fortran* to libhdf5hl_fortran in Makefile.am to match cmake install and the name pattern for other hl lib files.
  Change FAIL_PUTS_ERROR to TEST_ERROR.
  Cleaned up the parallel flush test and set t_pflush1 to always fail. Also set CMake to handle this.
  Update calls: H5Dcreate to H5Dcreate2, H5Acreate to H5Acreate2.
  Removed a comment.
  Fix comment
  Align H5Lcreate_ud behavior with documentation for NULL udata pointer
  Update toolchain comment
  Use c99 standard cmake variable
  Add PGI toolchain
  Correction based on code review.
  HDFFV-10664 add check for state before set call
  HDFFV-10546 refactor variable name
  ...
2019-01-05 07:41:12 -06:00
Jordan Henderson
b0feaecdb9 Merge pull request #1425 in HDFFV/hdf5 from ~JHENDERSON/hdf5:develop to develop
* commit '659dd9bccf69f32bfdd01dc49410116ec5c1b0bb':
  Align H5Lcreate_ud behavior with documentation for NULL udata pointer
2019-01-04 14:06:10 -06:00