Commit Graph

6681 Commits

Author SHA1 Message Date
Quincey Koziol
df7cf2a9d3 Refactor, standardize, and revise the async request callbacks. 2018-11-04 00:05:08 -05:00
Songyu Lu
6fa26d7e49 HDFFV-10601 Issues with chunk cache hash value calcuation:
1. H5D__chunk_hash_val: When the number of chunks in the fastest changing dimension is larger than the number of slots in the hash table, H5D__chunk_hash_val abandons the normal hash value calculation algorithm and simply uses the scaled dimension.  This will cause chunks a selection that cuts across chunks in dimensions other than the fastest changing to all have the same hash value, and they will therefore always evict each other from the cache, having an obvious major performance impact. Eliminated the check  for the number of slots in this function and always use the full algorithm.

2. H5D__chunk_init: When the scaled dimensions (number of chunks in each dimension) are calculated in H5D__chunk_init, a simple divide ("/") operator is used with the dataset size in elements and the chunk size in elements. While this is fine when the dataset size is an exact multiple of the chunk size, in other cases, since "/" rounds down, it results in a scaled dimension one less than it should (it ignores the partial edge chunk). This has trickle down effects on hash value calculation that can cause excess hash value collisions and therefore performance issues. Changed the calculation to (((dataset_size - 1) / chunk_size) + 1).

Tested the build with Autotool and CMake.
2018-11-02 14:53:35 -05:00
Quincey Koziol
4806d29d9f Refactor and standardize named datatype VOL callbacks. 2018-11-01 17:02:27 -05:00
Quincey Koziol
1b00d04080 Refactor and standardize object VOL callbacks. 2018-11-01 14:54:44 -05:00
Quincey Koziol
a5cc4e272b Refactor and standardize the link VOL callbacks. Also fix error stack for
testing.
2018-11-01 11:22:54 -05:00
Quincey Koziol
a39486a341 Refactor and standardize group VOL callbacks. 2018-10-31 19:17:37 -05:00
Quincey Koziol
d66d5e01c0 Refactor and standarize file object callbacks, including some fixes on the
dataset callbacks.
2018-10-31 18:18:45 -05:00
Ray Lu
3fcc1e7929 Merge pull request #1315 in HDFFV/hdf5 from ~SONGYULU/hdf5_ray:bugfix/ray_test to develop
* commit 'b72e926ed0c981d14d5219b32042c3d0b938d1e1':
  A trivial change to reformat a comment.  The purpose is simply to try out Git.
2018-10-31 10:43:22 -05:00
Songyu Lu
b72e926ed0 A trivial change to reformat a comment. The purpose is simply to try out Git. 2018-10-31 09:54:08 -05:00
Quincey Koziol
4c6d8d376c Remove erroneous call. 2018-10-29 17:26:33 -05:00
Quincey Koziol
8c37f531a9 Switch to using VOL callbacks, instead of direct calls to internal routines. 2018-10-29 17:24:33 -05:00
Quincey Koziol
aaaf1acc08 Add ref counter to VOL object wrapping in API contexts, so that re-entrant /
recursive routines work correctly.  Another minor cleanup in the attribute code.
2018-10-29 16:52:00 -05:00
Larry Knox
b24259ed8c Add files and changes to cross compile with CrayLinuxEnvironment and run
ctest tests with a batch job on knl compute nodes.
2018-10-29 14:24:08 -05:00
Quincey Koziol
ca5ff0c0c2 Convert dataset callbacks to standard form, for wrapping object IDs, etc.
Also, fix a few attribute issues that I missed.
2018-10-29 13:15:56 -05:00
Quincey Koziol
8fd2e1d36e Convert attribute VOL callbacks to a standard form that sets up the wrapper
object info in a single place.
2018-10-28 20:51:16 -05:00
Quincey Koziol
ad92351809 Clean up va_list properly on error. 2018-10-28 02:03:08 -05:00
Quincey Koziol
58202337dd Minor coding style cleanups 2018-10-28 02:02:45 -05:00
Quincey Koziol
8ee361f185 Move all callback-related routines into the callback source file, standardizing
their coding style
2018-10-28 01:31:28 -05:00
Quincey Koziol
9bc6644ed1 Duplicate name of VOL plugin class, to avoid continuing to refer to the caller's
string when making a copy of the class for internal use.
2018-10-27 16:15:39 -05:00
Quincey Koziol
9c7b96b42f Move (final?) file operation from directly calling into library code to using
the file_optional VOL callback.
2018-10-27 02:21:48 -05:00
Quincey Koziol
43ffc1deb5 Remove some VOL-specific internal coding. 2018-10-27 02:06:43 -05:00
Quincey Koziol
83e03772eb Add VOL object wrapping to API context when refreshing groups, named datatypes
and objects.
2018-10-27 01:55:31 -05:00
Quincey Koziol
6e84fd8327 Remove virtual dataset refresh code's direct use of native VOL plugin's ID. 2018-10-27 01:33:07 -05:00
Dana Robinson
62b92ddf40 Makes the installed header files the same between the autotools
and CMake.

* CMake will no longer install private generated headers and tools
  library headers.
* Several empty public header files (which represent internal packages)
  were removed. These were only installed by CMake.
* Autotools installs will install H5FDwindows.h.

Fixes HDFFV-10614.
2018-10-26 20:44:29 -07:00
Quincey Koziol
ce3d1ec9ea Eliminate last call to H5VL_native_register, shifting everything over to
the VOL plugin wrapping code.
2018-10-26 17:27:10 -05:00
Quincey Koziol
171deaa588 Checkpoint progress on stacking VOL plugins - all standalone regression tests
in the 'test' directory appear to be working.
2018-10-26 16:09:06 -05:00
Quincey Koziol
305997501d Updates after merging changes from develop. 2018-10-26 00:54:29 -05:00
Quincey Koziol
4a32895724 Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/hdffv/hdf5 into stackable_vol 2018-10-25 18:32:13 -05:00
Quincey Koziol
0df6e44a6e Squashed commit of private branch changes to support stackable VOL plugins.
modified:   test/dsets.c
2018-10-24 23:52:47 -05:00
Dana Robinson
46c5b059f1 Split H5VLnative.h into public and private files and updated 2018-10-20 13:06:37 -07:00
Allen Byrne
5f99891e6d Merge pull request #1299 in HDFFV/hdf5 from ~BYRN/hdf5_merge_adb:develop to develop
* commit '494d39f1c2f6225c732cb857c860d6b475d03226':
  Move options to root cmake file
  EED-258 Add modules uses toolchain files
2018-10-19 13:49:33 -05:00
Allen Byrne
494d39f1c2 Move options to root cmake file 2018-10-19 13:34:31 -05:00
Dana Robinson
4e575a0865 Merge pull request #1297 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:develop to develop
* commit '2d6d5cad0cc426cd53b9d4fda15e53d8021e4c7b':
  Updates to the VOL ID and object API calls.
2018-10-18 16:05:40 -05:00
Dana Robinson
2d6d5cad0c Updates to the VOL ID and object API calls.
This brings the H5VL code in line with the H5I code regarding
naming, parameter order, etc.

Several public API calls were affected by this change.

These changed names to reflect their use with VOL drivers:
    H5VLregister()          -->     H5VLregister_driver()
    H5VLregister_by_name()  -->     H5VLregister_driver_by_name()
    H5VLunregister()        -->     H5VLunregister_driver()
    H5VLis_registered()     -->     H5VLis_driver_registered()

This call was renamed to match H5Iregister(). The order of the first
two parameters also reversed and the object pointer is now const.
    H5VLobject_register()   -->     H5VLregister()
2018-10-17 02:06:28 -07:00
Allen Byrne
2e28fad2d0 TRILAB-34 remove extra arg 2018-10-16 16:46:45 -05:00
Allen Byrne
c39449fa12 Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit '1e8ef703cdc057211b3587be776a6e9f62e8f76f':
  Fixed a C++ style commenting issue and removed an unused field from H5F_trav_obj_ids_t.
  Fixed a memory issue in H5Drefresh() where the dataset's H5VL_object_t gets closed but we try to use it to find the driver.
2018-10-16 13:10:48 -05:00
Allen Byrne
6b65a1b78f TRILAB-34 Add batch script config command 2018-10-16 12:06:07 -05:00
Allen Byrne
77f284a175 TRILAB-36 Revert to target_file and correct property 2018-10-16 11:33:44 -05:00
Allen Byrne
db3fe90abd Add generated propoerty to H5lib_settings 2018-10-15 16:57:50 -05:00
Allen Byrne
689f184345 TRILAB-34 batch command options 2018-10-12 13:55:29 -05:00
Dana Robinson
b0b9eda809 Fixed a C++ style commenting issue and removed an unused field
from H5F_trav_obj_ids_t.
2018-10-12 10:47:47 -07:00
Dana Robinson
c3a813a37f Fixed a memory issue in H5Drefresh() where the dataset's H5VL_object_t
gets closed but we try to use it to find the driver.
2018-10-11 15:08:02 -07:00
Allen Byrne
46e7e5f8f2 Add more descriptive option name 2018-10-11 12:35:53 -05:00
Allen Byrne
4659e43a6b HDFFV-10603 fixup cmake usage 2018-10-11 12:24:15 -05:00
Allen Byrne
d69b541c20 TRILABS-34 add batch option. Also fix cmakedefine01 usage 2018-10-11 11:31:09 -05:00
Dana Robinson
e962df1591 VOL FEATURE 2018-10-10 08:10:15 -07:00
Dana Robinson
75b84b09bd Updated the H5I debug dump to not segfault when types are not initialized
or contain zero IDs in the skip list.
2018-10-05 22:16:51 -07:00
Dana Robinson
1b2c820b4b The H5I debug ID dump function is now always available instead
of hidden behind an H5_DEBUG_OUTPUT ifdef.
2018-10-05 21:47:51 -07:00
Allen Byrne
4442b5b743 TRILABS-34 improve cross compile with emulator 2018-10-04 16:06:31 -05:00
Jacob Smith
f5114fcddb Remove unnecessary H5_now() call; remove ohdr v2 mtime addition. 2018-10-02 10:50:37 -05:00