philipp/hdf5
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
22,473 Commits 73 Branches 125 Tags
9d07011f4e2d8c5fd870c61e855d13a688940a18
Commit Graph

232 Commits

Author SHA1 Message Date
Muqun Yang
83de84f88d Add the Makefile.am for the performance. 2021-06-29 10:42:11 -05:00
Dana Robinson
62868dabe1 Parallel updates and fixes for VFD SWMR (#705) 
* Brings CMake updates from develop

* Brings reduction in pedantic casts from develop

* Purges UFAIL from the library (#637)

* Committing clang-format changes

* Purges UFAIL from the library

* H5HL_insert change requested in PR

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Removes gratuitous (double)x.yF casts (#632)

* Committing clang-format changes

* Removes gratuitous (double)x.yF casts

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Committing clang-format changes

* Cleans up a const warning left over from previous constification (#633)

* Committing clang-format changes

* Adds consts to a few global variables

* Cleans up a const warning left over from previous constification

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Formatted source

* Bring over some VOL registration changes

* VOL cleanup

* H5VL_request_status_t substitutions

* H5F.c cleanup

* Minor API tweaks from develop

* Moves H5G package init/teardown to H5Gint.c

* H5G cleanup

* H5M cleanup

* H5SM cleanup

* H5T cleanup

* H5R cleanup

* H5Lpublic.h cleanup

* H5L cleanup

* H5O cleanup

* H5A, H5CS, and H5AC cleanup

* Moved H5A init/teardown code to H5Aint.c

* Moves H5D I/O functions to H5D.c

* H5D cleanup

* Misc minor cleanup

* H5P close cleanup

* H5Tpublic.h cleanup

* Fixes err_compat test

* H5PLpublic.h cleanup

* Updates H5Ppublic.h

* H5Fpublic.h updates

* H5A.c cleanup

* Brings over H5Aexists and related changes

* Brings CMake shell testing changes from develop

* Close callback changes

* H5R and H5Tcommit normalization

* err_compat test works now

* H5O tweaks

* Updates VOL registration code

* Brings over H5VL_create_object

* H5Tconv.c reformatting

* H5T.c tweaks

* Brings datatype and reference updates from develop

* Brings VOL plugin loading changes from develop

* Brings event sets from develop

* Brings async functions over

* Tools changes

* Brings over many tools changes from develop

* Brings VOL flags from develop

* Fixes h5dump double/float tests

* Updates h5repack tests

* Brings h5diff test changes from develop

* Last h5dump changes

* Brings test changes from develop

* Committing clang-format changes

* Tidied h5_testing()

* Brings chunk iteration code + misc from develop

* Updates vds test

* Enables external link parallel test

* Brings updated property lists from develop

* H5G changes from develop

* H5MF cleanup

* Brings vfd_swmr test back into CMake

* Updates threadsafe test

* Updates plist test

* Brings recent changes from develop

* Require a C++11 compiler to build the C++ wrappers (#693)

* Committing clang-format changes

* Requires a C++11 compiler to build the C++ wrappers

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Fix H5FD_mpio_Debug="rw" can report MPI_count instead of bytes (#699)

* Removes pre-C99 build and header cruft (#700)

* Assumes ANSI C headers exist
* Assumes stdbool.h, stdint.h, and inttypes.h are present
* Assumes the C++ compiler can handle stdint.h
* Removes all work-arounds for missing functionality, especially stdbool.h

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Updates release documents

* Brings parallel changes from develop

* Adds a VFD/parallel check program for the VFD SWMR shell test

* Updates MANIFEST

* Disable building VFD SWMR acceptance tests on CMake

* Removes err.h from genall5.c

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
 2021-05-28 12:04:18 -07:00
Dana Robinson
2ddf5fbd92 Much normalization with develop (#701) 
* Brings CMake updates from develop

* Brings reduction in pedantic casts from develop

* Purges UFAIL from the library (#637)

* Committing clang-format changes

* Purges UFAIL from the library

* H5HL_insert change requested in PR

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Removes gratuitous (double)x.yF casts (#632)

* Committing clang-format changes

* Removes gratuitous (double)x.yF casts

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Committing clang-format changes

* Cleans up a const warning left over from previous constification (#633)

* Committing clang-format changes

* Adds consts to a few global variables

* Cleans up a const warning left over from previous constification

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Formatted source

* Bring over some VOL registration changes

* VOL cleanup

* H5VL_request_status_t substitutions

* H5F.c cleanup

* Minor API tweaks from develop

* Moves H5G package init/teardown to H5Gint.c

* H5G cleanup

* H5M cleanup

* H5SM cleanup

* H5T cleanup

* H5R cleanup

* H5Lpublic.h cleanup

* H5L cleanup

* H5O cleanup

* H5A, H5CS, and H5AC cleanup

* Moved H5A init/teardown code to H5Aint.c

* Moves H5D I/O functions to H5D.c

* H5D cleanup

* Misc minor cleanup

* H5P close cleanup

* H5Tpublic.h cleanup

* Fixes err_compat test

* H5PLpublic.h cleanup

* Updates H5Ppublic.h

* H5Fpublic.h updates

* H5A.c cleanup

* Brings over H5Aexists and related changes

* Brings CMake shell testing changes from develop

* Close callback changes

* H5R and H5Tcommit normalization

* err_compat test works now

* H5O tweaks

* Updates VOL registration code

* Brings over H5VL_create_object

* H5Tconv.c reformatting

* H5T.c tweaks

* Brings datatype and reference updates from develop

* Brings VOL plugin loading changes from develop

* Brings event sets from develop

* Brings async functions over

* Tools changes

* Brings over many tools changes from develop

* Brings VOL flags from develop

* Fixes h5dump double/float tests

* Updates h5repack tests

* Brings h5diff test changes from develop

* Last h5dump changes

* Brings test changes from develop

* Committing clang-format changes

* Tidied h5_testing()

* Brings chunk iteration code + misc from develop

* Updates vds test

* Enables external link parallel test

* Brings updated property lists from develop

* H5G changes from develop

* H5MF cleanup

* Brings vfd_swmr test back into CMake

* Updates threadsafe test

* Updates plist test

* Brings recent changes from develop

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
 2021-05-28 07:10:51 -07:00
vchoi-hdfgroup
7ebc6f267d Merge pull request #626 from vchoi-hdfgroup/dsetops_feature_vfd_swmr 
Dsetops feature vfd swmr
 2021-05-20 11:29:58 -05:00
vchoi
a4cf8aea72 Add new test vfd_swmr_dsetops_writer.c to test writing operations for: 
Dataset types:
--dataset with compact layout
--dataset with contiguous layout
--datasets with chunked layout: single/implicit/fixed array/extensible array/btree2 indexes
Types of writes:
--Sequential writes
--Random writes
--Regular hyperslab writes
--Raw data modifications
 2021-05-04 18:04:58 -05:00
Dana Robinson
16eda679e3 Removes dead H5ST code 2021-05-03 16:23:33 -07:00
Dana Robinson
c83ca64bd0 Merge remote-tracking branch 'canonical/feature/vfd_swmr' into vfd_swmr/thg_standards 2021-05-03 10:11:25 -07:00
vchoi
74b1d42bbf Add new test vfd_swmr_attrdset_writer.c. 
It is to test attribute handling for different dataset types.
 2021-04-07 15:03:16 -05:00
Dana Robinson
17c7cf9475 Moves VFD and VOL make check targets to conclude.am 2021-04-07 09:29:27 -07:00
Dana Robinson
67cfa2396c Updates license url and history docs 2021-03-24 18:29:11 -07:00
David Young
dc97b9a165 Merge branch 'aug-develop' into aug-merge-attempt/feature/vfd_swmr 2020-08-04 10:22:47 -05:00
vchoi
4bc49d8b2b Fix for jira issue HDFFV-11080: 
(1) Patch up the file pointer when reading attribute of variable length datatype
(2) Test to verify the fix when doing multiple threads
 2020-07-14 10:56:33 -05:00
Quincey Koziol
e767f44e95 Merge remote-tracking branch 'origin/develop' into monotonic_timer 2020-06-26 18:57:38 -05:00
David Young
f8d469ca38 Add a test writer that creates groups under the root and a reader that polls 
until all of the expected groups appear.
 2020-05-27 10:47:04 -05:00
David Young
abced7dc4e Rebase repair: add stubs.c back to the test library sources. 2020-05-20 12:06:00 -05:00
Quincey Koziol
a96e6246d0 Align contents of CMake and autotools testfile cleanup lists. 2020-05-20 09:37:20 -05:00
Quincey Koziol
4ad7dca124 Avoid allocating a chunk index for datasets with 0-sized dimensions, until 
the dataset is extended.
 2020-05-20 09:37:20 -05:00
Jacob Smith
b11015c4cc Add Splitter VFD to library. 
* "Simultaneous and equivalent" Read-Write and Write-Only channels for
  file I/O.
* Only supports drivers with the H5FD_FEAT_DEFAULT_VFD_COMPATIBLE flag for
  now, preventing issues with multi-file drivers.

Add Mirror VFD to library.

* Write-only operations over a network.
* Uses TCP/IP sockets.
* Server and auxiliary server-shutdown programs provided in a new directory,
  `utils/mirror_vfd`.
* Automated testing via loopback ("remote" of localhost).
 2020-05-20 09:34:20 -05:00
David Young
e3ce3bf76a Take out the temporary performance tests. 2020-05-20 09:31:54 -05:00
David Young
66334f899d Temporarily add some code that measures the time to run the simplest possible 
H5T__copy_all()-like routine 10 million times and then measures the version
with FUNC_ENTER_STATIC/_LEAVE_NOAPI and a HGOTO_ERROR() statement.
 2020-05-20 09:31:54 -05:00
David Young
06dfd6025d Replace pthread_self_ulong() with H5TS_thread_id(). The POSIX Threads 
implementation ought to be portable to any system that has POSIX
Threads.  On Windows, I use the same API call as before.
 2020-05-20 09:31:54 -05:00
David Young
77144641df Revert "Make a squash commit of 'Quiet some warnings by adjusting warnings level and fixing some code.' (commit 5c911d8baf)" 2020-05-20 09:20:17 -05:00
David Young
b1be318a62 Undo accidental test deletion. 2020-05-20 09:19:37 -05:00
David Young
6b80036fde Straggler from previous commit: build and link the new .c files where I 
moved some previously-static variables and functions that were causing
unused-variable/function warnings.
 2020-05-20 09:19:37 -05:00
David Young
2b863a8119 Change some GCC warnings to errors. Fix code to quiet some warnings. 2020-05-20 09:17:27 -05:00
David Young
5d4b109abf Create a reader for the extensible datasets tests. 2020-05-07 12:09:15 -05:00
David Young
f8e494ba80 Add my work-in-progress dataset test. It writes a handful of datasets 
that expand in one or two dimensions, depending on the setting of the -d
option argument.
 2020-05-06 15:01:29 -05:00
Quincey Koziol
deadf25465 Align contents of CMake and autotools testfile cleanup lists. 2020-05-02 20:42:33 -05:00
Quincey Koziol
ac069841f3 Avoid allocating a chunk index for datasets with 0-sized dimensions, until 
the dataset is extended.
 2020-05-01 17:59:57 -05:00
David Young
9b5384b436 Add a reader personality to vfd_swmr_zoo_writer. Remove the disused 
`-t` option.  Disable the error stack across create_/validate_zoo()
calls.
 2020-03-31 13:14:59 -05:00
David Young
cb5e1ead29 Let tests provide a "hook" that runs after each step in create_zoo(). 
Provide a hook in vfd_swmr_zoo_writer that calls H5Fvfd_swmr_end_tick().
 2020-03-23 14:55:07 -05:00
David Young
51b8fa3ffe Take a stab at a "zoo" writer for VFD SWMR: write all of the same 
objects as the cache_image test does.  The zoo writer is a work in
progress.

This version is useful as a reproducer for the hang in the global heap
that I stumbled over, yesterday.  I run this to reproduce,

env HLOG="pbio=on" ./vfd_swmr_zoo_writer -W -a
 2020-03-20 15:20:51 -05:00
Jacob Smith
b65405439d Add Splitter VFD to library. 
* "Simultaneous and equivalent" Read-Write and Write-Only channels for
  file I/O.
* Only supports drivers with the H5FD_FEAT_DEFAULT_VFD_COMPATIBLE flag for
  now, preventing issues with multi-file drivers.

Add Mirror VFD to library.

* Write-only operations over a network.
* Uses TCP/IP sockets.
* Server and auxiliary server-shutdown programs provided in a new directory,
  `utils/mirror_vfd`.
* Automated testing via loopback ("remote" of localhost).
 2020-03-13 17:13:17 -05:00
David Young
ae351c2376 Add tests for the two expected failure modes for variable-length (VL) strings 
in VFD SWMR mode.
 2020-03-13 14:16:40 -05:00
David Young
4e4c1c3850 Add my work in progress on variable-length string test for VFD SWMR. 2020-03-10 17:48:32 -05:00
David Young
986c7451a0 Replace pthread_self_ulong() with H5TS_thread_id(). The POSIX Threads 
implementation ought to be portable to any system that has POSIX
Threads.  On Windows, I use the same API call as before.
 2020-02-03 11:59:14 -06:00
David Young
c8f533cfc3 Merge all of my changes from merge-back-to-feature-vfd_swmr-attempt-1, 
including the merge of `hdffv/hdf5/develop`, back to the branch that Vailin and
I share.

Now I need to put this branch on a fork with a less confusing name than
vchoi_fork!
 2019-12-09 10:30:58 -06:00
David Young
0108072f7a Revert "Make a squash commit of 'Quiet some warnings by adjusting warnings level and fixing some code.' (commit 5c911d8baf)" 2019-11-27 11:26:02 -06:00
David Young
bd77af437f Undo accidental test deletion. 2019-11-13 13:41:05 -06:00
David Young
943e601b10 Straggler from previous commit: build and link the new .c files where I 
moved some previously-static variables and functions that were causing
unused-variable/function warnings.
 2019-11-13 11:49:15 -06:00
David Young
34eff2aae5 Change some GCC warnings to errors. Fix code to quiet some warnings. 2019-11-05 12:14:26 -06:00
Jerome Soumagne
ae490016b9 Add new H5R API that abstracts object, region and attribute reference types 
Also support references to external files

Add new H5T_REF type and type conversion routines

Support conversion from H5T_REF_OBJ/DSET_REG to H5T_REF

Add H5Treclaim() API to reclaim memory of vlen/reference types

Deprecate H5Dvlen_reclaim()

Fix H5T_vlen_reclaim() and H5T_reclaim() to use private callback

Add H5T_ref_reclaim()

Move previous H5R APIs to H5Rdeprec.c

Clean up H5Ocopy

Separate H5O_copy_expand_ref() to H5Ocopy_ref()

Add support for copying new reference types

Clean up deprecated routines to go through VOL and same code path

Fix return codes in existing trefer.c test

Rename trefer.c to trefer_deprec.c

trefer.c is for new references

Add performance test for trefer

Add additional obj_copy_ref test

Make use of tokens and blobs to store references

Skip blob encoding for object references

Start adding new reference examples
 2019-10-08 14:30:24 -05:00
Binh-Minh Ribler
e06bf9d263 HDFFV-10677 and HDFFV-10661 
Description:
    - Added functions to query chunk information:
        H5Dget_num_chunks(dset_id, fspace_id, *nchunks)
            Gets the number of written chunks that intersect with the given
            dataspace.  However, in this version, the intersection is not
            yet completed.  Thus, the number of all written chunks will be
            returned.
        H5Dget_chunk_info_by_coord(dset_id, *offset, *filter_mask, *addr, *size)
            Given a chunk's logical coordinates, returns the chunk's filter,
            address, and size.
        H5Dget_chunk_info(dset_id, fspace_id, index, *offset, *filter_mask, *addr, *size)
            Given a chunk's index, returns the chunk's logical coordinates, filter,
            address, and size.  The chunk belongs to a set of chunks that have
            nonempty intersection with the specified dataspace.  However, in
            this version, the intersection is not yet completed, and the index
            is of all the written chunks.
        These functions comply with VOL.

    - Fixed some oversights found in the library for the tests in chunk_info.c
      to work correctly.  The returned value from a callback function was not
      checked in H5EA_iterate(), H5FA_iterate(), and H5D__none_idx_iterate().
      This oversight caused a callback function to continue iterating even though
      it's supposed to stop.
Platforms tested:
    Linux/64 (jelly)
    Linux/64 (platypus)
    Darwin (osx1011test)
 2019-08-28 10:08:12 -05:00
Allen Byrne
98c8c6e45b Mostly whitespace in testpar, addl changes for tmp dir 2019-08-15 10:28:40 -05:00
Larry Knox
ab6679315c Update COPYING file. 
Remove hdfs from list of vfds for check-vfd testing as it is read-only.
 2019-07-30 16:37:57 -05:00
Larry Knox
8008294578 Squashed commit of the following: 
Merge changes from update_merged_S3_HDFS branch into develop.

commit d5034315aea88629929ac0c9c59ebfafd5f21a31
Merge: 9c48823 d3fdcd8
Author: Larry Knox <lrknox@hdfgroup.org>
Date:   Thu Jul 25 08:24:53 2019 -0500

    Merge branch 'develop' into update_merged_S3_HDFS
 2019-07-25 11:47:12 -05:00
Songyu Lu
1d1e96f1dd Some coding style changes. 2019-04-11 15:51:21 -05:00
Songyu Lu
477dda3c0d HDFFV-10658 - setting and getting properties in API context: 
1. switched to use the existing H5F_prefix_open_t for enum type;
2. put the common private function used by external.c and external_env.c into external_common.c
 2019-04-09 16:20:19 -05:00
Songyu Lu
50d9a397ab This commit basically has the following changes: 
1. restored the datatype, dataspace, and LCPL of the dataset for VOL connector back to the properties.
2. splitted external.c and vds.c because they called HDsetenv in the program, instead using shell scripts to set the environment variables.
3. changed H5CX_get_vds_prefix and H5CX_get_ext_file_prefix to use H5P_peek instead of H5P_get.
 2019-04-04 11:03:05 -05:00
Vailin Choi
d3dff6efe6 Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/hdffv/hdf5 into develop 
Resolve conflicts:
Conflicts:
	test/Makefile.am
 2019-01-02 13:23:54 -06:00