533 lines
22 KiB
Plaintext
533 lines
22 KiB
Plaintext
HDF5 version 1.12.1-4 currently under development
|
|
================================================================================
|
|
|
|
|
|
INTRODUCTION
|
|
|
|
This document describes the new features introduced in the HDF5 1.12.0 release.
|
|
It contains information on the platforms tested and known problems in this
|
|
release. For more details check the HISTORY*.txt files in the HDF5 source.
|
|
|
|
Note that documentation in the links below will be updated at the time of the
|
|
release.
|
|
|
|
Links to HDF5 documentation can be found on The HDF5 web page:
|
|
|
|
https://portal.hdfgroup.org/display/HDF5/HDF5
|
|
|
|
The official HDF5 releases can be obtained from:
|
|
|
|
https://www.hdfgroup.org/downloads/hdf5/
|
|
|
|
More information about the new features can be found at:
|
|
|
|
https://portal.hdfgroup.org/display/HDF5/New+Features+in+HDF5+Release+1.12
|
|
|
|
If you have any questions or comments, please send them to the HDF Help Desk:
|
|
|
|
help@hdfgroup.org
|
|
|
|
|
|
CONTENTS
|
|
|
|
- New Features
|
|
- Support for new platforms and languages
|
|
- Bug Fixes since HDF5-1.12.0-alpha1
|
|
- Supported Platforms
|
|
- Tested Configuration Features Summary
|
|
- More Tested Platforms
|
|
- Known Problems
|
|
- CMake vs. Autotools installations
|
|
|
|
|
|
New Features
|
|
============
|
|
|
|
Configuration:
|
|
-------------
|
|
- Autotools and CMake target added to produce doxygen generated documentation
|
|
|
|
The default is OFF or disabled.
|
|
Autoconf option is '--enable-doxygen'
|
|
autotools make target is 'doxygen' and will build all doxygen targets
|
|
CMake configure option is 'HDF5_BUILD_DOC'.
|
|
CMake target is 'doxygen' for all available doxygen targets
|
|
CMake target is 'hdf5lib_doc' for the src subdirectory
|
|
|
|
(ADB - 2020/11/13)
|
|
|
|
- CMake option to use MSVC naming conventions with MinGW
|
|
|
|
HDF5_MSVC_NAMING_CONVENTION option enable to use MSVC naming conventions
|
|
when using a MinGW toolchain
|
|
|
|
(xan - 2020/10/30)
|
|
|
|
- CMake option to statically link gcc libs with MinGW
|
|
|
|
HDF5_MINGW_STATIC_GCC_LIBS allows to statically link libg/libstdc++
|
|
with the MinGW toolchain
|
|
|
|
(xan - 2020/10/30)
|
|
|
|
- CMake option to build the HDF filter plugins project as an external project
|
|
|
|
The HDF filter plugins project is a collection of registered compression
|
|
filters that can be dynamically loaded when needed to access data stored
|
|
in a hdf5 file. This CMake-only option allows the plugins to be built and
|
|
distributed with the hdf5 library and tools. Like the options for szip and
|
|
zlib, either a tgz file or a git repository can be specified for the source.
|
|
|
|
The necessary options are (see the INSTALL_CMake.txt file):
|
|
HDF5_ENABLE_PLUGIN_SUPPORT
|
|
PLUGIN_TGZ_NAME or PLUGIN_GIT_URL
|
|
There are more options necessary for various filters and the plugin project
|
|
documents should be referenced.
|
|
|
|
(ADB - 2020/10/16, OESS-98)
|
|
|
|
- Added CMake option to format source files
|
|
|
|
HDF5_ENABLE_FORMATTERS option will enable creation of targets using the
|
|
pattern - HDF5_*_SRC_FORMAT - where * corresponds to the source folder
|
|
or tool folder. All sources can be formatted by executing the format target;
|
|
make format
|
|
|
|
(ADB - 2020/09/24)
|
|
|
|
- CMake option to link the generated Fortran MOD files into the include
|
|
directory.
|
|
|
|
The Fortran generation of MOD files by a Fortran compile can produce
|
|
different binary files between SHARED and STATIC compiles with different
|
|
compilers and/or different platforms. Note that it has been found that
|
|
different versions of Fortran compilers will produce incompatible MOD
|
|
files. Currently, CMake will locate these MOD files in subfolders of
|
|
the include directory and add that path to the Fortran library target
|
|
in the CMake config file, which can be used by the CMake find library
|
|
process. For other build systems using the binary from a CMake install,
|
|
a new CMake configuration can be used to copy the pre-chosen version
|
|
of the Fortran MOD files into the install include directory.
|
|
|
|
The default will depend on the configuration of
|
|
BUILD_STATIC_LIBS and BUILD_SHARED_LIBS:
|
|
YES YES Default to SHARED
|
|
YES NO Default to STATIC
|
|
NO YES Default to SHARED
|
|
NO NO Default to SHARED
|
|
The defaults can be overriden by setting the config option
|
|
HDF5_INSTALL_MOD_FORTRAN to one of NO, SHARED, or STATIC
|
|
|
|
(ADB - 2020/07/9, HDFFV-11116)
|
|
|
|
- CMake option to use AEC (open source SZip) library instead of SZip
|
|
|
|
The open source AEC library is a replacement library for SZip. In
|
|
order to use it for hdf5 the libaec CMake source was changed to add
|
|
"-fPIC" and exclude test files. Autotools does not build the
|
|
compression libraries within hdf5 builds. New option USE_LIBAEC is
|
|
required to compensate for the different files produced by AEC build.
|
|
|
|
(ADB - 2020/04/22, OESS-65)
|
|
|
|
- CMake ConfigureChecks.cmake file now uses CHECK_STRUCT_HAS_MEMBER
|
|
|
|
Some handcrafted tests in HDFTests.c has been removed and the CMake
|
|
CHECK_STRUCT_HAS_MEMBER module has been used.
|
|
|
|
(ADB - 2020/03/24, TRILAB-24)
|
|
|
|
- Both build systems use same set of warnings flags
|
|
|
|
GNU C, C++ and gfortran warnings flags were moved to files in a config
|
|
sub-folder named gnu-warnings. Flags that only are available for a specific
|
|
version of the compiler are in files named with that version.
|
|
Clang C warnings flags were moved to files in a config sub-folder
|
|
named clang-warnings.
|
|
Intel C, Fortran warnings flags were moved to files in a config sub-folder
|
|
named intel-warnings.
|
|
|
|
There are flags in named "error-xxx" files with warnings that may
|
|
be promoted to errors. Some source files may still need fixes.
|
|
|
|
There are also pairs of files named "developer-xxx" and "no-developer-xxx"
|
|
that are chosen by the CMake option:HDF5_ENABLE_DEV_WARNINGS or the
|
|
configure option:--enable-developer-warnings.
|
|
|
|
In addition, CMake no longer applies these warnings for examples.
|
|
|
|
(ADB - 2020/03/24, TRILAB-192)
|
|
|
|
|
|
Library:
|
|
--------
|
|
- Added two new API routines for tracking library memory use:
|
|
H5get_alloc_stats() and H5get_free_list_sizes().
|
|
|
|
(QAK - 2020/03/25)
|
|
|
|
Java Library:
|
|
----------------
|
|
- Added new H5S functions.
|
|
|
|
H5Sselect_copy, H5Sselect_shape_same, H5Sselect_adjust,
|
|
H5Sselect_intersect_block, H5Sselect_project_intersection,
|
|
H5Scombine_hyperslab, H5Smodify_select, H5Scombine_select
|
|
wrapper functions added.
|
|
|
|
(ADB - 2020/10/27, HDFFV-10868)
|
|
|
|
|
|
Tools:
|
|
------
|
|
- h5repack added options to control how external links are handled.
|
|
|
|
Currently h5repack preserves external links and cannot copy and merge
|
|
data from the external files. Two options, merge and prune, were added to
|
|
control how to merge data from an external link into the resulting file.
|
|
--merge Follow external soft link recursively and merge data.
|
|
--prune Do not follow external soft links and remove link.
|
|
--merge --prune Follow external link, merge data and remove dangling link.
|
|
|
|
(ADB - 2020/08/05, HDFFV-9984)
|
|
|
|
|
|
Support for new platforms, languages and compilers.
|
|
=======================================
|
|
|
|
|
|
Bug Fixes since HDF5-1.12.0 release
|
|
==================================
|
|
Library
|
|
-------
|
|
- Fix bug and simplify collective metadata write operation when some ranks
|
|
have no entries to contribute. This fixes parallel regression test
|
|
failures with IBM SpectrumScale MPI on the Summit system at ORNL.
|
|
|
|
(QAK - 2020/09/02)
|
|
|
|
- Avoid setting up complex MPI types with 0-length vectors, which some
|
|
MPI implementations don't handle well. (In particular, IBM
|
|
SpectrumScale MPI on the Summit system at ORNL)
|
|
|
|
(QAK - 2020/08/21)
|
|
|
|
- Explicitly declared dlopen to use RTLD_LOCAL
|
|
|
|
dlopen documentation states that if neither RTLD_GLOBAL nor
|
|
RTLD_LOCAL are specified, then the default behavior is unspecified.
|
|
The default on linux is usually RTLD_LOCAL while macos will default
|
|
to RTLD_GLOBAL.
|
|
|
|
(ADB - 2020/08/12, HDFFV-11127)
|
|
|
|
- H5Sset_extent_none() sets the dataspace class to H5S_NO_CLASS which
|
|
causes asserts/errors when passed to other dataspace API calls.
|
|
|
|
H5S_NO_CLASS is an internal class value that should not have been
|
|
exposed via a public API call.
|
|
|
|
In debug builds of the library, this can cause asserts to trip. In
|
|
non-debug builds, it will produce normal library errors.
|
|
|
|
The new library behavior is for H5Sset_extent_none() to convert
|
|
the dataspace into one of type H5S_NULL, which is better handled
|
|
by the library and easier for developers to reason about.
|
|
|
|
(DER - 2020/07/27, HDFFV-11027)
|
|
|
|
- Fixed the segmentation fault when reading attributes with multiple threads
|
|
|
|
It was reported that the reading of attributes with variable length string
|
|
datatype will crash with segmentation fault particularly when the number of
|
|
threads is high (>16 threads). The problem was due to the file pointer that
|
|
was set in the variable length string datatype for the attribute. That file
|
|
pointer was already closed when the attribute was accessed.
|
|
|
|
The problem was fixed by setting the file pointer to the current opened file pointer
|
|
when the attribute was accessed. Similar patch up was done before when reading
|
|
dataset with variable length string datatype.
|
|
|
|
(VC - 2020/07/13, HDFFV-11080)
|
|
|
|
- Reduce overhead for H5open(), which is involved in public symbols like
|
|
H5T_NATIVE_INT, etc.
|
|
|
|
(QAK - 2020/06/18)
|
|
|
|
- Cache last ID looked up for an ID type (dataset, datatype, file, etc),
|
|
improving performance when accessing the same ID repeatedly.
|
|
|
|
(QAK - 2020/06/11)
|
|
|
|
- Streamline I/O to a single element, improving performance for record
|
|
appends to chunked datasets.
|
|
|
|
(QAK - 2020/06/11)
|
|
|
|
- Remove redundant tagging of metadata cache entries for some chunked
|
|
dataset operations, slightly improving performance for chunked
|
|
datasets.
|
|
|
|
(QAK - 2020/06/10)
|
|
|
|
- Better detect selections with the same shape, improving performance for
|
|
some uses of H5DOappend (and other situations).
|
|
|
|
(QAK - 2020/06/07)
|
|
|
|
- Don't allocate an empty (0-dimensioned) chunked dataset's chunk
|
|
index, until the dataset's dimensions are increased.
|
|
|
|
(QAK - 2020/05/07)
|
|
|
|
|
|
Tools:
|
|
------
|
|
|
|
|
|
Fortran API
|
|
-----------
|
|
|
|
- Fixed configure issue when building HDF5 with NAG Fortran 7.0.
|
|
|
|
HDF5 now accounts for the addition of half-precision floating-point
|
|
in NAG 7.0 with a KIND=16.
|
|
|
|
(MSB - 2020/02/28, HDFFV-11033)
|
|
|
|
High-Level Library
|
|
------------------
|
|
- Eliminated unnecessary code in H5DOappend(), improving its performance.
|
|
|
|
(QAK - 2020/06/05)
|
|
|
|
Testing
|
|
-------
|
|
- Stopped java/test/junit.sh.in installing libs for testing under ${prefix}
|
|
|
|
Lib files needed are now copied to a subdirectory in the java/test
|
|
directory, and on Macs the loader path for libhdf5.xxxs.so is changed
|
|
in the temporary copy of libhdf5_java.dylib.
|
|
|
|
(LRK, 2020/7/2, HDFFV-11063)
|
|
|
|
|
|
Supported Platforms
|
|
===================
|
|
|
|
Linux 2.6.32-696.16.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
|
|
#1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
|
|
(ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
|
|
IBM XL C/C++ V13.1
|
|
IBM XL Fortran V15.1
|
|
|
|
Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
(kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4)
|
|
Version 4.9.3, 5.2.0, 7.1.0
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 17.0.0.098 Build 20160721
|
|
MPICH 3.1.4
|
|
|
|
Linux-3.10.0- spectrum-mpi/rolling-release with cmake>3.10 and
|
|
862.14.4.1chaos.ch6.ppc64le clang/3.9,8.0
|
|
#1 SMP ppc64le GNU/Linux gcc/7.3
|
|
(ray) xl/2016,2019
|
|
|
|
Linux 3.10.0- openmpi/3.1,4.0 with cmake>3.10 and
|
|
957.12.2.1chaos.ch6.x86_64 clang 5.0
|
|
#1 SMP x86_64 GNU/Linux gcc/7.3,8.2
|
|
(serrano) intel/17.0,18.0/19.0
|
|
|
|
Linux 3.10.0- openmpi/3.1/4.0 with cmake>3.10 and
|
|
1062.1.1.1chaos.ch6.x86_64 clang/3.9,5.0,8.0
|
|
#1 SMP x86_64 GNU/Linux gcc/7.3,8.1,8.2
|
|
(chama,quartz) intel/16.0,18.0,19.0
|
|
|
|
Linux 4.4.180-94.100-default cray-mpich/7.7.6 with PrgEnv-*/6.0.5, cmake>3.10 and
|
|
#1 SMP x86_64 GNU/Linux gcc/7.2.0,8.2.0
|
|
(mutrino) intel/17.0,18.0
|
|
|
|
Linux 4.14.0- spectrum-mpi/rolling-release with cmake>3.10 and
|
|
49.18.1.bl6.ppc64le clang/6.0,8.0
|
|
#1 SMP ppc64le GNU/Linux gcc/7.3
|
|
(lassen) xl/2019
|
|
|
|
SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc
|
|
(emu) Sun Fortran 95 8.6 SunOS_sparc
|
|
Sun C++ 5.12 SunOS_sparc
|
|
|
|
Windows 7 x64 Visual Studio 2015 w/ Intel C, Fortran 2018 (cmake)
|
|
Visual Studio 2015 w/ MSMPI 10 (cmake)
|
|
|
|
Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 18 (cmake)
|
|
Visual Studio 2017 w/ Intel Fortran 19 (cmake)
|
|
Visual Studio 2019 w/ Intel Fortran 19 (cmake)
|
|
|
|
Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.1 from Xcode 7.0
|
|
64-bit gfortran GNU Fortran (GCC) 4.9.2
|
|
(osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
Mac OS X El Capitan 10.11.6 Apple clang/clang++ version 7.3.0 from Xcode 7.3
|
|
64-bit gfortran GNU Fortran (GCC) 5.2.0
|
|
(osx1011dev/osx1011test) Intel icc/icpc/ifort version 16.0.2
|
|
|
|
Mac OS High Sierra 10.13.6 Apple LLVM version 10.0.0 (clang/clang++-1000.10.44.4)
|
|
64-bit gfortran GNU Fortran (GCC) 6.3.0
|
|
(bear) Intel icc/icpc/ifort version 19.0.4
|
|
|
|
Mac OS Mojave 10.14.6 Apple LLVM version 10.0.1 (clang/clang++-1001.0.46.4)
|
|
64-bit gfortran GNU Fortran (GCC) 6.3.0
|
|
(bobcat) Intel icc/icpc/ifort version 19.0.4
|
|
|
|
|
|
Tested Configuration Features Summary
|
|
=====================================
|
|
|
|
In the tables below
|
|
y = tested
|
|
n = not tested in this release
|
|
C = Cluster
|
|
W = Workstation
|
|
x = not working in this release
|
|
dna = does not apply
|
|
( ) = footnote appears below second table
|
|
<blank> = testing incomplete on this feature or platform
|
|
|
|
Platform C F90/ F90 C++ zlib SZIP
|
|
parallel F2003 parallel
|
|
Solaris2.11 32-bit n y/y n y y y
|
|
Solaris2.11 64-bit n y/n n y y y
|
|
Windows 7 y y/y n y y y
|
|
Windows 7 x64 y y/y y y y y
|
|
Windows 7 Cygwin n y/n n y y y
|
|
Windows 7 x64 Cygwin n y/n n y y y
|
|
Windows 10 y y/y n y y y
|
|
Windows 10 x64 y y/y n y y y
|
|
Mac OS X Mountain Lion 10.8.5 64-bit n y/y n y y y
|
|
Mac OS X Mavericks 10.9.5 64-bit n y/y n y y ?
|
|
Mac OS X Yosemite 10.10.5 64-bit n y/y n y y ?
|
|
Mac OS X El Capitan 10.11.6 64-bit n y/y n y y ?
|
|
CentOS 6.7 Linux 2.6.18 x86_64 GNU n y/y n y y y
|
|
CentOS 6.7 Linux 2.6.18 x86_64 Intel n y/y n y y y
|
|
CentOS 6.7 Linux 2.6.32 x86_64 PGI n y/y n y y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 GNU y y/y y y y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 Intel n y/y n y y y
|
|
Linux 2.6.32-573.18.1.el6.ppc64 n y/n n y y y
|
|
|
|
|
|
Platform Shared Shared Shared Thread-
|
|
C libs F90 libs C++ libs safe
|
|
Solaris2.11 32-bit y y y y
|
|
Solaris2.11 64-bit y y y y
|
|
Windows 7 y y y y
|
|
Windows 7 x64 y y y y
|
|
Windows 7 Cygwin n n n y
|
|
Windows 7 x64 Cygwin n n n y
|
|
Windows 10 y y y y
|
|
Windows 10 x64 y y y y
|
|
Mac OS X Mountain Lion 10.8.5 64-bit y n y y
|
|
Mac OS X Mavericks 10.9.5 64-bit y n y y
|
|
Mac OS X Yosemite 10.10.5 64-bit y n y y
|
|
Mac OS X El Capitan 10.11.6 64-bit y n y y
|
|
CentOS 6.7 Linux 2.6.18 x86_64 GNU y y y y
|
|
CentOS 6.7 Linux 2.6.18 x86_64 Intel y y y n
|
|
CentOS 6.7 Linux 2.6.32 x86_64 PGI y y y n
|
|
CentOS 7.2 Linux 2.6.32 x86_64 GNU y y y n
|
|
CentOS 7.2 Linux 2.6.32 x86_64 Intel y y y n
|
|
Linux 2.6.32-573.18.1.el6.ppc64 y y y n
|
|
|
|
Compiler versions for each platform are listed in the preceding
|
|
"Supported Platforms" table.
|
|
|
|
|
|
More Tested Platforms
|
|
=====================
|
|
The following platforms are not supported but have been tested for this release.
|
|
|
|
Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
(mayll/platypus) Version 4.4.7 20120313
|
|
Version 4.9.3, 5.3.0, 6.2.0
|
|
PGI C, Fortran, C++ for 64-bit target on
|
|
x86-64;
|
|
Version 17.10-0
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 17.0.4.196 Build 20170411
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
|
|
Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers
|
|
#1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4)
|
|
(jelly) with NAG Fortran Compiler Release 6.1(Tozai)
|
|
GCC Version 7.1.0
|
|
OpenMPI 3.0.0-GCC-7.2.0-2.29
|
|
Intel(R) C (icc) and C++ (icpc) compilers
|
|
Version 17.0.0.098 Build 20160721
|
|
with NAG Fortran Compiler Release 6.1(Tozai)
|
|
PGI C (pgcc), C++ (pgc++), Fortran (pgf90)
|
|
compilers:
|
|
Version 18.4, 19.4
|
|
MPICH 3.3
|
|
OpenMPI 2.1.5, 3.1.3, 4.0.0
|
|
|
|
Fedora30 5.3.11-200.fc30.x86_64
|
|
#1 SMP x86_64 GNU/Linux GNU gcc (GCC) 9.2.1 20190827 (Red Hat 9.2.1 20190827)
|
|
GNU Fortran (GCC) 9.2.1 20190827 (Red Hat 9.2.1 20190827)
|
|
(cmake and autotools)
|
|
|
|
|
|
Known Problems
|
|
==============
|
|
CMake files do not behave correctly with paths containing spaces.
|
|
Do not use spaces in paths because the required escaping for handling spaces
|
|
results in very complex and fragile build files.
|
|
ADB - 2019/05/07
|
|
|
|
At present, metadata cache images may not be generated by parallel
|
|
applications. Parallel applications can read files with metadata cache
|
|
images, but since this is a collective operation, a deadlock is possible
|
|
if one or more processes do not participate.
|
|
|
|
Known problems in previous releases can be found in the HISTORY*.txt files
|
|
in the HDF5 source. Please report any new problems found to
|
|
help@hdfgroup.org.
|
|
|
|
|
|
CMake vs. Autotools installations
|
|
=================================
|
|
While both build systems produce similar results, there are differences.
|
|
Each system produces the same set of folders on linux (only CMake works
|
|
on standard Windows); bin, include, lib and share. Autotools places the
|
|
COPYING and RELEASE.txt file in the root folder, CMake places them in
|
|
the share folder.
|
|
|
|
The bin folder contains the tools and the build scripts. Additionally, CMake
|
|
creates dynamic versions of the tools with the suffix "-shared". Autotools
|
|
installs one set of tools depending on the "--enable-shared" configuration
|
|
option.
|
|
build scripts
|
|
-------------
|
|
Autotools: h5c++, h5cc, h5fc
|
|
CMake: h5c++, h5cc, h5hlc++, h5hlcc
|
|
|
|
The include folder holds the header files and the fortran mod files. CMake
|
|
places the fortran mod files into separate shared and static subfolders,
|
|
while Autotools places one set of mod files into the include folder. Because
|
|
CMake produces a tools library, the header files for tools will appear in
|
|
the include folder.
|
|
|
|
The lib folder contains the library files, and CMake adds the pkgconfig
|
|
subfolder with the hdf5*.pc files used by the bin/build scripts created by
|
|
the CMake build. CMake separates the C interface code from the fortran code by
|
|
creating C-stub libraries for each Fortran library. In addition, only CMake
|
|
installs the tools library. The names of the szip libraries are different
|
|
between the build systems.
|
|
|
|
The share folder will have the most differences because CMake builds include
|
|
a number of CMake specific files for support of CMake's find_package and support
|
|
for the HDF5 Examples CMake project.
|
|
|