* commit '54957d37f5aa73912763dbb6e308555e863c43f4':
Commit copyright header change for src/H5PLpkg.c which was added after running script to make changes.
Add new files in release_docs to MANIFEST. Cimmit changes to Makefile.in(s) and H5PL.c that resulted from running autogen.sh.
Merge pull request #407 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:hdf5_1_10_1 to hdf5_1_10_1
Change copyright headers to replace url referring to file to be removed and replace it with new url for COPYING file.
Also purged obsolete /*ARGSUSED*/ from the library.
Tested on:
Ubuntu 15.10 (Linux 4.2.0 x86_64) gcc 5.2.1
serial only
(these changes have been in revise_chunks for a long time)
Complete revamp of package initialization/shutdown mechanism in the library.
Each package now has a single init/term routine.
This new way should avoid packages being re-initialized during library
shutdown and is also be _much_ more proactive about giving feedback for
resource leaks internal to the library.
Introduces a new "module" header file for packages in the library
(e.g src/H5Fmodule.h) which sets up some necessary package configuration macros
for the FUNC_ENTER/LEAVE macros. (The VFL drivers have their own slightly
modified version of this header, src/H5FDdrvr_module.h)
Also cleaned up a bunch of resources leaks all across the library and tests,
along with addressing many warnings, as I encountered them.
Tested on:
MacOSX/64 10.10.5 (amazon) w/serial & parallel
Linux/64 3.10.x (kituo) w/serial & parallel
Linux/64 2.6.x (ostrich) w/serial
branch.
Removed the configure option that allows selective disabling of individual
internal filters (fletcher32, shuffle, etc.).
This feature mucked up the code with a lot of #ifdefs, saved very little space,
and was not scalable to a general scheme for library size reduction. We've
decided to remove the feature while we investigate a more general scheme for
decreasing the library size.
Part of: HDFFV-9086
Tested on: h5committest
Clean up more compiler warnings, plus merge a few Coverity bug fixes from
the hdf5_1_8_coverity branch back to the trunk:
r20877:
Purpose: Fix coverity issue 1723
Description:
Modified test_generate in hl/test_image to close file "f" before exit, even if
an error occurs.
r20879:
Issue 63: change check of return of H5Tget_nmembers to <=0. No need to go
futher if call fails as well as empty.
r20881:
Coverity #659 in Run 46: I changed the Line 442 where it tries to check whether
FLAG_PRINTED is TRUE. But it had just been set to FALSE. I took out the
condition check in the print statement.
Tested on:
Mac OSX/64 10.9.2 (amazon) w/C++, FORTRAN & Parallel
(too minor to require h5committest)
Refactor function name macros and simplify the FUNC_ENTER macros, to clear
away the cruft and prepare for further cleanups.
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug, production & parallel
Description:
Previously, there was no versioning for H5Z_class_t. This prevented applications
written for 1.6 using custom filters from being able to use the 1.8 library.
There is now an H5Z_class1_t and H5Z_class2_t to enable compatibility. H5Zregister is
*not* versioned, it determines which version of the struct has been passed in by the
value of the first field (id or version, both are ints).
Tested: jam, linew, smirom (h5committest), jam (--with-default-api-version=v16)
TO DO: szip, nbit and scale offset
NOTE: the symbol H5Z_SHUFFLE_TOTAL_NPARMS was made public
Tested: windows, teragrid with icc 8.1, linux (kagiso), solaris (linew)
Make H5Pget_filter_by_id() API versioned and switch internal usage
to H5Pget_filter_by_id2().
Add simple regression test for H5Pget_filter_by_id1().
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.4.10 (amazon) in debug mode
Tested platform:
Kagiso only since it is only a comment block change. If it works in one
machine, it should work in all, I hope. Still need to check the parallel
build on copper.
Code cleanup
Description:
Trim trailing whitespace, which is making 'diff'ing the two branches
difficult.
Solution:
Ran this script in each directory:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Bug fix
Description:
Correct bug where buffers that have only fractional elements (usually from
being compressed before being shuffled) would cause optimized algorithm to
dump core.
Solution:
Don't attempt to shuffle bytes unless we've got more than one element.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Code cleanup
Description:
Clean up various warnings reported by the Windows team.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Code optimization
Description:
Use Duff's Device to unroll shuffling loop a bit, for ~30% speedup.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Solaris 2.9 (shanti)
Description: Removed PABLO from the source
Solution:
Platforms tested: arabica with 64-bit, copper with parallel,
heping with GNU C and C++ and PGI fortran (but
I disabled hl, there is some weird problem only
on heping: F9XMODFLAG is not
propagated to the Makefile files
Misc. update:
Purpose:
HDF5 now supports SZIP with no encoder.
Description:
SZIP can be configured to have both encoder and decoder or just to have the decoder. HDF5 can now query the configuration of any filter, and will throw errors if users try to write using a filter with encoding disabled.
Solution:
Added H5Zget_filter_info function, changed API for H5Pget_filter and H5P_get_filter_by_id. See SZIP RFC.
Platforms tested:
Copper (fortran, C++, parallel), Sleipnir (C++), Arabica (fortran, C++), Verbena (fortran, C++)
Misc. update:
Code cleanup & minor optimization
Description:
Re-work the way interface initialization routines are specified in the
library to avoid the overhead of checking for them in routines where there is
no interface initialization routine. This cleans up warnings with gcc 3.4,
reduces the library binary size a bit (about 2-3%) and should speedup the
library's execution slightly.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/gcc34
h5committest
bug fix in H5Zshuffle.c
add more tests to h5repack that exposed the bug
Description:
when creating a dataset with the shuffle filter and duplicating it in a new dataset (file)
the call to H5Z_set_local_shuffle failed. this is because the value of cd_nelmts of the filter
structure is set to 1 (H5Z_SHUFFLE_TOTAL_NPARMS) when the original dataset is created, but when
the new dataset is created there is a checking instruction that fails if the value of
cd_nelmts is not 0 (its original value, H5Z_SHUFFLE_USER_NPARMS)
Solution:
just remove that check condition, since the value of cd_nelmts is not used anyway.
if we decide that the value of cd_nelmts is necessary, then the H5O_pline_copy function
must be changed to update this value (a different update for each filter)
Platforms tested:
linux
solaris
AIX
Misc. update:
Code cleanup/new features
Description:
Switch over to a new style for registering filters with the library -
instead passing in an ID, a string and a callback function to H5Zregister,
the client should pass in a single pointer to a H5Z_claass_t struct which
contains the ID, the description string and all the function callbacks as
fields.
Added support for a new "can apply" callback for each filter, which is
called when a dataset is created to check whether the parameters for that
filter apply correctly to the combination of the datatype and the chunk size
(i.e. dataspace) for the dataset.
Added support for a new "set local" callback for each filter, which
is called when a dataset is created (after the "can apply" filter callback)
and sets filter parameters that are specific to that particular dataset.
Switched the filters we ship over to use the new H5Z_class_t struct for
their internal registrations and also added "set local" callbacks to the
szip and shuffle filters and a "can apply" callback to the szip filter.
Lots of other code cleanups, etc. also
Solution:
Platforms tested:
FreeBSD 4.8 (sleipnir) w/szip
Linux 2.4 (sleipnir) w/szip
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/szip, FORTRAN & parallel
Misc. update:
Code cleanup
Description:
Added lots of comments to existing filters, so they are easier to understand
and use as examples for future filters. Cleaned up various bits of code, etc.
Solution:
Platforms tested:
FreeBSD 4.8 (sleipnir)
IRIX64 6.5 (modi4) w/parallel
Misc. update:
Code cleanup
Description:
Clean up miscellaneous warnings which have crept into the code.
Fix "_POSIX_C_SOURCE not defined" warning on FreeBSD.
Adjust gcc compiler flags to be more concise for production mode.
Refactor the H5O code so that there is a stronger boundary between code
in the H5O package and code in the library which just calls H5O routines.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel and gcc 2.95.4 & gcc 3.2.2
Misc. update:
Update MANIFEST if you add or remove any file.
Code cleanup/new feature.
Description:
Split FUNC_LEAVE into API and non-API specific versions. This allows a
solution to compiling this branch with C++, as well as reducing the size
of the binaries produced.
Platforms tested:
FreeBSD 4.7 (sleipnir) w/serial, parallel (including MPE) & thread-safe
Code Cleanup & New Feature
Description:
H5config.h.in:
Removed H5_HAVE_COMPRESSION & H5_HAVE_FILTER_GZIP flags.
Added H5_HAVE_FILTER_DEFLATE flag.
H5Z.c:
H5Zprivate.h:
H5Zpublic.h:
Switched from using H5_HAVE_COMPRESSION flag in favor of
H5_HAVE_FILTER_DEFLATE.
Added H5Zunregister & H5Zfilter_avail API functions.
Changed a numeric constant (256) to a symbolic constant
(H5Z_FILTER_RESERVED).
Automatically add the shuffling filter to the list of available filters
(when it is enabled).
Moved prototypes for H5Z_filter_deflate & H5Z_filter_shuffle from the
public header into the private header.
H5Zdeflate.c:
Switched from using H5_HAVE_COMPRESSION & H5_HAVE_FILTER_GZIP flags in
favor of H5_HAVE_FILTER_DEFLATE.
Cleaned up formatting & error reporting a bit.
H5Zshuffle.c:
Rewrote shuffling algorithm to be more efficient.
Added error checking & reporting.
Added standard Pablo information.
Added standard function header comment.
Added FUNC_ENTER & FUNC_LEAVE macros.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir)
Purpose:
Adding internal shuffle filter
Description:
With the combination of shuffling filter with general compression
algorithm, the compression ratio may be improved without adding much
encoding and decoding time for many real NASA datasets(especially
floating data) and other application
datasets(See techNotes).
Solution:
SHuffle the bytes within the data to utilize the locality.
Platforms tested:
arabica , eirene, modi4
Misc. update:
Update MANIFEST if you add or remove any file.
Update release_docs/RELEASE for bug fixes, new features, etc.
Update applicable document files too.
