Bring r22921 from trunk to 1.8 branch:
Review Coverity changes and bring them back to trunk. (QK & JK)
r20402:
Added #includes for h5tools.y or h5tools_utils.h as required to remedy implicit
function declarations which caused compiler warnings and coverity issues 703-4
and 708-11.
r20414:
This is related to the previous checkin r20399. There were incorrect updates
which caused incorrect behavior when no file was given. Also possible segfault
when handling hyperslab options. Simplify the code changes.
r20449:
Description: Modified H5E_walk2_cb to check return value of H5I_object_verify.
r20450:
fixed coverity 813
Tested on:
Mac OSX/64 10.8.2 (amazon) w/C++, FORTRAN, debug & threadsafe
(too minor to require h5committest)
Merge 1.8 and h5dump/tools and tests based on tools library from trunk.
Reduced warnings.
HDFFV-7949:
Remove duplicated functions in h5ls
Tested: local linux,h5committest
- HDFFV-5928 - GMQS: h5diff problem and improvement on comparsing the same objects
Description:
Merged from HDF5 trunk r20676.
Fixed:
1) adding h5tools_is_obj_same() function to check if two given IDs or paths point to the same object. This function can be very useful for other tools and applications.
2) using h5tools_is_obj_same() at h5diff() and diff() in h5diff.c. If two paths point to the same object, there is no need to check the details of the object since we know there is no difference. The fix will increase the performance by skipping the content comparison. It also fixed the problem of reporting difference for some cases of comparing the same file, e.g. empty files or files with incomparable objects the same file.
Test update:
Updat prvious test cases (171, 172, 530) affected by this fix, so they
still perfrom originally intended testing without bypassing.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), cmake
Fixed Bug 2216 - GMQS: h5diff - memory leak when compares vlen string in
dataset or attributes
Description:
Merged from HDF5 trunk r20266, r20270 and r20285.
Test for dataset : valgrind --leak-check=full ./h5diff -v h5diff_dset1.h5 h5diff_dset2.h5 /g1/VLstring
Test for attr : valgrind --leak-check=full ./h5diff h5diff_attr1.h5 h5diff_attr2.h5
Both test cases are in testing script.
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), Cmake - jam
Added back test for H5Tdetect_class of H5T_VLEN after each instance of above function to catch all VLEN types in h5dump.
bring back from trunk r20247
Tested: local linux
Bring r19252 & 19253 from trunk to 1.8 branch:
Bring Coverity changes from branch to trunk:
r19161:
Fixed the part for matching the subset info with dataset
r19189:
BZ1646: h5dump does not check number of dimensions for subsetting parameters against the dataset
Changed subset_t structure from holding hsize_t pointers to holding new subset_d pointers, which hold the original hsize_t pointer + len. this len is then checked against dataset ndims in the handle_dataset function of h5dump.
Changed all references to use new data structure.
Added tests for each subset parameter.
r19190:
Added new h5dump ddl files
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & production
(h5committested on Coverity branch)
any more. Changed algorithm of creating mask which now sits
at the least significant bits. Display data by down shift
first and then mask it for display. Changed the variable
names to reflect the new purpose better.
Tested: AlbertPax (linux)
Install packed-bits feature for h5dump.
tools/h5dump/h5dump.c:
Added lots of error checking into parse_mask_list() to catch and
flagged packed-bits argument errors. For now, the packed bits
are limited to 1 byte (8 bits) range.
tools/h5dump/testh5dump.sh.in
Added 10 new packed-bits test to verify normal operation and argument
error detection.
tools/lib/h5tools.h:
tools/lib/h5tools_str.c:
tools/lib/h5tools.c:
Only name changes of the packed-bit global variables to reflect
the nature of the variable better, IMO.
Tested: albertPax (linux)
------------------------------------------------------------------------
Install packed-bits feature for h5dump.
Allen has installed the feature in the NPOESS feature branch.
I ported it back to the v1.8 branch for v1.8.5 release.
tools/h5dump/testh5dump.sh.in:
Added tests for the packed-bits feature.
tools/h5dump/h5dump.c
tools/h5dump/h5dump.h
Added code to parse and handled the packed-bits request.
tools/lib/h5tools.h
tools/lib/h5tools_str.c
tools/lib/h5tools.c
Added code to print packed-bits.
tools/testfiles/tpackedbits2.ddl
tools/testfiles/tnofilename-with-packed-bits.ddl
tools/testfiles/tpackedbits.ddl
Expected output files for packed-bits feature.
Tested:
h5committested the NPOESS branch.
Tested in the linux VM machine in my Dove.
use of
H5_PRINTF_LL_WIDTH
in the definition of
#define HSIZE_T_FORMAT "%"H5_PRINTF_LL_WIDTH"u"
the printing of "unknow u" in tg-login3.
for unsigned long long was a configure bug fixed by rev 15942
tested: tg-login3
for some reason , the use of H5_PRINTF_LL_WIDTH to convert to a unsigned long long format, is not working in ia64 linux (tg-login3)
extra characters are printed, this time in h5ls
hardcoded the format defined in HSIZE_T_FORMAT to "%llu"
tested: windows, linux (kagiso), solaris, ia64 linux (tg-login3)
Remove trailing whitespace from C/C++ source files, with the following
script:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Tested on:
Mac OS X/32 10.5.5 (amazon)
No need for h5committest, just whitespace changes...
Move H5Giterate() and H5G_iterate_t to deprecated symbols section,
replacing them with H5Literate and H5L_iterate_t respectively.
Tested on:
FreeBSD/32 6.2 (duty)
FreeBSD/64 6.2 (liberty)
Linux/32 2.6 (kagiso)
Linux/64 2.6 (smirom)
AIX/32 5.3 (copper)
Solaris/32 5.10 (linew)
Mac OS X/32 10.4.10 (amazon)
Changed macro names, conflict with new compiler
-- changed DATASET to H5_TOOLS_DATASET
-- changed DATATYPE to H5_TOOLS _DATATYPE
-- changed GROUP to H5_TOOLS_GROUP
In mingw's version of winsock2.h they define:
typedef unsigned int GROUP;
remove STORAGELAYOUT macro, it was not used
tested: linux
h5repack and h5diff hyperslab I/O
changed the limit on which hyperslab I/O is done from 1GB to 128MB
h5repack currently tests this feature by defining a dataset with dimensions of 128MB + 1byte (the datum being 1 byte integer), in which a 1Kb hyperslab was written.
tested: linux
there were some errors in running the tests, but these were in the library, due to a recent check-in that will be fixed shortly (at least we hope :-) )
Add small 'h5mkgrp' tool to create groups in an HDF5 file from the command
line, allowing the group structure for a file to be created in a script. This
tool closely follows the 'mkdir' command line tool in UNIX/Linux.
Allow tool library applications to pass a FAPL to the h5tool_fopen() call,
giving some additional flexibility to tools which are adding objects to an
existing HDF5 file (like h5copy & h5mkgrp).
Fix missing files in MANIFEST from previous checkin(s).
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Add feature to h5copy to allow it to add an object to an existing file,
instead of blowing away existing file.
Modify h5tools_fopen() routine to take access flags, so it can be used
to open an existing file for writing.
Added check to h5copy test script that verifies it has produced a file
with the correct structure.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Tested platform:
Kagiso only since it is only a comment block change. If it works in one
machine, it should work in all, I hope. Still need to check the parallel
build on copper.
fix for bugzilla bug #551
several programming errors contributed to this bug
1) the parsing of subsetting was using atoi to convert the parameter to an int,
which caused problems for numbers greater that int. Substitute with atof
2) several index counters were declared as int, use hsize_t instead
3) the numerical format passed for printf was %lu, defined one compatible with
hsize_t instead (unsigned long long)
Fixes for bugs 676, 228
676: both h5repack and h5diff use H5Dread. In the case of a "big"
dataset, use read/write by hyperslabs the same way h5dump uses. An
arbitrary value of 1GB was defined for "big", i.e, if the dataset is
greater than 1GB, then read/write by hyperslabs
228: use the file type in read/write by default. A new switch -n was
introduced if the user wants to use a native type, which was the
previous use by default.
Added a new test for h5repack that repacks a 1GB dataset
Tested: heping (serial, parallel), sol, copper
Code cleanup.
Description:
Removed argc and argv from the function arguments of h5tools_get_fapl() and
h5tools_fopen(). They were used to call MPI_Init() which was no longer
needed.
Tested:
heping (serial and parallel).
new feature
Description:
added support for h5dump to dump binary data using the file type format
added one test to the test script that tests this
Solution:
Platforms tested:
mir
shanti
copper
Misc. update:
new feature. h5dump output of binary data
Description:
a new switch -b FILE_NAME that dumps the contents of memory data to file FILE_NAME in binary form
new program binread.c that reads the contents of this file and outputs it to stdout
added a test for the h5dump shell script that does a run of -b
the binread.c program reads the data used in this run, usage is ./binread FILE_NAME
Solution:
Platforms tested:
linux
solaris
AIX
Misc. update:
Code cleanup
Description:
Check in some of the code cleanups from working on the external link
support. (This doesn't include any of the external link features)
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OSX.4 (amazon)
Linux 2.4
Code cleanup
Description:
Trim trailing whitespace, which is making 'diff'ing the two branches
difficult.
Solution:
Ran this script in each directory:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
bug fix 366
Description:
the printing of the array indices was done relatively to the stripmine data (data read
by hyperslabs when its memory requiremnts are too large)
this was causing an incorrect numbering of the array indices (the next read would initialize
the indices to zero)
Solution:
added a field to the print context that keeps track of the stripmine position
and pass to the rendering function the total element position
Platforms tested:
Linux
Misc. update:
Purpose:
bug fix, new test file
Description:
h5dump was not properly displaying array indices > 3D
Solution:
added the same algorythm and data structure that h5diff uses to calculate the array index
from a element number position
Platforms tested:
linux
solaris
Misc. update:
Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-)
Description:
Generally speaking, this is the "signed->unsigned" change to selections.
However, in the process of merging code back, things got stickier and stickier
until I ended up doing a big "sync the two branches up" operation. So... I
brought back all the "infrastructure" fixes from the development branch to the
release branch (which I think were actually making some improvement in
performance) as well as fixed several bugs which had been fixed in one branch,
but not the other.
I've also tagged the repository before making this checkin with the label
"before_signed_unsigned_changes".
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel & fphdf5
FreeBSD 4.10 (sleipnir) w/threadsafe
FreeBSD 4.10 (sleipnir) w/backward compatibility
Solaris 2.7 (arabica) w/"purify options"
Solaris 2.8 (sol) w/FORTRAN & C++
AIX 5.x (copper) w/parallel & FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN
Linux 2.4 (heping) w/FORTRAN & C++
Misc. update:
Fix SZIP filter to dynmically detect encoder.
Description:
Solution:
See:
http://hdf.ncsa.uiuc.edu/RFC/SZIP/Szip_dynamic_12_Oct.pdf
Changes to h5repack tests, contingent on detecting SZIP encoder.
Note new program:
testh5repack_detect_szip
Checks fo rencoder, prints out "yes" or "no". Used by hrepack.sh
to detect encoder. Can also be used for windows tests. This is
only used as part of the tests.
Had to modify Makefile to build and clean this program.
Refactor code
Description:
Refactor common code for determining the native type for using in the
tools into separate routine.
Also, reduce diffs between the two branches and bring back some fixes from
the development branch to the release branch.
Platforms tested:
FreeBSD 4.10 (sleipnir)
too minor to require h5committest
h5dump output change, new tests
Description:
the storage layout output format the storage layout output format had some changes
same for the user defined filter
add an option (-y) for not printing the array indices (default is print indices )
the option for escaping non printable characters covers all characters (default is not escape)
(this might be not very portable, the test files are tstring.ddl and tstringe.ddl )
add tests for the new options
Solution:
Platforms tested:
linux
solaris
AIX
Misc. update:
Description:
added the code for print strings with new line and display the path of references (new source files h5tools_ref.c and .h )
added a test suite in testh5dump.sh.in for
( note : to create testh5dump.sh , one must redo ./configure; this detects the availability of filters
and generates testh5dump.sh accordingly)
1) storage layout
2) fill value
3) print reference with path
4) print strings with new lines
5) filters
Solution:
Platforms tested:
linux
solaris
AIX
Misc. update:
h5dump new version
Description:
added the changes already made for 1.6
support for dumping of
1) filters
2) storage layout
3) fill value
4) comments
5) superblock
6) file contents
7) array indices
Solution:
Platforms tested:
linux
solaris
AIX
Misc. update:
1) new function for tools library
2) new test script for h5repack
Description:
1) currently all the tools (h5dump, h5diff, etc) do not check if a filter is available
for reading some dataset that might have a filter not available on the current configuration (the behaviour
of the tools until now was to trigger a library error, saying that the dataset cannot be read
due to the lack of the filter)
Solution:
1) added a new function h5tools_canreadf that checks if a dataset can be read
depending on the availability of filters.
this function was added in calls for h5diff and h5repack.
instead of triggering the library error, a message is printed, saying that the dataset
cannot be read (the print is optional, it is on on verbose mode)
2) added a shell script that tests the commannd line tool behaviour of h5repack
the script does a series of runs of h5repack with several options on the same file (this file test4.h5
was added to the testfiles dir).
then, it runs the h5diff tool, with the input and output files , in each run.
the goal of the test is also to check item 1) . the binary file was saved with filters
that might not be available on other configurations
Platforms tested:
linux (all filters enabled)
linux (some filters disabled)
solaris (some filters disabled)
AIX (some filters disabled)
windows (all filters on and off )
Misc. update:
Bug fix
Description:
Fix h5tools routines to not try to call MPI_Init() unless an MPI-based
VFD is actually used.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel
Linux 2.4 (verbena) w/parallel
