Fix Bug1672 - Display garbage value on LE machine for BE data.
Description:
Casuing by calling diff_dataset twice when -r or no option was given.
Change to call it once which fix the problem.
It also improves the performance. (diffing time in half)
According further test, it also occurred on BE machine as well and it
seems occruing only with the BE data attached to Bugzilla.
Don't know how the file was created. Anyway this fix will prevent
from diffing with garbage values in similiar potential case.
Tested on:
linux32 (jam) , linux64 (almani), solaris (linew)
CFLAGS overhaul
Description:
Modified the way configure handles CFLAGS.
(note: all changes also apply to FCFLAGS, CXXFLAGS, CPPFLAGS, and LDFLAGS).
1. The configure process will now always preserve a user's CFLAGS
environment variable setup. Any additional flags necessary for compilation
added at configure time will be passed into the Makefiles as AM_CFLAGS,
which is an automake construct to be used in addition to CFLAGS.
This will allow a user to have the final say, as CFLAGS will always appear
later in the compile line than AM_CFLAGS. Additionally, setting CFLAGS
during make will no longer completely erase all flags set by configure,
since they're maintained in AM_CFLAGS.
2. Additionally, where possible, flags previously being assigned directly
into CFLAGS (and thus propagating into h5cc) have now been redirected into
H5_CFLAGS, so they're used ONLY for compiling hdf5, and not embedded into
the h5cc wrapper script as well.
*Note that H5_CFLAGS ultimately is assigned into AM_CFLAGS for use in the
Makefiles. Complete description of changes and build process will be
included in a Configure Document that Elena and I are working on.
3. Removed unsupported config files. This includes:
config/dec-osf*
config/hpux11.00
config/irix5.x
config/powerpc-ibm-aix4.x
config/rs6000-ibm-aix5.x
config/unicos*
4. Modified configure summary to display additional values. Specifically,
appropriate AM_* variables are being shown, as well as H5_FCFLAGS and
H5_CXXFLAGS, which were for some reason not already present.
Tested:
- H5committest
- Tested on all THG / NCSA machines, using several combinations of the more
prominent configure options (c++, fortran, szip, threadsafe, parallel,
et cetera). (Thanks to Quincey for rysnc testing setup!)
- With regards to new automated testing, anything *necessary* for
compilation will be caught by the daily tests as it stands now. (i.e.,
if LDFLAGS is not properly set when szip is used, linking will fail).
Additionally, with regards to which flags get into h5cc, if any
*necessary* flags have been improperly removed, then daily tests should
fail during make installcheck. Additional machine-specific desired
behaviors and/or checks may have to be set up separately within the
daily tests, so this is something to work on.
Added missing H5Tclose in region reference routines.
H5DS.c checked for other locations which need to account for platform dependent types in dealing with ds_list_t. Added generator program for test files of LE and BE dimension scales.
Tested: local linux, h5committest
Bring most of Vailin's changes to the fixed array data structure back
to the trunk, including new regression test for data structure.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.8 (amazon) in debug mode
Mac OS X/32 10.5.8 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
" Use "--use-system-epsilon" for system EPSILON
" Use "-p" or "-d" for whatever user's choice of epsilon
" Use "-p 0" or "-d 0" for strict equality (same as default)
Correct a 'hsize_t *' to 'hsize_t' and vice versa.
Tested on:
Mac OS X/32 10.5.8 (amazon) in debug mode
Mac OS X/32 10.5.8 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
(too minor to require h5committest)
Updating autotools
Description:
Installed new autotools and used them to reconfigure HDF5.
- Automake upgraded to 1.11
- Autoconf upgraded to 2.64
- bin/reconfigure script edited to use new versions (on jam), and
run to generate new configure script and Makefile.in's.
- configure.in script edited to add "_cv_" to all AC_CACHE_VAL strings
(in order to comply with new autoconf standard).
Tested:
Tested on machines jam, smirom, liberty, linew.
Tested w/ features c++, fortran, parallel.
Tested w/ compilers gcc, pgcc, icc.
Further testing via Daily Tests should catch any other outliers. Upon
passing DT's, I'll propogate the new tools into 1.8, hdf4, et cetera.
The file used for input is located in the common source tools for testfiles, in tools/testfiles
Modified the h5repack shell script to read files from this location (h5repack reads its input files from a dedicated testfiles location in h5repack/testfiles)
Changed the h5diff open file call to use h5tools_fopen, so that it can open all file drivers
Tested: windows, linux, solaris
- Updated bin/reconfigure to use latest version of automake (1.10.2).
Re-generated Makefile.in's by running bin/reconfigure.
- Added libtool version numbers to c++, fortran, hl, hl c++, and hl fortran
libraries.
Tested:
jam, liberty, smirom
Description:
Changed Skip list package to use a deterministic skip list. This allows the
skip list package to avoid calling rand() and srand(), even on machines without
rand_r(). There is no longer a p-value or maximum level for skip lists.
Tested: jam, smirom, linew (h5committest)
ISSUE : the tools use the following formula to read by hyperslabs: hyperslab_size[i] = MIN( dim_size[i], H5TOOLS_BUFSIZE / datum_size) where H5TOOLS_BUFSIZE is a constant defined of 1024K. This is OK as long as the datum_size does not exceed 1024K, otherwise we have a hyperslab size of 0 (since 1024K/(greater than 1024K) = 0). This affects h5dump. h5repack, h5diff
SOLUTION: add a check for a 0 size and define as 1 if so.
TEST FOR H5DUMP: Defined a case in the h5dump test generator program of such a type (an array type of doubles with a large array dimension, that was the case the user reported). Since the written file commited in svn would be around 1024K, opted for not writing the data (the part of the code where the hyperslab is defined is executed, since h5dump always reads the files). Defined a macro WRITE_ARRAY to enable such writing if needed. Added a run on the h5dump shell script. Added 2 new files to svn: tools/testfiles/tarray8.ddl, tools/testfiles/tarray8.h5. NOTE: while doing this I thought of adding this dataset case to an existing file, but that would add the large array output to those files (the ddls). The issue is that the file list is increasing.
TEST FOR H5DIFF: for h5diff the check for reading by hyperslabs is H5TOOLS_MALLOCSIZE (128 * H5TOOLS_BUFSIZE) or 128 Mb. This makes it not possible to add such a file to svn, so used the same method as h5dump (only write the dataset if WRITE_ARRAY is defined). As opposed to h5dump, the hyperslab code is NOT executed when the dataset is empty (dataset is not read). Added the new dataset to existing files and shell run (tools/h5diff/testfiles/h5diff_dset1.h5 and tools/h5diff/testfiles/h5diff_dset2.h5 and output in tools/h5diff/testfiles/h5diff_80.txt).
TEST FOR H5REPACK: similar issue as h5diff with the difference that the hyperslab code is run. Added a run to the shell script (with a filter, otherwise the code uses H5Ocopy).
tested: linux (h5commitest failed , apparently it did not detect the code changes in /tools/lib that fix the bug: the error in an assertion in the hyperslab of 0. I am sure that making h5ccomitest --distclean will detect the new code , but don't want to wait more 3 hours :-) )
Bug Fix
Description:
Fixing BZ #1381. The --includedir=DIR configure option, which is used
to specify the installation location of C header files, did not work
correctly as the path was hard-coded in config/commence.am. I'm presuming
this is because an older version of automake didn't know where to put
c header files. In any case, removing this line now defaults the includedir
to the same directory that it is currently hard-coded to, and also fixes
the configure flag to allow customization of this value.
Tested:
jam, liberty
the way h5ls prints types, it starts searching for NATIVE types first. One solution would be h5ls not to detect these native types, using for example the same print datatype function that h5dump does, that would make the output look the same on all platforms ("32-bit little-endian integer" would be printed instead). Drawback, this "native" information would not be available. Other solution is to have not one but 2 expected outputs and make the shell script detect the endianess and compare with one output or other
tested: h5committest
Solution: for compound types, recursively apply that check
Two new cases are added
1) the compound type has a different number of members. Message printed is
<obj1> has X members <obj2> has Y members
Where X and Y are the number of members of each compound type being compared
2) the compound type has not comparable types (for example a double and an int at the same index)
In this case the message
Comparison not possible: object1 is of class1 and object2 is of class2
Is replaced with
Comparison not possible: object1 has a class1 and object2 has a class2
Modified the test generator program to have these 2 cases
Added a shell run for these 2 cases
Tested: windows, h5committest
-N, --nan Avoid NaNs detection
Note: there is no shell script run for datasets with NaN because the output is non portable (different results and NaN strings for different systems)
Tested: windows, linux
PG compiler complains about array out of bounds (a rank of zero was not checked)
Adding a scalar dataset to the test generator program. this case is run on a previous existing run, the case was added to 2 existing files
Tested: windows, linux
* Pedro Vicente, pvn@hdfgroup.org, Nov 4, 2008
* Compare the graph and make h5diff return 1 for difference if
* 1) the number of objects in file1 is not the same as in file2
* 2) the graph does not match, i.e same names (absolute path)
* 3) objects with the same name are not of the same type
Introduced a new feature in the tools library regarding command line parsing
In the definition of arguments, an "*" means that the switch can or can not have an optional argument. This "*" is put in the code regarding the letter definition, and it is transparent to the user (e.g b* instead of the previous b: ), where ":" notes a required argument after the letter (and no ":" or "*" notes no argument, mandatory)
Used for the h5dump binary option -b
It can be now
1) -b (defaults to NATIVE)
2) - b NATIVE
3) - b FILE
4) -b LE
5) -b BE
Note: the keyword NATIVE replaces MEMORY
This feature (-b with no argument) was tested with the sequence of h5dump to binary (NATIVE) then h5import to generate an HDF5 file from the binary file and h5diff to compare the 2 HDF5 files
Tested: linux
Description:
- Remove need to set LD_LIBRARY_PATH when using shared szip library.
- Libtool 2.2.6a is now used to generate libraries.
- 'make check install' dependency bug is fixed, and should no longer
break the build.
- removed hard coding of shell in config/commence.am, as this causes
problems on Solaris with the new version of libtool.
- RELEASE.txt with appropriate changes.
Tested:
- kagiso, smirom, linew (merged from 1.8, pretty quick tests)
Description: Improved external link traversal of h5dump. h5dump will now
properly avoid all cycles, even those spanning multiple files. Improvement
to the output of committed datatypes. Committed datatypes are now checked
for uniqueness (like other objects). Tests added for these cases.
Tested: kagiso, linew, smirom (h5committest)
Description: Adds capability to h5ls to traverse external links when the -r
(recursive) option is given. Changes to the way absolute path names are patched
in h5trav.c. Changes to the way recursive traversal starting from a non-root
group is handled (which also fixes some preexisting issues). Tests added for
these cases.
Tested: kagiso, smirom, linew (h5committest)
Remove trailing whitespace from C/C++ source files, with the following
script:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Tested on:
Mac OS X/32 10.5.5 (amazon)
No need for h5committest, just whitespace changes...
