philipp/hdf5
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
a9c79d6b6168f6833a3683660ae2a15da4e2ec4e
hdf5/test/ohdr.c
Quincey Koziol 9a433b99a5 [svn-r6252] Purpose: 
Lots of performance improvements & a couple new internal API interfaces.

Description:
    Performance Improvements:
        - Cached file offset & length sizes in shared file struct, to avoid
            constantly looking them up in the FCPL.
        - Generic property improvements:
            - Added "revision" number to generic property classes to speed
                up comparisons.
            - Changed method of storing properties from using a hash-table
                to the TBBT routines in the library.
            - Share the propery names between classes and the lists derived
                from them.
            - Removed redundant 'def_value' buffer from each property.
            - Switching code to use a "copy on write" strategy for
                properties in each list, where the properties in each list
                are shared with the properties in the class, until a
                property's value is changed in a list.
        - Fixed error in layout code which was allocating too many buffers.
        - Redefined public macros of the form (H5open()/H5check, <variable>)
            internally to only be (<variable>), avoiding innumerable useless
            calls to H5open() and H5check_version().
        - Reuse already zeroed buffers in H5F_contig_fill instead of
            constantly re-zeroing them.
        - Don't write fill values if writing entire dataset.
        - Use gettimeofday() system call instead of time() system when
            checking the modification time of a dataset.
        - Added reference counted string API and use it for tracking the
            names of objects opening in a file (for the ID->name code).
        - Removed redundant H5P_get() calls in B-tree routines.
        - Redefine H5T datatype macros internally to the library, to avoid
            calling H5check redundantly.
        - Keep dataspace information for dataset locally instead of reading
            from disk each time.  Added new module to track open objects
            in a file, to allow this (which will be useful eventually for
            some FPH5 metadata caching issues).
        - Remove H5AC_find macro which was inlining metadata cache lookups,
            and call function instead.
        - Remove redundant memset() calls from H5G_namei() routine.
        - Remove redundant checking of object type when locating objects
            in metadata cache and rely on the address only.
        - Create default dataset object to use when default dataset creation
            property list is used to create datasets, bypassing querying
            for all the property list values.
        - Use default I/O vector size when performing raw data with the
            default dataset transfer property list, instead of querying for
            I/O vector size.
        - Remove H5P_DEFAULT internally to the library, replacing it with
            more specific default property list based on the type of
            property list needed.
        - Remove redundant memset() calls in object header message (H5O*)
            routines.
        - Remove redunant memset() calls in data I/O routines.
        - Split free-list allocation routines into malloc() and calloc()-
            like routines, instead of one combined routine.
        - Remove lots of indirection in H5O*() routines.
        - Simplify metadata cache entry comparison routine (used when
            flushing entire cache out).
        - Only enable metadata cache statistics when H5AC_DEBUG is turned
            on, instead of always tracking them.
        - Simplify address comparison macro (H5F_addr_eq).
        - Remove redundant metadata cache entry protections during dataset
            creation by protecting the object header once and making all
            the modifications necessary for the dataset creation before
            unprotecting it.
        - Reduce # of "number of element in extent" computations performed
            by computing and storing the value during dataspace creation.
        - Simplify checking for group location's file information, when file
            has not been involving in file-mounting operations.
        - Use binary encoding for modification time, instead of ASCII.
        - Hoist H5HL_peek calls (to get information in a local heap)
            out of loops in many group routine.
        - Use static variable for iterators of selections, instead of
            dynamically allocation them each time.
        - Lookup & insert new entries in one step, avoiding traversing
            group's B-tree twice.
        - Fixed memory leak in H5Gget_objname_idx() routine (tangential to
            performance improvements, but fixed along the way).
        - Use free-list for reference counted strings.
        - Don't bother copying object names into cached group entries,
            since they are re-created when an object is opened.

        The benchmark I used to measure these results created several thousand
        small (2K) datasets in a file and wrote out the data for them.  This is
        Elena's "regular.c" benchmark.

        These changes resulted in approximately ~4.3x speedup of the
        development branch when compared to the previous code in the
        development branch and ~1.4x speedup compared to the release
        branch.

        Additionally, these changes reduce the total memory used (code and
        data) by the development branch by ~800KB, bringing the development
        branch back into the same ballpark as the release branch.

        I'll send out a more detailed description of the benchmark results
        as a followup note.

    New internal API routines:
        Added "reference counted strings" API for tracking strings that get
            used by multiple owners without duplicating the strings.
        Added "ternary search tree" API for text->object mappings.

Platforms tested:
    Tested h5committest {arabica (fortran), eirene (fortran, C++)
	modi4 (parallel, fortran)}
    Other platforms/configurations tested?
        FreeBSD 4.7 (sleipnir) serial & parallel
        Solaris 2.6 (baldric) serial
2003-01-09 12:20:03 -05:00

280 lines
6.3 KiB
C
Raw Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/*
* Copyright © 1998 NCSA
* All rights reserved.
*
* Programmer: Robb Matzke <matzke@llnl.gov>
* Tuesday, November 24, 1998
*/
#include "h5test.h"
#include "H5Iprivate.h"
#include "H5Oprivate.h"
/*
* This file needs to access private datatypes from the H5G package.
*/
#define H5G_PACKAGE
#include "H5Gpkg.h"
const char *FILENAME[] = {
"ohdr",
NULL
};
/*-------------------------------------------------------------------------
* Function: main
*
* Purpose:
*
* Return: Success:
*
* Failure:
*
* Programmer: Robb Matzke
* Tuesday, November 24, 1998
*
* Modifications:
*
*-------------------------------------------------------------------------
*/
int
main(void)
{
hid_t fapl=-1, file=-1;
H5F_t *f=NULL;
char filename[1024];
H5G_entry_t oh_ent;
H5O_stab_t stab, ro;
int i;
/* Reset library */
h5_reset();
fapl = h5_fileaccess();
h5_fixname(FILENAME[0], fapl, filename, sizeof filename);
if ((file=H5Fcreate(filename, H5F_ACC_TRUNC, H5P_DEFAULT, fapl))<0)
goto error;
if (NULL==(f=H5I_object(file))) {
H5Eprint(stdout);
goto error;
}
/*
* Test object header creation
*/
TESTING("object header creation");
HDmemset(&oh_ent,0,sizeof(H5G_entry_t));
if (H5O_create(f, 64, &oh_ent/*out*/)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
PASSED();
/* create a new message */
TESTING("message creation");
stab.btree_addr = 11111111;
stab.heap_addr = 22222222;
if (H5O_modify(&oh_ent, H5O_STAB, H5O_NEW_MESG, 0, 1, &stab)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (H5AC_flush(f, NULL, HADDR_UNDEF, TRUE)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (NULL==H5O_read(&oh_ent, H5O_STAB, 0, &ro)) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (H5F_addr_ne(ro.btree_addr, stab.btree_addr) ||
H5F_addr_ne(ro.heap_addr, stab.heap_addr)) {
H5_FAILED();
HDfprintf(stdout, " got: {%a, %a}\n",
ro.btree_addr, ro.heap_addr);
HDfprintf(stdout, " ans: {%a, %a}\n",
stab.btree_addr, stab.heap_addr);
goto error;
}
PASSED();
/*
* Test modification of an existing message.
*/
TESTING("message modification");
stab.btree_addr = 33333333;
stab.heap_addr = 44444444;
if (H5O_modify(&oh_ent, H5O_STAB, 0, 0, 1, &stab)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (H5AC_flush(f, NULL, HADDR_UNDEF, TRUE)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (NULL==H5O_read(&oh_ent, H5O_STAB, 0, &ro)) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (H5F_addr_ne(ro.btree_addr, stab.btree_addr) ||
H5F_addr_ne(ro.heap_addr, stab.heap_addr)) {
H5_FAILED();
HDfprintf(stdout, " got: {%a, %a}\n",
ro.btree_addr, ro.heap_addr);
HDfprintf(stdout, " ans: {%a, %a}\n",
stab.btree_addr, stab.heap_addr);
goto error;
}
PASSED();
/*
* Test creation of a second message of the same type.
*/
TESTING("duplicate message creation");
stab.btree_addr = 55555555;
stab.heap_addr = 66666666;
if (H5O_modify(&oh_ent, H5O_STAB, H5O_NEW_MESG, 0, 1, &stab)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (H5AC_flush(f, NULL, HADDR_UNDEF, TRUE)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (NULL==H5O_read(&oh_ent, H5O_STAB, 1, &ro)) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (H5F_addr_ne(ro.btree_addr, stab.btree_addr) ||
H5F_addr_ne(ro.heap_addr, stab.heap_addr)) {
H5_FAILED();
HDfprintf(stdout, " got: {%a, %a}\n",
ro.btree_addr, ro.heap_addr);
HDfprintf(stdout, " ans: {%a, %a}\n",
stab.btree_addr, stab.heap_addr);
goto error;
}
PASSED();
/*
* Test modification of the second message with a symbol table.
*/
TESTING("duplicate message modification");
stab.btree_addr = 77777777;
stab.heap_addr = 88888888;
if (H5O_modify(&oh_ent, H5O_STAB, 1, 0, 1, &stab)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (H5AC_flush(f, NULL, HADDR_UNDEF, TRUE)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (NULL==H5O_read(&oh_ent, H5O_STAB, 1, &ro)) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (H5F_addr_ne(ro.btree_addr, stab.btree_addr) ||
H5F_addr_ne(ro.heap_addr, stab.heap_addr)) {
H5_FAILED();
HDfprintf(stdout, " got: {%a, %a}\n",
ro.btree_addr, ro.heap_addr);
HDfprintf(stdout, " ans: {%a, %a}\n",
stab.btree_addr, stab.heap_addr);
goto error;
}
PASSED();
/*
* Test creation of a bunch of messages one after another to see
* what happens when the object header overflows in core.
*/
TESTING("object header overflow in memory");
for (i=0; i<40; i++) {
stab.btree_addr = (i+1)*1000+1;
stab.heap_addr = (i+1)*1000+2;
if (H5O_modify(&oh_ent, H5O_STAB, H5O_NEW_MESG, 0, 1, &stab)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
}
if (H5AC_flush(f, NULL, HADDR_UNDEF, TRUE)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
PASSED();
/*
* Test creation of a bunch of messages one after another to see
* what happens when the object header overflows on disk.
*/
TESTING("object header overflow on disk");
for (i=0; i<10; i++) {
stab.btree_addr = (i + 1) * 1000 + 10;
stab.heap_addr = (i + 1) * 1000 + 20;
if (H5O_modify(&oh_ent, H5O_STAB, H5O_NEW_MESG, 0, 1, &stab)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (H5AC_flush(f, NULL, HADDR_UNDEF, TRUE)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
}
PASSED();
/*
* Delete all symbol table messages.
*/
TESTING("message deletion");
if (H5O_remove(&oh_ent, H5O_STAB, H5O_ALL)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (H5O_read(&oh_ent, H5O_STAB, 0, &ro)) {
H5_FAILED();
puts(" H5O_read() should have failed but didn't");
H5Eclear();
goto error;
}
PASSED();
/* release resources */
TESTING("object header closing");
if (H5O_close(&oh_ent)<0) {
H5_FAILED();
H5Eprint(stdout);
goto error;
}
if (H5Fclose(file)<0) goto error;
PASSED();
puts("All object header tests passed.");
h5_cleanup(FILENAME, fapl);
return 0;
error:
H5E_BEGIN_TRY {
H5Fclose(file);
} H5E_END_TRY;
return 1;
}
Reference in New Issue View Git Blame Copy Permalink