philipp/hdf5
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[svn-r1736] HDF5 Tutorial: First version of Tutorial to be checked in for distribution.

Browse Source
This commit is contained in:
Frank Baker
1999-10-08 18:03:05 -05:00
parent 59b430fbdd
commit 3c2fc7a967
75 changed files with 6823 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

314
doc/html/Tutor/extend.html Normal file
View File

@@ -0,0 +1,314 @@
<HTML><HEAD>
<TITLE>HDF5 Tutorial - Chunking and Extendible Datasets
</TITLE>
</HEAD>
<body bgcolor="#ffffff">
<!-- BEGIN MAIN BODY -->
<A HREF="http://www.ncsa.uiuc.edu/"><img border=0
src="http://www.ncsa.uiuc.edu/Images/NCSAhome/footerlogo.gif"
width=78 height=27 alt="NCSA"><P></A>
[ <A HREF="title.html"><I>HDF5 Tutorial Top</I></A> ]
<H1>
<BIG><BIG><BIG><FONT COLOR="#c101cd">Chunking and Extendible Datasets</FONT>
</BIG></BIG></BIG></H1>
<hr noshade size=1>
<BODY>
<H2>Contents:</H2>
<UL>
<LI><A HREF="#def">Creating an Extendible Dataset</A>
<LI>Programming Example
<UL>
<LI> <A HREF="#desc">Description</A>
<LI> <A HREF="#rem">Remarks</A>
<!--
<LI> <A HREF="#fc">File Contents</A>
<LI> <A HREF="#ddl">Dataset Definition in DDL</A>
-->
</UL>
</UL>
<HR>
<A NAME="def">
<H2>Creating an Extendible Dataset</H2>
An extendible dataset is one whose dimensions can grow. In HDF5, it is possible to define a dataset to have certain initial dimensions, then later
to increase the size of any of the initial dimensions.
<P>
HDF5 requires you to use chunking in order to define extendible datasets. Chunking makes it possible to extend datasets efficiently, without
having to reorganize storage excessively.
<P>
The following operations are required in order to write an extendible dataset:
<OL>
<LI>Declare the dataspace of the dataset to have unlimited dimensions for all dimensions that might eventually be extended.
<LI>Set dataset creation properties to enable chunking and create a dataset.
<LI>Extend the size of the dataset.
</OL>
<H2> Programming Example</H2>
<A NAME="desc">
<H3><U>Description</U></H3>
This example shows how to create a 3 x 3 extendible dataset, write to that
dataset, extend the dataset to 10x3, and write to the dataset again.
[<A HREF="examples/h5_extend.c">Download h5_extend.c</A>]
<PRE>
/**************************************************************
*
* This example shows how to work with extendible datasets.
* In the current version of the library a dataset MUST be
* chunked in order to be extendible.
*
* This example is derived from the h5_extend_write.c and
* h5_read_chunk.c examples that are in the "Introduction
* to HDF5".
*
*************************************************************/
#include "hdf5.h"
#define FILE "ext.h5"
#define DATASETNAME "ExtendibleArray"
#define RANK 2
int
main (void)
{
hid_t file; /* handles */
hid_t dataspace, dataset;
hid_t filespace;
hid_t cparms;
hid_t memspace;
hsize_t dims[2] = { 3, 3}; /* dataset dimensions
at creation time */
hsize_t dims1[2] = { 3, 3}; /* data1 dimensions */
hsize_t dims2[2] = { 7, 1}; /* data2 dimensions */
hsize_t maxdims[2] = {H5S_UNLIMITED, H5S_UNLIMITED};
hsize_t size[2];
hssize_t offset[2];
hsize_t i,j;
herr_t status, status_n;
int data1[3][3] = { {1, 1, 1}, /* data to write */
{1, 1, 1},
{1, 1, 1} };
int data2[7] = { 2, 2, 2, 2, 2, 2, 2};
/* Variables used in reading data back */
hsize_t chunk_dims[2] ={2, 5};
hsize_t chunk_dimsr[2];
hsize_t dimsr[2];
int data_out[10][3];
int rank, rank_chunk;
/* Create the data space with unlimited dimensions. */
dataspace = H5Screate_simple (RANK, dims, maxdims);
/* Create a new file. If file exists its contents will be overwritten. */
file = H5Fcreate (FILE, H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT);
/* Modify dataset creation properties, i.e. enable chunking */
cparms = H5Pcreate (H5P_DATASET_CREATE);
status = H5Pset_chunk ( cparms, RANK, chunk_dims);
/* Create a new dataset within the file using cparms
creation properties. */
dataset = H5Dcreate (file, DATASETNAME, H5T_NATIVE_INT, dataspace,
cparms);
/* Extend the dataset. This call assures that dataset is 3 x 3.*/
size[0] = 3;
size[1] = 3;
status = H5Dextend (dataset, size);
/* Select a hyperslab */
filespace = H5Dget_space (dataset);
offset[0] = 0;
offset[1] = 0;
status = H5Sselect_hyperslab (filespace, H5S_SELECT_SET, offset, NULL,
dims1, NULL);
/* Write the data to the hyperslab */
status = H5Dwrite (dataset, H5T_NATIVE_INT, dataspace, filespace,
H5P_DEFAULT, data1);
/* Extend the dataset. Dataset becomes 10 x 3 */
dims[0] = dims1[0] + dims2[0];
size[0] = dims[0];
size[1] = dims[1];
status = H5Dextend (dataset, size);
/* Select a hyperslab */
filespace = H5Dget_space (dataset);
offset[0] = 3;
offset[1] = 0;
status = H5Sselect_hyperslab (filespace, H5S_SELECT_SET, offset, NULL,
dims2, NULL);
/* Define memory space */
dataspace = H5Screate_simple (RANK, dims2, NULL);
/* Write the data to the hyperslab */
status = H5Dwrite (dataset, H5T_NATIVE_INT, dataspace, filespace,
H5P_DEFAULT, data2);
/* Close resources */
status = H5Dclose (dataset);
status = H5Sclose (dataspace);
status = H5Sclose (filespace);
status = H5Fclose (file);
/****************************************************************
Read the data back
***************************************************************/
file = H5Fopen (FILE, H5F_ACC_RDONLY, H5P_DEFAULT);
dataset = H5Dopen (file, DATASETNAME);
filespace = H5Dget_space (dataset);
rank = H5Sget_simple_extent_ndims (filespace);
status_n = H5Sget_simple_extent_dims (filespace, dimsr, NULL);
cparms = H5Dget_create_plist (dataset);
if (H5D_CHUNKED == H5Pget_layout (cparms))
{
rank_chunk = H5Pget_chunk (cparms, 2, chunk_dimsr);
}
memspace = H5Screate_simple (rank,dimsr,NULL);
status = H5Dread (dataset, H5T_NATIVE_INT, memspace, filespace,
H5P_DEFAULT, data_out);
printf("\n");
printf("Dataset: \n");
for (j = 0; j &lt; dimsr[0]; j++)
{
for (i = 0; i &lt; dimsr[1]; i++)
printf("%d ", data_out[j][i]);
printf("\n");
}
status = H5Pclose (cparms);
status = H5Dclose (dataset);
status = H5Sclose (filespace);
status = H5Sclose (memspace);
status = H5Fclose (file);
}
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
</PRE>
<A NAME="rem">
<H3><U>Remarks</U></H3>
<P>
<UL>
<LI>The function H5Pcreate creates a new property as an instance of
a property list. The signature of this function is as follows:
<PRE>
hid_t H5Pcreate ( H5P_class_t type )
</PRE>
<UL>
<LI>The parameter <I>type</I> is the type of property list to create.<BR>
The class types are: H5P_FILE_CREATE, H5P_FILE_ACCESS, H5P_DATASET_CREATE,
H5P_DATASET_XFER, and H5P_MOUNT
</UL>
<P>
<LI>The function H5Pset_chunk sets the size of the chunks used
to store a chunked layout dataset.
The signature of this function is as follows:
<PRE>
herr_t H5Pset_chunk ( hid_t plist, int ndims, const hsize_t * dim )
</PRE>
<UL>
<LI>The first parameter, <I>plist</I>, is the identifier for the property
list to query.
<LI>The second parameter, <I>ndims</I>, is the number of dimensions of
each chunk.
<LI>The third parameter, <I>dim</I>, is an array containing the size of
each chunk.
</UL>
<P>
A non-negative value is returned if successful; otherwise a negative
value is returned.
<P>
<LI>The function H5Dextend extends a dataset that has an unlimited
dimension. The signature is as follows:
<PRE>
herr_t H5Dextend ( hid_t dataset_id, const hsize_t * size )
</PRE>
<UL>
<LI>The first parameter, <I>dataset_id</I>, is the identifier of
the dataset.
<LI>The second parater, <I>size</I>, is an array containing the
new magnitude of each dimension.
</UL>
<P>
This function returns a non-negative value if successful; otherwise
it returns a negative value.
<P>
<LI>The H5Dget_create_plist function returns an identifier for a
copy of the dataset creation property list for a dataset.
<P>
<LI>The H5Pget_layout function returns the layout of the raw data for a
dataset. Valid types are H5D_COMPACT, H5D_CONTIGUOUS, and H5D_CHUNKED.
<P>
<LI>The H5Pget_chunk function retrieves the size of chunks for the
raw data of a chunked layout dataset.
The signature of this function is:
<PRE>
int H5Pget_chunk ( hid_t plist, int max_ndims, hsize_t * dims )
</PRE>
<UL>
<LI>The first parameter, <I>plist</I>, is the identifier of the
property list to query.
<LI>The second parameter, <I>max_ndims</I>, is the size of the <I>dims</I>
array.
<LI>The third parameter, <I>dims</I>, is the array to store the chunk
dimensions
</UL>
<P>
<LI>The H5Pclose function terminates access to a property list.
The signature of this function is:
<PRE>
herr_t H5Pclose ( hid_t plist )
</PRE>
where <I>plist</I> is the identifier of the property list to terminate
access to.
</UL>
<!--
<A NAME="fc">
<H3><U>File Contents</U></H3>
-->
<!-- BEGIN FOOTER INFO -->
<P><hr noshade size=1>
<font face="arial,helvetica" size="-1">
<a href="http://www.ncsa.uiuc.edu/"><img border=0
src="http://www.ncsa.uiuc.edu/Images/NCSAhome/footerlogo.gif"
width=78 height=27 alt="NCSA"><br>
The National Center for Supercomputing Applications</A><br>
<a href="http://www.uiuc.edu/">University of Illinois
at Urbana-Champaign</a><br>
<br>
<!-- <A HREF="helpdesk.mail.html"> -->
<A HREF="mailto:hdfhelp@ncsa.uiuc.edu">
hdfhelp@ncsa.uiuc.edu</A>
<BR> <H6>Last Modified: August 27, 1999</H6><BR>
<!-- modified by Barbara Jones - bljones@ncsa.uiuc.edu -->
</FONT>
<BR>
<!-- <A HREF="mailto:hdfhelp@ncsa.uiuc.edu"> -->
</BODY>
</HTML>