(→Example) |
(→Test Example) |
Revision as of 22:46, 3 February 2009
Contents |
General Remarks
HDF5 is a library for managing large numerical data sets. It is appropriate for saving and retrieving data to/from structured large files. It also supports parallel access to files in HDF5 format, in particular within the MPI environment. HDF5 library can be used from C, C++ and Fortran. Files can be saved optionally in text format, binary and compressed format (if zlib is available).
HDF5 is recommended by the developers of FFTW http://www.fftw.org/fftw3.3alpha_doc/Simple-MPI-example.html as a means to output and synchronize data from different processes.
Some popular numerical packages, like MATLAB, Mathematica, Octave and ROOT, already have native support for the HDF5 format. If you want to quickly experiment with HDF5 files, you can use those programs to see how it works.
For example, in Mathematica
m = RandomInteger[255, {5, 5}] Export[ "matrix.h5", m] mLoad = Import["matrix.h5", {"Datasets", "/Dataset1"}]
will create a binary file called "matrix.h5" with the matrix data.
Install
The following are instructions to install HDF5 in different systems. Note that, as of HDF5 version 1.9 there is no way to use the MPI version and the C++ interfaces together.
Ubuntu
HDF5 1.6.6 can be installed directly in Ubuntu 8.10 be doing:
sudo apt-get install libhdf5-serial-dev
This will install the C, C++ and Fortran versions of the library and development (header) files, but it will not include the MPI version.
The MPI version, (which will remove the previous serial version) can be installed by:
sudo apt-get install libhdf5-mpich-dev
where 'mpich' can be replaced by 'openmpi' or 'lam'. Unfortunately then the C++ interface is not provided for this MPI version. The two version are incompatible.
Build and Installation from Sources
We will try to install the parallel version of HDF5 1.9 in our user space. (HDF5 1.8 --official release-- does not play well compiling with gcc4.)
mk $HOME/usr
from a download location
mk $HOME/soft cd $HOME/soft wget ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v19/hdf5-1.9.29.tar.gz tar -zxvf hdf5-1.9.29.tar.gz cd hdf5-1.9.29
then we can configure:
CC=mpicc ./configure --prefix=$HOME/usr --enable-parallel
Other options are described in ./configure --help. The opton --enable-cxx can be specified but not together with --enable-parallel. For non-parallel version adding "CC=mpicc" (or equivalent) is not necessary. Check that your MPI compiler by doing, for example:
which mpicc
The we can make and install
make make install
The compilation takes ~5 minutes and several warning messages will appear. Many header files will be installed in ~/usr/include and ~/usr/lib
~/usr/include/H5*.h (around 40 files) ~/usr/include/hdf5[|_hl].h ~/usr/lib/libhdf5[|_hl].[a|la]
The most for us are hdf5.h and libhdf5.a.
Test Example
The source files contain examples on the usage of HDF5, including C++ examples. (See directories ./examples, ./hl/examples, ./c++/examples. and ./hl/c++/examples)
Simple introductory examples are also provided online, but they outdated and are slightly incompatible with this version which can be very confusing. It is better to use the examples contained in the distribution file and use the online documentation to read the details of the examples. In any case here I provide the sources I used for testing the installation h5_test.tar.gz. Use the example as follows:
wget -O h5_test.tar.gz tar -zxvf h5_test.tar.gz cd h5_test make test
A write and read program will be compiled. The write will create a binary file named SDS.h5 with the data of a certain array, then this array will be read from the file and printed.