Contents |
General Remarks
HDF5 is a library for managing large numerical data sets. It is appropriate for saving and retrieving data to/from structured large files. It also supports parallel access to files in HDF5 format, in particular within the MPI environment. HDF5 library can be used from C, C++ and Fortran. Files can be saved optionally in text format, binary and compressed format (if zlib is available).
HDF5 is recommended by the developers of FFTW http://www.fftw.org/fftw3.3alpha_doc/Simple-MPI-example.html as a means to output and synchronize data from different processes.
Some popular numerical packages, like MATLAB, Mathematica, Octave and ROOT, already have native support for the HDF5 format. If you want to quickly experiment with HDF5 files, you can use those programs to see how it works.
For example, in Mathematica
m = RandomInteger[255, {5, 5}] Export[ "matrix.h5", m] mLoad = Import["matrix.h5", {"Datasets", "/Dataset1"}]
will create a binary file called "matrix.h5" with the matrix data.
Install
The following are instructions to install HDF5 in different systems. Note that, as of HDF5 version 1.9 there is no way to use the MPI version and the C++ interfaces together.
Ubuntu
HDF5 1.6.6 can be installed directly in Ubuntu 8.10 be doing:
sudo apt-get install libhdf5-serial-dev
This will install the C, C++ and Fortran versions of the library and development (header) files, but it will not include the MPI version.
The MPI version, (which will remove the previous serial version) can be installed by:
sudo apt-get install libhdf5-mpich-dev
where 'mpich' can be replaced by 'openmpi' or 'lam'. Unfortunately then the C++ interface is not provided for this MPI version. The two version are incompatible.
Build and Installation from Sources
We will try to install the parallel version of HDF5 1.9 in our user space. (HDF5 1.8 --official release-- does not play well compiling with gcc4.)
mk $HOME/usr
from a download location
mk $HOME/soft cd $HOME/soft wget ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v19/hdf5-1.9.29.tar.gz tar -zxvf hdf5-1.9.29.tar.gz cd hdf5-1.9.29
then we can configure:
CC=mpicc ./configure --prefix=$HOME/usr --enable-parallel
Other options are described in ./configure --help. The opton --enable-cxx can be specified but not together with --enable-parallel. For non-parallel version adding "CC=mpicc" (or equivalent) is not necessary. Check that your MPI compiler by doing, for example:
which mpicc
The we can make and install
make make install
The compilation takes ~5 minutes and several warning messages will appear. Many header files will be installed in ~/usr/include and ~/usr/lib
~/usr/include/H5*.h (around 40 files) ~/usr/include/hdf5[|_hl].h ~/usr/lib/libhdf5[|_hl].[a|la]
The most for us are hdf5.h and libhdf5.a.