h5py

Running

pip install h5py

gives the error message

h5py/defs.c:8:22: fatal error: pyconfig.h: No such file or directory

this was due to the python development package not being installed.

Subsequently running

pip install h5py

throws the error message

/usr/include/H5public.h:57:20: fatal error: mpi.h: No such file or directory

this was due to pip not finding the mpi package. It can be resolved by specifying the CFLAGS shell variable.

Eventually running

env CFLAGS=-I/usr/lib/openmpi/include pip install h5py

did the trick.

An alternative could be

pip install --global-option "env CFLAGS=-I/usr/lib/openmpi/include" h5py

The alternative did not work for me (Manuel, python2, @ww8stud6) — Manuel Leimberger 2014/08/28 11:19

Required is a HDF5 library compiled with MPI. Ask your admin or compile it yourself. If that's the case, execute this is in your bash:

export CC=mpicc
export HDF5_MPI="ON"
pip install h5py --no-binary=h5py h5py

If it has finished without any (major) complaints, test it with this script:

from mpi4py import MPI
import h5py
 
rank = MPI.COMM_WORLD.Get_rank()
N = MPI.COMM_WORLD.Get_size()
 
f = h5py.File('parallel.hdf5', 'w', driver='mpio', comm=MPI.COMM_WORLD)
dset = f.create_dataset('test', (N,), dtype='i')
dset[rank] = rank
f.close()

by executing: </code> If it has finished without any (major) complaints, test it with this script:

mpiexec -n <number of processors on your computer> python test-h5py-parallel.py

You should have now a hdf5 file of the name “parallel.hdf5” with one dataset inside with the number of processors in c-indexing in sequential order. So for four processors the entries are 0,1,2,3.