Revision as of 13:47, 1 October 2008 (view source) Correaa (Talk) (→‎Installation in Ubuntu) ← Older edit

Revision as of 13:50, 1 October 2008 (view source) Correaa (Talk) (→‎General Remarks) Newer edit →

---

Revision as of 13:50, 1 October 2008

In this document, we describe how to compile QBOX program on wcr.stanford.edu (Linux parallel cluster).

I designed this wiki such that you could in principle copy and paste the code part in your terminal. Custom files are linked from this wiki and can be downloaded from the command line with 'wget'. I encourage to do this to other writers of the group wiki. In any case, be aware of error messages, please report if something it is not working from this instructions or edit this wiki for other obvious typos.

Contents 1 General Remarks 2 Install FFTW 3 Install BLACS 4 Install Scalapack 5 Install Xerces 6 Ready to Install Qbox 7 Installation in Ubuntu

General Remarks

Based on the Qbox build instructions page we will give some additional details. The difficult part of the installation is to correctly build the dependence libraries. In particular

1. BLACS http://www.netlib.org/blacs
2. ScaLAPACK http://www.netlib.org/scalapack
3. FFTW 2.1.5 http://www.fftw.org
4. Apache Xerces-C http://xerces.apache.org/xerces-c

these libraries will have to be installed in your user space directory. To achieve this I use to create a directory called ~/usr where libraries and header files will stay:

 mk $HOME/usr
 mk $HOME/usr/local
 mk $HOME/local/bin

I also use to download software packages to ~/soft. This will make easier to establish absolute compilation paths for BLACS and Scalapack which have a non-standard installation procedure.

 mk $HOME/soft

I will assume that

1. BLAS (or ATLAS) http://www.netlib.org or http://math-atlas.sourceforge.net
2. LAPACK http://www.netlib.org/lapack

are installed system wide already (as they are currently in wcr.stanford).

This document does not cover the usage of Qbox but just the compilation, a manual of Qbox is provided in the Qbox home page.

Install FFTW

The easiest library to install is FFTW 2.1.5 (FFTW3 is available but to the best of my knowledge incompatible with FFTW2). Simple download it from the FFTW download page and decompress it:

 cd ~/soft
 wget http://www.fftw.org/fftw-2.1.5.tar.gz
 tar -zxvf fftw-2.1.5.tar.gz
 cd fftw-2.1.5

Then we do the usual configure and make install procedure, also indicating that we want a private installation in our ~/usr directory

 ./configure --prefix=$HOME/usr
 make
 make install

This will install the FFTW2 libraries in ~/usr/include and ~/usr/lib.

Install BLACS

Next download and untar BLACS from BLACS papers page:

 cd ~/soft
 wget http://www.netlib.org/blacs/mpiblacs.tgz
 tar -zxvf mpiblacs.tgz
 cd BLACS

We have to have our own version of a file called Bmake.inc in the ~/soft/BLACS directory. The following file will serve for BLACS/Bmake.inc

 wget http://micro.stanford.edu/mediawiki-1.11.0/images/BLACS_Bmake.inc.txt -O Bmake.inc

The file is just the same as BMAKES/Bmake.MPI-LINUX with the following changes in the variables definition:

 < BTOPdir = $(HOME)/soft/BLACS
 < MPIdir = /opt/mpich/gnu

Then change to the source directory and make, then copy the compiled libraries to ~/usr/local/lib/

 cd SRC/MPI
 make
 cp ~/soft/BLACS/LIB/*.a ~/usr/local/lib

Install Scalapack

Now it is the turn for Scalapack, get it from the Scalapack home page, untar it

 cd ~/soft
 wget http://www.netlib.org/scalapack/scalapack-1.8.0.tgz
 tar -zxvf scalapack-1.8.0
 cd scalapack

and put this compilation input file scalapack/SLmake.inc in the directory

 wget http://micro.stanford.edu/mediawiki-1.11.0/images/Scalapack_SLmake.inc.txt -O SLmake.inc

which is just the same as SLmake.inc.example with the following changes:

 < home          = $(HOME)/soft/scalapack-1.8.0
 < F77           = /opt/mpich/gnu/bin/mpif77
 < CC            = /opt/mpich/gnu/bin/mpicc

then make and copy the resulting library to ~/usr/local/lib

 cp ~/soft/scalapack-1.8.0/libscalapack.a ~/usr/local/lib

Install Xerces

Now the Xerces XML library, get it from the Xerces download page and untar it, also define an enviroment variable were the source of xerces is located:

 wget http://apache.siamwebhosting.com/xerces/c/2/sources/xerces-c-src_2_8_0.tar.gz
 tar -zxvf xerces-c-src_2_8_0.tar.gz
 cd xerces-c-src_2_8_0
 export XERCESCROOT=$PWD
 cd src/xercesc

then run this script and the usual make install

 ./runConfigure -plinux -cgcc -xg++ -minmem -nsocket -tnative -rpthread -P $HOME/usr
 make
 make install

(there are some clues that indicate that we may need to use -rnone but it is not for sure.) This will make xerces installed in ~/usr/lib and ~/usr/include/xerces.

Note that this will produce the dynamic version of the libxerces-c libraries. For the static version add the -s option, you can have both installed by repeating the last step with the -s option for runConfigure.

Ready to Install Qbox

Having the previous libraries installed in our own directory ~/usr and the rest of them already installed by the system administrator we shouldn't have any problems compiling Qbox

First step is to get the source code from Qbox download page and decompress it.

 cd ~/soft
 wget http://fpmd.ucdavis.edu/software/qbox/qbox-1.45.0.tgz
 tar -zxvf qbox-1.45.0.tgz
 cd qbox-1.45.0/src 

We have to have a makefile input file for our own system (wcr) in the current directory; for that we have created qbox-1.45.0/src/wcr.mk, we will also need to specify our target system (according to the filename wcr.mk)

 wget http://micro.stanford.edu/mediawiki-1.11.0/images/Qbox_wcr.mk.txt -O wcr.mk
 export TARGET=wcr

wcr.mk file distributed here is based on x8664_gcc.mk with the following changes:

 <  MPIDIR=/opt/mpich/gnu
 <  XERCESCDIR=$(HOME)/usr
 <  FFTWDIR=$(HOME)/usr
 <  BLASDIR=$(HOME)/usr/local/lib
 <  LAPACKDIR=$(HOME)/usr/local/lib
 <  CXX=/opt/mpich/gnu/bin/mpicxx
 <          -llapack -lf77blas -latlas -lm  \
 <          -Xlinker -lxerces-c -Bstatic \
 <           -lc -static-libgcc -lmpich  \
 <  # gfortran
 <  PLAT=LINUX
 <  BLACSdir      = $(HOME)/usr/local/lib
 <  SCALAPACK_DIR = $(HOME)/usr/local/lib

Note the subtle change, by putting -lxerces-c before -Bstatic, the linking will be done to the dynamic version of libxerces-c which is the one available after the compilation of xerces and in most systems. For compiling the static version of Xerces see last note in "Xerces Compilation" section above.

The following is not documented but very useful to free us from defining system variables in out rc files such as 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/usr/lib:$HOME/usr/local/lib'. We have to define this variable BEFORE compilation occurs (there is no need to define it before each run of Qbox):

 export LD_RUN_PATH=~/usr/lib

This will enforce the compiler to store the absolute location of the libraries (e.g. xerces) in the executable itself. Now we are ready to make

 make

This can take a couple of minutes and generate an executable named 'qb' . We can check that the qb is properly compiler by doing

 ldd ./qb

and see whether all necessary libraries can be found in the system. Afterwards we can try to run a small example,

 cd ~/soft/qbox-1.45.0/examples/ch4
 /opt/mpich/gnu/bin/mpirun -np 2 ~/soft/qbox-1.45.0/src/qb gs.i

If it works we can copy the executable to our bin directory. Since the qbox versions change rather often it is convenient to keep track of the executable version and make a link to the latest version available.

 cp ~/soft/qbox-1.45.0/src ~/usr/local/bin/qbox-1.45.0
 ln -s ~/usr/local/bin/qbox-1.45.0 ~/usr/local/bin/qbox

Installation in Ubuntu

This part is about the installation in Ubuntu. All necessary libraries are provided by the Ubuntu repositories. Therefore for a quick local installation for testing and debugging it can be useful to just install the compiled libraries. I tryed as much as possible to use libraries already provided by Ubuntu, and follow Ubuntu (Debian?) conventions.

First install the necessary compiler (or compiler script) and development files:

 sudo apt-get install mpich-bin libmpich1.0gf libmpich1.0-dev mpi-doc
 sudo apt-get install fftw-dev fftw2
 sudo apt-get install libxerces28 libxerces28-dev libxerces28-doc
 sudo apt-get install blacs1gf-mpich 
 sudo apt-get install scalapack1-mpich scalapack-doc

All the necessary libraries and compilers should be installed by now. We still have to tell the makefile were the libraries are installed in Ubuntu; for that we have created qbox-1.45.0/src/x8664_gcc_ubuntu.mk, we will also need to specify our target system (according to the filename)

 wget http://micro.stanford.edu/mediawiki-1.11.0/images/qbox_x8664_gcc_mpi_ubuntu.mk.txt -O x8664_gcc_mpi_ubuntu.mk
 export TARGET=x8664_gcc_mpi_ubuntu

x8664_gcc_ubuntu.mk file distributed here is based on x8664_gcc.mk with the following changes:

 <  MPIDIR=/usr/lib/mpich
 <  XERCESCDIR=/usr/lib
 <  FFTWDIR=/usr/lib
 <  BLASDIR=/usr/lib
 <  LAPACKDIR=/usr/lib
 <  CXX=/usr/lib/mpich/bin/mpiCC
 <          -Xlinker -lxerces-c -Bstatic \
 <           -lc -lgfortran -static-libgcc -lmpich  \
 <          -Xlinker -Bdynamic 
 <  BLACSdir      = /usr/lib
 <  BLACSFINIT    = $(BLACSdir)/libblacsF77init-mpich.so.1
 <  BLACSCINIT    = $(BLACSdir)/libblacsCinit-mpich.so.1
 <  BLACSLIB      = $(BLACSdir)/libblacs-mpich.so.1
 <  SCALAPACK_DIR = /usr/lib
 <  SCALAPACKLIB  = $(SCALAPACK_DIR)/libscalapack-mpich.so.1

Now we can compile

 make

A buch of messages 'warning: deprecated conversion from string constant to ‘char*’' will appear and can be ignored.

And then check proper library linking for the newly generated executable 'qb'

 ldd ./qb

supposedly this will two parallel processes of qb in the current machine:

/usr/lib/mpich/bin/mpirun -np 2 ./qb

we can check this by checking the output line <mpi_processes count="2">. In principle this can be useful in a machine with dual core.

(end of compilation tutorial)