(→Bulk modulus of Au) |
(→Computing Bulk Modulus of Au) |
Revision as of 13:48, 8 October 2008
In this document, we describe how to compile QBOX program on wcr.stanford.edu (Linux parallel cluster).
I designed this wiki such that you could in principle copy and paste the code part in your terminal. Custom files are linked from this wiki and can be downloaded from the command line with 'wget'. I encourage to do this to other writers of the group wiki. In any case, be aware of error messages, please report if something it is not working from this instructions or edit this wiki for other obvious typos.
Contents |
General Remarks
Based on the Qbox build instructions page we will give some additional details. The difficult part of the installation is to correctly build the dependence libraries. In particular
- BLACS http://www.netlib.org/blacs
- ScaLAPACK http://www.netlib.org/scalapack
- FFTW 2.1.5 http://www.fftw.org
- Apache Xerces-C http://xerces.apache.org/xerces-c
these libraries will have to be installed in your user space directory. To achieve this I use to create a directory called ~/usr where libraries and header files will stay:
mk $HOME/usr mk $HOME/usr/local mk $HOME/local/bin
I also use to download software packages to ~/soft. This will make easier to establish absolute compilation paths for BLACS and Scalapack which have a non-standard installation procedure.
mk $HOME/soft
I will assume that
- BLAS (or ATLAS) http://www.netlib.org or http://math-atlas.sourceforge.net
- LAPACK http://www.netlib.org/lapack
are installed system wide already (as they are currently in wcr.stanford).
This document does not cover the usage of Qbox but just the compilation, a manual of Qbox is provided in the Qbox home page.
Install FFTW
The easiest library to install is FFTW 2.1.5 (FFTW3 is available but to the best of my knowledge incompatible with FFTW2). Simple download it from the FFTW download page and decompress it:
cd ~/soft wget http://www.fftw.org/fftw-2.1.5.tar.gz tar -zxvf fftw-2.1.5.tar.gz cd fftw-2.1.5
Then we do the usual configure and make install procedure, also indicating that we want a private installation in our ~/usr directory
./configure --prefix=$HOME/usr make make install
This will install the FFTW2 libraries in ~/usr/include and ~/usr/lib.
Install BLACS
Next download and untar BLACS from BLACS papers page:
cd ~/soft wget http://www.netlib.org/blacs/mpiblacs.tgz tar -zxvf mpiblacs.tgz cd BLACS
We have to have our own version of a file called Bmake.inc in the ~/soft/BLACS directory. The following file will serve for BLACS/Bmake.inc
wget http://micro.stanford.edu/mediawiki-1.11.0/images/BLACS_Bmake.inc.txt -O Bmake.inc
The file is just the same as BMAKES/Bmake.MPI-LINUX with the following changes in the variables definition:
< BTOPdir = $(HOME)/soft/BLACS < MPIdir = /opt/mpich/gnu
Then change to the source directory and make, then copy the compiled libraries to ~/usr/local/lib/
cd SRC/MPI make cp ~/soft/BLACS/LIB/*.a ~/usr/local/lib
Install Scalapack
Now it is the turn for Scalapack, get it from the Scalapack home page, untar it
cd ~/soft wget http://www.netlib.org/scalapack/scalapack-1.8.0.tgz tar -zxvf scalapack-1.8.0.tgz cd scalapack-1.8.0
and put this compilation input file scalapack/SLmake.inc in the directory
wget http://micro.stanford.edu/mediawiki-1.11.0/images/Scalapack_SLmake.inc.txt -O SLmake.inc
which is just the same as 'SLmake.inc.example' with the following changes:
< home = $(HOME)/soft/scalapack-1.8.0 < F77 = /opt/mpich/gnu/bin/mpif77 < CC = /opt/mpich/gnu/bin/mpicc
then
make
and copy the resulting library to ~/usr/local/lib
cp ~/soft/scalapack-1.8.0/libscalapack.a ~/usr/local/lib
Install Xerces
Now the Xerces XML library, get it from the Xerces download page and untar it, also define an enviroment variable were the source of xerces is located:
wget http://apache.siamwebhosting.com/xerces/c/2/sources/xerces-c-src_2_8_0.tar.gz tar -zxvf xerces-c-src_2_8_0.tar.gz cd xerces-c-src_2_8_0 export XERCESCROOT=$PWD cd src/xercesc
then run this script and the usual make install
./runConfigure -plinux -cgcc -xg++ -minmem -nsocket -tnative -rpthread -P $HOME/usr -s make make install
This will make static xerces libraries installed in ~/usr/lib and ~/usr/include/xerces.
Ready to compile Qbox
Having the previous libraries installed in our own directory ~/usr and the rest of them already installed by the system administrator we shouldn't have any problems compiling Qbox
First step is to get the source code from Qbox download page and decompress it.
cd ~/soft wget http://fpmd.ucdavis.edu/software/qbox/qbox-1.45.0.tgz tar -zxvf qbox-1.45.0.tgz cd qbox-1.45.0/src
We have to have a makefile input file for our own system (wcr) in the current directory; for that we have created qbox-1.45.0/src/wcr.mk, we will also need to specify our target system (according to the filename wcr.mk)
wget http://micro.stanford.edu/mediawiki-1.11.0/images/Qbox_wcr.mk.txt -O wcr.mk export TARGET=wcr
wcr.mk file distributed here is based on x8664_gcc.mk with the following changes:
< MPIDIR=/opt/mpich/gnu < XERCESCDIR=$(HOME)/usr < FFTWDIR=$(HOME)/usr < BLASDIR=$(HOME)/usr/local/lib < LAPACKDIR=$(HOME)/usr/local/lib < CXX=/opt/mpich/gnu/bin/mpicxx < -llapack -lf77blas -latlas -lm \ < -Xlinker -Bstatic \ < -lc -static-libgcc -lmpich -lxerces-c \ < # gfortran < PLAT=LINUX < BLACSdir = $(HOME)/usr/local/lib < SCALAPACK_DIR = $(HOME)/usr/local/lib
Now we are ready to make
make
This can take a couple of minutes and generate an executable named 'qb' . We can check that the qb is properly compiler by doing
ldd ./qb
and see whether all necessary libraries can be found in the system. Afterwards we can try to run a small example,
cd ~/soft/qbox-1.45.0/examples/ch4 /opt/mpich/gnu/bin/mpirun -np 2 ~/soft/qbox-1.45.0/src/qb gs.i
If it works we can copy the executable to our bin directory. Since the qbox versions change rather often it is convenient to keep track of the executable version and make a link to the latest version available.
cp ~/soft/qbox-1.45.0/src ~/usr/local/bin/qbox-1.45.0 ln -s ~/usr/local/bin/qbox-1.45.0 ~/usr/local/bin/qbox
Since the libraries are statically linked 'qb' executable does not depend in any actual file in our /usr/lib.
Quick compilation in Ubuntu
It is possible to compile Qbox using the libraries and development file provided by Ubuntu. The downside is that it seems that it does not work with the amd64 version (either with the provided Scalapack or with a manually compiled one). In the 32bit version of Ubuntu just install the following packages from the standard repositories.
Install compilers and compiler libraries
sudo apt-get install cpp-4.2 g++-4.2 gcc-4.2 libstdc++6-4.2-dev sudo apt-get install gfortran-4.2
Install MPICH environment and intercommunication tools
sudo apt-get install libmpich1.0-dev libmpich1.0gf sudo apt-get install mpich-bin sudo apt-get install openssh-server ssh
Install numerical and parsing libraries
sudo apt-get install fftw-dev fftw2 sudo apt-get install libxerces-c2-dev libxerces-c28
Install parallel and serial linear algebra libraries
sudo apt-get install blacs1gf-mpich blacsgf-mpich-dev sudo apt-get install libblas3gf libgfortran3 sudo apt-get install liblapack3gf liblapack-dev sudo apt-get install scalapack-mpich-dev scalapack1-mpich
Now download the following prepared file qbox-1.45.0/src/Ubuntux86.mk and compile:
cd ~/soft/qbox-1.45.0 wget http://micro.stanford.edu/mediawiki-1.11.0/images/Qbox_ubuntux86.mk.txt -O Ubuntux86.mk export TARGET=Ubuntux86 make
after successful compilation run a small example
cd ../examples/ch4 /usr/lib/mpich/bin/mpirun -np 2 ../../qb gs.i
Notes: Qbox-<1.45 does not compile with g++-4.3 (this is the default compiler in Ubuntu 8.10), therefore g++-4.2 was explicitly specified in the provided makefile. Compilation warnings about 'char*' can be ignored. Some systems have MPI-LAM installed also, so make sure you use the mpich version of mpirun. If passwordless ssh is not enabled mpirun will request login password even for the localhost, to enable it follow the instruccion for passwordless ssh.
Computing Bulk Modulus of Au
This part of the tutorial is dedicated to a non trivial calculation of Bulk modulus with Qbox. We performed this calculation on a dual-core Ubuntu installation (see above).
We will compute the bulk modulus by compiting stress at two different volumes and taking finite differences. The calculation will be done with one atom per cell and many k-points. Qbox does not checks for convergence so it is our responsability to verify for this, including energy cutoff, iteration convergence, k-point grid, system size, and pseudopotential quality.
For computing the bulk modulus of gold we will need at least three files (1) a pseudopotential for Gold, (2) input files for defining cell parameters other specifications (3) a list of k-points for the FCC cell. We will put all of them in one empty directory:
mkdir au_bulk cd au_bulk
We will omit the generation of the pseudopotential, we will only mention that the pseudopotential has to be consistent with the functional choosen (in this case GGA-PBE) and the file is directly given here: au_bulk/au-pbe.TM.xml
wget -O au-pbe.TM.xml
Qbox was designed to be an interactive code, so each input line y actually a command given to the Qbox engine to set internal variables or perform a wavefunction optimization. The input file we will use is this au_bulk/au_stress.i . This input file calls another input file with the k-point sampling set au_bulk/kp8881fcc.i .
wget -O au_stress.i wget -O kp8881fcc.i
What follows is a explanation line by line of the input file au_bulk/au_stress.i. Pound sign '#' are comment lines.
## this input file compute the stress tensor for FCC Au # define cell
set cell 3.9 3.9 0.000 0.000 3.9 3.9 3.9 0.000 3.9
# define reference cell (always larger than cell), # meant to smooth out discrete changes on pw basis on volume change set ref_cell 4 4 0 0 4 4 4 0 4
# declare a pseudopotential file and give name to such species species gold au-pbe.TM.xml # add one atom to the cell atom Au gold 0 0 0
# set number of empty states, needed for metals, nempty is 0 by default. set nempty 8 # energy cut off, always check convergece against this variable. set ecut 90
# define kpoints using file kp8881fcc.i # this will call an external file that should be in the current directory # if file is not found Qbox will ignore the line and continue! kp8881fcc.i # activates the calculation of stress tensor. set stress ON
# effective cutoff for stress calculation, improves smoothness relative to change in pw basis set set ecuts 85
# set functional, default is LDA set xc PBE
#set optimization algorithm in non-selfconsistent loop set wf_dyn PSD
# energy cutoff for the wf optimization algorith # must be lower than actual ecut, # lower values can accelerate converge but increase instabilities set ecutprec 10
# real calculations start here # randomize initial wavefunction randomize_wf
# updating of charge density self consistent loop, can improve convergence set charge_mix_coeff 0.2
# set smearing of fermi surface, can improve convergence for metals set fermi_temp 300
# run 10 series of 5 self consistent loops with 5 non-self consistent loop run 10 5 5
# save atom position and wavefunction for later use save gs.xml
Now we can run the calculation
mpirun -np 2 ~/soft/qbox-1.45.0/src/qbox au_stress.i | tee au_stress.o
'au_bulk/au_stress.o' will contain all the output from qbox, exploring is the only way to check for convergence. The simples check is to see (and plot) the total energy converge:
grep etotal au_stress.o
Other check can include stress tensor or electron occupancy. Once we checked converge we can look at the final output of the stress tensor. For example in our case we will see
$ grep '<sigma_xx' au_stress_a3.9.o -A 5 | tail -6
<sigma_xx> 4.85104411 </sigma_xx>
<sigma_yy> 4.85104541 </sigma_yy>
<sigma_zz> 4.85104410 </sigma_zz>
<sigma_xy> 0.00000018 </sigma_xy>
<sigma_yz> 0.00000010 </sigma_yz>
<sigma_xz> -0.00000064 </sigma_xz>
That means that the resulting stress is diagonal (as it should be) and the magnitude is 4.85104 GPa. The we can rerun an input file with a different cell:
set cell 3.855 3.855 0.000 0.000 3.855 3.855 3.855 0.000 3.855
Then read the final stress in the same way. In this way we obtain two volumes 114.58 and 118.64 and two corresponding pressures 11.441950 and 4.85104411 Gpa. The volumes are given in atomic units but we do not need the units. A numerical approximation of the bulf modulus is simply
(11.441950-4.85104411)/(114.58 - 118.64)* ((114.58 + 118.64)/2)
which results numerically in 189.30 in units of GPa.
Possible improvements not shown here for simplicity:
(1) It is possible to write a script (bash or otherwise) to automatize the process. (2) It is possible to reuse the wavefunction of the first calculation in the second by loading the gs.xml wavefunction before resetting the cell size (and removing the 'randomize_wf' line)
load gs.xml set cell 3.855 3.855 0 3.855 0 3.855 0 0 3.855 # randomize_wf
Further work to convert this tutorial in a serious result:
Check that the result is converged by increasing the value of ecut. Take finite differences with smaller volume differences. Calculating the bulk modulus using total energies at three different volumes instead of pressures. Change the k-point grid, k-point grids can be generated or just converted from other codes k-point grid outputs. For example from VASP, the only difference is that the k-point weights (see kpt8881fcc.i) must be normalized to one.
(end of compilation tutorial)