How to compile VASP

Revision as of 12:26, 14 November 2008

In this document, we describe how to compile VASP program on several computer clusters in the Mechanical Engineering Department of Stanford University. These include mc-cc, wcr, and su-ahpcrc, all of which are linux clusters. We then give examples of how to use VASP to compute the bulk modulus of Au and ZrO2.

Notice that in both makefiles, we use the /opt/mpich/intel/bin/mpif90 compiler. Different clusters have different mpi compilers and they have different speeds. Intel compilers usually perform better than generic compilers on intel linux clusters. Make sure in your directories, you do not have another file named makefile, which takes precedence over Makefile.

The binary (executable) file vasp can run in both serial mode (e.g. ./vasp) and parallel mode (e.g. mpiexec -np 4 vasp in a PBS script). The following table compares the time to run a simple benchmark case (one Au atom, LDA, ENCUT=400, ISMEAR=1, SIGMA=0.1, KPOINTS=21x21x21) using our executable here and the one available at /share/apps/vasp.4.6/bin/vasp. Our executable is about 70% faster.

Number of CPUs	`vasp compiled here`	`/share/apps/vasp.4.6/bin/vasp`
1	68 (seconds)	116 (seconds)
2	50 (seconds)	86 (seconds)
4	56 (seconds)	(cannot run -- killed)

VASP on WCR

The procecure is similar to that on MC-CC, except that different compilers need to be used.

First, the command mpi-selector allows us to choose among different MPI compilers installed on the cluster.

$ mpi-selector --list
mvapich_gcc-0.9.9
mvapich_gcc-1.0
mvapich_intel-0.9.9
mvapich_intel-1.0
mvapich_pgi-0.9.9
mvapich_pgi-1.0
openmpi_gcc-1.2.2
openmpi_intel-1.2.2
openmpi_pgi-1.2.2

This gives us a list of choices. Next, we choose mvapich_intel-0.9.9 by

$ mpi-selector --set mvapich_intel-0.9.9

You can double-check that your choice has been made by

$ mpi-selector --query
default:mvapich_intel-0.9.9
level:user

Now you need to log-out of the cluster. When you log-in again, all MPI library paths will be correctly set up for you, e.g.

$ which mpif90
/usr/mpi/intel/mvapich-0.9.9/bin/mpif90

Now we go to the vasp.4.lib/ directory and execute the following commands.

rm makefile
wget http://micro.stanford.edu/mediawiki-1.11.0/images/Vasp.4.lib_makefile.wcr.txt -O Makefile
make clean
make

This will create libdmy.a in this directory.

Next, go to vasp.4.6/ directory and execute the following commands.

rm makefile
wget http://micro.stanford.edu/mediawiki-1.11.0/images/Vasp.4.6_makefile.wcr.txt -O Makefile
make clean
make

This will create executable vasp in this directory. This time the executable vasp can not run interactively, but can only run in the queue through a PBS script (e.g. mpiexec -np 4 vasp). Make sure in your directories, you do not have another file named makefile, which takes precedence over Makefile.

There is another executable at /share/apps/vasp.4.6/vasp, which can execute in both serial and parallel mode. To see that this executable contains MPI functions, use command

nm /share/apps/vasp.4.6/vasp | grep -i "MPI"

The following table compares the time to run the same benchmark case as above using both executables. Our executable shows speed up with multiple CPUs.

Number of CPUs	`vasp compiled here`	`/share/apps/vasp.4.6/vasp`	`vasp` another build (below)
1	76 (seconds)	75 (seconds)	48 (seconds)
2	59 (seconds)	72 (seconds)	34 (seconds)
4	35 (seconds)	64 (seconds)	29 (seconds)
8	37 (seconds)	65 (seconds)	31 (seconds)

Another way to compile vasp on WCR is the following. First

$ mpi-selector --unset

Log out WCR and log in again. Go to vasp.4.lib/.

rm makefile
wget http://micro.stanford.edu/mediawiki-1.11.0/images/Vasp.4.lib_makefile.wcr-intel.txt -O Makefile
make clean
make

This will re-create the libdmy.a in this directory. Next, go to vasp.4.6/.

rm makefile
wget http://micro.stanford.edu/mediawiki-1.11.0/images/Vasp.4.6_makefile.wcr-intel.txt -O Makefile
make clean
make

This will create executable vasp in this directory. The performance of this executable is listed in the fourth column of the table above.

VASP on SU-AHPCRC

The procecure is similar to that on WCR. First, we choose the mvapich_intel-0.9.9 compiler by

$ mpi-selector --set mvapich_intel-0.9.9

Now you need to log-out of the cluster and log-in again to have the MPI library paths correctly set up. Next Go to the vasp.4.lib/ directory and execute the following commands.

rm makefile
wget http://micro.stanford.edu/mediawiki-1.11.0/images/Vasp.4.lib_makefile.su-ahpcrc.txt -O Makefile
make clean
make

This will create libdmy.a in this directory.

Next, go to vasp.4.6/ directory and execute the following commands.

rm makefile
wget http://micro.stanford.edu/mediawiki-1.11.0/images/Vasp.4.6_makefile.su-ahpcrc.txt -O Makefile
make clean
make

This will create executable vasp in this directory. This time the executable vasp can not run interactively, but can only run in the queue through a PBS script (e.g. mpiexec --comm=ib -np 4 vasp). Notice that here we need to specify the communication channel. (For mvapich1 we need to use --comm=ib and for mvapich2 use --comm=pmi.)

The following table provides the timing information for the same benchmark case studied above.

Number of CPUs	`vasp compiled here`
1	32 (seconds)
2	23 (seconds)
4	19 (seconds)
6	28 (seconds)
8	28 (seconds)
16	203 (seconds)

A Word of Caution: Make sure to run a few test cases to confirm your executable not only runs but produces the correct numerical results. For example, we have found that on su-ahpcrc, function BRMIX (broyden.f) was giving serious errors. This was solved by changing the compilation options to "OFLAG=-O1 -mtune core2 -axW -unroll".

Computing Bulk Modulus of Au

In the following, we give an example of how to use VASP to compute the bulk modulus of LDA-Au. We performed this calculation on MC-CC in serial model in the ~/Codes/VASP/runs/Au/LDA/perfect.21x21x21 directory. This directory contains the following files.

INCAR

ENCUT  =  400
ISMEAR  = 1
SIGMA = 0.1

KPOINTS

21x21x21
0        0 = automatic generation of k-points
Monkhorst
21 21 21
0 0 0

POSCAR

POSCAR for FCC Au (created manually)
   4.068 
   0    0.5  0.5
   0.5  0    0.5
   0.5  0.5  0
   1 
Cartesian  (real coordinates r)
   0    0    0

To do this calculation, you also need to put the LDA pseudopotential file as POTCAR in this directory.

Now we are ready to run

vasp

To compute the equilibrium lattice constant, cohesive energy and bulk modulus, we use the following script auto.B.serial to run vasp repeated with different lattice constants.

#!/bin/bash
rm WAVECAR 
for a in 4.056 4.058 4.060 4.062 4.064 4.066 4.068
do
cat > POSCAR << FIN
POSCAR for FCC Au (created manually)
   $a 
   0    0.5  0.5
   0.5  0    0.5
   0.5  0.5  0
   1 
Cartesian  (real coordinates r)
   0    0    0
FIN

echo "a=$a"
./vasp

E=`tail -1 OSZICAR`
echo $a $E | sed -s 's/F=//; s/E0=//; s/d E =//;' >> Elatt.B.dat

p=`grep pressure OUTCAR | cut -b 25-34`
echo $a $p >> platt.B.dat

done

After running it as ./auto.B.serial, it will create data files Elatt.B.dat and platt.B.dat.

Launch octave and run the following functions fit_a0EB.m and fit_a0B.m,

fit_a0EB('Elatt.B.dat');
fit_a0B ('platt.B.dat');

The first line fits the energy data to a quadratic curve and computes the equilibrium lattice constant, cohesive energy and bulk modulus. The second line fits the pressure data to a linear curve and computes the equilibrium lattice constant and bulk modulus. In this example, the result is a0 = 4.60 angstrom, Ecoh = -4.39 eV, B = 190 GPa.

To run vasp in parallel, you need to submit vasp.pbs as

qsub vasp.pbs

Computing Bulk Modulus of ZrO2

In the following, we give an example of how to use VASP to compute the bulk modulus of GGA-ZrO2. We performed this calculation on SU-AHPCRC in parallel mode (8 CPUs). The running directory contains the following files.

INCAR

ZrO2 calculation

ISMEAR = 0
ISIF   = 2
PREC   = High

KPOINTS

6x6x6
0        0 = automatic generation of k-points
Monkhorst
6 6 6
0 0 0

B.pbs

#!/bin/bash
#PBS -N ZrO2_bulk
#PBS -j oe
#PBS -l nodes=1:ppn=8,walltime=6:00:00
#PBS -V

### ---------------------------------------
### BEGINNING OF EXECUTION
### ---------------------------------------

echo The master node of this job is `hostname`
echo The working directory is `echo $PBS_O_WORKDIR`
echo This job runs on the following nodes:
echo `cat $PBS_NODEFILE`

ncpu=`cat $PBS_NODEFILE | wc -w`
echo "Number of processors = $ncpu "

### end of information preamble

cd $PBS_O_WORKDIR

echo $PWD

./auto.B.par $ncpu

auto.B.par

#!/bin/bash

ncpu=$1

rm WAVECAR 
for a in 5.130 5.135 5.140 5.145 5.150 5.155
do
cat > POSCAR << FIN
POSCAR for ZrO2 (created by auto.B.par)
 $a
 1.00   0.00    0.00
 0.00   1.00    0.00
 0.00   0.00    1.00
 4  8
Direct
0       0       0
0       0.5     0.5
0.5     0       0.5
0.5     0.5     0
0.25    0.25    0.25
0.25    0.25    0.75
0.25    0.75    0.25
0.25    0.75    0.75
0.75    0.25    0.25
0.75    0.25    0.75
0.75    0.75    0.25
0.75    0.75    0.75
FIN

echo "a=$a  ncpu=$ncpu"

mpiexec --comm=ib -np $ncpu ../../../bin/vasp

E=`tail -1 OSZICAR`
echo $a $E | sed -s 's/F=//; s/E0=//; s/d E =//;' >> Elatt.B.dat

p=`grep pressure OUTCAR | cut -b 25-34`
echo $a $p >> platt.B.dat

done

You also need to put the pseudopotential file (for Zr and O together) as POTCAR in this directory. Submit the calculation by

qsub B.pbs

The job will create data files Elatt.B.dat and platt.B.dat. Similar to the case of Au, we can compute the bulk modulus from these data by fitting the energy - lattice constant relation to a quadratic curve. In this example, the result is a0 = 5.146 angstrom, Ecoh = -9.47 eV, B = 239 GPa. You will need to make n=12; and N=12; if you want to use Matlab function fit_a0EB.m. Allowing the Zr and O sub-lattices to relax relative to each other doesn't seem to change the results. Note that in reality the cubic zirconia phase studied here is unstable (against the tetragonal phase). The cubic phase is only stablized if Yttrium dopants are added, forming yttria-stablized-zirconia (YSZ). (See http://www.springerlink.com/content/g72mw270608t757t/fulltext.pdf)

Revision as of 12:26, 14 November 2008

Contents

VASP on MC-CC

VASP on WCR

VASP on SU-AHPCRC

Computing Bulk Modulus of Au

Computing Bulk Modulus of ZrO2