Gamess (US) frequently asked questions Part 4: The rungms script

Third guest post by Kirill Berezovsky (Petrozadovsk State University).

Gamess (US) is run though a script provided in the installation package. This script MUST be adapted by the user. The rungms-script below has been adapted to be used with the Gamess (US) installation described in earlier posts. Happy computing :-)

RUNGMS script

#!/bin/csh

set SCR=/scr/$USER

set USERSCR=~$USER/scr

set GMSPATH=/usr/local/gamess

set JOB=$1

set VERNO=$2

if ($VERNO == cpu) set TARGET=mpi

if ($VERNO == gpu) set TARGET=ga

if (null$VERNO == null) set VERNO=cpu

set master=`hostname`

printf "\n * Started at:           `date`"

set DSK=`df -m $SCR | awk 'NR==2{print$4}'`

printf "\n * Available disk space: $DSK MB"

printf "\n * Temporary files in:   $SCR and in $USERSCR\n\n"

limit stacksize 8192

if ($JOB:r.inp == $JOB) set JOB=$JOB:r

if (-e $JOB.inp) then

   cp  $JOB.inp  $SCR/$JOB.F05

else

   echo "Input file $JOB.inp not found"

   exit 4

endif

source $GMSPATH/gms-files.csh

if (-e $HOME/.gmsrc) source $HOME/.gmsrc

set ngddi=`grep -i '^ \$GDDI' $SCR/$JOB.F05 | grep -iv 'NGROUP=0 ' | wc -l`

if ($ngddi > 0) then

   set GDDIjob=true

   echo "This is a GDDI run, keeping various output files on local disks"

   set echo

   setenv  OUTPUT $SCR/$JOB.F06

   setenv   PUNCH $SCR/$JOB.F07

   unset echo

else

   set GDDIjob=false

endif

if ((-e $PUNCH) || (-e $MAKEFP) || (-e $TRAJECT) || (-e $RESTART) ) then

   echo "Please save, rename, or erase these files from a previous run:"

   echo "     $PUNCH,"

   echo "     $TRAJECT,"

   echo "     $RESTART, and/or"

   echo "     $MAKEFP,"

   echo "and then resubmit this computation."

   exit 4

endif

#----------------------------------------------------------------------

if ($TARGET == mpi) then

 set NCPUS=`grep cores /proc/cpuinfo | wc -l`

 echo " * CPU cores:   $NCPUS "

 echo " * GPU devices: not used"

 echo " "

 setenv TRAJECT $USERSCR/$JOB.trj

 setenv RESTART $USERSCR/$JOB.rst

 setenv INPUT $SCR/$JOB.F05

 setenv PUNCH $USERSCR/$JOB.dat

 if ( -e $TRAJECT ) rm $TRAJECT

 if ( -e  $PUNCH ) rm $PUNCH

 if ( -e  $RESTART ) rm $RESTART

 setenv LD_LIBRARY_PATH /opt/intel/impi/4.0.2.003/intel64/lib:$LD_LIBRARY_PATH

 set path= ( /opt/intel/impi/4.0.2.003/intel64/bin $path )

mpdboot

 mpiexec -n $NCPUS $GMSPATH/gamess.$VERNO.x

mpdallexit

 cp $PUNCH .

endif

#----------------------------------------------------------------------

if ($TARGET == ga) then

 set PPN=1

 set NCPUS=1

 @ NPROCS = $NCPUS

 setenv HOSTFILE $SCR/$JOB.nodes.mpd

 if (-e $HOSTFILE) rm $HOSTFILE

 touch $HOSTFILE

  

 echo `hostname` >> $HOSTFILE

 set NNODES=1

 setenv PROCFILE $SCR/$JOB.processes.mpd

 if (-e $PROCFILE) rm $PROCFILE

 touch $PROCFILE

 echo "-n $NPROCS -host `hostname` $GMSPATH/gamess.$VERNO.x" >> $PROCFILE

 set path=(/opt/intel/impi/4.0.2.003/intel64/bin $path)

   setenv I_MPI_WAIT_MODE enable

   setenv I_MPI_PIN disable

   setenv I_MPI_DEBUG 0

   setenv I_MPI_STATS 0

   setenv I_MPI_DEVICE sock

   setenv I_MPI_NETMASK ib0

 setenv LD_LIBRARY_PATH /opt/intel/impi/4.0.2.003/intel64/lib:$LD_LIBRARY_PATH

 setenv LD_LIBRARY_PATH /opt/intel/composerxe-2011.4.191/compiler/lib/intel64:$LD_LIBRARY_PATH

 setenv LD_LIBRARY_PATH /opt/intel/composer_xe_2013.0.079/mkl/lib/intel64:$LD_LIBRARY_PATH

 setenv MKL_SERIAL YES

 setenv MKL_NUM_THREADS 1

 setenv LD_LIBRARY_PATH /usr/local/cuda/lib64:$LD_LIBRARY_PATH

 setenv GMS_CCHEM '1'

 @ NUMGPU=1

 setenv CCHEM 'devices=0;memory=4g'

 echo " * CPU cores:   `grep cores /proc/cpuinfo | wc -l` "

 echo " * GPU devices: $NUMGPU (with settings: $CCHEM)"

 echo " "

 chdir $SCR

 set echo

 mpdboot --rsh=ssh -n $NNODES -f $HOSTFILE

  mpiexec -configfile $PROCFILE < /dev/null

 mpdallexit

 unset echo

 rm -f $PROCFILE

endif

#----------------------------------------------------------------------

echo ----- accounting info -----

if ($GDDIjob == true) cp $SCR/$JOB.F07 ~/scr/$JOB.dat

echo Files used on the master node $master were:

ls -lF $SCR/$JOB.*

rm -f  $SCR/$JOB.F*

if (-e $SCR/$JOB.V84)        mv $SCR/$JOB.V84     $USERSCR

if (-e $SCR/$JOB.V80)        rm -f $SCR/$JOB.V*

if (-e $SCR/$JOB.TEMP02)     rm -f $SCR/$JOB.TEMP*

if (-e $SCR/$JOB.orb)        mv $SCR/$JOB.orb     $USERSCR

if (-e $SCR/$JOB.vec)        mv $SCR/$JOB.vec     $USERSCR

if (-e $SCR/$JOB.mol)        mv $SCR/$JOB.mol     $USERSCR

if (-e $SCR/$JOB.molf)       mv $SCR/$JOB.molf    $USERSCR

if (-e $SCR/$JOB.mkl)        mv $SCR/$JOB.mkl     $USERSCR

if (-e $SCR/$JOB.xyz)        mv $SCR/$JOB.xyz     $USERSCR

ls $SCR/${JOB}-*.cube > $SCR/${JOB}.lis

if (! -z $SCR/${JOB}.lis) mv $SCR/${JOB}*.cube $USERSCR

rm -f $SCR/${JOB}.lis

ls $SCR/${JOB}-*.grd > $SCR/${JOB}.lis

if (! -z $SCR/${JOB}.lis) mv $SCR/${JOB}*.grd $USERSCR

rm -f $SCR/${JOB}.lis

ls $SCR/${JOB}-*.csv > $SCR/${JOB}.lis

if (! -z $SCR/${JOB}.lis) mv $SCR/${JOB}*.csv $USERSCR

rm -f $SCR/${JOB}.lis

if ($TARGET == mpi) then

   set nnodes=`wc -l $HOSTFILE`

   set nnodes=$nnodes[1]

   @ n=1

   set master=`hostname`

   set master=$master:r

   while ($n <= $nnodes)

      set host=`sed -n -e "$n p" $HOSTFILE`

      set host=$host[1]

      if ($host != $master) then

         echo Files used on node $host were:

         ssh $host -l $USER "ls -l $SCR/$JOB.*"

         ssh $host -l $USER "rm -f $SCR/$JOB.*"

      endif

      @ n++

   end

   rm -f $HOSTFILE

   if ($?I_MPI_STATS) then

      if ($I_MPI_STATS > 0) mv $SCR/stats.txt ~/$JOB.$NCPUS.stats

   endif

endif

date

time

exit

Gamess (US) frequently asked questions Part 3: Compiling Gamess (US) in a GPU-enabled environment

A second guest post by Kirill Berezovsky (Petrozadovsk State University)


Gamess (US) may use GPUs for MP2 andCCSD(T) jobs. If you will use NVIDIA GPU, then additionally install these packages:

1.     NVIDIA CUDA 4.1

https://developer.nvidia.com/cuda-downloads

2.     HDF5 1.8.7

http://www.hdfgroup.org/ftp/HDF5/releases/hdf5-1.8.7/src/

But, first install NVIDIA video driver by terminal (it should be 304.64 and upper):

1.     Get driver:                    http://www.nvidia.ru/Download/index.aspx

This site allows the manual download of the required drivers, but may also detect automatically the drivers your system needs. Neat ;-)

2.     Run in terminal:

o    echo “blacklist nvidiafb” >> /etc/modprobe.d/blacklist.conf

o    echo “blacklist nouveau” >> /etc/modprobe.d/blacklist.conf

o    echo “blacklist rivafb” >> /etc/modprobe.d/blacklist.conf

o    echo “blacklist rivatv” >> /etc/modprobe.d/blacklist.conf

o    echo “blacklist vga16fb” >> /etc/modprobe.d/blacklist.conf

o    echo “options nouveau modeset=0” >> /etc/modprobe.d/blacklist.conf

3.     Next remove all pre-installed nvidia-packages (in terminal):

o    apt-get --purge remove nvidia*

4.     Restart your machine and press Ctrl-Alt-F1 (or the equivalent key sequence for your machine) and enter as root. Kill GUI by:

o    killall gdm3       (for Debian)

o    killall gdm         (for Ubuntu 10.xx)

o    killall lightgdm   (for Ubuntu 11.xx)

o    killall lighdm     (for Ubuntu 12.xx)

5.     Go to folder, where NVIDIA-driver places and:

o    chmod +x NVIDIA-Linux-x86_64-304.64.run

o    ./ NVIDIA-Linux-x86_64-304.64.run

6.     When you install driver, restart your machine, and install NVIDIA CUDA.

7.     HDF5 compiled by:

o    tar xvf hdf5-1.8.7.tar.bz2

o    cd hdf5-1.8.7

o    ./configure --prefix=/usr/local/hdf5 --enable-fortran --enable-parallel FC=mpiifort

o    make

o    make install

8.     Add in ~/.bashrc new variables:

# CUDA

export PATH=/usr/local/cuda/bin:$PATH

export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

export LD_LIBRARY_PATH=/usr/local/cuda/lib:$LD_LIBRARY_PATH

9.     Re-run GAMESS (US) ./config script again, but at this time answer “yes” for LIBCCHEM-question, and choose paths for CUDA and HDF5.

10.  Go to libcchem folder and run script ./build-cchem.

11.  Wait…

12.  Go upper folder and link new executable of GAMESS (US):

o    ./lked gamess gpu

13.  Great! Now you can run MP2 and CCSD(T) jobs on gpu by command using my script:

gamess Inputfile.input gpu

Gamess (US) frequently asked questions Part 2: Installation in Linux boxes

This is a guest post by Kirill Berezovsky (Petrozavodsk State university), based on instructions he posted some time ago in the Gamess (US) list



First, you need to get some software!
• Fortran compiler: gfortran or Intel Fortran (ifort);
• Math library's: ACML (for AMD CPU’s), ATLAS or MKL (Intel Math Kernel Library);
• (optionally) MPI: Intel MPI, OpenMPI, MVAPICH2, ... (if you using MPI);
(bold is better)

• I’m not recommending the use of OpenMPI because it’s really slower than Intel MPI.
  But, if you want to try use it, go to this site. This is a very short and informative solution for building 64-bit OpenMPI.

This software for non-commercial using you can get on
http://software.intel.com/en-us/non-commercial-software-development
(Intel Fortran and MKL places in Intel® Fortran Composer XE 2013 for Linux)

As GAMESS (US) developers, I’m using:
• ifort 12.0.4 (places in Intel Fortran Composer XE 2011 update 4, “l_fcompxe_2011.4.191.tgz”)
• MKL 11.0 (places in Intel Fortran Composer XE 2013 initial release, “l_fcompxe_2013.0.079.tgz”).

Be aware, ifort 13.0.0 can’t compile some GAMESS (US) objects!

Anyway, you can install separate software from these archives – look hard when installing. It will safe HDD space and protect from some mistakes, maybe.




Great, when you have these packages, let’s start to configure your system!
You should be root, as I think that is better.

And before starting, configure memory by terminal commands. First, answer the question – how many RAM does your PC have? For example, my PC has 4 GB RAM, so in bytes it will be:

4 GB = 4*1024 MB = 4*1024*1024 KB = 4*1024*1024*1024 bytes = 4294967296 bytes.

And go on: 4294967296 bytes / 2 = 2147483648 bytes.
This number shows maximum size of 1 segment of shared memory.

Next, total size of shared memory in pages will be: 2147483648 / 4096 = 524288 pages.

This numbers you need to write in /etc/sysctl.conf:
• echo “kernel.shmmax=2147483648” >> /etc/sysctl.conf
• echo “kernel.shmall=524288” >> /etc/sysctl.conf
And restart your machine.

Install the packages!
1. It's better to use 64-bit Linux, whatever you like. I’m using Debian 6.0.7;

2. In terminal, install these packages (just in case):
apt-get install tcsh gcc g++ gfortran build-essential dpkg-dev binutils zlib1g-dev

3. Install Intel Fortan Composer XE 2011;
1. Unpack downloaded archive by: tar xvf l_fcompxe_2011.4.191.tar
2. Goto unpacked folder by: cd l_fcompxe_2011.4.191
3. Install by run: ./install.sh
4. ...And follow the instructions

4. Install GAMESS (US);
1. Get the GAMESS-archive;
2. Unpack it (default in /usr/local/);
3. Go to the GAMESS-folder and run script: ./config
4. Answer the questions:
1. Target machine name: linux64
2. GAMESS location: /usr/local/gamess
3. Build location: /usr/local/gamess
4. GAMESS executable version name (any name you want. In this example, we will use "cpu" as the name): cpu
5. Fortran compiler (choose what you use):

•  ifort --> version : 12
• gfortran --> version (like 4.4) you can get if you run in other terminal by: gfortran -v

6. Math library: mkl
7. Math library location: /opt/intel/composerxe_2011/mkl (verify it for your installation!)
8. When it shows string which contains 'bin' and 'lib' then type: skip
9. If you are not using MPI then type: sockets and you'll finish configuration;

10. Else, if you are using MPI type: mpi
1. Next, choose MPI-program: impi
2. Select MPI location directory, and go on.

11. Answer “no” for “LIBCCHEM”-question. If you are using NVIDIA GPU for calculations, anyway answer “no” at this first configuration time go on and don’t forget to read important note after these steps.

5. Goto ddi folder by: cd ddi
6. Edit 'compddi'-script by: gedit compddi
7. Find and change strings:
1. set MAXCPUS=4 (number of cores, is it 4 in your machine?)
2. set MAXNODES=1 (for single node)
8. Run 'compddi'-script by: ./compddi
• If you're NOT using MPI, there will be file ddikick.x - move in by: mv ddikick.x .. (two dots means upper folder)

9. Go upper folder: cd ..
10. Then compile GAMESS (US) by: ./compall

11. Link by: ./lked gamess cpu which creates gamess.cpu.x file. Of course, you can name it as you want, not only 'cpu'


12. Edit 'rungms'-script:

• See here the rungms-script rewritten by Kirill.

Next you should create folders:

• mkdir /scr
• mkdir /scr/root
• mkdir /root/scr

Add system variables into the ~/.bashrc file:

# iFort
export PATH=/opt/intel/composerxe-2011.4.191/bin/intel64:$PATH
export LD_LIBRARY_PATH=/opt/intel/composerxe-2011.4.191/compiler/lib/intel64:$LD_LIBRARY_PATH

# iMPI
export PATH=/opt/intel/impi/4.0.2.003/intel64/bin:$PATH
export LD_LIBRARY_PATH=/opt/intel/impi/4.0.2.003/intel64/lib:$LD_LIBRARY_PATH

# MKL
export LD_LIBRARY_PATH=/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64:$LD_LIBRARY_PATH


To run GAMESS (US) just type:

/usr/local/gamess/rungms [input file] [optionally,  the 'version name' of your gamess file ("cpu" in this example)]
just like this:

/usr/loca/gamess/rungms BSi85H95.inp
/usr/loca/gamess/rungms BSi85H95.inp cpu   will run exactly the same.

For simple run edit ~/.bashrc like:
• gedit ~/.bashrc
• add line alias gamess=’/usr/local/gamess/rungms’
• apply changes by source ~/.bashrc
• and now you can run it by:

gamess BSi85H95.inp cpu

END!

Science by press release

I woke up today with the news that researchers at the University of Aveiro had, "for the first time", altered the translational apparatus of an organism. I was outraged with the news: not with the science itself, but with the mindless hype surrounding it: actually, such a modification had already been performed in 2011 in C. elegans . I first thought that the "first time evah" pitch had been added by ignorant journalists, but the hype was already present in the press release from Univ. Aveiro!
The research publicized today is good and interesting, no doubt about that, but the quest for "good press" should never come at the expense of the truth. There is no excuse for that. Every bit of "good press" achieved with hype/exageration unfairly benefits those institutions and/or researchers with no moral qualms, leaving those researchers who are honest enough to not misrepresent their results in a disadvantage.

I've always disliked "science by press release", because (all other things being equal) it disproportionately benefits those who have access to the mass media, or who can afford publicists. Hyped press releases are even worse. And this can only end when science journalists stop relying on press releases to decide what is newsworthy. Though I strongly believe that such a day will not happen in the next 5 * 10⁹ years.


Addendum: Previous reports all reassigned a STOP codon to an unnatural aminoacid. The report from Univ. Aveiro is indeed the first time that a non-STOP codon has been reassigned in an organism. This difference is unfortunately not present in the press release. I still stand by all other points on my post.

Gamess (US) frequently asked questions Part 1: SCF convergence

In spite of the very high quality of the Gamess(US) documentation, the Gamess(US) list is very often flooded with requests from new users regarding the lack of convergence of the SCF procedure. A few words of advice:

When your SCF does not converge,  you should re-run the job including a $guess guess=moread $end line, as well as the complete $VEC group present in the output PUNCH file (usually called <jobname>.dat, and present in you scratch directory).

Addendum:

Whenever you read a $VEC group from a UHF run you must assign NORB in the $GUESS group. An additional problem is that by default the $VEC group only includes the occupied orbitals, and this means that in UHF runs the $VEC group does not include equal numbers of alpha and beta orbitals (e.g., a run with 41 electrons and MULT=2) will have 21 alpha orbitals and 20 beta orbitals. Therefore, if you include

$guess guess=moread NORB=21 $end

Gamess will crash because there are not 21 beta orbitals, and if you input

$guess guess=moread NORB=20 $end

there will be another error, since there are more than 20 alpha orbitals. In these cases, you should check the number of alpha and beta orbitals. Then , copy the coefficients of the extra alpha orbitals to the end of the beta orbitals. In my example above

$guess guess=moread NORB=21 $end

will yield no problems, since the modification of the VEC group yields equal numbers of alpha and beta orbitals. There is also an option to PUNCH every orbital (occupied+virtuals) at every step. In this case, Gamess always punches a full $VEC group, making it very easy to assign NORB as one can simply inspect the output file to learn the number of orbitals. However, this yields gigantic PUNCH files, and may therefore not be feasible.

You should also experiment with changing convergers, damping, etc. Some systems are notoriously hard to converge, and may require several re-iterations of the whole process. 

Advances in peptide chemistry

Protein synthesis is nowadays achieved through molecular biology techniques: the relevant gene is cloned in an appropriate vector, over-expressed with e.g. a poly-histidine tag, and then purified through high affinity chromatography. Peptide chemistry is therefore often forgotten by biochemists, unless we need to order a short customized peptide from a commercial source.
Danishefsky et al. have now combined solid phase peptide synthesis, native chemical ligation and metal-free dethyilation to synthesize a number of analogues of human parathormone. Their strategy afforded native parathormone with higher purity than obtained from commercial sources, as well as pure analogues not achievable by any other means. These analogues were shown to be much more stable (10% decomposition in 7 days) than parathormone ,(>90% loss in 7 days), and to be as active as parathormone when injected to mice.
This is a very interesting work, which should pave the way towards the synthesis of long-lived synthetic peptide hormones, thus potentially decreasing the number of injections needed to control hormone levels in patients suffering from impaired endocrine function.