Showing posts with label Gamess(US). Show all posts
Showing posts with label Gamess(US). Show all posts

Wednesday, June 26, 2013

Gamess (US) frequently asked questions Part 3: Compiling Gamess (US) in a GPU-enabled environment

A second guest post by Kirill Berezovsky (Petrozadovsk State University)


Gamess (US) may use GPUs for MP2 andCCSD(T) jobs. If you will use NVIDIA GPU, then additionally install these packages:

1.     NVIDIA CUDA 4.1
https://developer.nvidia.com/cuda-downloads

2.     HDF5 1.8.7
http://www.hdfgroup.org/ftp/HDF5/releases/hdf5-1.8.7/src/

But, first install NVIDIA video driver by terminal (it should be 304.64 and upper):

1.     Get driver:                    http://www.nvidia.ru/Download/index.aspx
This site allows the manual download of the required drivers, but may also detect automatically the drivers your system needs. Neat ;-)


2.     Run in terminal:
o    echo “blacklist nvidiafb” >> /etc/modprobe.d/blacklist.conf
o    echo “blacklist nouveau” >> /etc/modprobe.d/blacklist.conf
o    echo “blacklist rivafb” >> /etc/modprobe.d/blacklist.conf
o    echo “blacklist rivatv” >> /etc/modprobe.d/blacklist.conf
o    echo “blacklist vga16fb” >> /etc/modprobe.d/blacklist.conf
o    echo “options nouveau modeset=0” >> /etc/modprobe.d/blacklist.conf

3.     Next remove all pre-installed nvidia-packages (in terminal):
o    apt-get --purge remove nvidia*

4.     Restart your machine and press Ctrl-Alt-F1 (or the equivalent key sequence for your machine) and enter as root. Kill GUI by:
o    killall gdm3       (for Debian)
o    killall gdm         (for Ubuntu 10.xx)
o    killall lightgdm   (for Ubuntu 11.xx)
o    killall lighdm     (for Ubuntu 12.xx)

5.     Go to folder, where NVIDIA-driver places and:
o    chmod +x NVIDIA-Linux-x86_64-304.64.run
o    ./ NVIDIA-Linux-x86_64-304.64.run

6.     When you install driver, restart your machine, and install NVIDIA CUDA.

7.     HDF5 compiled by:
o    tar xvf hdf5-1.8.7.tar.bz2
o    cd hdf5-1.8.7
o    ./configure --prefix=/usr/local/hdf5 --enable-fortran --enable-parallel FC=mpiifort
o    make
o    make install

8.     Add in ~/.bashrc new variables:

# CUDA
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib:$LD_LIBRARY_PATH

9.     Re-run GAMESS (US) ./config script again, but at this time answer “yes” for LIBCCHEM-question, and choose paths for CUDA and HDF5.

10.  Go to libcchem folder and run script ./build-cchem.

11.  Wait…

12.  Go upper folder and link new executable of GAMESS (US):
o    ./lked gamess gpu

13.  Great! Now you can run MP2 and CCSD(T) jobs on gpu by command using my script:
gamess Inputfile.input gpu

Gamess (US) frequently asked questions Part 2: Installation in Linux boxes


This is a guest post by Kirill Berezovsky (Petrozavodsk State university), based on instructions he posted some time ago in the Gamess (US) list



First, you need to get some software!
• Fortran compiler: gfortran or Intel Fortran (ifort);
• Math library's: ACML (for AMD CPU’s), ATLAS or MKL (Intel Math Kernel Library);
• (optionally) MPI: Intel MPI, OpenMPI, MVAPICH2, ... (if you using MPI);
(bold is better)

  • I’m not recommending the use of OpenMPI because it’s really slower than Intel MPI.
    But, if you want to try use it, go to this site. This is a very short and informative solution for building 64-bit OpenMPI.

This software for non-commercial using you can get on
http://software.intel.com/en-us/non-commercial-software-development
(Intel Fortran and MKL places in Intel® Fortran Composer XE 2013 for Linux)

As GAMESS (US) developers, I’m using:
• ifort 12.0.4 (places in Intel Fortran Composer XE 2011 update 4, “l_fcompxe_2011.4.191.tgz”)
• MKL 11.0 (places in Intel Fortran Composer XE 2013 initial release, “l_fcompxe_2013.0.079.tgz”).

Be aware, ifort 13.0.0 can’t compile some GAMESS (US) objects!

Anyway, you can install separate software from these archives – look hard when installing. It will safe HDD space and protect from some mistakes, maybe.




Great, when you have these packages, let’s start to configure your system!
You should be root, as I think that is better.

And before starting, configure memory by terminal commands. First, answer the question – how many RAM does your PC have? For example, my PC has 4 GB RAM, so in bytes it will be:

4 GB = 4*1024 MB = 4*1024*1024 KB = 4*1024*1024*1024 bytes = 4294967296 bytes.

And go on: 4294967296 bytes / 2 = 2147483648 bytes.
This number shows maximum size of 1 segment of shared memory.

Next, total size of shared memory in pages will be: 2147483648 / 4096 = 524288 pages.

This numbers you need to write in /etc/sysctl.conf:
echo “kernel.shmmax=2147483648” >> /etc/sysctl.conf
echo “kernel.shmall=524288” >> /etc/sysctl.conf
And restart your machine.

Install the packages!
1. It's better to use 64-bit Linux, whatever you like. I’m using Debian 6.0.7;

2. In terminal, install these packages (just in case):
apt-get install tcsh gcc g++ gfortran build-essential dpkg-dev binutils zlib1g-dev

3. Install Intel Fortan Composer XE 2011;
1. Unpack downloaded archive by: tar xvf l_fcompxe_2011.4.191.tar
2. Goto unpacked folder by: cd l_fcompxe_2011.4.191
3. Install by run: ./install.sh
4. ...And follow the instructions

4. Install GAMESS (US);
1. Get the GAMESS-archive;
2. Unpack it (default in /usr/local/);
3. Go to the GAMESS-folder and run script: ./config
4. Answer the questions:
1. Target machine name: linux64
2. GAMESS location: /usr/local/gamess
3. Build location: /usr/local/gamess
4. GAMESS executable version name (any name you want. In this example, we will use "cpu" as the name): cpu
5. Fortran compiler (choose what you use):
  •  ifort --> version : 12
  • gfortran --> version (like 4.4) you can get if you run in other terminal by: gfortran -v
6. Math library: mkl
7. Math library location: /opt/intel/composerxe_2011/mkl (verify it for your installation!)
8. When it shows string which contains 'bin' and 'lib' then type: skip
9. If you are not using MPI then type: sockets and you'll finish configuration;

10. Else, if you are using MPI type: mpi
1. Next, choose MPI-program: impi
2. Select MPI location directory, and go on.

11. Answer “no” for “LIBCCHEM”-question. If you are using NVIDIA GPU for calculations, anyway answer “no” at this first configuration time go on and don’t forget to read important note after these steps.

5. Goto ddi folder by: cd ddi
6. Edit 'compddi'-script by: gedit compddi
7. Find and change strings:
1. set MAXCPUS=4 (number of cores, is it 4 in your machine?)
2. set MAXNODES=1 (for single node)
8. Run 'compddi'-script by: ./compddi
• If you're NOT using MPI, there will be file ddikick.x - move in by: mv ddikick.x .. (two dots means upper folder)

9. Go upper folder: cd ..
10. Then compile GAMESS (US) by: ./compall

11. Link by: ./lked gamess cpu which creates gamess.cpu.x file. Of course, you can name it as you want, not only 'cpu'


12. Edit 'rungms'-script:
  • See here the rungms-script rewritten by Kirill.

Next you should create folders:
  • mkdir /scr
  • mkdir /scr/root
  • mkdir /root/scr

Add system variables into the ~/.bashrc file:

# iFort
export PATH=/opt/intel/composerxe-2011.4.191/bin/intel64:$PATH
export LD_LIBRARY_PATH=/opt/intel/composerxe-2011.4.191/compiler/lib/intel64:$LD_LIBRARY_PATH

# iMPI
export PATH=/opt/intel/impi/4.0.2.003/intel64/bin:$PATH
export LD_LIBRARY_PATH=/opt/intel/impi/4.0.2.003/intel64/lib:$LD_LIBRARY_PATH

# MKL
export LD_LIBRARY_PATH=/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64:$LD_LIBRARY_PATH


To run GAMESS (US) just type:

/usr/local/gamess/rungms [input file] [optionally,  the 'version name' of your gamess file ("cpu" in this example)]
just like this:

/usr/loca/gamess/rungms BSi85H95.inp
/usr/loca/gamess/rungms BSi85H95.inp cpu   will run exactly the same.

For simple run edit ~/.bashrc like:
gedit ~/.bashrc
• add line alias gamess=’/usr/local/gamess/rungms’
• apply changes by source ~/.bashrc
• and now you can run it by:

gamess BSi85H95.inp cpu

END!

Wednesday, June 19, 2013

Gamess (US) frequently asked questions Part 1: SCF convergence

In spite of the very high quality of the Gamess(US) documentation, the Gamess(US) list is very often flooded with requests from new users regarding the lack of convergence of the SCF procedure. A few words of advice:

When your SCF does not converge,  you should re-run the job including a $guess guess=moread $end line, as well as the complete $VEC group present in the output PUNCH file (usually called <jobname>.dat, and present in you scratch directory).

    Addendum:

    Whenever you read a $VEC group from a UHF run you must assign NORB in the $GUESS group. An additional problem is that by default the $VEC group only includes the occupied orbitals, and this means that in UHF runs the $VEC group does not include equal numbers of alpha and beta orbitals (e.g., a run with 41 electrons and MULT=2) will have 21 alpha orbitals and 20 beta orbitals. Therefore, if you include

    $guess guess=moread NORB=21 $end

    Gamess will crash because there are not 21 beta orbitals, and if you input

    $guess guess=moread NORB=20 $end

    there will be another error, since there are more than 20 alpha orbitals. In these cases, you should check the number of alpha and beta orbitals. Then , copy the coefficients of the extra alpha orbitals to the end of the beta orbitals. In my example above

    $guess guess=moread NORB=21 $end

    will yield no problems, since the modification of the VEC group yields equal numbers of alpha and beta orbitals. There is also an option to PUNCH every orbital (occupied+virtuals) at every step. In this case, Gamess always punches a full $VEC group, making it very easy to assign NORB as one can simply inspect the output file to learn the number of orbitals. However, this yields gigantic PUNCH files, and may therefore not be feasible.




You should also experiment with changing convergers, damping, etc. Some systems are notoriously hard to converge, and may require several re-iterations of the whole process.