High Performance Computing

Modules are designed as a convenient and simple way to manage the various environments needed by different pieces of software. On the Nottingham system, they allow different compilers and parallel programming/execution environments to be used, as well as the selection of a particular cluster (or tower) for running jobs.

Users who need to compile programs to use on the cluster should familiarise themselves with the modules relevant to the various compilers on the system. Those interested in numerical libraries may wish to use the intel Math Kernel Library (MKL), and a module is provided for their convenience.

Users of parallel programs which run over more than one compute node, using MPI will need to set their environments to use one of the versions of MPI on the system. Currently, openmpi can be used on towers A-D, and openmpi or mvapich on towerE. No more than one MPI module should be loaded at the same time.

Users of commercial programs, e.g Fluent or Abaqus , are unlikely to have to set programming environments using modules, and will only need the module command in order to select the cluster to run their jobs on. Some of these programs will use their own parallel environments, but are unlikely to need these to be set by the user.

To see which modules are available on the system type 'module avail', for example:

> module avail

------------------------ /usr/share/Modules/modulefiles ------------------------
ddt                   mvapich/gcc/2-1.2p1   openmpi/1.4/gnu
fftw2-mpi/gcc         mvapich/intel/2-1.2p1 openmpi/1.4/intel
fftw2-mpi/intel       mvapich/pgi/2-1.2p1   pgi/7.1
gmp5                  open-mx               sunhpc/8.2.1/gnu
intel/compiler/101-64 openmpi/1.3.4-1/gnu   sunhpc/8.2.1/intel
intel/compiler/111    openmpi/1.3.4-1/intel sunhpc/8.2.1/pgi
intel/mkl/111         openmpi/1.3.4-1/pgi

-------------------------- /usr/share/Modules/towers ---------------------------
jupitera jupiterb jupiterc jupiterd jupitere

Typically, a tower will be chosen so that jobs may be submitted, e.g. to use Tower A type

module load jupitera

An alternative tower can be selected later thus:

module switch jupitera jupitere

Users who have wish to compile code, or run code they have compiled, may have to load compiler and library modules, and for parallel programs, MPI modules as well, e.g.

module load intel/compiler/111
module load intel/mkl/111
module load openmpi/1.3.4-1/intel

Note the multiple versions of openmpi modules, one for each available compiler.

If a module is no longer needed, it can be unloaded:

module unload intel/mkl/111

To see which modules are currently loaded, use the 'list' option, e.g:

> module list
Currently Loaded Modulefiles:
  1) jupitere                2) intel/compiler/111      3) openmpi/1.3.4-1/intel

To see the details of what loading a module does, use the 'show' option, e.g:

> module show intel/mkl/111     
-------------------------------------------------------------------
/usr/share/Modules/modulefiles/intel/mkl/111:

module-whatis    adds `  em64t' to your PATH environment variable 
prepend-path     LD_LIBRARY_PATH /opt/intel/Compiler/11.1/mkl/lib/em64t:
/opt/intel/Compiler/11.1/mkl/lib/32: 
prepend-path     LPATH /opt/intel/Compiler/11.1/mkl/lib/em64t:
/opt/intel/Compiler/11.1/mkl/lib/32: 
-------------------------------------------------------------------