This section explains how to make use of the system-wide available software stack which is organized in modules.

A module is a self-contained description of a software package - it contains the settings required to run a software package and, usually, encodes required dependencies on other software packages.

There are a number of different environment module implementations commonly used on HPC systems: the two most common are TCL modules and Lmod. Both of these use similar syntax and the concepts are the same so learning to use one will allow you to use whichever is installed on the system you are using.

Info
Both ALICE and SHARK, use Lmod.

The module system makes it easy to switch between different software for different jobs and access multiple versions of the same software.

Table of Contents

minLevel	1
maxLevel	7

Basic usage

This table provides a short overview of the most commonly used options for the command module. Check the help for module to see all commands with a description.

Command	Description
`module -h` or `module --help`	Get help message
`module list`	Get a list of all currently loaded modules
`module avail` or `module av`	Get list of all available modules (See below for details)
`module avail <string>` or `module av <string>`	Get list of all available modules which contain `<string>` in their name (See below for details)
`module spider <string>`	Get list of all modules matching `<string>` and their description
`module whatis <module>`	Get the description of the given `<module>`
`module load <module>`	Load the specified `<module>` (See below for details)
`module unload <module>`	Unload the module `<module>` (See below for details)
`module reset`	Reset module environment to default list of modules (See below for details)
`module purge`	Unload all loaded modules (See below for details)
`module save`	Save the current list of modules, e.g., as a default environment
`module restore`	Restore module environment to a default list of modules

The module structure on ALICE and SHARK

ALICE and SHARK have different module structures. This is partly related to how the system-wide software stack is being managed on both clusters. However, the basic usage of environment modules is the same on both clusters.

Once you are familiar with how modules are organized on the cluster that you are using, it will not be difficult for you to find the modules that you are looking for.

ALICE

On ALICE modules are organized as different module trees. The software behind each module is build and maintained using EasyBuild (https://easybuild.io/).

In each tree, the modules are organized as <name>/<version> where <version> can also include additional information about what was used to build the module <name>

The default trees are

Tree

Description

/cm/shared/easybuild/modules/all

or

/cm/shared/easybuild/GenuineIntel/modules/all

Default tree, containing all of the system-wide software build with EasyBuild for nodes with Intel CPU. Available by default or after loading the ALICE/Intel module or on Intel-powered nodes the ALICE/default module.

/cm/shared/modulefiles

Containing the modules for Slurm and LUMC/FSL software

Additionally available trees that need to be loaded manually are:

Tree	Description
`/cm/shared/LUMC/modulefiles`	Containing all LUMC-provided modules including FSL. Available after loading the module `LUMC`
`/cm/shared/easybuild/AuthenticAMD/modules/all`	Containing all EasyBuild-build modules for nodes with AMD CPUs. Available after loading the `ALICE/AMD` module or on AMD-powered nodes the `ALICE/default` module

SHARK

On SHARK, all software is contained in the tree /share/modulefiles

Modules are being organized in a hierarchical structure, i.e., <moduleclass>/<name>/<version> where <moduleclass> is the class to which the module belong to, <name> the name of the module, and <version> the installed version of the module <name>

The different module classes currently are:

benchmark
bioimage
bioinformatics
container
cryogenicEM
genomics
graphics
gwas
library
mathematical
medicalImaging
neuroImaging
pharmaceutical
statistical
system
tools

List currently loaded modules

On login, you might start out with a default set of modules.

The module list command shows which modules you currently have loaded in your environment.

ALICE

This is what you will most likely see after logging into ALICE.

Code Block
[me@nodelogin02~]$ module list Currently Loaded Modules: 1) shared 2) DefaultModules 3) gcc/8.2.0 4) slurm/19.05.1

You can see that by default the module for Slurm and the gcc compiler is loaded. The number behind the slash sign represent the version number.

SHARK

On SHARK, no modules are being loaded by default. After login, you would see

Code Block
[me@res-hpc-lo02 ~]$ module list No modules loaded

List available modules

To see the available modules, use module -d avail. With the -d option, you will only get the default versions of the modules. For various software packages, there are also older/other versions available, that might be used if necessary. You can see all version by omitting the -d option.

ALICE

Code Block

 [me@nodelogin02~]$ module -d avail
 ---------------------------------------------------------------------- /cm/shared/easybuild/GenuineIntel/modules/all ----------------------------------------------------------------------
   4ti2/1.6.9-GCC-8.2.0-2.31.1                                     Miniconda2/4.7.10                                      flatbuffers/1.12.0-GCCcore-10.2.0
   ALICE/default                                           (L)     Miniconda3/4.9.2                                       flex/2.6.4
   AMUSE-GPU/12.0.0-foss-2018a-Python-2.7.14                       MotionCor2/1.3.0-GCCcore-8.2.0                         fontconfig/2.13.93-GCCcore-10.3.0
   AMUSE-Miniconda2/4.7.10                                         MrBayes/3.2.6-foss-2017a                               foss/2021a
   AMUSE-VADER/12.0.0-foss-2018a-Python-2.7.14                     MultiQC/1.7-foss-2018b-Python-3.6.6                    fosscuda/2020b
   AMUSE/2021.7-Miniconda3-4.9.2                                   NASM/2.15.05-GCCcore-10.3.0                            freeglut/3.2.1-GCCcore-8.3.0
   ANTs/2.3.5-foss-2019b-Python-3.7.4                              NCCL/2.10.3-GCCcore-10.3.0-CUDA-11.3.1                 freetype/2.10.4-GCCcore-10.3.0
   APR-util/1.6.1-GCCcore-8.2.0                                    NGS/2.10.0-GCCcore-8.2.0-Java-11                       gc/7.6.12-GCCcore-9.3.0
   APR/1.7.0-GCCcore-8.2.0                                         NLTK/3.2.4-foss-2019a-Python-3.7.2                     gcccuda/2020b
   ARPwARP/8.0                                                     NLopt/2.6.2-GCCcore-10.2.0                             gensim/3.7.3-foss-2019a-Python-3.7.2
# etc...

SHARK

Code Block

[rfschulz@res-hpc-lo02 ~]$ module -d avail
----------------------------------------------------------------------------------- /share/modulefiles ------------------------------------------------------------------------------------
   benckmarks/disk/ior/3.4.0/gcc-8.3.1                     library/cuda/11.0/gcc.8.3.1                             neuroImaging/MRtrix3/3.0.3/gcc-8.3.1
   benckmarks/disk/iozone/3.491/gcc-8.3.1                  library/cuda/11.1/gcc.8.3.1                             neuroImaging/SimpleElastix/0.10.0/python3.6.8
   bioimage/CellProfiler/4.2.1/gcc-8.3.1                   library/cuda/11.2/gcc.8.3.1                             neuroImaging/dsi-studio/2021.10.05/gcc-8.3.1
   bioimage/QuPath/0.2.3                                   library/cuda/11.3/gcc.8.3.1                             neuroImaging/freesurfer/7.2.0
   bioimage/bftools/6.8.1                                  library/cuda/11.4.1/gcc.8.3.1                           neuroImaging/fsl/5.0.10
   bioimage/slideToolkit/1.1                               library/cuda/11.6.1/gcc.8.3.1                           neuroImaging/fsl/5.0.11
   bioinformatics/tools/SNP/flashpca/2.1/gcc-8.3.1         library/cuda/7.5/gcc.8.3.1                              neuroImaging/fsl/5.0.9
   bioinformatics/tools/assembly-stats/1.0.1/gcc-8.3.1     library/cuda/8.0/gcc.8.3.1                              neuroImaging/fsl/6.0.0
   bioinformatics/tools/ncbi/sra/2.10.9                    library/cuda/9.0/gcc.8.3.1                              neuroImaging/fsl/6.0.1
   container/singularity/3.10.0/gcc.8.5.0                  library/cuda/9.1/gcc.8.3.1                              neuroImaging/fsl/6.0.2
# etc...

Finding specific modules

If you are searching for a specific software package or tool you can search for the full module name like this:

ALICE

Code Block

[me@nodelogin02~]$ module avail python
  ---------------------------------------- /cm/shared/easybuild/modules/all -----------------------------------------
  AMUSE-GPU/12.0.0-foss-2018a-Python-2.7.14
  AMUSE-VADER/12.0.0-foss-2018a-Python-2.7.14
  AMUSE/12.0.0-foss-2018a-Python-2.7.14
  AMUSE/13.1.0-foss-2018a-Python-3.6.4                    (D)
  Biopython/1.71-foss-2018a-Python-2.7.14
  Biopython/1.73-foss-2019a
  Biopython/1.75-foss-2019b-Python-3.7.4                  (D)
  Boost.Python/1.67.0-foss-2018b-Python-3.6.6
  CGAL/4.11.1-foss-2018b-Python-3.6.6
  CONCOCT/1.1.0-foss-2019a-Python-2.7.15
  CheckM/1.0.18-foss-2019a-Python-2.7.15
  Cython/0.25.2-foss-2018a-Python-2.7.14
  Cython/0.25.2-foss-2018a-Python-3.6.4
  Cython/0.29.3-foss-2019a-Python-3.7.2                   (D)
  DAS_Tool/1.1.1-foss-2019a-R-3.6.0-Python-3.7.2
  Docutils/0.9.1-foss-2018a-Python-2.7.14
  Docutils/0.9.1-foss-2018a-Python-3.6.4                  (D)
  GObject-Introspection/1.60.1-GCCcore-8.2.0-Python-3.7.2
  IPython/5.7.0-foss-2018a-Python-2.7.14
  IPython/6.4.0-foss-2018a-Python-3.6.4
  IPython/7.7.0-foss-2019a-Python-3.7.2                   (D)
  Keras/2.2.4-foss-2019a-Python-3.7.2
  Mako/1.0.7-foss-2017b-Python-2.7.14
  Mako/1.0.7-foss-2018a-Python-2.7.14
  Mako/1.0.7-foss-2018b-Python-2.7.15
  Meson/0.50.0-GCCcore-8.2.0-Python-3.7.2
  Meson/0.51.2-GCCcore-8.3.0-Python-3.7.4                 (D)
  MultiQC/1.7-foss-2018b-Python-3.6.6
  NLTK/3.2.4-foss-2018a-Python-3.6.4
  NLTK/3.2.4-foss-2019a-Python-3.7.2                      (D)
  PyCairo/1.18.0-foss-2018b-Python-3.6.6
  PyTorch/1.3.1-fosscuda-2019b-Python-3.7.4
  PyYAML/3.13-foss-2018b-Python-3.6.6
  Python/2.7.14-foss-2017b
  Python/2.7.14-foss-2018a
  Python/2.7.14-GCCcore-6.4.0-bare
  Python/2.7.15-foss-2018b
  Python/2.7.15-GCCcore-7.3.0-bare
  Python/2.7.15-GCCcore-8.2.0
  Python/2.7.16-GCCcore-8.3.0
  Python/3.6.4-foss-2018a
  Python/3.6.6-foss-2018b
  Python/3.7.2-GCCcore-8.2.0
  Python/3.7.4-GCCcore-8.3.0                              (D)
  etc etc etc .......

To only search for the module Python itself, use

Code Block
module -r avail '^Python/'

SHARK

Code Block

me@res-hpc-lo02 ~]$ module avail python

--------------------- /share/modulefiles ----------------------
   genomics/ngs/tools/liftoff/1.5.1/python-3.8
   library/wxpython/4.1.0/gcc-8.3.1
   library/wxpython/4.1.1/gcc-8.3.1
   neuroImaging/SimpleElastix/0.10.0/python3.6.8
   system/python/2.7.17
   system/python/3.7.6
   system/python/3.8.1
   system/python/3.9.12
   system/python/3.9.13
   system/python/3.10.2                          (L,D)
   tools/miniconda/python2.7/4.7.12
   tools/miniconda/python3.8/4.8.5
   tools/miniconda/python3.8/4.9.2               (D)
   tools/miniconda/python3.9/4.12.0

In the above output, you can see that there are modules with the flag "(D)". This indicates that this is the default module for a software package for which modules for different versions exists.

If you want to get more information about a specific module, you can use the whatis sub-command, e.g.

ALICE

Code Block
[me@nodelogin02 ~]$ module whatis Python/3.7.4-GCCcore-8.3.0

SHARK

Code Block
[me@res-hpc-lo02 ~]$ module whatis system/python/3.10.2

Load modules

To load a module, use module load.

In the example below, we will use R to demonstrate how the module system works and what it does to your user environment.

Initially, R is not loaded and therefore the command Rscript is not available for use. We can test this by using the command which that looks for programs the same way that Bash does. We can use it to tell us where a particular piece of software is stored.

ALICE

Code Block

[me@nodelogin02 ~]$ which Rscript
/usr/bin/which: no Rscript in (/cm/shared/apps/slurm/20.11.9/sbin:/cm/shared/apps/slurm/20.11.9/bin:/cm/local/apps/gcc/8.2.0/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/me/.local/bin:/home/me/bin)

SHARK

Code Block

[me@res-hpc-lo02 ~]$ which Rscript
/usr/bin/which: no Rscript in (/share/software/system/python/3.10.2/bin:/home/me/.local/bin:/home/me/bin:/opt/TurboVNC/bin:/opt/VirtualGL/bin:/share/software/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/me/.dotnet/tools)

The list of paths in the brackets after “no Rscript in” comes from the environment variable $PATH.

Let us first find the a module for R which contains the Rscript command

ALICE

Code Block

[me@nodelogin02 ~]$ module -r avail '^R/'

----------------------------------------------------------------------------- /cm/shared/easybuild/modules/all -----------------------------------------------------------------------------
   R/3.6.0-foss-2019a-Python-3.7.2    R/3.6.0-foss-2019a    R/3.6.2-foss-2019b    R/3.6.2-fosscuda-2019b    R/4.0.0-foss-2020a    R/4.0.5-foss-2020b (D)

SHARK

Code Block

[me@res-hpc-lo02 ~]$ module -r avail '/R/'

------------------------------------------------------------------------------------ /share/modulefiles ------------------------------------------------------------------------------------
   statistical/R/3.4.4/gcc.8.3.1    statistical/R/3.6.0/gcc.8.3.1    statistical/R/3.6.3/gcc.8.3.1    statistical/R/4.1.0/gcc.8.3.1
   statistical/R/3.5.3/gcc.8.3.1    statistical/R/3.6.2/gcc.8.3.1    statistical/R/4.0.2/gcc.8.3.1    statistical/R/4.1.2/gcc.8.3.1

Now that we have found the module, we can load it, list all loaded modules just to make sure and check whether Rscript is now available:

ALICE

Code Block

[schulzrf@nodelogin02 ~]$ module load R/4.0.5-foss-2020b
[schulzrf@nodelogin02 ~]$ module list

Currently Loaded Modules:
  1) shared                            23) foss/2020b                         45) gettext/0.21-GCCcore-10.2.0         67) Szip/2.1.1-GCCcore-10.2.0
  2) DefaultModules                    24) bzip2/1.0.8-GCCcore-10.2.0         46) PCRE/8.44-GCCcore-10.2.0            68) HDF5/1.10.7-gompi-2020b
  3) gcc/8.2.0                         25) expat/2.2.9-GCCcore-10.2.0         47) GLib/2.66.1-GCCcore-10.2.0          69) UDUNITS/2.2.26-GCCcore-10.2.0
  4) slurm/20.11.9                     26) libpng/1.6.37-GCCcore-10.2.0       48) cairo/1.16.0-GCCcore-10.2.0         70) GSL/2.6-GCC-10.2.0
  5) GCCcore/10.2.0                    27) Brotli/1.0.9-GCCcore-10.2.0        49) libreadline/8.0-GCCcore-10.2.0      71) Ghostscript/9.53.3-GCCcore-10.2.0
  6) zlib/1.2.11-GCCcore-10.2.0        28) freetype/2.10.3-GCCcore-10.2.0     50) Tcl/8.6.10-GCCcore-10.2.0           72) JasPer/2.0.24-GCCcore-10.2.0
  7) binutils/2.35-GCCcore-10.2.0      29) ncurses/6.2-GCCcore-10.2.0         51) SQLite/3.33.0-GCCcore-10.2.0        73) LittleCMS/2.11-GCCcore-10.2.0
  8) GCC/10.2.0                        30) util-linux/2.36-GCCcore-10.2.0     52) PCRE2/10.35-GCCcore-10.2.0          74) ImageMagick/7.0.10-35-GCCcore-10.2.0
  9) numactl/2.0.13-GCCcore-10.2.0     31) fontconfig/2.13.92-GCCcore-10.2.0  53) NASM/2.15.05-GCCcore-10.2.0         75) GLPK/4.65-GCCcore-10.2.0
 10) XZ/5.2.5-GCCcore-10.2.0           32) xorg-macros/1.19.2-GCCcore-10.2.0  54) libjpeg-turbo/2.0.5-GCCcore-10.2.0  76) nodejs/12.19.0-GCCcore-10.2.0
 11) libxml2/2.9.10-GCCcore-10.2.0     33) X11/20201008-GCCcore-10.2.0        55) jbigkit/2.1-GCCcore-10.2.0          77) Python/3.8.6-GCCcore-10.2.0
 12) libpciaccess/0.16-GCCcore-10.2.0  34) gzip/1.10-GCCcore-10.2.0           56) LibTIFF/4.1.0-GCCcore-10.2.0        78) netCDF/4.7.4-gompi-2020b
 13) hwloc/2.2.0-GCCcore-10.2.0        35) lz4/1.9.2-GCCcore-10.2.0           57) Java/11.0.2                         79) GEOS/3.9.1-GCC-10.2.0
 14) libevent/2.1.12-GCCcore-10.2.0    36) zstd/1.4.5-GCCcore-10.2.0          58) Tk/8.6.10-GCCcore-10.2.0            80) PROJ/7.2.1-GCCcore-10.2.0
 15) UCX/1.9.0-GCCcore-10.2.0          37) libdrm/2.4.102-GCCcore-10.2.0      59) cURL/7.72.0-GCCcore-10.2.0          81) libgeotiff/1.6.0-GCCcore-10.2.0
 16) libfabric/1.11.0-GCCcore-10.2.0   38) libglvnd/1.3.2-GCCcore-10.2.0      60) GMP/6.2.0-GCCcore-10.2.0            82) pybind11/2.6.0-GCCcore-10.2.0
 17) PMIx/3.1.5-GCCcore-10.2.0         39) libunwind/1.4.0-GCCcore-10.2.0     61) NLopt/2.6.2-GCCcore-10.2.0          83) SciPy-bundle/2020.11-foss-2020b
 18) OpenMPI/4.0.5-GCC-10.2.0          40) LLVM/11.0.0-GCCcore-10.2.0         62) libogg/1.3.4-GCCcore-10.2.0         84) libtirpc/1.3.1-GCCcore-10.2.0
 19) OpenBLAS/0.3.12-GCC-10.2.0        41) Mesa/20.2.1-GCCcore-10.2.0         63) FLAC/1.3.3-GCCcore-10.2.0           85) HDF/4.2.15-GCCcore-10.2.0
 20) gompi/2020b                       42) libGLU/9.0.1-GCCcore-10.2.0        64) libvorbis/1.3.7-GCCcore-10.2.0      86) GDAL/3.2.1-foss-2020b
 21) FFTW/3.3.8-gompi-2020b            43) pixman/0.40.0-GCCcore-10.2.0       65) libsndfile/1.0.28-GCCcore-10.2.0    87) R/4.0.5-foss-2020b
 22) ScaLAPACK/2.1.0-gompi-2020b       44) libffi/3.3-GCCcore-10.2.0          66) ICU/67.1-GCCcore-10.2.0
 
[me@nodelogin02 ~]$ which Rscript
/cm/shared/easybuild/GenuineIntel/software/R/4.0.5-foss-2020b/bin/Rscript

Note
While you can also just use `module load R`, this is not recommended because this command will always load the default version of a module and the default may change when newer versions are being added.

Info
On ALICE, the long list of modules is completely fine because the module R depends on several other modules which have to be loaded in order for R to properly work.

SHARK

Code Block

[me@res-hpc-lo02 ~]$ module load statistical/R/4.1.2
[me@res-hpc-lo02 ~]$ module list

Currently Loaded Modules:

statistical/R/4.1.2/gcc.8.3.1

[em@res-hpc-lo02 ~]$ which Rscript
/share/software/statistical/R/R-4.1.2/bin/Rscript

So what just happened? To understand the output, first we need to understand the nature of the $PATH environment variable. $PATH is a special environment variable that controls where a Linux operating system (OS) looks for software. Specifically $PATH is a list of directories (separated by :) that the OS searches through for a command. As with all environment variables, we can print it using echo.

ALICE

This is how $PATH looks like after loading R on ALICE

Code Block

[me@nodelogin02 ~]$ echo $PATH
/cm/shared/easybuild/GenuineIntel/software/R/4.0.5-foss-2020b/bin:/cm/shared/easybuild/GenuineIntel/software/GDAL/3.2.1-foss-2020b/bin:/cm/shared/easybuild/GenuineIntel/software/HDF/4.2.15-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/SciPy-bundle/2020.11-foss-2020b/bin:/cm/shared/easybuild/GenuineIntel/software/pybind11/2.6.0-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/libgeotiff/1.6.0-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/PROJ/7.2.1-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/GEOS/3.9.1-GCC-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/netCDF/4.7.4-gompi-2020b/bin:/cm/shared/easybuild/GenuineIntel/software/Python/3.8.6-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/nodejs/12.19.0-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/GLPK/4.65-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/ImageMagick/7.0.10-35-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/LittleCMS/2.11-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/JasPer/2.0.24-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/Ghostscript/9.53.3-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/GSL/2.6-GCC-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/UDUNITS/2.2.26-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/HDF5/1.10.7-gompi-2020b/bin:/cm/shared/easybuild/GenuineIntel/software/ICU/67.1-GCCcore-10.2.0/sbin:/cm/shared/easybuild/GenuineIntel/software/ICU/67.1-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/libsndfile/1.0.28-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/FLAC/1.3.3-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/cURL/7.72.0-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/Tk/8.6.10-GCCcore-10.2.0/bin:/cm/shared/easybuild/software/Java/11.0.2:/cm/shared/easybuild/software/Java/11.0.2/bin:/cm/shared/easybuild/GenuineIntel/software/LibTIFF/4.1.0-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/jbigkit/2.1-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/libjpeg-turbo/2.0.5-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/NASM/2.15.05-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/PCRE2/10.35-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/SQLite/3.33.0-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/Tcl/8.6.10-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/cairo/1.16.0-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/GLib/2.66.1-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/PCRE/8.44-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/gettext/0.21-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/LLVM/11.0.0-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/zstd/1.4.5-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/lz4/1.9.2-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/gzip/1.10-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/X11/20201008-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/fontconfig/2.13.92-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/util-linux/2.36-GCCcore-10.2.0/sbin:/cm/shared/easybuild/GenuineIntel/software/util-linux/2.36-GCCcore-10.2.0/bin:/cm/shared/easybuild/software/ncurses/6.2-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/freetype/2.10.3-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/Brotli/1.0.9-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/libpng/1.6.37-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/expat/2.2.9-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/bzip2/1.0.8-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/FFTW/3.3.8-gompi-2020b/bin:/cm/shared/easybuild/GenuineIntel/software/OpenMPI/4.0.5-GCC-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/PMIx/3.1.5-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/libfabric/1.11.0-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/UCX/1.9.0-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/libevent/2.1.12-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/hwloc/2.2.0-GCCcore-10.2.0/sbin:/cm/shared/easybuild/GenuineIntel/software/hwloc/2.2.0-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/libxml2/2.9.10-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/XZ/5.2.5-GCCcore-10.2.0/bin:/cm/shared/easybuild/GenuineIntel/software/numactl/2.0.13-GCCcore-10.2.0/bin:/cm/shared/easybuild/software/binutils/2.35-GCCcore-10.2.0/bin:/cm/shared/easybuild/software/GCCcore/10.2.0/bin:/cm/shared/apps/slurm/20.11.9/sbin:/cm/shared/apps/slurm/20.11.9/bin:/cm/local/apps/gcc/8.2.0/bin:/usr/lib/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/me/.local/bin:/home/me/bin

SHARK

This is how $PATH looks like after loading R on SHARK

Code Block

[rfschulz@res-hpc-lo02 ~]$ echo $PATH
/share/software/statistical/R/R-4.1.2/bin:/home/me/.local/bin:/home/me/bin:/opt/TurboVNC/bin:/opt/VirtualGL/bin:/share/software/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/me/.dotnet/tools

After loading the module for R, the $PATH variable has been changed to include the directories holding executables of the newly loaded modules.

Taking this to its conclusion, module load adds software to your $PATH. It “loads” software.

Note
Careful when you load several modules. It is possible that their dependencies can cause conflicts in particular with respect to different compilers which can create problems later on. It is best to always check what other modules have been automatically loaded.

Also a note of warning: When you load several modules, it is possible that their dependencies can cause conflicts and problems later on. It is best to always check what other modules have been automatically loaded.

Unload modules

The command module unload “un-loads” a module. Using the above example of R

ALICE

Code Block

[m@nodelogin02 ~]$ module unload R/4.0.5-foss-2020b
[me@nodelogin02 ~]$ module list

Currently Loaded Modules:
  1) shared                            23) foss/2020b                         45) gettext/0.21-GCCcore-10.2.0         67) HDF5/1.10.7-gompi-2020b
  2) DefaultModules                    24) bzip2/1.0.8-GCCcore-10.2.0         46) PCRE/8.44-GCCcore-10.2.0            68) UDUNITS/2.2.26-GCCcore-10.2.0
  3) gcc/8.2.0                         25) expat/2.2.9-GCCcore-10.2.0         47) GLib/2.66.1-GCCcore-10.2.0          69) GSL/2.6-GCC-10.2.0
  4) slurm/20.11.9                     26) libpng/1.6.37-GCCcore-10.2.0       48) cairo/1.16.0-GCCcore-10.2.0         70) Ghostscript/9.53.3-GCCcore-10.2.0
  5) GCCcore/10.2.0                    27) Brotli/1.0.9-GCCcore-10.2.0        49) libreadline/8.0-GCCcore-10.2.0      71) JasPer/2.0.24-GCCcore-10.2.0
  6) zlib/1.2.11-GCCcore-10.2.0        28) freetype/2.10.3-GCCcore-10.2.0     50) Tcl/8.6.10-GCCcore-10.2.0           72) LittleCMS/2.11-GCCcore-10.2.0
  7) binutils/2.35-GCCcore-10.2.0      29) ncurses/6.2-GCCcore-10.2.0         51) SQLite/3.33.0-GCCcore-10.2.0        73) ImageMagick/7.0.10-35-GCCcore-10.2.0
  8) GCC/10.2.0                        30) util-linux/2.36-GCCcore-10.2.0     52) PCRE2/10.35-GCCcore-10.2.0          74) GLPK/4.65-GCCcore-10.2.0
  9) numactl/2.0.13-GCCcore-10.2.0     31) fontconfig/2.13.92-GCCcore-10.2.0  53) NASM/2.15.05-GCCcore-10.2.0         75) nodejs/12.19.0-GCCcore-10.2.0
 10) XZ/5.2.5-GCCcore-10.2.0           32) xorg-macros/1.19.2-GCCcore-10.2.0  54) libjpeg-turbo/2.0.5-GCCcore-10.2.0  76) Python/3.8.6-GCCcore-10.2.0
 11) libxml2/2.9.10-GCCcore-10.2.0     33) X11/20201008-GCCcore-10.2.0        55) jbigkit/2.1-GCCcore-10.2.0          77) netCDF/4.7.4-gompi-2020b
 12) libpciaccess/0.16-GCCcore-10.2.0  34) gzip/1.10-GCCcore-10.2.0           56) LibTIFF/4.1.0-GCCcore-10.2.0        78) GEOS/3.9.1-GCC-10.2.0
 13) hwloc/2.2.0-GCCcore-10.2.0        35) lz4/1.9.2-GCCcore-10.2.0           57) Tk/8.6.10-GCCcore-10.2.0            79) PROJ/7.2.1-GCCcore-10.2.0
 14) libevent/2.1.12-GCCcore-10.2.0    36) zstd/1.4.5-GCCcore-10.2.0          58) cURL/7.72.0-GCCcore-10.2.0          80) libgeotiff/1.6.0-GCCcore-10.2.0
 15) UCX/1.9.0-GCCcore-10.2.0          37) libdrm/2.4.102-GCCcore-10.2.0      59) GMP/6.2.0-GCCcore-10.2.0            81) pybind11/2.6.0-GCCcore-10.2.0
 16) libfabric/1.11.0-GCCcore-10.2.0   38) libglvnd/1.3.2-GCCcore-10.2.0      60) NLopt/2.6.2-GCCcore-10.2.0          82) SciPy-bundle/2020.11-foss-2020b
 17) PMIx/3.1.5-GCCcore-10.2.0         39) libunwind/1.4.0-GCCcore-10.2.0     61) libogg/1.3.4-GCCcore-10.2.0         83) libtirpc/1.3.1-GCCcore-10.2.0
 18) OpenMPI/4.0.5-GCC-10.2.0          40) LLVM/11.0.0-GCCcore-10.2.0         62) FLAC/1.3.3-GCCcore-10.2.0           84) HDF/4.2.15-GCCcore-10.2.0
 19) OpenBLAS/0.3.12-GCC-10.2.0        41) Mesa/20.2.1-GCCcore-10.2.0         63) libvorbis/1.3.7-GCCcore-10.2.0      85) GDAL/3.2.1-foss-2020b
 20) gompi/2020b                       42) libGLU/9.0.1-GCCcore-10.2.0        64) libsndfile/1.0.28-GCCcore-10.2.0
 21) FFTW/3.3.8-gompi-2020b            43) pixman/0.40.0-GCCcore-10.2.0       65) ICU/67.1-GCCcore-10.2.0
 22) ScaLAPACK/2.1.0-gompi-2020b       44) libffi/3.3-GCCcore-10.2.0          66) Szip/2.1.1-GCCcore-10.2.0

As you can see, the module for R was “unloaded”.

SHARK

Code Block
[me@res-hpc-lo02 ~]$ module unload statistical/R/4.1.2/gcc.8.3.1 [me@res-hpc-lo02 ~]$ module list No modules loaded

Note
IMPORTANT: Unloading a module does not unload its dependencies.

If you want to unload all currently loaded modules, you can use the command module purge.

Note

For ALICE users

Using module purge will also remove the module for Slurm, but you can quickly load it again by running module load slurm (There is only one version of slurm available on the cluster, so no need to specify the version here)

Info

Pro tip

You can also use module purge in your batch script before loading the modules that you need in order to make sure that no other modules can interfere with your job environment.

Saving and restoring (default) modules

If you work with a standard set of modules quite a lot, you can save and restore to the list of modules at any time.

Let’s start from a clean module environment:

Code Block
module purge

ALICE

Note
On ALICE, you should always include the module `slurm`. Otherwise, you will not be able to access Slurm commands when you restore the module environment

Load a set of modules and save them as default

Code Block
[me@nodelogin02 ~]$ module load slurm Python/3.10.4-GCCcore-11.3.0 git/2.36.0-GCCcore-11.3.0-nodocs [me@nodelogin02 ~]$ module save Saved current collection of modules to: "default"

If you need to restore them:

Code Block
[me@nodelogin02 ~]$ module restore Restoring modules from user's default

To test the effect of restoring them, either load another module or purge the module environment. You can check with module list that the default modules have been loaded.

You can can also define multiple different module environments by specifying a name when saving the module list:

Code Block
[me@nodelogin02 ~]$ module load slurm Python/3.10.4-GCCcore-11.3.0 git/2.36.0-GCCcore-11.3.0-nodocs [me@nodelogin02 ~]$ module save python Saved current collection of modules to: "python"

And restoring works like this:

Code Block
[me@nodelogin02 ~]$ module restore python Restoring modules from user's python

SHARK

Load a set of modules and save them as default

Code Block
[me@res-hpc-lo02 ~]$ module load system/python/3.10.2 [me@res-hpc-lo02 ~]$ module save Saved current collection of modules to: "default"

If you need to restore them:

Code Block
[me@res-hpc-lo02 ~]$ module restore Restoring modules from user's default

To test the effect of restoring them, either load another module or purge the module environment. You can check with module list that the default modules have been loaded.

You can can also define multiple different module environments by specifying a name when saving the module list:

Code Block
[me@res-hpc-lo02 ~]$ module load system/python/3.10.2 [me@res-hpc-lo02 ~]$ module save python Saved current collection of modules to: "python"

And restoring works like this:

Code Block
[me@res-hpc-lo02 ~]$ module restore python Restoring modules from user's python

Note

Note that using module restore in your batch scripts has an impact on the reproducibility of your batch scripts. When you want to check later on what modules (and versions) were loaded for a job, this information is no longer available in your batch script.

If you want to make use of module restore, we recommend to add the command module list after the restore command in your batch script. This way, the Slurm output file for the job will contain the list of modules and versions used.

Note
The command `module save` will always save the full list of modules with the version used even if you loaded the modules without specifying a version.

You can get a full list of saved module environments like this:

Code Block
module savelist

Disabling an saved collection of modules can be done with

Code Block
module disable <name>

where <name> should be replaced by the name of the collection. This will not entirely remove the file though. This has to be done manually.

The various collections of modules are stored in your home directory in: ~/.lmod.d

Versions Compared

Old Version 1

New Version 2

Key

Basic usage

The module structure on ALICE and SHARK

List currently loaded modules

List available modules

Finding specific modules

Load modules

Unload modules

Saving and restoring (default) modules

Page Comparison

Versions Compared

Old Version 1

New Version 2

Key

Basic usage

The module structure on ALICE and SHARK

List currently loaded modules

List available modules

Finding specific modules

Load modules

Unload modules

Saving and restoring (default) modules