-
Notifications
You must be signed in to change notification settings - Fork 2
Modules
In order to provide users with access to a range of software, the ACAD servers provide a number of software modules via lmod. Typically, the user will need to load a module or modules prior to running an analyses. Please read the lmod documentation for the complete details on how the module system functions. Below is a basic introduction, including a more specific description on how modules are used on the ACAD servers.
Avoid modules with foss-2016a in the name, instead favour modules with foss-2016b.
Add the following to your ~/.bash_profile
export MODULEPATH=/data/acad/apps/modules/all:$MODULEPATH
Delete module cache, which may be required the first time you update your MODULEPATH.
$ rm -r $HOME/.lmod.d
List all available modules
$ module avail
Search for modules named 'python'
$ module spider python
Load a python module
$ module load Python
Load a specific version of the python module
$ module load Python/2.7.13-foss-2016b
Unload the python module
$ module unload Python
List currently loaded modules
$ module list
When you type a command in your Terminal
, there is some software known as the shell which interprets the command. The default shell on Mac and many Linux systems is bash
, but other shells behave similarly. A command typed into the shell may refer to either a shell builtin (such as cd
and echo
), a shell alias/function, or a standalone program (such as vim
or gzip
). The shell first compares your command with its list of builtins, if not found it then compares the command with aliases and functions, and if still not found it looks for standalone programs in your PATH
. If the command exists in multiple places, the first one takes precedence (builtin,alias,function,order in PATH) and if no corresponding command is found, an error is reported.
PATH
is a special environment variable containing a colon separated list of directories. You can look at the value of an environment variable using the echo
builtin. Note that when referring to environment variables, the variable has a dollar symbol prefix.
$ echo $PATH /usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin
The above is a standard set of locations for binary files on Unix-like systems. The type
builtin can be used to determine whether a command is a builtin, alias, function, or a standalone program.
$ type type type is a shell builtin $ type ls ls is aliased to `ls --color=auto' $ type vim vim is /usr/bin/vim $ type fixworldpoverty -bash: type: fixworldpoverty: not found
Note that when we asked for the type
of command vim
, we were told its location. For the ls
command, we're told its an alias and what the alias is. An alias is a simple translation of one command into another command, and in this case ls
is an alias to ls --color=auto
. In fact, ls
is a standalone program and the alias gives it a default parameter without it having to be typed in full each time. We can find the location of ls
in the PATH too, using the which
command.
$ which ls alias ls='ls --color=auto' /bin/ls
The module
command can be used to load or unload software. Loading software causes new directories to be prepended to your PATH
.
$ echo $PATH /usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin $ module load Python $ echo $PATH /apps/software/Python/2.7.13-foss-2016b/bin:/apps/software/SQLite/3.13.0-foss-2016b/bin:/apps/software/Tcl/8.6.5-foss-2016b/bin:/apps/software/libreadline/6.3-foss-2016b/bin:/apps/software/ncurses/6.0-foss-2016b/bin:/apps/software/bzip2/1.0.6-foss-2016b/bin:/apps/software/FFTW/3.3.4-gompi-2016b/bin:/apps/software/OpenBLAS/0.2.18-GCC-5.4.0-2.26-LAPACK-3.6.1/bin:/apps/software/OpenMPI/1.10.3-GCC-5.4.0-2.26/bin:/apps/software/hwloc/1.11.3-GCC-5.4.0-2.26/sbin:/apps/software/hwloc/1.11.3-GCC-5.4.0-2.26/bin:/apps/software/numactl/2.0.11-GCC-5.4.0-2.26/bin:/apps/software/binutils/2.26-GCCcore-5.4.0/bin:/apps/software/GCCcore/5.4.0/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin
$ type module module is a function module () { eval $($LMOD_CMD bash "$@"); [ $? = 0 ] && eval $(${LMOD_SETTARG_CMD:-:} -s sh) } $ echo $LMOD_CMD /usr/share/lmod/lmod/libexec/lmod $ file /usr/share/lmod/lmod/libexec/lmod /usr/share/lmod/lmod/libexec/lmod: a /usr/bin/lua script text executable
The module
command is a shell function. The shell is a fully turing-complete programming language, and as such functions can be written in this language. type
shows us the function definition. This function evaluates whatever is in the LMOD_CMD
environment variable and passes it the 'bash' parameter and also any additional parameters given to the module command ($@
). file
shows the type of file for the lmod
executable. Lmod is written in the Lua programming language.
Looking at the lmod documentation, we find that lmod looks for modules in directories listed in the MODULEPATH
environment variable. Actually, when we load a module, lmod looks in the MODULEPATH directories for a script (written in either the TCL or Lua programming languages) with the corresponding name and then runs that script. The module script for the software is responsible for modifying PATH and other environment variables, and may load additional modules which are dependencies. So the software might really be installed anywhere, as long as the module script for that software modifies the PATH
accordingly.
$ echo $MODULEPATH /apps/modules/all:/usr/share/modulefiles/Linux:/usr/share/modulefiles/Core:/usr/share/lmod/lmod/modulefiles/Core $ ls /apps/modules/all AdapterRemoval BLAST FastQC HDF5 metafast QIIME AdmixTools BLAST+ FASTX-Toolkit HTSlib NASM R Anaconda2 Boost FFTW hwloc ncurses RAxML Anaconda3 Bowtie flex Java netCDF SAMtools angsd Bowtie2 foss Kaiju numactl ScaLAPACK ART BWA GATK libcerf OpenBLAS seqtk Autoconf bwa-meth GCC libevent OpenMPI SQLite Automake bwa-pssm GCCcore libgtextutils paleomix Szip Autotools bzip2 GDAL libjpeg-turbo Pango tabix BCFtools cairo GDB libpng parallel taxator-tk bcl2fastq capnproto gdsl libreadline PCRE Tcl bedops CMake gettext LibTIFF Perl Tk BEDOPS cURL GHC libtool picard tmux BEDTools cutadapt ghostscript libxml2 PileOMeth toolshed binutils Doxygen GMAP-GSNAP libxslt PLINK treemix bioawk EasyBuild GMP M4 preseq VCFtools Biopython EIGENSOFT gnuplot MAFFT PROJ XZ Bismark ExaML gompi mapDamage Pysam zlib Bison expat GSL mash Python $ ls /apps/modules/all/Python/ 2.7.11-foss-2016a 2.7.13-foss-2016b.lua
By modifying the MODULEPATH
, we can tell lmod to search additional locations for module scripts.
$ ls /data/acad/apps/modules/all AdapterRemoval Boost HTSlib mapDamage SAMtools ART BWA HUMAnN paleomix-meta SCons BCFtools freetype Kaiju Python-packages texlive BEDTools grg-utils LMAT R-packages $ ls /data/acad/apps/modules/all/LMAT 1.2.6-foss-2016b.lua $ export MODULEPATH=/data/acad/apps/modules/all:$MODULEPATH $ module load LMAT/1.2.6-foss-2016b
You can save your changes to the MODULEPATH
by adding the export line to the file $HOME/.bash_profile
, a script that is run by your shell when you logon.