Software and the Module System

DeepThought comes with both pre-built software packages (set up by IDS Research Success) and you can also run your own software as long as it doesn’t require root/admin access.

Note

Control groups have been put in place on the Login Node to prevent applications from being able to inadvertently impact the experience of other users. Each user is restricted to 4 CPUs (2 Core/4 Thread) and 16GB of RAM so we recommend requesting resources via SLURM when compiling or running your software

The Module Load System

DeepThought uses the LMod (Load MODules) system to provide most of the software that we have installed. The best way to think of a Module is a singular program version and all of its associated dependencies to run correctly.

module load allows you to load/unload applications in the command line as you need them instead of keeping everything loaded at all times. Any software modules you use frequently can be loaded on login using the .bash_profile file in your home directory.

A list of modules available on DeepThought is available via the Open OnDemand Web Portal’s module browser. This only lists visible modules (we hide dependencies to reduce visual clutter) so remember to also check module --show-hidden avail or use module spider <name if the package you want isn’t listed.

Tip

Modules required for a job should be added to your SLURM script as all modules will be automatically purged as part of starting each job to give you a clean slate to work with

Using Python / Conda

If the code or software you want to run is mostly Python-based, you can use the inbuilt package management tools (such as pip) inside your virtual environments to install the packages you need.

Tip

You can use module load conda to load the pre-packaged version of Miniconda3 using Lmod

You can also use the Jupyter Notebook app in the Open OnDemand Web Portal to quickly run code without needing to directly work in the command line.

Important

Python, Conda and pip all create folders in your home directory by default and do not clear out their cache directories. As your home directory has a quota it is recommended that you regularly clear out the caches to save space