Setting Up Your Environment
This page explains how to set up your bash or Python environment in a Slurm job runtime. We adopt a modular approach to software and package distribution.
The information reported here applies to all jobs submitted both from the Magellano shell and the OnDemand interface. See also User Interfaces.
Environment Modules (module)
The environment-modules utility allows you to load packaged and libraries from a shared location.
The job runtime environment is a minimal AlmaLinux9 instance.
Only a small set of packages are installed in that environment.
With the module you can browse an load software into your session.
You can get a list of available modules with the following command:
module avail
This will print a list of softwares grouped by platform and/or type.
You can load one or more modules into your environment. For example, to load the latest installed version of Python:
module load python
This will append the correct binary paths to your PATH environment variable, so that Python will be available in your shell:
$ which python
/cvmfs/software.pi.infn.it/spack/prod/linux-x86_64_v3/python-3.13.5-jvkjfssfgkupkpfczeupvfat7w6nekjv/bin/python
You can specify the exact version of the module you want to load:
$ module load python/3.13.5-gcc-11.5.0
$ python --version
Python 3.13.5
Finally, you can unload a module as follows:
module unload python
Modules can be loaded in the job runtime by adding the proper module commands in your batch script.
Note
Modules are loaded into your current session. When starting a new session, you will have to reload them.
You can request the installation of new modules by issuing a ticket to our Service Desk. For more info about Environment Modules, see the official documentation.
Python Virtual Environments (venv)
A virtual environment is an isolated directory tree where you can install the dependencies of your Python code.
In Magellano, Python versions are distributed as modules, as explained above. You can install Python packages by loading the desired Python version and creating a virtual environment.
First, create a virtual environment in a chosen path (VENV_ROOT).
A typical choice is to put it in a subdirectory of your home directory:
module load python
python -m venv $HOME/my_environment
In this case VENV_ROOT is $HOME/my_environment.
After creating the environment, you can activate it by sourcing the activation script in the VENV_ROOT/bin directory:
source $VENV_ROOT/bin/activate
When your virtual environment is active, the name of the environment will appear in your command prompt between brackets (for example, (my_environment)).
Hint
After creation, you won’t need to load the Python module again.
The path to the correct Python version is coded in the activate script.
With your environment activated, you can install Python packages using PyPI. For example:
pip install pandas
You can easily check that the packages have been installed under VENV_ROOT:
(my_environment) $ python -c "import pandas; print(pandas.__file__)"
/gpfs/ddn/users/neo/my_environment/lib/python3.13/site-packages/pandas/__init__.py
You can deactivate your environment with the deactivate command.
Hint
It’s a good practice to have separated Virtual Environments for different use cases.
You can request the installation of shared Python Virtual Environments by issuing a ticket to our Service Desk. Learn more about Python Virtual Environments in the official documentation.