Building With Spack
Magellano supports software installation using the Spack package manager.
Spack is a multi-platform package manager designed for supercomputing centers. It allows unprivileged users to build and install multiple versions of scientific software on a variety of platforms and environments. Learn more about Spack in the official documentation.
Note
You should always perform build workload inside a Slurm job runtime. Never run Spack on the Magellano login shell!
Quick Start
To begin using Spack, load the environment module with the following command:
module load spack
If this is the first time you use Spack on Magellano, start by bootstrapping its dependencies. Use the following command:
spack bootstrap now
Note
The first command you run with Spack may take a while, as Spack builds caches to speed up future commands.
When Spack is done compiling, check the status as follows:
spack bootstrap status
If everything went fine, you should see both Core Functionalities and Binary packages in [PASS] status:
Spack v1.0.4 - python@3.9
[PASS] Core Functionalities
[PASS] Binary packages
Now, get a list of available packages from the source repository:
spack list
Try to install your first package:
spack install cowsay
After the installation, load and test the package:
$ spack load cowsay
$ cowsay ciao!
_______
< ciao! >
-------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
Configuration
Spack data are stored by default under the $HOME/.spack path.
This path contains the user’s custom configuration, the source cache and the build products.
You can configure the behavior of your Spack with the spack config command.
To get a list of all configuration sections:
spack config list
You can edit specific configuration sections. For example:
spack config edit config
This command will open the corresponding configuration file in an editor.
Alternatively, use the add and remove subcommands to edit specific parameters.
All parameters set via spack config will override the defaults.
Read the official documentation to learn more about Spack configuration.
Examples
Change the path of the Spack build products (install_tree):
spack config add 'config:install_tree:root: /my/custom/installation/path'
See the current configuration:
$ spack config --scope user get config
config:
install_tree:
root: /my/custom/installation/path
Remove a custom parameter:
spack config remove 'config:install_tree:root'
Switch the granulatiry of the target architectures from the default microarchitectures (e.g. zen4) to generic (e.g. x86_64_v3 ):
spack config add 'concretizer:targets:granularity: generic'
See the current configuration:
$ spack config --scope user get concretizer
concretizer:
targets:
granularity: generic
Print the aggregated (default + user) configuration section:
spack config get concretizer
Basic usage
The metadata and sources of Spack packages are stored on its remote package repository.
Whenever you issue the spack install command, Spack solves the full chain of dependencies.
Then, is compiles the dependencies and the requested package.
Finally, the build products are stored and organized under the Spack install_tree.
To get a full list of available packages, use the following command:
spack list
To get information about a specific package (e.g. openmpi), run:
spack info openmpi
The output provides the list of available versions, the full set of build flags (variants) and the package dependencies. When installing a package, you can optionally specify a version, a set of build flags and the target architecture.
With the spec command, you can check the dependency tree without installing:
spack spec openmpi@5.0.8 fabrics=auto schedulers=slurm +internal-hwloc target=x86_64_v3
If you are satisfied with the setup, issue the install command:
spack install openmpi@5.0.8 fabrics=auto schedulers=slurm +internal-hwloc target=x86_64_v3
Load the package and check the path:
$ spack load openmpi
$ which mpiexec
/home/users/neo/.spack/opt/spack/linux-x86_64_v4/openmpi-5.0.8-ybmk5fessnyfa4ix5f2htywye4iwe6qp/bin/mpiexec
To unload a packages:
spack unload openmpi
Finally, to remove an obsolete package:
spack uninstall openmpi@5.0.8
Learn more Spack fundamentals on the official documentation.
Spack Environments
Spack environments are self-contained and reproducible software collections, similar to Python’s virtual environments.
They do not contain copies of the software, but they reference installations in the Spack install_tree to reduce duplication.
Hint
Altough Spack is capable of installing Python packages, the recommended tool for managing Python environments
is Python’s native venv tool. See Setting Up Your Environment.
Create and activate a new Spack environment as follows:
spack env create myenv
spack env activate myenv
To verify that you are currently inside the environment:
$ spack env status
==> In environment myenv
You can now add packages to this environment:
spack add openmpi@5.0.8 openblas@0.3.29 julia
Solve the proper set of dependencies for your requested packages:
spack concretize
Install everything at once:
spack install
To deactivate a Spack managed environment:
spack env deactivate
Finally, to remove an existing environment:
spack env rm myenv
Learn more about Spack managed environments in the official documentation.