Building With Spack

Magellano supports software installation using the Spack package manager.

Spack is a multi-platform package manager designed for supercomputing centers. It allows unprivileged users to build and install multiple versions of scientific software on a variety of platforms and environments. Learn more about Spack in the official documentation.

Note

You should always perform build workload inside a Slurm job runtime. Never run Spack on the Magellano login shell!

Quick Start

To begin using Spack, load the environment module with the following command:

module load spack

If this is the first time you use Spack on Magellano, start by bootstrapping its dependencies. Use the following command:

spack bootstrap now

Note

The first command you run with Spack may take a while, as Spack builds caches to speed up future commands.

When Spack is done compiling, check the status as follows:

spack bootstrap status

If everything went fine, you should see both Core Functionalities and Binary packages in [PASS] status:

Spack v1.0.4 - python@3.9

[PASS] Core Functionalities

[PASS] Binary packages

Now, get a list of available packages from the source repository:

spack list

Try to install your first package:

spack install cowsay

After the installation, load and test the package:

$ spack load cowsay
$ cowsay ciao!
_______
< ciao! >
-------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Configuration

Spack data are stored by default under the $HOME/.spack path. This path contains the user’s custom configuration, the source cache and the build products.

You can configure the behavior of your Spack with the spack config command. To get a list of all configuration sections:

spack config list

You can edit specific configuration sections. For example:

spack config edit config

This command will open the corresponding configuration file in an editor. Alternatively, use the add and remove subcommands to edit specific parameters. All parameters set via spack config will override the defaults.

Read the official documentation to learn more about Spack configuration.

Examples

Change the path of the Spack build products (install_tree):

spack config add 'config:install_tree:root: /my/custom/installation/path'

See the current configuration:

$ spack config --scope user get config
config:
  install_tree:
    root: /my/custom/installation/path

Remove a custom parameter:

spack config remove 'config:install_tree:root'

Switch the granulatiry of the target architectures from the default microarchitectures (e.g. zen4) to generic (e.g. x86_64_v3 ):

spack config add 'concretizer:targets:granularity: generic'

See the current configuration:

$ spack config --scope user get concretizer
concretizer:
  targets:
    granularity: generic

Print the aggregated (default + user) configuration section:

spack config get concretizer

Basic usage

The metadata and sources of Spack packages are stored on its remote package repository. Whenever you issue the spack install command, Spack solves the full chain of dependencies. Then, is compiles the dependencies and the requested package. Finally, the build products are stored and organized under the Spack install_tree.

To get a full list of available packages, use the following command:

spack list

To get information about a specific package (e.g. openmpi), run:

spack info openmpi

The output provides the list of available versions, the full set of build flags (variants) and the package dependencies. When installing a package, you can optionally specify a version, a set of build flags and the target architecture.

With the spec command, you can check the dependency tree without installing:

spack spec openmpi@5.0.8 fabrics=auto schedulers=slurm +internal-hwloc target=x86_64_v3

If you are satisfied with the setup, issue the install command:

spack install openmpi@5.0.8 fabrics=auto schedulers=slurm +internal-hwloc target=x86_64_v3

Load the package and check the path:

$ spack load openmpi
$ which mpiexec
/home/users/neo/.spack/opt/spack/linux-x86_64_v4/openmpi-5.0.8-ybmk5fessnyfa4ix5f2htywye4iwe6qp/bin/mpiexec

To unload a packages:

spack unload openmpi

Finally, to remove an obsolete package:

spack uninstall openmpi@5.0.8

Learn more Spack fundamentals on the official documentation.

Spack Environments

Spack environments are self-contained and reproducible software collections, similar to Python’s virtual environments. They do not contain copies of the software, but they reference installations in the Spack install_tree to reduce duplication.

Hint

Altough Spack is capable of installing Python packages, the recommended tool for managing Python environments is Python’s native venv tool. See Setting Up Your Environment.

Create and activate a new Spack environment as follows:

spack env create myenv
spack env activate myenv

To verify that you are currently inside the environment:

$ spack env status
==> In environment myenv

You can now add packages to this environment:

spack add openmpi@5.0.8 openblas@0.3.29 julia

Solve the proper set of dependencies for your requested packages:

spack concretize

Install everything at once:

spack install

To deactivate a Spack managed environment:

spack env deactivate

Finally, to remove an existing environment:

spack env rm myenv

Learn more about Spack managed environments in the official documentation.