Skip to content

Containers

The container represents an individual unit with the appropriate set of software, including the appropriate libraries and settings for efficient operation in the environment. The technology enables reliability, portability, isolation, security, and launch of container software inside or outside the command line. The latter allows the software to be adapted to the supercomputer environment. Individual distributions enable different approaches and mechanisms that provide flexibility and configuration of the environment. The Vega cluster already has the Apptainer container platform installed and adapted to work on supercomputers (HPC).

On HPC Vega, containers are available on $PATH: /ceph/hpc/software/containers/singularity

Apptainer containers

Apptainer is a platform designed to create and run containers within a supercomputer environment. Containers have access to a common operating system, file system, and software installed on nodes. The latter takes into account the privileges of the operating system (in our case RHEL), which allows the use and transfer of the appropriate rights of the system user account. If we decide to create our own container, we can install software, libraries and adapt the environment within it according to our needs. Due to truncated isolation, we need to pay attention to the adequacy, the compatibility of the environment we establish. Otherwise, the container will not work properly or will not work at all. Apptainer provides a high level of portability – Singularity Image Format (SIF) and repeatability of containers, with the help of definition or description files (.def / .dsc).

Apptainer (Formerly Singularity) is an open source fork of Singularity. They are mostly interchangeable, so you can easily adapt your singularity workflows to work with apptainer aswell.

More information is available in the official documentation: https://apptainer.org/documentation/

Apptainer Commands

If you do not want to build your own container, you can use pre-built containers from public repositories (e.g. Docker Hub, Sylabs Library, etc.).

The apptainer pull command downloads the selected container to the VEGA file system and saves it as a .sif file, which can later be used to run applications.

General command syntax:

apptainer pull <image-name>.sif <hub>://<image>[:tag]

where:

  • <image-name>.sif specifies the name of the file that will be created in your account,
  • <hub> specifies the source repository (e.g. docker, library),
  • <image> specifies the container image name from the selected repository,
  • [:tag] is an optional container version tag (e.g. latest, 2.15, cuda12).

The following command downloads the latest version of the TensorFlow container from Docker Hub:

apptainer pull tensorflow.sif docker://tensorflow/tensorflow:latest

After a successful download, a .sif file will be created in the current directory, for example container.sif. This file can then be used to run applications inside the container.

Running a Container

Once you have downloaded a container as a .sif file, you can run it in several ways, depending on what you want to do.

The run command starts the default application configured within the container:

apptainer run container.sif

To exit the container, use the exit command.

The exec command executes a specific command inside the container:

apptainer exec container.sif <command>

Example:

apptainer exec tensorflow.sif python --version

You can also use the exec command to run scripts inside the container.

Example:

apptainer exec tensorflow.sif python script.py

Using exec is the most common way of working with containers on HPC systems because it allows individual programs to be executed inside the container.

An interactive shell inside the container can be started with the shell command:

apptainer shell container.sif

You can then execute commands directly inside the container.

Example:

Apptainer> python --version
Apptainer> ls
Apptainer> pwd

To exit the container, use the exit command.

Inspecting Container Information

To display basic information about a container, use:

apptainer inspect container.sif

To display the command that is executed when using apptainer run, use:

apptainer inspect --runscript container.sif

To display the container definition file, use:

apptainer inspect --deffile container.sif

Some containers include additional usage instructions, which can be displayed with:

apptainer run-help container.sif

To view all available commands, use:

apptainer help

To display help for a specific command, use:

apptainer help <command>

Creating Your Own Container

Preparing your own container is possible in several ways, including combining them. You can also use pre-prepared containers, arrange them and adapt them to your needs. However, if this does not suit you, it is possible to create your own container with the help of a definition file. The latter allows a higher level of repeatability of the container construction itself (you always get the same result, it is necessary to be careful not to use the latest tags).

Templates are available from various bootstrap repositories, including:

A complete list of supported bootstrap repositories is available in the official documentation: https://apptainer.org/docs/user/latest/definition_files.html

Additional Features and Commonly Used Options

The Singularity Image Format (SIF) is a read-only container format designed for portability. If modifications to a container are required, it can be converted into a sandbox format or extended using the Persistent Overlays feature, which is available through the --overlay option.

More information is available in the https://apptainer.org/docs/user/latest/persistent_overlays.html.

If a container has been created using the --sandbox option, its contents can be modified directly by using the --writable option.

System directories can be mounted into a container using the --bind $PATH option.

More information is available here.

Building

A writable container (sandbox) can be created from an existing image in a selected repository:

apptainer build --sandbox --fix-perms <container>/ <hub>://<image>:[tag]

where: - <container> specifies the directory where the container will be created, - <hub> specifies the source repository (e.g. docker, library), - <image> specifies the container image from the selected repository, - [:tag] is an optional version tag of the container image.

The following example creates a writable container based on Ubuntu 24.04:

apptainer build --sandbox --fix-perms ubuntu_container/ docker://ubuntu:24.04

Shell

If you are using a writable sandbox container, you can open it in an interactive shell that allows modifications to the container contents:

apptainer shell --writable --fakeroot container/

Fakeroot is a functionality that allows non-privileged users to obtain the appropriate "root" rights within containers with the --fakeroot switch. Host filesystem (FS) rights are mapped appropriately within the container. Additionally, it is recommended to use the --fix-perms switch, which regulates and adjusts the rights inside the container accordingly.

More information is available at the link.

Note

Fakeroot for all users is set to root-mapped user namespace mode by default. If you require rootless mode for the purpose of building your own containers forward the request to enable it to support@sling.si.

Execution

If you want to execute a specific command inside a container without opening an interactive shell, use the exec command:

apptainer exec <container> <command>

The following command executes whoami inside a writable sandbox container:

apptainer exec --fakeroot container/ whoami

Cache

Apptainer temporarily stores files in a cache during container downloads and builds. This avoids downloading the same data repeatedly, improving performance and reducing network traffic.

To view the files currently stored in the cache, use:

apptainer cache list

To free up space on the file system, you can clear the cache:

apptainer cache clean --type=all

Creating a Definition File

A definition file contains the instructions used to build a container. It specifies the base image, software installation steps, environment variables, and other configuration settings. A definition file can be created using a text editor:

$ vim example.def

A definition file consists of several sections. The first section defines the base image from which the container will be built. Example:

Bootstrap: docker
From: ubuntu:24.04

In this example, Ubuntu 24.04 from Docker Hub is used as the base image.

The %environment section defines environment variables that will be available every time the container is started. Example:

%environment
    export LC_ALL=C

The %post section is used to install software, libraries, and create directories during the container build process. Example:

%post
   apt-get -y update
   apt-get -y install <package-name>
   apt-get clean

The %runscript section defines what is executed when the apptainer run command is used. Example:

%runscript
    echo "This is what happens when you run the container."

The %labels section can be used to store additional information about the container. Example:

%labels
   Maintainer Y
   Version 1.0

The %help section can be used to include usage instructions for the container. Example:

%help
    Description of help section.

An example of a complete definition file is shown below:

bootstrap: docker
from: ubuntu:latest

%environment
    export LC_ALL=C
%runscript
    echo "This is what happens when you run the container.."
%post 
   apt-get -y update
   apt-get -y install <package-name>
   apt-get clean
%labels
   Maintainer Y
   Version 1.0
%help
    Description of help section.

Read-only Container

A read-only container (compressed squashfs) uses the SIF (Singularity Image Format) format. It can be converted into a sandbox container for modification. Unlike sandbox directories, writable mode does not provide permanent or persistent changes.

To modify an existing SIF container, first convert it to sandbox format:

$ apptainer build --sandbox --fakeroot  container/ container.sif

After the conversion, the container can be opened in writable mode:

$ apptainer build --sandbox --fakeroot  container.simg container.sif

Using Containers on HPC Vega

Various containers are available on HPC Vega in the following directory: /ceph/hpc/software/containers/singularity

The directory contains:

  • .sif images for directly running software inside containers;
  • .def definition files that can be customized to meet your specific requirements.

Direct Use of Apptainer Images on HPC Vega

If a container already includes all the required software, it can be run directly using the apptainer run command.

Example:

apptainer run /ceph/hpc/software/containers/singularity/images/tensorflow-23.09-tf2-py3.sif

or

apptainer exec /ceph/hpc/software/containers/singularity/images/tensorflow-23.09-tf2-py3.sif python3 ...

hese commands are typically executed with Slurm.

Using Apptainer Definition Files

Definition files can be customized for your own use case (software versions, additional tools, etc.). In this case, the .def file should be copied from /ceph/hpc/software/containers/singularity/def/ to your home directory, modified as needed, and used there with apptainer build, for example:

apptainer build --fakeroot my_tensorflow-23.09-tf2-py3.def my_container.sif

Example of an Apptainer Definition File for an MPI Application

When using host MPI, the application inside the container must be compiled with the same MPI version used on the host system.

The following example demonstrates building a container with OpenMPI 4.1.6 and compiling an MPI application using the same MPI version:

Bootstrap: docker 
From: ubuntu:22.04 
%post 
    apt-get update apt-get install -y \ build-essential wget ca-certificates 

    # Installing OpenMPI 4.1.6 
    wget https://download.open-mpi.org/release/open-mpi/v4.1/openmpi-4.1.6.tar.gz 
    tar xf openmpi-4.1.6.tar.gz 
    cd openmpi-4.1.6 
    ./configure --prefix=/opt/openmpi 
    make -j$(nproc) 
    make install 

    export PATH=/opt/openmpi/bin:$PATH 
    export LD_LIBRARY_PATH=/opt/openmpi/lib:$LD_LIBRARY_PATH 

    # Compile the application using the same MPI version 
    cd /tmp 
    tar xf myprogram.tar.gz 
    cd myprogram 
    ./configure CC=mpicc 
    make 
    make install 

%environment 
    export PATH=/opt/openmpi/bin:$PATH 
    export LD_LIBRARY_PATH=/opt/openmpi/lib:$LD_LIBRARY_PATH

After transferring the container to Vega, load the same MPI version:

module load OpenMPI/4.1.6

Then start the application using MPI:

mpirun -np 4 apptainer exec my_container.sif /usr/local/bin/myprogram

Important: The MPI version inside the container must match the MPI version available on Vega (e.g. OpenMPI 4.1.6). If the versions differ, application startup failures or compatibility issues may occur.

Using GPU Applications in an Apptainer Container (CUDA)

Apptainer supports the use of NVIDIA GPUs inside containers through the --nv option. When this option is used, the required NVIDIA libraries are made available inside the container from the host system (Vega), while the container itself provides the user-level CUDA libraries and application.

Important: The CUDA version inside the container must be compatible with the NVIDIA driver installed on Vega. NVIDIA drivers should not be installed inside the container.

Example Definition File

This example uses a CUDA 12.4 environment and an application compiled with nvcc and installed in /usr/local/bin. The definition file is stored in cuda.def:

Bootstrap: docker
From: nvidia/cuda:12.4.1-devel-ubuntu22.04

%post
    apt-get update
    apt-get install -y build-essential cmake git

    # Check the CUDA environment inside the container
    nvcc --version

    # Download the GPU application source code
    cd /opt
    git clone https://github.com/example/mycudaapp.git

    # Change to the application directory
    cd /opt/mycudaapp

    # Create a separate build directory
    mkdir build
    cd build

    # Configure and build the application
    cmake ..
    make -j$(nproc)

    # Copy the compiled executable to a directory available in PATH
    cp mycudaapp /usr/local/bin/

%environment
    export PATH=/usr/local/cuda/bin:/usr/local/bin:$PATH
    export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

%runscript
    exec /usr/local/bin/mycudaapp "$@"

The example uses a fictitious CUDA application called mycudaapp. Before building the container, users must modify several lines according to their own application:

  • Replace https://github.com/example/mycudaapp.git with the URL of your application's repository.
  • Replace cd /opt/mycudaapp with the name of your application's directory.
  • This example assumes that the project uses CMake. If your application uses a different build process, modify the commands in the %post section accordingly.
  • The line cp mycudaapp /usr/local/bin/ assumes that the compilation process produces an executable named mycudaapp. If your executable has a different name, replace it accordingly.
  • In the %runscript section, replace mycudaapp with the name of your executable file.

Build the container: apptainer build mycudaapp.sif cuda.def

To run the application using a GPU container on Vega, first create a Slurm job that uses the GPU partition. In the sbatch script, load the module corresponding to the required CUDA version before starting the application: module load CUDA/12.4

For more information, refer to the official documentation: