The container represents an individual unit with the appropriate set of software, including the appropriate libraries and settings for efficient operation in the environment. The technology enables reliability, portability, isolation, security, and launch of container software inside or outside the command line. The latter allows the software to be adapted to the supercomputer environment. Individual distributions enable different approaches and mechanisms that provide flexibility and configuration of the environment. The Vega cluster already has a Singularity container platform installed and adapted to work on supercomputers (HPC).
On HPC Vega, containers are available on $PATH: /ceph/hpc/software/containers/singularity
Singularity is a platform designed to create and run containers within a supercomputer environment. Containers have access to a common operating system, file system, and software installed on nodes. The latter takes into account the privileges of the operating system (in our case RHEL), which allows the use and transfer of the appropriate rights of the system user account. If we decide to create our own container, we can install software, libraries and adapt the environment within it according to our needs. Due to truncated isolation, we need to pay attention to the adequacy, the compatibility of the environment we establish. Otherwise, the container will not work properly or will not work at all. Singularity provides a high level of portability – Singularity Image Format (SIF) and repeatability of containers, with the help of definition or description files (.def / .dsc).
More information is available in the official documentation: https://sylabs.io/docs/
Download pre-prepared containers – downloading a container with the "pull" command from repositories (library, docker, sbub, etc.):
$ singularity pull <image-name>.sif <hub>://<image>[:tag] $ singularity pull container.sif docker://tensorflow/tensorflow:latest
To run a container:
$ singularity run container.sif
To run a command in the local container:
$ singularity exec container.sif <command>
Local container shell:
$ singularity shell container.sif
Print out local container data:
$ singularity inspect container.sif $ singularity inspect --runscript container.sif $ singularity inspect --deffile container.sif
Printout of the “Help” section:
$ singularity run-help container.sif
Additional commands can be found in the “help” section:
$ singularity help
Preparing your own container is possible in several ways, including combining them. You can also use pre-prepared containers, arrange them and adapt them to your needs. However, if this does not suit you, it is possible to create your own container with the help of a definition file. The latter allows a higher level of repeatability of the container construction itself (you always get the same result, it is necessary to be careful not to use the latest tags).
Templates are available in various bootstrap repositories such as:
- The Singularity Container Library: https://cloud.sylabs.io/library
- Docker Hub: https://hub.docker.com/
- Singularity Hub: https://singularity-hub.org/
A complete list of repositories is available at: User guide
Additional functionalities and use of various switches:
The Singularity Image Format (SIF) is a read-only container that facilitates portability. If changes are needed within the container, it can be converted into a “sandbox” or use the additional functionality “Persistent Overlays”, which is available with the
More information is available at the link.
If the container is prepared with the
--sandbox switch, the container itself can be edited directly with the
System directories can be mounted inside the container using the
--bind $PATH switch.
More information is available at the link
Preparing your own container:
$ singularity build --sandbox --fix-perms container/ docker://centos:latest
$ singularity shell container.sif $ singularity shell --writable --fakeroot container/
$ singularity exec --fakeroot container/ whoami
$ singularity cache list $ singularity cache clean –type=all
Preparing a definition file
$ vim example.def bootstrap: docker from: ubuntu:latest %environment export LC_ALL=C %runscript echo "This is what happens when you run the container.." %post apt-get -y update apt-get -y install <package-name> apt-get clean %labels Maintainer Y Version 1.0 %help Description of help section.
Fakeroot is a functionality that allows non-privileged users to obtain the appropriate "root" rights within containers with the
-–fakeroot switch. Host filesystem (FS) rights are mapped appropriately within the container. Additionally, it is recommended to use the
--fix-perms switch, which regulates and adjusts the rights inside the container accordingly.
More information is available at the link
$ singularity build --fix-perms –sandbox --fakeroot container/ example.def
The read-only container (compressed squashfs) is a SIF (Singularity Image Format) format that can be converted to an image with the --writable switch or the sandbox switch to sandbox directories. The writable switch, unlike sandbox directories, does not allow permanent/persistent changes.
$ singularity build --sandbox --fakeroot container/ container.sif $ singularity build --sandbox --fakeroot container.simg container.sif