Setting up a Stratum 1¶
Setting up a Stratum 1 involves the following steps:
- set up the Stratum 1, preferably by running the Ansible playbook that we provide;
- request a Stratum 0 firewall exception for your Stratum 1 server;
- request a
<your site>.stratum1.cvmfs.eessi-infra.orgDNS entry;
- open a pull request to include the URL to your Stratum 1 in the EESSI configuration.
The last two steps can be skipped if you want to host a "private" Stratum 1 for your site.
Requirements for a Stratum 1¶
The main requirements for a Stratum 1 server are a good network connection to the clients it is going to serve, and sufficient disk space. For the EESSI pilot, a few hundred gigabytes should suffice, but for production environments at least 1 TB would be recommended.
In terms of cores and memory, a machine with just a few (~4) cores and 4-8 GB of memory should suffice.
Various Linux distributions are supported, but we recommend one based on RHEL 7 or 8.
Finally, make sure that ports 80 (for the Apache web server) and 8000 are open.
Step 1: set up the Stratum 1¶
The recommended way for setting up an EESSI Stratum 1 is by running the Ansible playbook
from the filesystem-layer repository on GitHub.
Installing a Stratum 1 requires a GEO API license key, which will be used to find the (geographically) closest Stratum 1 server for your client and proxies. More information on how to (freely) obtain this key is available in the CVMFS documentation: https://cvmfs.readthedocs.io/en/stable/cpt-replica.html#geo-api-setup.
You can put your license key in the local configuration file
Furthermore, the Stratum 1 runs a Squid server. The template configuration file can be found at
If you want to customize it, for instance for limiting the access to the Stratum 1, you can make your own version of this template file
and point to it by setting
See the comments in the example file for more details.
Start by installing Ansible:
sudo yum install -y ansible
Then install Ansible roles for EESSI:
ansible-galaxy role install -r requirements.yml -p ./roles --force
Make sure you have enough space in
/srv (on the Stratum 1) since the snapshot of the Stratum 0
will end up there by default. To alter the directory where the snapshot gets copied to you can add
this variable in
Make sure that you have added the hostname or IP address of your server to the
inventory/hosts file. Finally, install the Stratum 1 using one of the two following options.
# -b to run as root, optionally use -K if a sudo password is required ansible-playbook -b [-K] -e @inventory/local_site_specific_vars.yml stratum1.yml
Create a ssh key pair and make sure the
ansible-host-keys.pub is in the
$HOME/.ssh/authorized_keys file on your Stratum 1 server.
ssh-keygen -b 2048 -t rsa -f ~/.ssh/ansible-host-keys -q -N ""
Then run the playbook:
ansible-playbook -b --private-key ~/.ssh/ansible-host-keys -e @inventory/local_site_specific_vars.yml stratum1.yml
Running the playbook will automatically make replicas of all the repositories defined in
Step 2: request a firewall exception¶
(This step is not implemented yet and can be skipped)
You can request a firewall exception rule to be added for your Stratum 1 server by opening an issue on the GitHub page of the filesystem layer repository.
Make sure to include the IP address of your server.
Step 3: Verification of the Stratum 1¶
When the playbook has finished your Stratum 1 should be ready. In order to test your Stratum 1, even
without a client installed, you can use
curl --head http://<url-or-ip-to-your-stratum1>/cvmfs/pilot.eessi-hpc.org/.cvmfspublished
HTTP/1.1 200 OK ... X-Cache: MISS from <url-or-ip-to-your-stratum1>
The second time you run it, you should get a cache hit:
X-Cache: HIT from <url-or-ip-to-your-stratum1>
Example with the Norwegian Stratum 1:
curl --head http://bgo-no.stratum1.cvmfs.eessi-infra.org/cvmfs/pilot.eessi-hpc.org/.cvmfspublished
You can also test access to your Stratum 1 from a client, for which you will have to install the CVMFS client.
Then run the following command to add your newly created Stratum 1 to the existing list of EESSI Stratum 1 servers by creating a local CVMFS configuration file:
echo 'CVMFS_SERVER_URL="http://<url-or-ip-to-your-stratum1>/cvmfs/@fqrn@;$CVMFS_SERVER_URL"' | sudo tee -a /etc/cvmfs/domain.d/eessi-hpc.org.local
If this is the first time you set up the client you now run:
sudo cvmfs_config setup
If you already had configured the client before, you can simply reload the config:
sudo cvmfs_config reload -c pilot.eessi-hpc.org
Finally, verify that the client connects to your new Stratum 1 by running:
cvmfs_config stat -v pilot.eessi-hpc.org
Assuming that your new Stratum 1 is the geographically closest one to your client, this should return:
Connection: http://<url-or-ip-to-your-stratum1>/cvmfs/pilot.eessi-hpc.org through proxy DIRECT (online)
Step 4: request an EESSI DNS name¶
In order to keep the configuration clean and easy, all the EESSI Stratum 1 servers have a DNS name
<your site>.stratum1.cvmfs.eessi-infra.org, where
<your site> is often a short name or
abbreviation followed by the country code (e.g.
bgo-no). You can request this for
your Stratum 1 by mentioning this in the issue that you created in Step 2, or by opening another
Step 5: include your Stratum 1 in the EESSI configuration¶
If you want to include your Stratum 1 in the EESSI configuration, i.e. allow any (nearby) client to be able to use it,
you can open a pull request with updated configuration files. You will only have to add the URL to your Stratum 1 to the
urls list of the
eessi_cvmfs_server_urls variable in the