Shortest Guide to Success#

This guide provides a short list of all the required commands to set up Autosubmit and auto-icon, create an experiment and run your first experiment. See the linked guides for more detailed information.
The guide is specifically designed for the art case. For other cases, check out the project Readme.

Relevant placeholders and key terms#

  • ‘local’: your laptop/workstation you are currently working on; Autosubmit will run on this computer

  • ‘remote’: the HPC system, you connect to

  • <HPCARC>: the HPC system to use. Fully tested are LEVANTE and HOREKA.

  • <expid>: experiment identifier of the form a123, provided by auto-icon

Prerequisites#

Hint

These have to be run only once for all experiments.

1. Set up Gitlab access from HPC system#

Note

If you use the public version of ICON, you can skip this step.

Important

To be executed on the remote system.

Create a personal access token (PAT) for the DKRZ Gitlab: In the DKRZ Gitlab, select on the left sidebar -> avatar -> Edit profile. On the left sidebar, go to Access Tokens and click Add new token. Enter a name, expiry date and read_repository for the scope. Copy and save the created PAT. Leave the window open, you will need it again.

Now, set up the netrc credentials manager on the remote HPC machine, by running the commands and substituting the placeholders <...>:

cat > ~/.netrc << EOF
machine gitlab.dkrz.de
login <DKRZ user name>
password <PAT>
EOF

git config --global credential.helper "netrc"

2. Set up HPC system#

Important

To be executed on the remote system.

Levante 
if [ ! -f ~/.bashrc ]; then
    cp /etc/skel/.bashrc ~/
fi
cat >> ~/.bashrc << EOF
source /sw/etc/profile.levante
mkdir -p /work/$(id -gn)/$USER
export WORK=/work/$(id -gn)
export SCRATCH=/scratch/${USER:0:1}/$USER
EOF
HoreKa 
ws_allocate auto-icon 60
Different HPC system

After step 2 of the Experiment setup, you have to add your platform to auto-icon, following these instructions.

3. Set up Autosubmit#

Important

To be executed on the local system.

KIT users

KIT users should connect to the Autosubmit server with their KIT username (e.g. ab1234) and password (only within KIT network or VPN):

ssh -X <username>@auto-icon.scc.kit.edu
Non-KIT users

Autosubmit needs to be installed: Official Autosubmit installation instructions

Install the prerequisites bash, python3 >= 3.9, pip >= 24.0, sqlite3, git-scm > 1.8.2, subversion, dialog, curl, python-tk (tkinter in centOS), graphviz >= 2.41 and rsync on the ‘local’ system with your package manager.

Install Autosubmit:

pip install autosubmit
autosubmit configure
autosubmit install

4. Create an SSH key#

Important

To be executed on the local system.

ssh-keygen -t ed25519 -f ~/.ssh/key_as2hpc -C "Autosubmit access to HPC"

Add the public key ~/.ssh/key_as2hpc.pub to the HPC system. This depends on the HPC system (instructions for Levante and HoreKa).

Then unlock the key (this will prompt for the key passphrase):

eval $(ssh-agent -s)
ssh-add ~/.ssh/key_as2hpc

Experiment setup#

Important

To be executed on the local system.

1. Add an experiment and initialize it#

Remember to substitute <HPCARCH> with your HPC system (case sensitive, usually ALL CAPS!). The option -b sets the auto-icon branch, public defaults to the public ICON version, main to the internal ICON version.

autosubmit expid -min -repo https://gitlab.dkrz.de/auto-icon/auto-icon.git -conf "conf/art.yml" \
                 -b public -d "Test run of LAM_OEM with auto-icon." -H <HPCARC>
Autosubmit is running with 4.0.84
The new experiment "a00j" has been registered.
Generating folder structure...
Experiment folder: /pool/experiments/a00j
Generating config files...
Experiment a00j created

This prints an experiment id (in the line “Experiment created” and of the form a123, short expid) and experiment folder. The next step is to initialize the experiment with auto-icon:

cd <experiment_folder>
autosubmit create <expid> -np
./proj/git_project/auto-icon-init -e LAM_OEM [--share [<directory]]

The last command initializes the experiment to use the LAM_OEM case, which should run out of the box. After this step you would usually do the manual configuration to adjust the case to your needs. Details can be found in the step by step guide.
You might want to add the option --share to share the ICON model and Python virtual environment between different experiments. It will be located in ~/auto-icon-share per default but can be changed by adding a path to the share option. Details can be found here.

2. Continue setup#

Hint

These have to be run only once for all experiments.

cp proj/git_project/examples/platforms_user.yml ~/.asconf/platforms_user.yml

Modify ~/.asconf/platforms_user.yml e.g. with nano: enter your user and project names for the respective HPC systems in the open fields. On Levante, the SCRATCH_PROJECT_DIR field needs to be set to the first letter of your user/project (the one in /home/<letter>/<user>).

3. Create the workflow and run the experiment#

Remember to substitute <expid> with your experiment id!

autosubmit create <expid> -np
nohup autosubmit run <expid> &

If you get connection issues, check your connection again. Also the ssh-agent has to be set up again, each time you log off and on.

You can follow the current status of your experiment with:

autosubmit monitor <expid> [--txt]

The option --txt prints text output only, while leaving out the option provides you with a graphical representation.

Hint

You can also monitor your progress via the Autosubmit GUI.

Success! How to go on?#

The recommended steps depend on whether you know the model already or not.
If you do so:

  1. Check out the Setup/Setting up auto-icon first (optional, but it sets up everything to also use the non-public versions of the model code).

  2. Go through the Step-by-Step guide, especially the sections about configuring the model. You can rerun the model with an updated config to get used to it.

  3. Follow the transformation guide to create a configuration that resembles one of your runscripts. It is recommended to create a new experiment for this.

If you don’t know the model:

  1. Go through the Step-by-Step guide. You can rerun the model with an updated config to get used to the various options you have and you can also check out different templates.

  2. Check out the Setup/Setting up auto-icon first (optional, but it sets up everything to also use the non-public versions of the model code).

  3. Learn more on how the ICON model works with the ICON guide.