Overview of Open OnDemand

features

Open OnDemand (OOD) gives you a web interface to the ALICE and SHARK HPC clusters with the following options:

features:

• Dashboard

  • Message of the day (ALICE only)

  • Storage quota overview (ALICE only)

  • Pending and running job count (ALICE only)

  • Cluster load (ALICE only)

• Access Files

• Jobs

  • Active/Completed Jobs

  • Job Composer (batch jobs)

• Clusters

  • Cluster Shell Access (command line)

• Interactive Apps

  • Shark cluster DesktopA full desktop

  • RStudio Server (SHARK only)

  • Jupyter Notebook (with GPU support)

The Interactive Apps in Open OnDemand only work through Slurm job allocation. An example of how this works is given below.

Accessing

Open OnDemand

For SHARK Users

Open OnDemand on SHARK is only available from within the LUMC network or if you are already logged in on SHARK.

Then, you can open a browser of your choice and go to the URL: https://res-hpc-ood01.researchlumc.nlIf you are within the LUMC network:

1. Open a browser

2. go to the URL: https://ood.lumc.nl/

If you are outside of the LUMC network:

1. Connect to SHARK using X2Go

2. Start a browser

3. Go to the URL: https://ood.lumc.nl/

Once you opened the Open OnDemand portal, login with your SHARK username and password.

Image Added

For ALICE users

Open OnDemand on ALICE is an experimental new feature which is currently in a testing phase. You are welcome to try it out and provide feedback to us.

Because of this, access to Open OnDemand is currently only possible from nodelogin02 and not from outside of ALICE.

If you want to give Open OnDemand a try

1. Login to nodelogin02 using either X2Go Getting a remote desktop on ALICE or SHARK with X2Go or RDP Getting a remote desktop on ALICE with RDP

2. Start Firefox

3. Go to the url https://ood.alice.universiteitleiden.nl/

Once you opened the Open OnDemand portal, login with your ALICE username and password:

Image Added

Using Open OnDemand

Here, we will provide a quick overview of using Open OnDemand on ALICE and SHARK.

The Dashboard

The dashboard is the first page that you see when you login. At the top you have the general navigation bar in Open OnDemand with which you can access various features. The main window shows you different information depending on the cluster.

Accessing Files

If you click on “Files” in the navigation bar, you will see a drop-down menu with different options. Clicking on one of them will open a file explorer in Open OnDemand showing you the files and directories for the selected option:

The file explore in Open OnDemand allows you

• to create directories and new files

• move and copy files to other locations

• delete files

• open the specific directory in a terminal

• upload and download the content of the directory

• on the left side, you can find the shortcuts from the files menu

Jobs with OpenOnDemand

Open Ondemand has several features with respect to jobs on the cluster:

• allows you to view and remove/stop pending and currently running jobs. Recently completed jobs will also be listed briefly

• compose and submit new batch jobs

• run certain pre-defined interactive jobs

Accessing active jobs

In order to get an overview of all your current jobs, click on “Jobs” in the navigation bar and select “Active Jobs”

Because the layout of the page “Active Jobs” is the same on both clusters, we show an example from ALICE here (SHARK users would see the same on SHARK):

Image Added

If you click on the red bin-button in the column “Actions”, you can cancel a pending or active job

If you click on the arrow next to the job id, you can get detailed information about your job, access the location of where the job was submitted and get a command line terminal:

Image Added

Submitting new batch jobs

The Job Composer in Open OnDemand is a powerful tool that allows you to set up and submit your batch jobs from the browser. You can either start from scratch with your job or load and edit existing templates either from the system or from you.

You can access the Job Composer through the “Job” button in the navigation bar and then by selecting “Job Composer”. This will open a new tab in your browser which looks like this (again, the job composer looks the same on SHARK and ALICE, so we only show one cluster here)

Image Added

Let’s create a new job with the Job Composer. First, click on the “New Job” button.

You can create a job using

• “From Default Template” which is a very basic test job

  • ALICE users: The job is based on the “My first bash job” from the tutorial on the wiki. You can just use it and run it.

  • SHARK users: A basic job script that you can use with no sbatch settings in the script, so default settings would be used or you add sbatch settings.

• “From Template” which opens a window with a selection of templates.

  • ALICE users: System templates are based on the “My first job” tutorial of the wiki. You can run them while going through the tutorial on the wiki. You can also add your own templates

  • SHARK users: This option is only available after you created a template either by using the button “Create Template” from an existing job or by select “Template” in the navigation bar and clicking on “New Template”

• “From Specified Path” which allows you to open a job directory on the cluster

• “From Selected Job” which will create a new job based on past job of yours from the list in the Job Composer. Here, you can also use the button “Create Template”

Once you created a job, Open OnDemand will automatically set up a directory and add all the files from the template to it.

If you do not want to use the default storage location, you can use “Access Files” to move the entire directory. However, this means that you will not be able to continue using Job Composer unless you add a symbolic link in the default storage location that points to the new location.

With the template in place, you can edit all files either by clicking on them in “Folder Contents” of the section “Job details” (see above) using the button “Access files” (from the Job Composer menu) or “Open Dir” (from “Job Details”). If you click on “Open Editor” in “Job Details”, it will directly open the batch script.

Once you are happy with your job, you can click on the button “Submit” to submit it to Slurm.

A running job can be stopped using the button “Stop”.

You can delete a job directory using the button “Delete” and after confirming the warning message. This will delete the entire directory and all its files.

Interactive Apps

Interactive Apps are basically Slurm jobs that run a specific program in as an interactive job. You can access them through the button “Interactive Apps” in the main navigation bar

The button to the right of “Interactive Apps” gets you to an overview of all your interactive sessions.

Once you have selected an Interactive App, a page will open with a form to specify the parameters of your job. In most cases, those correspond to sbatch settings. As an example, we will show here the form for the Jupyter Notebook app from both:

If you are happy with the settings for your job, click on “Submit”. This will bring you to an overview of all your interactive jobs and show you the state of the job that was just submitted:

Command line access with Open OnDemand

You can open shell from within Open OnDemand if you want to do some work on the command line. This will connect you to a login node on the clusters

Getting help within Open OnDemand

The “Help” menu in the navigation bar provides different options for ALICE and SHARK: