Storage solutions

• HPC Isilon storage.
  This is fast storage for direct access to your data on the cluster, which can be purchased from the IT&DI department through Topdesk. Once purchased this storage will be NFS v4 mounted on all the nodes on the cluster. The default mountpoint will be /exports/.
  Access to this mountpoint is handled by an Active Directory group. The default mount access rights are set by an Ansible playbook. To grant users access to this share you need to have them added to the Active Directory group attached to the share. To find out which group is attached to your data storage use the following command ls -aldh /exports/<storage-share-name> | awk '{print $4}'
  No backups or snapshots are available for the HPC Isilon storage.

• research LTS Isilon storage
  This is slow storage for archiving data which can be purchased from the IT&DI department through Topdesk. Once purchased this storage will be NFS v4 mounted on all the execution/gpu/mem nodes on the cluster with read only access, on the login nodes you will have read and write access. The default mountpoint will be /exports/archive/. Access to this mountpoint is handled by an Active Directory group. The default mount access rights are set by an Ansible playbook. To grant users access to this share you need to have them added to the Active Directory group attached to the share. To find out which group is attached to your data storage use the following command ls -aldh /exports/archive/<storage-share-name> | awk '{print $4}'

How to stage your data:

• From within your submit script

• Create a directory on the local /tmp mkdir -p /tmp/$USER/$SLURM_JOB_ID

• Change permissions so only the owner can view the files chmod 700 /tmp/$USER

• Copy your data cp <path>/to/Data /tmp/$USER/$SLURM_JOB_ID/

• Execute your binary and save output to /tmp/$USER/$SLURM_JOB_ID/

• At the end of your sbatch script move/copy your output to your HPC/LTS storage.

• Now remove the complete directory on the staged directory `rm -Rf /tmp/$USER/$SLURM_JOB_ID/'

Importand information for /tmp directory

• Data is scratch and will be treated as such

• If needed data can be removed by the admins without notice !!

• There is no backup for /tmp

• Data and empty folders will be removed automatically

• There is no quota do not abuse this FileSystem, data can and will be removed if necessary

• Set the security to read,write,execute for the owner only chmod 700

Special directories on the cluster

• /bam-export/ This directory is created for sharing your Binary Alignment/Map files. This is a temporarel share, first in first out principle. Data here should not contain any patient related data and can be deleted at any time (make sure you have a copy somewhere else). This directory can be used in the UCSC Genome Browser to view your data tracks. The files here are accessible through a webbrowser with the following URL https://barmsijs.lumc.nl/. The files can only be accessed if you know the exact file name and the files are on the /bam-export/ directory word readable.

• /home The /home directory is an Isilon HPC export mounted on the /home directory for all the nodes in the cluster. This export is limited to 10 Gb per person. Your home directory is automatically created the first time you log into the cluster and is the same as your username. By default your /home/ directory is world readable and world executable. This directory should not be used for data storage.

How to get access to a HPC or LTS share.

• The shares are mounted on a folder on Shark and an Active Directory group grants access to the folder where the share is mounted.

• To get access to the storage share only your Department ACPr (Automation Contact Person) can grant access to the share through a self service desk form Aanvraag: Mutaties netwerkrechten of folders (add user to a security group)