{"type":"doc","content":[{"type":"layoutSection","content":[{"type":"layoutColumn","attrs":{"width":50.0},"content":[{"type":"heading","attrs":{"level":1},"content":[{"text":"About this tutorial","type":"text"}]},{"type":"paragraph","content":[{"text":"This tutorial will guide you through running a job using a GPU on ALICE or SHARK. Here, we will use a scripts from NVIDIA’s CUDA samples repository to run on the GPU.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"What you will learn?","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Setting up the batch script for a job using GPUs","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Loading the necessary modules","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Submitting your job","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Collect information about your job","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"What this example will not cover?","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Using multiple GPUs","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Writing and compiling code for GPU","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"What you should know before starting?","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Basic knowledge of how to use a Linux OS from the command line.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"How to connect to ALICE or SHARK: ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://pubappslu.atlassian.net/wiki/spaces/HPCWIKI/pages/37748771"}}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"How to move files to and from ALICE or SHARK: ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://pubappslu.atlassian.net/wiki/spaces/HPCWIKI/pages/37749117"}}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"How to setup a simple batch job as shown in: ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://pubappslu.atlassian.net/wiki/spaces/HPCWIKI/pages/37027928"}},{"text":" ","type":"text"}]}]}]}]},{"type":"layoutColumn","attrs":{"width":50.0},"content":[{"type":"extension","attrs":{"layout":"default","extensionType":"com.atlassian.confluence.macro.core","extensionKey":"toc","parameters":{"macroParams":{"minLevel":{"value":"1"},"maxLevel":{"value":"7"},"exclude":{"value":"Contents"}},"macroMetadata":{"macroId":{"value":"8fe3dd20-5842-440b-b0e3-8754dc465f54"},"schemaVersion":{"value":"1"},"title":"Table of Contents"}},"localId":"d38ab1da-7c7c-455b-8ef7-b82c628a5bbc"}}]}]},{"type":"heading","attrs":{"level":1},"content":[{"text":"CUDA on ALICE and SHARK","type":"text"}]},{"type":"paragraph","content":[{"text":"CUDA is available on both clusters in different versions. You can get an overview by running:","type":"text"}]},{"type":"layoutSection","content":[{"type":"layoutColumn","attrs":{"width":50.0},"content":[{"type":"rule"},{"type":"paragraph","content":[{"text":"ALICE","type":"text","marks":[{"type":"strong"}]}]},{"type":"codeBlock","content":[{"text":"module -r avail ^CUDA","type":"text"}]},{"type":"panel","attrs":{"panelType":"info"},"content":[{"type":"paragraph","content":[{"text":"Various modules on ALICE have been built with CUDA (e.g., pytorch, tensorflow). When you load these modules, the version of CUDA that was used to built the module will be loaded automatically.","type":"text"}]}]}]},{"type":"layoutColumn","attrs":{"width":50.0},"content":[{"type":"rule"},{"type":"paragraph","content":[{"text":"SHARK","type":"text","marks":[{"type":"strong"}]}]},{"type":"codeBlock","content":[{"text":"module avail /cuda/","type":"text"}]}]}]},{"type":"rule"},{"type":"paragraph","content":[{"text":"In this tutorial, we will make use of CUDA 11.3.","type":"text"}]},{"type":"heading","attrs":{"level":1},"content":[{"text":"Preparations","type":"text"}]},{"type":"paragraph","content":[{"text":"As usual, it is always helpful to check the current cluster status and load. The GPU nodes are being used quite extensively at the moment. Therefore, it might take longer for your job to be scheduled. This makes it even more important define the resources in your bash script as much as possible to help Slurm schedule your job.","type":"text"}]},{"type":"paragraph","content":[{"text":"If you have been following the previous tutorial, you should already have a directory called ","type":"text"},{"text":"user_guide_tutorials","type":"text","marks":[{"type":"code"}]},{"text":" in your ","type":"text"},{"text":"$HOME","type":"text","marks":[{"type":"code"}]},{"text":". Let's create a directory for this job and change into it:","type":"text"}]},{"type":"codeBlock","content":[{"text":" mkdir -p $HOME/user_guide_tutorials/first_gpu_job\n cd $HOME/user_guide_tutorials/first_gpu_job","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"CUDA samples","type":"text"}]},{"type":"paragraph","content":[{"text":"The CUDA samples for each CUDA release are available on ","type":"text"},{"text":"NVIDIA/cuda-samples","type":"text","marks":[{"type":"link","attrs":{"href":"https://github.com/NVIDIA/cuda-samples/"}}]},{"text":". In principle, you can download the release of CUDA samples specific to the CUDA version that you will use, but here we will just clone the repository and get the latest version. This should work fine, but if you encounter any issues, please let us know.","type":"text"}]},{"type":"codeBlock","content":[{"text":"git clone https://github.com/NVIDIA/cuda-samples.git","type":"text"}]},{"type":"paragraph","content":[{"text":"This should have created a directory called ","type":"text"},{"text":"cuda-samples","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]},{"type":"paragraph","content":[{"text":"Next, we need to build one sample script so that we can run it. Here, we will build ","type":"text"},{"text":"cuda-samples/Samples/6_Performance/transpose/","type":"text","marks":[{"type":"code"}]},{"text":". There two more samples and in this directory and you are welcome to try them all if you like.","type":"text"}]},{"type":"panel","attrs":{"panelType":"warning"},"content":[{"type":"paragraph","content":[{"text":"As of writing of this tutorial, the CUDA samples cannot be compiled with CUDA samples 10 or higher.","type":"text"}]}]},{"type":"layoutSection","content":[{"type":"layoutColumn","attrs":{"width":50.0},"content":[{"type":"rule"},{"type":"paragraph","content":[{"text":"ALICE","type":"text","marks":[{"type":"strong"}]}]},{"type":"paragraph","content":[{"text":"On ALICE, you can build the CUDA samples this way:","type":"text"}]},{"type":"codeBlock","content":[{"text":"module purge\nmodule load ALICE/default\nmodule load CUDA/12.3.2\nmodule load GCC/11.3.0\ncd cuda-samples/Samples/6_Performance/transpose/\nmake SMS=\"75 80\"","type":"text"}]}]},{"type":"layoutColumn","attrs":{"width":50.0},"content":[{"type":"rule"},{"type":"paragraph","content":[{"text":"SHARK","type":"text","marks":[{"type":"strong"}]}]},{"type":"paragraph","content":[{"text":"On SHARK, you can build the CUDA samples like this","type":"text"}]},{"type":"codeBlock","content":[{"text":"module purge\nmodule load library/cuda/11.3/gcc.8.3.1\n\ncd cuda-samples/Samples/6_Performance/transpose/\nmake SMS=\"75 80\"","type":"text"}]}]}]},{"type":"rule"},{"type":"heading","attrs":{"level":2},"content":[{"text":"The batch script","type":"text"}]},{"type":"paragraph","content":[{"text":"Next, we are going to set up or batch script. To this purpose, we are going to change back to the top-level directory for this job with ","type":"text"},{"text":"cd $HOME/user_guide_tutorials/first_gpu_job","type":"text","marks":[{"type":"code"}]},{"text":". ","type":"text"}]},{"type":"paragraph","content":[{"text":"You can copy the content below directly into a text file, which we name here ","type":"text"},{"text":"test_gpu.slurm","type":"text","marks":[{"type":"code"}]},{"text":". The batch file is again a bit more elaborate than perhaps necessary, but it is always helpful to have a few log messages more in the beginning. ","type":"text"}]},{"type":"layoutSection","content":[{"type":"layoutColumn","attrs":{"width":50.0},"content":[{"type":"rule"},{"type":"paragraph","content":[{"text":"ALICE","type":"text","marks":[{"type":"strong"}]}]},{"type":"codeBlock","content":[{"text":"#!/bin/bash\n#SBATCH --job-name=test_gpu\n#SBATCH --mail-user=\"\"\n#SBATCH --mail-type=\"ALL\"\n#SBATCH --time=00:01:00\n#SBATCH --partition=testing\n#SBATCH --output=%x_%j.out\n#SBATCH --ntasks=1\n#SBATCH --cpus-per-task=1\n#SBATCH --mem=2G\n#SBATCH --gres=gpu:2080ti:1\n\n# making sure we start with a clean module environment\nmodule purge\n\necho \"## Starting GPU test on $HOSTNAME\"\n\necho \"## Loading module\"\nmodule load ALICE/default\nmodule load slurm\nmodule load CUDA/12.3.2\nmodule load GCC/11.3.0\n\nTEST_DIR=$(pwd)\necho \"## Current dircectory $TEST_DIR\"\n\necho \"## Available CUDA devices: $CUDA_VISIBLE_DEVICES\"\n\necho \"## Checking status of CUDA device with nvidia-smi\"\nnvidia-smi\n\necho \"## Running test\"\n$HOME/user_guide_tutorials/first_gpu_job/cuda-samples/Samples/6_Performance/transpose/transpose\n\necho \"## Test finished. Goodbye\"","type":"text"}]}]},{"type":"layoutColumn","attrs":{"width":50.0},"content":[{"type":"rule"},{"type":"paragraph","content":[{"text":"SHARK","type":"text","marks":[{"type":"strong"}]}]},{"type":"codeBlock","content":[{"text":"#!/bin/bash\n#SBATCH --job-name=test_gpu\n#SBATCH --mail-user=\"\"\n#SBATCH --mail-type=\"ALL\"\n#SBATCH --time=00:01:00\n#SBATCH --partition=gpu\n#SBATCH --output=%x_%j.out\n#SBATCH --ntasks=1\n#SBATCH --cpus-per-task=1\n#SBATCH --mem=2G\n#SBATCH --gres=gpu:1\n\n# making sure we start with a clean module environment\nmodule purge\n\necho \"## Starting GPU test on $HOSTNAME\"\n\necho \"## Loading module\"\nmodule load library/cuda/11.3/gcc.8.3.1\n\nTEST_DIR=$(pwd)\necho \"## Current dircectory $TEST_DIR\"\n\necho \"## Number of available CUDA devices: $CUDA_VISIBLE_DEVICES\"\n\necho \"## Checking status of CUDA device with nvidia-smi\"\nnvidia-smi\n\necho \"## Running test\"\n$HOME/user_guide_tutorials/first_gpu_job/cuda-samples/Samples/6_Performance/transpose/transpose\n\necho \"## Test finished. Goodbye\"","type":"text"}]}]}]},{"type":"rule"},{"type":"heading","attrs":{"level":1},"content":[{"text":"Running your job","type":"text"}]},{"type":"paragraph","content":[{"text":"Now, we have everything that we need to run this job. Please make sure that you are in the same directory where the script are. If not, then change into ","type":"text"}]},{"type":"codeBlock","content":[{"text":" cd $HOME/user_guide_tutorials/first_gpu_job","type":"text"}]},{"type":"paragraph","content":[{"text":"You are ready to submit your job like this:","type":"text"}]},{"type":"codeBlock","content":[{"text":" sbatch test_gpu.slurm","type":"text"}]},{"type":"paragraph","content":[{"text":"Immediately after you have submitted it, you should see something like this:","type":"text"}]},{"type":"codeBlock","content":[{"text":" [me@ first_gpu_job]$ sbatch test_gpu.slurm\n Submitted batch job ","type":"text"}]},{"type":"heading","attrs":{"level":1},"content":[{"text":"Job output","type":"text"}]},{"type":"paragraph","content":[{"text":"In the directory where you launched your job, there should be new file created by Slurm: ","type":"text"},{"text":"test_gpu_.out","type":"text","marks":[{"type":"code"}]},{"text":". It contains all the output from your job which would have been normally written to the command line. Check the file for any possible error messages. The content of the file should look something like this:","type":"text"}]},{"type":"codeBlock","content":[{"text":"## Starting GPU test on \n## Loading module\n## Current dircectory /home//user_guide_tutorials/first_gpu_job\n## Number of available CUDA devices: 0\n## Checking status of CUDA device with nvidia-smi\nThu Aug 11 14:15:44 2022\n+-----------------------------------------------------------------------------+\n| NVIDIA-SMI 470.103.01 Driver Version: 470.103.01 CUDA Version: 11.4 |\n|-------------------------------+----------------------+----------------------+\n| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n| | | MIG M. |\n|===============================+======================+======================|\n| 0 GRID V100D-16Q On | 00000000:02:00.0 Off | N/A |\n| N/A N/A P0 N/A / N/A | 1104MiB / 16384MiB | 0% Default |\n| | | N/A |\n+-------------------------------+----------------------+----------------------+\n\n+-----------------------------------------------------------------------------+\n| Processes: |\n| GPU GI CI PID Type Process name GPU Memory |\n| ID ID Usage |\n|=============================================================================|\n| No running processes found |\n+-----------------------------------------------------------------------------+\n## Running test\nTranspose Starting...\n\nGPU Device 0: \"Volta\" with compute capability 7.0\n\n> Device 0: \"GRID V100D-16Q\"\n> SM Capability 7.0 detected:\n> [GRID V100D-16Q] has 80 MP(s) x 64 (Cores/MP) = 5120 (Cores)\n> Compute performance scaling factor = 1.00\n\nMatrix size: 1024x1024 (64x64 tiles), tile size: 16x16, block size: 16x16\n\ntranspose simple copy , Throughput = 386.2985 GB/s, Time = 0.02022 ms, Size = 1048576 fp32 elements, NumDevsUsed = 1, Workgroup = 256\ntranspose shared memory copy, Throughput = 378.6300 GB/s, Time = 0.02063 ms, Size = 1048576 fp32 elements, NumDevsUsed = 1, Workgroup = 256\ntranspose naive , Throughput = 269.5899 GB/s, Time = 0.02898 ms, Size = 1048576 fp32 elements, NumDevsUsed = 1, Workgroup = 256\ntranspose coalesced , Throughput = 400.9141 GB/s, Time = 0.01949 ms, Size = 1048576 fp32 elements, NumDevsUsed = 1, Workgroup = 256\ntranspose optimized , Throughput = 393.8769 GB/s, Time = 0.01983 ms, Size = 1048576 fp32 elements, NumDevsUsed = 1, Workgroup = 256\ntranspose coarse-grained , Throughput = 389.4536 GB/s, Time = 0.02006 ms, Size = 1048576 fp32 elements, NumDevsUsed = 1, Workgroup = 256\ntranspose fine-grained , Throughput = 399.0269 GB/s, Time = 0.01958 ms, Size = 1048576 fp32 elements, NumDevsUsed = 1, Workgroup = 256\ntranspose diagonal , Throughput = 379.7667 GB/s, Time = 0.02057 ms, Size = 1048576 fp32 elements, NumDevsUsed = 1, Workgroup = 256\nTest passed\n## Test finished. Goodbye","type":"text"}]},{"type":"paragraph","content":[{"text":"You can get a quick overview of the resources actually used by your job by running:","type":"text"}]},{"type":"codeBlock","content":[{"text":" seff ","type":"text"}]},{"type":"paragraph","content":[{"text":"It might look something like this:","type":"text"}]},{"type":"codeBlock","content":[{"text":"Job ID: \nCluster: \nUser/Group: /\nState: COMPLETED (exit code 0)\nCores: 1\nCPU Utilized: 00:00:01\nCPU Efficiency: 0.00% of 00:00:00 core-walltime\nJob Wall-clock time: 00:00:00\nMemory Utilized: 1.35 MB\nMemory Efficiency: 1.35% of 100.00 MB","type":"text"}]},{"type":"paragraph","content":[{"text":"As you can see this job runs very quickly and does almost nothing on the CPU.","type":"text"}]},{"type":"heading","attrs":{"level":1},"content":[{"text":"Cancelling your job","type":"text"}]},{"type":"paragraph","content":[{"text":"In case you need to cancel the job that you have submitted, you can use the following command","type":"text"}]},{"type":"codeBlock","content":[{"text":" scancel ","type":"text"}]},{"type":"paragraph","content":[{"text":"You can use it to cancel the job at any stage in the queue, i.e., pending or running. ","type":"text"}]},{"type":"paragraph","content":[{"text":"Note that you might not be able to cancel the job in this example, because it has already finished.\n","type":"text"}]}],"version":1}

Browser not supported