About

This page provides instructions for installing ollama.

The instructions below will use the manual install method for ollama (Ollama - Manual install) and using a specific release. Running the install shell script from the github requires sudo which is not possible for users.

The instructions should work on ALICE and SHARK.

If you notice any issues, please let us know.

1 About
2 Instructions
3 Notes
4 Slurm batch script

Instructions

The lib dir for ollama is about 3.5G, so as long as you have sufficient space, you can install it in your home. Otherwise, install it in your shared scratch directory (ALICE) or your group share (SHARK):
mkdir $HOME/ollama; cd $HOME/ollama
Check the latest release on the github page and adjust the link to download it accordingly. Here, we use release v0.5.7.
curl -L https://github.com/ollama/ollama/releases/download/v0.5.7/ollama-linux-amd64.tgz -o ollama-linux-amd64.tgz
Extract the archive
tar -xzf ollama-linux-amd64.tgz
Make a link to the executable to a directory that is in your $PATH, e.g.,
ln -s $HOME/ollama/bin/ollama /home/$USER/bin/ollama

Notes

It seems that ollama creates a directory in $HOME/.ollama. If it tries to store large amounts of data there, then you might run into a quota issues very quickly. If this is the case, then it is best to move the .ollama directory to your directory on the shared scratch (ALICE) or a your group share (SHARK).
the instructions for ollama on the github page show an interactive workflow. However, this is usually less efficient on a cluster. It is possible to run ollama in a non-interactive batch job (see below)

Slurm batch script

Here is a basic Slurm batch script to run ollama as batch job on the cluster. It requires a python script for which we will add an example soon.

We want to thank an ALICE user for providing the batch script.

Please adjust as the batch settings, modules and paths needed:

#!/bin/bash
#SBATCH --job-name=ollama_serve
#SBATCH --output=%x_%j.log
#SBATCH --error=%x_%j.err
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=4
#SBATCH --mem=16G
#SBATCH --time=02:00:00 # 2 hours runtime
#SBATCH --partition=gpu-short # Changed from short-gpu to gpu
#SBATCH --gres=gpu:1 # Request 1 GPU
 
# Load CUDA module
module load CUDA/12.3.2 # Adjust version as needed
 
# Set absolute path to Ollama
OLLAMA_PATH="$(pwd)/bin/ollama"
 
# Debug information
echo "Current PATH: $PATH"
echo "Ollama path: $OLLAMA_PATH"
echo "CUDA version: $(nvidia-smi)"
 
# Verify Ollama exists
if [ ! -f "$OLLAMA_PATH" ]; then
echo "Error: Ollama not found at $OLLAMA_PATH"
exit 1
fi
 
# Make sure Ollama is executable
chmod +x "$OLLAMA_PATH"

# check that the port by ollama is not already in use
function test_ollama_port_closed {
  for i in {11434..11454} # will test start port number 11434 till end port number 11454 if ollama is already running on that port
  do 
    TEST_OLLAMA_PORT=$(nc -z 127.0.0.1 "$i" && echo open || echo closed) #this returns "open" if the port is open and "closed if the port is closed
    if [ "$TEST_OLLAMA_PORT" != "open" ] 
    then
      echo $i
      break
    fi
  done
}
 
#Run Llama
 
OLLAMA_PORT=$(test_ollama_port_closed) #find the first open port from the test_ollama_port function
echo "Running ollame as: OLLAMA_HOST=127.0.0.1:${OLLAMA_PORT} ollama serve &"
OLLAMA_HOST=127.0.0.1:${OLLAMA_PORT} ollama serve &
 
# Wait for the server to start
sleep 10 # Adjust this value if needed
 
# Run the model
"$OLLAMA_PATH" pull deepseek-r1:7b
 
# call the model with ollama python API (do what you want to do)
python generate_cot.py
 
# Wait for the Python script to finish
wait