Skip to content

Open WebUI and Ollama

Open WebUI is a frontend that provides access and interactability for LLM models. Ollama is a backend that provides and envirnmont to load and run LLM models.

Run Open WebUI and Ollama on Vega

Login to Vega and allocate a node, then connect to it.

salloc -n 1 --gres=gpu:1 --partition=gpu -t 00:30:00
ssh node

Activate Conda and create a new environment.

If this is not your first time using Open WebUI and have already created the environment, you can skip these steps and just enable the environment.

module load Anaconda3

conda create -n owui python=3.11 -y

conda activate owui

Download and extract and clear Ollama into our env bin and lib folder.

curl -L https://ollama.com/download/ollama-linux-amd64.tgz -o $CONDA_PREFIX/ollama.tgz && \
tar -xzvf $CONDA_PREFIX/ollama.tgz -C $CONDA_PREFIX && \
rm $CONDA_PREFIX/ollama.tgz

Add Open WebUI to your environment.

pip install open-webui

By default Ollama will store models under the ~/.ollama/models directory. If you want to change that you can modify the default path Ollama uses for models by setting the environment variable.

conda env config vars set OLLAMA_MODELS="$(pwd)/models"

Changes only apply after you re-enable the environment.

Lists of all available env var configs are accessible at owui and ollama.

Start both services in the background.

ollama serve > ollama.log 2>&1 &
open-webui serve > owui.log 2>&1 &

To access Open WebUI on Windows in your local browser you will have to create a tunnel to your node. If using PuTTY on Windows you can set this on the right side under Connection -> SSH -> Tunnels.

Set whatever source port you would like to use on your local machine (for example 8080) and set the destination to \:8080. Set the node to the hostname of whatever node was allocated.

On Mac and Linux you can simply run this command: 

ssh -N -f -L <port>:gnXX:8080 <your-username>@login.vega.izum.si

You can now access Open WebUI in the browser on http://localhost:8080 (replacing 8080 with the port you used).

Singularity Container for Open WebUI and Ollama

Below is the provided singularity image definition file. You can use it to build the image yourself or use the one provided by Vega.

Note that you require permissions to use --fakeroot to build singularity images on Vega, which is disabled by default. To enable this for your account contact support@sling.si.

Bootstrap: docker
From: ghcr.io/open-webui/open-webui:main

%post
    curl -L https://ollama.com/download/ollama-linux-amd64.tgz -o /tmp/ollama.tgz
    tar -xzf /tmp/ollama.tgz -C /usr
    rm /tmp/ollama.tgz

    mkdir /models /data
    chmod 777 /models /data
    chmod 777 -R /app

    ln -s /data /usr/share/ollama

%environment
    # OLLAMA
    export OLLAMA_MODELS="/models"
    export OLLAMA_HOST="127.0.0.1:11434"

    # OPENWEB UI
    export DATA_DIR="/data"
    export OLLAMA_BASE_URL="http://127.0.0.1:11434"
    export ENABLE_OPENAI_API
    export PORT=8080
    export WEBUI_SECRET_KEY="key"
    export NVIDIA_VISIBLE_DEVICES=all

%test
    echo "Checking for Ollama installation..."
    test -f /usr/bin/ollama || exit 1
    echo "Checking for Open WebUI installation..."
    test -d /app/backend || exit 1

%runscript
    echo "Starting Open WebUI / Ollama stack"
    echo "Starting Ollama server ..."
    ollama serve > /data/ollama.log 2>&1 &

    echo "Starting Open WebUI on localhost:8080 ..."
    bash /app/backend/start.sh

%startscript
    echo "Starting Open WebUI / Ollama stack"

    echo "Starting Ollama server ..."
    ollama serve > /data/ollama.log 2>&1 &

    echo "Starting Open WebUI on localhost:8080 ..."
    bash /app/backend/start.sh > /data/owui.log 2>&1 &

%help
    This container runs a bundled Ollama and Open WebUI stack.
    Replace the ./models and ./data locations with whatever path you want to use as model and data persistent storage locations
    Usage:
        singularity run --nv --writable-tmpfs -B ./models:/models -B ./data:/data owui.sif owui

We provide a prebuilt Singularity container containing both the Open WebUI and Ollama services. To run the container you first have to make sure to create folders to store model data for Ollama and a persistent storage location for Open WebUI data.

You can use the commands below to create both folders. If you already downloaded some models using Ollama you can skip creating a new model folder and just use the existing one.

mkdir -p owui/models
mkdir -p owui/data
cd owui

Before you run the service you must first secure a compute node and connect to it.

salloc -n 1 --gres=gpu:1 --partition=gpu -t 01:00:00
ssh {allocated_node}

To run the container in the terminal you can use this command, replacing the bind locations to whatever path you decided to use.

singularity run --nv --writable-tmpfs -B ./owui/models:/models -B ./owui/data:/data owui.sif

The logs of Ollama will also be available in the /data folder you created.

Any service native setting can be set using the --env flag in the Singularity run command. To change the Open WebUI port for example you can use in the singularity run command:

--env PORT=9090

To access Open WebUI on Windows in your local browser you will have to create a tunnel to your node. If using PuTTY on Windows you can set this on the right side under Connection -> SSH -> Tunnels.

Set whatever source port you would like to use on your local machine (for example 8080) and set the destination to \:8080. Set the node to the hostname of whatever node was allocated.

On Mac and Linux you can simply run this command: 

ssh -N -f -L <port>:gnXX:8080 <your-username>@login.vega.izum.si