Run an LLM on a VPS

RAM decides whether a model runs at all, CPU cores decide how fast it answers, and disk only decides how long the download takes. The model's weights must fit in memory with room left for the operating system and the context window.

• RAM: the model's file size plus about 1.5–2 GB for the system and context.
• vCPU: four cores is a practical floor for CPU-only inference; speed improves with more cores and newer CPUs.
• Disk: NVMe, with 20 GB or more for two small models; spinning disks make each model load slowly.
• Swap: 2 GB as a safety net so a model that overshoots RAM does not freeze the SSH session.
• GPU (optional): turns seconds-per-answer into near-instant replies; needed for larger models at chat speed.

Run Ollama's install script, which places the binary and a systemd service; the API then listens on 127.0.0.1:11434, reachable only from the server itself. Use a fresh Ubuntu 24.04 server with sudo access.

sudo apt update && sudo apt upgrade -y
curl -fsSL https://ollama.com/install.sh | sh

# Check the service and the listening address
systemctl status ollama
ss -lntp | grep 11434      # should show 127.0.0.1:11434
ollama --version

If ss shows 0.0.0.0:11434, someone has changed OLLAMA_HOST; set it back before going further, because the Ollama API has no authentication.

Run Open WebUI in Docker on the same server, connect it to Ollama on 127.0.0.1, and publish it only through Nginx with HTTPS. Host networking lets the container reach Ollama's loopback address without exposing Ollama.

# Install Docker (official convenience script)
curl -fsSL https://get.docker.com | sh

# Run Open WebUI on the host network; the UI listens on port 8080
docker run -d --network=host \
  -e OLLAMA_BASE_URL=http://127.0.0.1:11434 \
  -v open-webui:/app/backend/data \
  --name open-webui --restart always \
  ghcr.io/open-webui/open-webui:main

Keep port 8080 closed in the firewall and proxy a domain such as ai.example.com to http://127.0.0.1:8080 in Nginx, with a certificate from Certbot. Avoid the common -p 3000:8080 form on a public server: Docker's published ports bypass ufw rules. Create the first account yourself immediately, because the first sign-up becomes the administrator.

On four to eight modern cores, expect roughly 5–15 tokens a second from a 7B model and 2–6 tokens a second from a 13B model; real figures depend on the CPU generation, memory bandwidth and context length. That is fine for summaries, classification and background jobs, and slow for long interactive chats.

• Keep models loaded: set OLLAMA_KEEP_ALIVE=30m in the service environment so the next request does not wait for a reload (the default is 5 minutes).
• Trim context: lower num_ctx to what prompts actually use; large contexts cost memory and time.
• Batch work: run bulk jobs overnight rather than expecting chat speed from a CPU box.
• Use a GPU when speed matters: see CPU vs GPU and WebJi® GPU VPS options.