Ollama + Open WebUI + Qdrant

The chat stack above plus a vector database. Open WebUI keeps the embeddings of every uploaded document in Qdrant instead of its built-in store, so the index outlives the chat container and you can open it in Qdrant's dashboard.

Run it

  1. Save the compose file

    curl -fsSL https://archestack.com/stacks/local-rag-ollama-open-webui-qdrant/compose.yaml -o compose.yaml
  2. Write a .env with fresh values for WEBUI_SECRET_KEY

    printf 'WEBUI_SECRET_KEY=%s\n' "$(openssl rand -hex 32)" > .env
  3. Start it

    docker compose up -d
  4. Open http://127.0.0.1:3000: Open WebUI (Qdrant dashboard on 6333/dashboard). First: docker compose exec ollama ollama pull llama3.2.

Needs Docker Compose v2 (started here with v5.6). Every port is bound to 127.0.0.1, so nothing is reachable from your network until you put a reverse proxy in front on purpose. For a GPU, add the GPU reservation to the ollama service. The file itself.

What each part does

Why these together: Open WebUI documents Qdrant as a vector store. Source ↗ (opens in a new tab)

Pinned images

  • ollama/ollama:0.40.16
  • qdrant/qdrant:v1.20.2
  • ghcr.io/open-webui/open-webui:v0.13.4

A weekly job checks that each tag still exists and flags a pin two or more major versions behind its project’s latest release. A newer release is not tested here: change a tag on purpose, not by latest.

What we checked

Started on ubuntu-lan (x86-64, CPU only, Docker 29.9.2, Compose v5.6.1) on from this exact file, with fresh volumes. Every container came up healthy and each check below passed. The box already runs its own services on ports 3000 and 4000, so the run shifted the host ports by 10000; nothing else differed. No model was pulled, so no chat or document upload was tried.

  • Open WebUI answers on 3000
  • Qdrant is ready on 6333
  • Open WebUI reaches Qdrant

Get it set up (opens in a new tab)Opens Archestack Studio with Ollama named. Our own paid service; it never changes a rank.