Ollama + Open WebUI + Qdrant
The chat stack above plus a vector database. Open WebUI keeps the embeddings of every uploaded document in Qdrant instead of its built-in store, so the index outlives the chat container and you can open it in Qdrant's dashboard.
Run it
Save the compose file
curl -fsSL https://archestack.com/stacks/local-rag-ollama-open-webui-qdrant/compose.yaml -o compose.yamlWrite a .env with fresh values for WEBUI_SECRET_KEY
printf 'WEBUI_SECRET_KEY=%s\n' "$(openssl rand -hex 32)" > .envStart it
docker compose up -dOpen
http://127.0.0.1:3000: Open WebUI (Qdrant dashboard on 6333/dashboard). First:docker compose exec ollama ollama pull llama3.2.
Needs Docker Compose v2 (started here with v5.6). Every port is bound to 127.0.0.1, so nothing is reachable from your network until you put a reverse proxy in front on purpose. For a GPU, add the GPU reservation to the ollama service. The file itself.
What each part does
- Ollama: runs the models. MIT, score 92 on Best of Open-Source AI.
- Open WebUI: chat UI and document upload. custom license, score 89 on Best of Open-Source AI.
- Qdrant: stores the document vectors. Apache-2.0, score 85 on Best of Open-Source AI.
Why these together: Open WebUI documents Qdrant as a vector store. Source ↗ (opens in a new tab)
Pinned images
ollama/ollama:0.40.16qdrant/qdrant:v1.20.2ghcr.io/open-webui/open-webui:v0.13.4
A weekly job checks that each tag still exists and flags a pin two or more major versions behind its project’s latest release. A newer release is not tested here: change a tag on purpose, not by latest.
What we checked
Started on ubuntu-lan (x86-64, CPU only, Docker 29.9.2, Compose v5.6.1) on from this exact file, with fresh volumes. Every container came up healthy and each check below passed. The box already runs its own services on ports 3000 and 4000, so the run shifted the host ports by 10000; nothing else differed. No model was pulled, so no chat or document upload was tried.
- Open WebUI answers on 3000
- Qdrant is ready on 6333
- Open WebUI reaches Qdrant
Get it set up (opens in a new tab)Opens Archestack Studio with Ollama named. Our own paid service; it never changes a rank.