AnythingLLM + Ollama + Qdrant

AnythingLLM is the workspace app (documents, agents, chat); Ollama is the model; Qdrant holds the vectors. All three configured by environment variables, so the first screen is already wired.

Run it

  1. Save the compose file

    curl -fsSL https://archestack.com/stacks/anything-llm-ollama-qdrant/compose.yaml -o compose.yaml
  2. Write a .env with fresh values for JWT_SECRET

    printf 'JWT_SECRET=%s\n' "$(openssl rand -hex 32)" > .env
  3. Start it

    docker compose up -d
  4. Open http://127.0.0.1:3001: AnythingLLM. First: docker compose exec ollama ollama pull llama3.2, then pick it in AnythingLLM's first-run screen.

Needs Docker Compose v2 (started here with v5.6). Every port is bound to 127.0.0.1, so nothing is reachable from your network until you put a reverse proxy in front on purpose. For a GPU, add the GPU reservation to the ollama service. The file itself.

What each part does

Why these together: AnythingLLM's own docs list Ollama as an LLM provider and Qdrant as a vector database. Source ↗ (opens in a new tab)

Pinned images

  • ollama/ollama:0.40.16
  • qdrant/qdrant:v1.20.2
  • mintplexlabs/anythingllm:1.20.0

A weekly job checks that each tag still exists and flags a pin two or more major versions behind its project’s latest release. A newer release is not tested here: change a tag on purpose, not by latest.

What we checked

Started on ubuntu-lan (x86-64, CPU only, Docker 29.9.2, Compose v5.6.1) on from this exact file, with fresh volumes. Every container came up healthy and each check below passed. The box already runs its own services on ports 3000 and 4000, so the run shifted the host ports by 10000; nothing else differed. No model was pulled, so no chat or document upload was tried.

  • AnythingLLM answers on 3001
  • AnythingLLM reaches Ollama
  • AnythingLLM reaches Qdrant

Get it set up (opens in a new tab)Opens Archestack Studio with AnythingLLM named. Our own paid service; it never changes a rank.