#1515th of 23 in RAG and knowledge
localgpt
Private document Q&A with hybrid search and reranking over Ollama models
- Stars
- 22.2k
- License
- MIT
- Last commit
- Aug 2026
- Language
- Python
Missed the latest GitHub check; these facts may be older.
Overview
LocalGPT indexes PDF, DOCX, HTML, Markdown and TXT files with Docling and answers questions over them using LanceDB hybrid search (dense vectors plus full-text, fused with RRF) and a Qwen3 cross-encoder reranker. A router chooses between RAG and a direct LLM answer, and generation runs through Ollama. It ships a Next.js UI, a REST backend on port 8000 and a RAG API on port 8001.
Who it is for: Self-hosters who want private Q&A over their own documents
Strengths
- Hybrid retrieval: LanceDB vector plus full-text, RRF fusion, reranking on by default
- Defaults chosen from documented ablations in eval/decisions; late chunking and verification are optional
- Every UI action is an HTTP call; CLI and scripts can also build indexes
- Embedding and reranking pick CUDA, then Apple MPS, then CPU automatically
Weaknesses
- Needs Python, Node.js 20+ and Ollama, running four separate services
- Reranker is about 7.5 GB and downloads on the first reranked query
- Changing the embedding model invalidates existing indexes; they must be rebuilt
- Vision and multimodal models are not integrated; scanned PDFs rely on OCR engines
What it needs
- RAM ≥ 8 GB
- GPU optional
- Docker + Compose
- Needs Ollama, Python 3.10+, Node.js 20+, LanceDB, Docling, SQLite
- Models: Ollama models (qwen3.5:9b default), HuggingFace embedding models, Qwen3-Reranker, watsonx
- port 3000
Also in RAG and knowledge
See all 23| Rank | Project | Score |
|---|---|---|
| 1 | deeptutorSelf-hosted AI tutor with knowledge bases, guided learning, books and quizzes | 79 out of 100 |
| 2 | Open NotebookSelf-hosted NotebookLM alternative with podcasts and 20+ model providers | 75 out of 100 |
| 3 | LightRAGGraph-based RAG framework with a REST API server and web UI | 75 out of 100 |
| #4 | RAGFlowRAG engine with deep document parsing, template chunking and agentic retrieval | 74 out of 100 |
| #5 | MaxKBEnterprise knowledge-base agent platform with RAG, workflows and MCP tools | 69 out of 100 |