Async FastAPI and Langchain service that ingests documents, embeds them under a file_id, and answers similarity queries against pgvector or Atlas MongoDB. Built mainly as the RAG backend for LibreChat, but usable for any ID-based retrieval. Reads and deletes are scoped to an owner taken from a verified JWT.
Who it is for: LibreChat self-hosters and devs needing a file-ID-based RAG API
Strengths
Many embedding providers: OpenAI, Azure, Bedrock, Google GenAI, VertexAI, Hugging Face, TEI, Ollama
Reads and deletes are scoped by JWT owner before ranking
Batched async ingestion with bounded memory and rollback on failure
Optional RAG_DISTANCE_THRESHOLD drops weak matches to cut downstream LLM tokens
Weaknesses
entity_id is caller-asserted; untrusted callers need upstream authorization
Upgrading the service before the client orphans agent knowledge-base files on delete
Chunks with no user_id become unreadable after upgrade unless backfilled with SQL
Omitting JWT_SECRET disables authentication and stamps all chunks as public
What it needs
Python, OpenAI, Azure OpenAI, AWS Bedrock, Google GenAI
Needs PostgreSQL with pgvector (or Atlas MongoDB), Embedding provider API (OpenAI by default)
xerjElasticsearch-compatible search server that auto-indexes folders for coding agents
Live demo ↗
(opens in a new tab)18.9k stars+15,411 (+437%) in 30 daysApache-2.0
86 out of 100score
2
llm-appYAML-configured RAG server that keeps a vector database synced with a document folder58.7k starsMIT