#1414th of 20 in Data and scraping for AI
cocoindex
Incremental Python pipelines that keep vector indexes and agent context fresh
Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 11.7k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- Oct 2026
- Language
- Rust
6,554 on 31 Mar 202611,734 on 20 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
CocoIndex is a Python framework with a Rust core for declaring how source data (files, code, PDFs, Slack, databases) maps to a target such as Postgres/pgvector or LanceDB. It memoizes by input and code hash, so after a change only the affected rows are recomputed and re-embedded. It also tracks lineage from each target row back to its source.
Who it is for: Python engineers keeping RAG and agent indexes in sync with changing sources
Strengths
- Memoization keyed on input and code hash re-embeds only changed files
- Pipelines are plain Python functions, not a DAG or YAML config
- Per-row lineage traces target vectors back to source content
- Rust core with retries, back-off and dead-letter queues, per the README
Weaknesses
- Library only: no UI, you write and run the pipeline code yourself
- Python 3.10-3.13 supported; other languages are not mentioned
- Quickstart needs an external target store such as Postgres
- README is marketing-heavy; connector and scale details are only in the docs
What it needs
- Needs Python 3.10-3.13, Postgres (pgvector) or LanceDB as target
- Models: sentence-transformers
Also in Data and scraping for AI
See all 20| Rank | Project | Score |
|---|---|---|
| 1 | mineruDocument parser that converts PDFs, images and Office files to Markdown and JSON | 80 out of 100 |
| 2 | Crawl4AIPython web crawler that converts pages to LLM-ready Markdown and JSON | 78 out of 100 |
| 3 | doclingPython library and CLI that converts PDFs, Office files and more into structured text | 78 out of 100 |
| #4 | label-studioMulti-user data labeling tool for audio, text, images, video and time series | 72 out of 100 |
| #5 | markitdownPython tool that converts PDF, Office, HTML and audio files to Markdown for LLMs | 71 out of 100 |