#77th of 13 in Data and scraping for AI
scrapegraph-ai
Python library that builds LLM-driven scraping pipelines from a prompt and a URL
Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 31.8k
- License
- MIT
- Last commit
- Oct 2026
- Last release
- Oct 2026
- Language
- Python
22,849 on 25 Mar 202631,779 on 14 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
ScrapeGraphAI is a Python library that extracts structured data from websites and local files (XML, HTML, JSON, Markdown) given a natural-language prompt. It provides graph pipelines such as SmartScraperGraph, SearchGraph and ScriptCreatorGraph, with multi-page variants that call the LLM in parallel. It works with OpenAI, Groq, Azure, Gemini, MiniMax or local models through Ollama, and uses Playwright to fetch pages.
Who it is for: Python developers extracting structured web data with LLMs
Strengths
- Prompt plus URL returns a JSON dict; no per-site selectors to write
- Runs against local Ollama models or hosted APIs by changing the llm config
- Several pipelines: single page, multi-page, search results, Python script generation
- Multi-page graph variants run LLM calls in parallel
Weaknesses
- Proxies, anti-bot handling and scaling are left to the user in the open-source version
- Needs a separate `playwright install` step to fetch page content
- Anonymous telemetry is on by default; opt out via SCRAPEGRAPHAI_TELEMETRY_ENABLED=false
- README steers users to the paid managed API for crawl and monitor features
What it needs
- GPU optional
- Docker + Compose
- Needs Playwright, Ollama (for local models), LLM provider API key (for hosted models)
- Models: Ollama, OpenAI, Groq, Azure, Gemini
Also in Data and scraping for AI
See all 13| Rank | Project | Score |
|---|---|---|
| 1 | mineruDocument parser that converts PDFs, images and Office files to Markdown and JSON | 78 out of 100 |
| 2 | Crawl4AIPython web crawler that converts pages to LLM-ready Markdown and JSON | 77 out of 100 |
| 3 | doclingPython library and CLI that converts PDFs, Office files and more into structured text | 76 out of 100 |
| #4 | markitdownPython tool that converts PDF, Office, HTML and audio files to Markdown for LLMs | 71 out of 100 |
| #5 | FirecrawlWeb scraping and crawling API that returns LLM-ready markdown | 67 out of 100 |