#1414th of 16 in Observability
ragas
Python library for scoring LLM and RAG application outputs with metrics
- Stars
- 16k
- License
- Apache-2.0
- Last commit
- Feb 2026
- Last release
- Jan 2026
- Language
- Python
Missed the latest GitHub check; these facts may be older.
Overview
Ragas is a Python package for evaluating LLM applications. It ships pre-built and custom metrics, including LLM-judged ones such as DiscreteMetric, and can generate test datasets when none exist. It integrates with LangChain and observability tools, and a `ragas quickstart rag_eval` command scaffolds an example RAG evaluation project.
Who it is for: Developers measuring the quality of LLM and RAG apps in Python
Strengths
- Installs with pip; quickstart command scaffolds a RAG evaluation project
- Custom metrics defined by a prompt and allowed values, scored asynchronously
- Generates test datasets when none are available
- Apache-2.0 licensed; telemetry can be disabled with RAGAS_DO_NOT_TRACK
Weaknesses
- LLM-based metrics need an external model, such as an OpenAI key in the example
- Only the rag_eval quickstart template exists; other templates are listed as coming soon
- Collects anonymized usage analytics by default; opt-out required
- No UI or hosted dashboard described in the README
What it needs
- no GPU
- Needs LLM provider API (OpenAI in example)
- Models: OpenAI, LangChain
Also in Observability
See all 16| Rank | Project | Score |
|---|---|---|
| 1 | LangfuseTracing, prompt management and evals for LLM apps on ClickHouse | 75 out of 100 |
| 2 | MLflowTracing, evals, prompt registry and AI gateway plus classic ML tracking | 72 out of 100 |
| 3 | promptfooCLI for evaluating and red-teaming prompts, agents and RAG | 72 out of 100 |
| #4 | PhoenixLLM tracing, evaluation and prompt management built on OpenTelemetry | 71 out of 100 |
| #5 | OpikTrace, evaluate and monitor LLM apps and agents, Apache-2.0 end to end | 66 out of 100 |