#1919th of 42 in Gateways and observability
giskard-oss
Python library for evals and red-team scans of LLM agents
- Stars
- 5.9k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- Oct 2026
- Language
- Python
5,180 on 3 Apr 20265,938 on 30 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
Giskard v3 is a modular Python library for testing agentic systems. giskard-checks provides a scenario API with string, regex, semantic-similarity and LLM-as-judge checks (Groundedness, Conformity, LLMJudge) for multi-turn evals. giskard-scan generates adversarial suites from a plain-language agent description, covering prompt injection, harmful content and misinformation, and adds RAG knowledge-base quality evaluation.
Who it is for: Developers testing and red-teaming LLM agents and RAG apps in Python
Strengths
- Wraps any sync or async callable, so LLMs, black-box agents and pipelines are all testable
- Supports multi-turn scenarios and suites, not just single-prompt assertions
- Scan generates adversarial tests across OWASP LLM Top-10 categories from a text description
- Split into packages, so installs carry only the dependencies needed
Weaknesses
- LLM judges and scan generators need a provider extra and API key; default judge is openai/gpt-4o-mini
- Requires Python 3.12+
- Tabular/ML model scan and Hub exist only in v2, which is no longer actively maintained
- Optional telemetry is on unless disabled via environment variable
What it needs
- no GPU
- Needs Python 3.12+, LLM provider API key (e.g. OpenAI) for judges and scan generators
- Models: OpenAI, Anthropic, other providers via giskard-llm
Quick answers
Is giskard-oss open source?
Yes. Its code is on GitHub under the Apache-2.0 license.
Does giskard-oss need a GPU?
No, it runs without one.
Also in Gateways and observability
See all 42| Rank | Project | Score |
|---|---|---|
| 1 | OmniRouteLLM gateway routing one OpenAI-compatible endpoint across 372 providers | 96 out of 100 |
| 2 | new-apiGateway that unifies LLM providers behind OpenAI, Anthropic and Gemini APIs | 90 out of 100 |
| 3 | MLflowTracing, evals, prompt registry and AI gateway plus classic ML tracking | 89 out of 100 |
| #4 | LiteLLMOpenAI-format gateway and Python SDK for calling 100+ LLM providers | 87 out of 100 |
| #5 | freellmapiOpenAI-compatible gateway pooling free-tier LLM providers with automatic failover | 86 out of 100 |