#66th of 9 in Security
garak
Command-line scanner that probes LLMs for jailbreaks, leakage and injection
Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 9.6k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- Oct 2026
- Language
- Python
7,338 on 1 Apr 20269,634 on 21 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
garak runs a library of probes against a target LLM (Hugging Face, OpenAI, Bedrock, NIM, Replicate, Cohere, Groq, ggml, LiteLLM or any REST endpoint) and scores the outputs with detectors. Probes cover prompt injection, DAN-style jailbreaks, encoding attacks, toxicity, training data replay, package hallucination, malware generation and XSS. It is a pip-installed CLI that writes a log, a JSONL run report and a hit log.
Who it is for: Security engineers and ML teams red-teaming LLMs and chat applications
Strengths
- Broad built-in probe set: promptinject, dan, encoding, glitch, leakreplay, malwaregen, xss and more
- Many generator backends, plus a configurable REST generator for arbitrary endpoints
- Plugin layout (probes, detectors, generators, harnesses) with base classes for custom additions
- Per-run JSONL report and hit log show which prompts caused failures
Weaknesses
- Command-line only; the README describes no web UI
- Default run uses all probes with 10 generations per prompt, which is slow and costly on paid APIs
- Detectors are automated, so FAIL results still need manual review
- PyPI release is updated periodically and can lag the main branch
What it needs
- GPU optional
- Needs Python 3.11-3.14, API keys for hosted targets (OpenAI, Replicate, Cohere, Groq, NIM, Bedrock)
- Models: Hugging Face, OpenAI, AWS Bedrock, Replicate, Cohere
Also in Security
See all 9| Rank | Project | Score |
|---|---|---|
| 1 | shannonAutonomous pentester that exploits web apps and APIs using source code analysis | 79 out of 100 |
| 2 | StrixAutonomous AI pentesting agents that validate vulnerabilities with working exploits | 75 out of 100 |
| 3 | pentagiAutonomous multi-agent penetration testing system with sandboxed security tools | 75 out of 100 |
| #4 | ifixaiPython CLI that audits AI agents for policy, access and reliability failures | 65 out of 100 |
| #5 | guardrailsProgrammable input, output and dialog guardrails for LLM applications | 55 out of 100 |