--- title: "giskard-oss" type: "tool" slug: "giskard-ai-giskard-oss" canonical_url: "https://www.graphcanon.com/tools/giskard-ai-giskard-oss" github_url: "https://github.com/Giskard-AI/giskard-oss" homepage_url: "https://docs.giskard.ai" stars: 5498 forks: 481 primary_language: "Python" license: "Apache-2.0" categories: ["evaluation-observability"] tags: ["llm-eval", "agent-evaluation", "llm", "fairness-ai", "ai-security", "ai-red-team", "ai-testing", "llm-evaluation"] updated_at: "2026-07-07T18:39:53.869225+00:00" --- # giskard-oss > Evaluation & Testing library for LLM Agents Giskard is an open-source Python library developed to test and evaluate agentic systems. It offers dynamic, multi-turn testing capabilities with enhanced vulnerability scanning and RAG evaluation features. ## Facts - Repository: https://github.com/Giskard-AI/giskard-oss - Homepage: https://docs.giskard.ai - Stars: 5,498 · Forks: 481 · Open issues: 73 · Watchers: 40 - Primary language: Python - License: Apache-2.0 - Last pushed: 2026-07-07T10:13:58+00:00 ## Categories - [Evaluation & Observability](/categories/evaluation-observability.md) ## Tags llm-eval, agent-evaluation, llm, fairness-ai, ai-security, ai-red-team, ai-testing, llm-evaluation ## Related tools - [langflow](/tools/langflow-ai-langflow.md) - Langflow is a powerful platform for building and deploying AI-powered agents and workflows. (★ 151,298) - [graphrag](/tools/microsoft-graphrag.md) - A modular graph-based Retrieval-Augmented Generation (RAG) system (★ 34,239) - [langfuse](/tools/langfuse-langfuse.md) - Langfuse: Open source AI engineering platform for LLM evaluation, observability, and prompt management. (★ 30,626) - [RAG_Techniques](/tools/nirdiamant-rag-techniques.md) - Advanced RAG Techniques (★ 28,398) - [mlflow](/tools/mlflow-mlflow.md) - The open source AI engineering platform for agents, LLMs, and ML models. (★ 26,921) - [llm-action](/tools/liguodongiot-llm-action.md) - 分享大模型技术原理和实战经验，涵盖工程化、应用落地 (★ 24,670) - [awesome-n8n-templates](/tools/enescingoz-awesome-n8n-templates.md) - The largest open-source collection of n8n automation templates on GitHub. (★ 23,714) - [promptfoo](/tools/promptfoo-promptfoo.md) - Test your prompts, agents, and RAGs. Evaluate LLM performance with red teaming/pentesting/vulnerability scanning. (★ 23,003) ## README (excerpt) ```text

giskardlogo giskardlogo

Evals, Red Teaming and Test Generation for Agentic Systems

Modular, Lightweight, Dynamic and Async-first

Docs • Website • Community


> [!IMPORTANT] > **Giskard v3** is a fresh rewrite designed for dynamic, multi-turn testing of AI agents. This release drops heavy dependencies for better efficiency while introducing a more powerful AI vulnerability scanner and enhanced RAG evaluation capabilities. For now, the vulnerability scanner and RAG evaluation still rely on Giskard v2. > **Giskard v2 remains available but is no longer actively maintained.** > Follow progress → [Read the v3 Announcement](https://github.com/orgs/Giskard-AI/discussions/2250) · [Roadmap](https://github.com/Giskard-AI/giskard-oss/issues/2252) ## Install ```sh pip install giskard ``` Requires Python 3.12+. **Telemetry:** Libraries built on `giskard-core` (including `giskard-checks`) may send **optional, aggregated usage analytics** to help improve the product. No prompts, model outputs, or scenario text are included. See [what is collected and how to opt out](libs/giskard-core/README.md#telemetry). --- Giskard is an open-source Python library for **testing and evaluating agentic systems**. The v3 architecture is a modular set of focused packages — each carrying only the dependencies it needs — built from scratch to wrap anything: an LLM, a black-box agent, or a multi-step pipeline. | Status | Package | Description | | -------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | ✅ Beta | `giskard-checks` | Testing & evaluation — scenario API, built-in checks, LLM-as-judge | | ✅ Beta | `giskard-scan` | Agent vulnerability scanner — red teaming, prompt injection, data leakage (successor of [v2 Scan](https://legacy-docs.giskard.ai/en/stable/open_source/scan/index.html)) | | 📋 Planned | `giskard-rag` | RAG evaluation & synthetic data generation (successor of [v2 RAGET](https://legacy-docs.giskard.ai/en/stable/open_source/testset_generation/index.html)) | ## Giskard Checks — create and apply evals for testing agents ```sh pip install giskard-checks ``` **[Giskard Checks](https://docs.giskard.ai/oss/checks)** is a lightweight library for creating evaluations (evals) that test LLM-based systems — from simple assertions to LLM-as-judge assessments. Unlike traditional unit tests, evals are designed for **non-deterministic outputs** where the same input can produce different valid responses. Use Giskard Checks to: - **Catch regressions** — verify your system still behaves correctly after changes - **Validate RAG quality** — check if answers are grounded in retrieved context - **Enforce safety rules** — ensure outputs conform to your content policies - **Evaluate multi-turn agents** — test full conversations, not just single exchanges Built-in evals include string matching, comparisons, regex, semantic similarity, and LLM-as-judge checks (`Groundedness`, `Conformity`, `LLMJudge`). ### Quickstart ```python f ``` --- **Machine-readable endpoints** - JSON: [`/api/graphcanon/tools/giskard-ai-giskard-oss`](/api/graphcanon/tools/giskard-ai-giskard-oss) - LLM index: [/llms.txt](/llms.txt) - Full corpus: [/llms-full.txt](/llms-full.txt) _GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_