{"data":{"slug":"giskard-ai-giskard-oss","name":"giskard-oss","tagline":"Open-Source Evaluation & Testing library for LLM Agents","github_url":"https://github.com/Giskard-AI/giskard-oss","owner":"Giskard-AI","repo":"giskard-oss","owner_avatar_url":"https://avatars.githubusercontent.com/u/71782571?v=4","primary_language":"Python","stars":5824,"forks":537,"topics":["agent-evaluation","ai-red-team","ai-security","ai-testing","fairness-ai","llm","llm-eval","llm-evaluation","llm-security","llmops","ml-testing","ml-validation","mlops","rag-evaluation","red-team-tools","responsible-ai","trustworthy-ai"],"archived":false,"github_pushed_at":"2026-09-18T07:09:06+00:00","maintenance_label":"Very active","stars_delta_30d":97,"url":"https://www.graphcanon.com/tools/giskard-ai-giskard-oss","markdown_url":"https://www.graphcanon.com/tools/giskard-ai-giskard-oss.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/giskard-ai-giskard-oss","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=giskard-ai-giskard-oss","description":"🐢 Open-Source Evaluation & Testing library for LLM Agents","homepage_url":"https://docs.giskard.ai","license":"Apache-2.0","open_issues":65,"watchers":42,"ai_summary":"Giskard is an open-source Python library for testing and evaluating agentic systems, including LLMs, black-box agents, and multi-step pipelines. It offers modules for checks, scans, and provider-agnostic LLM routing.","readme_excerpt":"## Install\n\n```sh\npip install giskard           # checks (+ agents, llm, core)\npip install \"giskard[scan]\"   # + vulnerability / quality scan\npip install \"giskard[openai]\" # provider SDK for LLM judges / generators\n```\n\nRequires Python 3.12+.\n\n| Extra | Adds |\n| --- | --- |\n| *(none)* | `giskard-checks` and dependencies |\n| `scan` | `giskard-scan` |\n| `openai` / `anthropic` / … | provider SDKs (see `pyproject.toml` optional deps) |\n\n**Telemetry:** optional aggregated analytics via `giskard-core`. No prompts or outputs are sent.\nOpt out with `export DO_NOT_TRACK=1` or `export GISKARD_TELEMETRY_DISABLED=1` (or the same keys in a `.env` file in the working directory). Set them before import to skip creating `~/.giskard/id`; setting them later still stops further sends.\nDetails: [`giskard-core` README](libs/giskard-core/README.md#telemetry).\n\n---\n\nGiskard is an open-source Python library for **testing and evaluating agentic systems**. The v3 architecture is a modular set of focused packages — each carrying only the dependencies it needs — built from scratch to wrap anything: an LLM, a black-box agent, or a multi-step pipeline.\n\n| Status         | Package          | Description                                                                                                                                                              |\n| -------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |\n| ✅ Stable      | `giskard-checks` | Testing & evaluation — scenario API, built-in checks, LLM-as-judge                                                                                                       |\n| ✅ Stable      | `giskard-scan`   | Agent vulnerability scanner + RAG/quality evaluation — red teaming, prompt injection, jailbreaks & harmful content (`vulnerability_scan`, successor of [v2 Scan](https://legacy-docs.giskard.ai/en/stable/open_source/scan/index.html)), plus knowledge-base quality eval (`quality_scan`, successor of [v2 RAGET](https://legacy-docs.giskard.ai/en/stable/open_source/testset_generation/index.html)) |\n\nThese build on three foundational libraries — `giskard-core` (shared utilities & telemetry), `giskard-llm` (provider-agnostic LLM routing), and `giskard-agents` (agent & workflow orchestration) — which are pulled in automatically and rarely used directly.","github_created_at":"2022-03-06T21:45:37+00:00","created_at":"2026-07-07T17:42:05.159294+00:00","updated_at":"2026-09-20T04:30:19.10392+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"}],"tags":[{"slug":"agent-evaluation","name":"agent-evaluation"},{"slug":"ai-red-team","name":"ai-red-team"},{"slug":"ai-security","name":"ai-security"},{"slug":"ai-testing","name":"ai-testing"},{"slug":"fairness-ai","name":"fairness-ai"},{"slug":"llm-eval","name":"llm-eval"},{"slug":"llm-security","name":"llm-security"},{"slug":"ml-testing","name":"ml-testing"}],"trust":{"provenance":{"is_fork":false,"github_id":466864356,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-18T08:22:32.605Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":26,"days_since_push":0,"last_release_at":"2026-09-14T09:49:47Z","stars_delta_30d":97,"open_issues_delta_30d":-27},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-09-18T08:22:34.736Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-18T08:22:31.519Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-09-18T08:22:31.519Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-18T08:22:31.519Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When you need a comprehensive testing and evaluation library for agentic systems, including LLMs, black-box agents, and multi-step pipelines.","If you require a tool that supports both vulnerability scanning and quality evaluation for RAG systems.","When you are working with Python 3.12+ and want to leverage provider-agnostic LLM routing.","If you are interested in using LLMs as judges or generators and need the corresponding provider SDKs."],"when_not_to_use":["If you are working with Python versions below 3.12, as Giskard-oss requires Python 3.12+.","When you prefer not to use tools that collect telemetry data, even if it is aggregated and does not include prompts or outputs.","If you are looking for a tool that does not offer a modular architecture, as Giskard-oss is built with a modular set of focused packages.","When you need a tool that does not support open-source licensing under Apache-2.0."],"source":"enrich:decision_facts","observed_at":"2026-09-18T11:17:41.779Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"Giskard-oss is an open-source Python library for testing and evaluating agentic systems, including LLMs, black-box agents, and multi-step pipelines. It offers modules for checks, scans, and provider-agnostic LLM routing."}]}}