{"data":{"slug":"ethz-spylab-agentdojo","name":"agentdojo","tagline":"A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents","github_url":"https://github.com/ethz-spylab/agentdojo","owner":"ethz-spylab","repo":"agentdojo","owner_avatar_url":"https://avatars.githubusercontent.com/u/106388551?v=4","primary_language":"Python","stars":659,"forks":168,"topics":["benchmark","large-language-models","prompt-injection","security"],"archived":false,"github_pushed_at":"2026-06-02T10:01:31+00:00","maintenance_label":"Steady","url":"https://www.graphcanon.com/tools/ethz-spylab-agentdojo","markdown_url":"https://www.graphcanon.com/tools/ethz-spylab-agentdojo.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/ethz-spylab-agentdojo","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=ethz-spylab-agentdojo","description":"A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents.","homepage_url":"https://agentdojo.spylab.ai/","license":"MIT","open_issues":33,"watchers":7,"ai_summary":"AgentDojo provides a benchmarking environment to evaluate security attacks (like prompt injection) and defenses for large language model agents.","readme_excerpt":"<center>\n\n# AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents\n\n \n   \n\n\n\n[Edoardo Debenedetti](https://edoardo.science)<sup>1</sup>, [Jie Zhang](https://zj-jayzhang.github.io)<sup>1</sup>, [Mislav Balunović](https://www.sri.inf.ethz.ch/people/mislav)<sup>1,2</sup>, [Luca Beurer-Kellner](https://www.sri.inf.ethz.ch/people/luca)<sup>1,2</sup>, [Marc Fischer](https://marcfischer.at)<sup>1,2</sup>, [Florian Tramèr](https://floriantramer.com)<sup>1</sup>\n\n<sup>1</sup>ETH Zurich and <sup>2</sup>Invariant Labs\n\n[Read Paper](https://arxiv.org/abs/2406.13352) | [Inspect Results](https://agentdojo.spylab.ai/results/)\n</center>\n\n## Quickstart\n\n```bash\npip install agentdojo\n```\n\n> [!IMPORTANT]\n> Note that the API of the package is still under development and might change in the future.\n\nIf you want to use the prompt injection detector, you need to install the `transformers` extra:\n\n```bash\npip install \"agentdojo[transformers]\"\n```\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attack with tool knowlege, run the following command:\n\n```bash\npython -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \\\n    -ut user_task_1 --model gpt-4o-2024-05-13 \\\n    --defense tool_filter --attack tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \\\n    --defense tool_filter --attack tool_knowledge\n```\n\n## Inspect the results\n\nTo inspect the results, go to the dedicated [results page](https://agentdojo.spylab.ai/results/) of the documentation. AgentDojo results are also listed in the [Invariant Benchmark Registry](https://explorer.invariantlabs.ai/benchmarks/).Agent\n\n## Documentation of the Dojo\n\nTake a look at our [documentation](https://agentdojo.spylab.ai/).\n\n## Development set-up\n\nTake a look at the [development set-up](https://agentdojo.spylab.ai/development/) docs.\n\n## Citing\n\nIf you use AgentDojo in your research, please consider citing our paper:\n\n```bibtex\n@inproceedings{\n debenedetti2024agentdojo,\n title={AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for {LLM} Agents},\n author={Edoardo Debenedetti and Jie Zhang and Mislav Balunovic and Luca Beurer-Kellner and Marc Fischer and Florian Tram{\\`e}r},\n booktitle={The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track},\n year={2024},\n url={https://openreview.net/forum?id=m1YYAQjO3w}\n}\n```","github_created_at":"2024-02-29T10:47:28+00:00","created_at":"2026-07-11T23:42:10.344174+00:00","updated_at":"2026-07-12T00:57:16.370908+00:00","categories":[{"slug":"ai-agents","name":"AI Agents","url":"https://www.graphcanon.com/categories/ai-agents","markdown_url":"https://www.graphcanon.com/categories/ai-agents.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/ai-agents"},{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"}],"tags":[{"slug":"prompt-injection","name":"prompt-injection"},{"slug":"benchmark","name":"benchmark"},{"slug":"large-language-models","name":"large-language-models"},{"slug":"security","name":"security"}],"trust":{"provenance":{"is_fork":false,"github_id":765133369,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T23:42:18.572Z","maintenance":{"label":"Steady","score":60,"methodology":"github_public_v1","releases_90d":0,"days_since_push":39,"last_release_at":"2025-10-27T18:09:56Z"},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T23:42:19.100Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-12T00:56:57.718Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-07-12T00:56:57.718Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-07-12T00:56:57.718Z"}},"decision_facts":{"hosting":null,"pricing":{"model":"freemium","summary":"Open-source under the MIT License. Some advanced features might require additional libraries or APIs."},"requirements":{"min_ram_gb":8,"requires_docker":false},"constraints":{"min_ram_gb":8,"pricing_model":"freemium","requires_docker":false},"when_to_use":[],"when_not_to_use":[],"source":"enrich:decision_facts","observed_at":"2026-07-12T00:57:16.100Z"},"constraint_facets":{"min_ram_gb":8,"pricing_model":"freemium","requires_docker":false},"decision_summary":[{"label":"Pricing","value":"freemium - Open-source under the MIT License. Some advanced features might require additional libraries or APIs."},{"label":"Requirements","value":"Min 8 GB RAM"},{"label":"Adopt for","value":"AgentDojo serves as a benchmarking environment to evaluate security attacks, like prompt injection, and defenses for Large Language Model (LLM) agents."}]}}