{"data":{"slug":"liu00222-open-prompt-injection","name":"Open-Prompt-Injection","tagline":"This repository provides a benchmark for prompt injection attacks and defenses in LLMs","github_url":"https://github.com/liu00222/Open-Prompt-Injection","owner":"liu00222","repo":"Open-Prompt-Injection","owner_avatar_url":"https://avatars.githubusercontent.com/u/42081599?v=4","primary_language":"Python","stars":464,"forks":74,"topics":["llm","llm-security","llms","prompt-injection","prompt-injection-tool","security-and-privacy"],"archived":false,"github_pushed_at":"2025-10-29T17:11:34+00:00","maintenance_label":"Slowing","url":"https://www.graphcanon.com/tools/liu00222-open-prompt-injection","markdown_url":"https://www.graphcanon.com/tools/liu00222-open-prompt-injection.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/liu00222-open-prompt-injection","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=liu00222-open-prompt-injection","description":"This repository provides a benchmark for prompt injection attacks and defenses in LLMs","homepage_url":null,"license":"MIT","open_issues":14,"watchers":3,"ai_summary":null,"readme_excerpt":"# Open-Prompt-Injection\n\n\n## Introduction\n\nThis repository is an open-source toolkit for prompt injection attacks and defenses. It enables implementation, evaluation, and extension of attacks, defenses, and LLM-integrated applications and agents. For a deeper dive into prompt injection, see [these slides](https://people.duke.edu/~zg70/code/PromptInjection.pdf), an extended version of a presentation given at the Safer with Google Summit 2025. \n\n\n\n## Required Python packages\n\nPre-requisite: [conda](https://www.anaconda.com/docs/getting-started/miniconda/install)\n\nInstall the environment using the following command: \n\n```\nconda env create -f environment.yml --name my_custom_env\n```\n\nThen activate the environment:\n\n```\nconda activate my_custom_env\n```\n\n## Usage\n\n### A simple demo\n\nBefore you start, go to './configs/model\\_configs/palm2\\_config.json' and replace the API keys with your real keys. Please refer to Google's official site for how to obtain an API key for PaLM2. For Meta's Llama models and OpenAI's GPT models, please also refer to their websites for registration details. \n\nThe following code snippet creates a model and queries the model with the prompt \"Write a poem about monkeys\":\n\n```python\nimport OpenPromptInjection as PI\nfrom OpenPromptInjection.utils import open_config\n\nmodel_config_path = './configs/model_configs/palm2_config.json'\nmodel_config = open_config(config_path=model_config_path)\nmodel = PI.create_model(config=model_config)\nmodel.print_model_info()\n\nmsg = \"Write a poem about monkeys\"\nprint(model.query(msg))\n```\n\n### Combined attack\n\nThe following code snippet evaluates the ASV of the scenario where the target task is sentiment analysis (i.e., the target data is sst2), the injected task is spam detection (i.e., the injected data is spam detection), the model is PaLM2, and no defense is applied:\n\n```python\nimport OpenPromptInjection as PI\nfrom OpenPromptInjection.utils import open_config\n\n# Create the target task\ntarget_task = PI.create_task(open_config(config_path='./configs/task_configs/sst2_config.json'), 100)\n\n# Create the model\nmodel_config = open_config(config_path='./configs/model_configs/palm2_config.json')\nmodel = PI.create_model(config=model_config)\n\n# Create the injected task\ninject_task = PI.create_task(open_config(config_path='./configs/task_configs/sms_spam_config.json'), 100, for_injection=True)\nattacker = PI.create_attacker('combine', inject_task)\n\n# Create the LLM-integrated App\ntarget_app = PI.create_app(target_task, model, defense='no')\n\n# Queries the model with the attacked data prompt and save the responses\nattack_responses = list()\nfor i, (data_prompt, ground_truth_label) in enumerate(target_app):\n    data_prompt_after_attack = attacker.inject(data_prompt, i, target_task=target_task.task)\n    response = target_app.query(data_prompt_after_attack, verbose=1, idx=i, total=len(target_app))\n    attack_responses.append(response)\n\n# Create an evaluator to calculate the ASV\nevaluator = PI.create_evaluator(\n    target_task_responses=None, \n    target_task=target_task,\n    injected_task_responses=None, \n    injected_task=attacker.task,\n    attack_responses=attack_responses\n)\n\nprint(f\"ASV = {evaluator.asv}\")\n```\n\nTo evaluate another defense (or attack strategy, target task, etc.), clients can change the config files and the path passed into the factory methods. \n\nTo run the experiments reported in the paper, please execute:\n\n```\npython3 run.py\n```\nafter changing the configurations in the \"run.py\" file. This script will call \"main.py\" which is the main file for the experiments. Clients can also feel free to check how \"main.py\" uses the major classes, factory methods, and utils for their own use cases. \n\n\n### Prompt Injection Detection with DataSentinel\n\nHere is an example of using DataSentinel as prompt injection detector. You may download the fine-tuned checkpoint from [this link](https://drive.google.com/file/d/1B0w5r5udH3I_aiZL0_-2a8WzBAqjuLsn/view?usp=sharing).\n\n```python\nimport OpenPromptInjec","github_created_at":"2023-10-19T04:15:27+00:00","created_at":"2026-07-11T23:40:03.770177+00:00","updated_at":"2026-07-11T23:40:12.923505+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"ai-agents","name":"AI Agents","url":"https://www.graphcanon.com/categories/ai-agents","markdown_url":"https://www.graphcanon.com/categories/ai-agents.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/ai-agents"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"llms","name":"llms"},{"slug":"prompt-injection","name":"prompt-injection"},{"slug":"llm","name":"llm"},{"slug":"python","name":"python"},{"slug":"prompt-injection-tool","name":"prompt-injection-tool"},{"slug":"llm-security","name":"llm-security"},{"slug":"security-and-privacy","name":"security-and-privacy"}],"trust":{"provenance":{"is_fork":false,"github_id":707020503,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T23:40:04.914Z","maintenance":{"label":"Slowing","score":36,"methodology":"github_public_v1","releases_90d":0,"days_since_push":255,"last_release_at":null},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T23:40:05.376Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T23:40:04.670Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-07-11T23:40:04.670Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-07-11T23:40:04.670Z"}}}}