PromptAttack logo

PromptAttack

GodXuxilie/PromptAttack

An LLM can Fool Itself: A Prompt-Based Adversarial Attack

GraphCanon updated Oct 5, 2026 · GitHub synced Oct 5, 2026

33views this month

117 stars17 forksLast push Jan 21, 2025 Python

Decision brief

PromptAttack is an LLM-targeted adversarial attack tool that leverages prompt engineering to generate adversarial samples keeping semantic intact but misclassifying outputs.

Good fit when

  • For targeted analysis of adversarial robustness in specific language models.
  • When needing a detailed understanding of how perturbations in input prompts affect model classification performance.

Avoid when

  • If the focus is on general model improvement rather than adversarial testing.
  • When working with proprietary or sensitive data that cannot be manipulated via external prompt tools, given potential data leakage concerns.

Observed Jul 17, 2026 · Source: enrich:decision_facts

Verify the decision

Maintenance and security

Full trust report
Maintenance
Dormant (622d since push)
As of Oct 5, 2026
Provenance
Not a fork · Personal account
As of Oct 5, 2026
Security (OSV)
241 low (241 low)
As of Jul 11, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

pip install PromptAttack
PyPI

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

This repository contains the source code for the ICLR 2024 paper that discusses a prompt-based adversarial attack on language models, allowing users to generate adversarial samples targeting various LLMs.

Capability facts

Languages
python

Source: github.language · Oct 5, 2026

Categories

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Oct 5, 2026)

pip install -r requirements.txt
Source link
Works with ChatGPTChatGPT

Source: README excerpt (regex_v1, Oct 5, 2026)

Logs of querying *ChatGPT*:
Source link

Tags

README

An LLM can Fool Itself: A Prompt Based Adversarial Attack This is the source code for the ICLR 2024 paper "An LLM can Fool Itself: A Prompt Based Adversarial Attack", <br Xilie Xu (NUS), Keyi Kong (SDU), Ning Liu (SDU), Lizhen Cui (SDU), Di Wang (KAUST), Jingfeng Zhang (Universit...

For agents

This page has a .md twin and JSON over the API.

Was this helpful?

Anonymous feedback helps us improve pages and translations.