baseline-defenses
Research code for evaluating defenses against adversarial attacks on aligned language models
GraphCanon updated 2w · GitHub synced 2w · 25 views this month
Decision brief
A toolkit for evaluating defenses against adversarial attacks on aligned language models, focusing on perplexity filter and paraphrase defense strategies.
Good fit when
- - When you need to evaluate the effectiveness of baseline defenses such as the perplexity filter or paraphrase defense in protecting aligned language models from adversarial attacks.
- - If your work involves examining how input preprocessing, like paraphrasing prompts, can mitigate risks associated with adversarial inputs.
Avoid when
- - Do not use if you require comprehensive coverage of all possible defensive measures. This tool specifically lacks detailed code for retokenization defenses involving BPE-dropout.
- - If your scenario demands more advanced or specialized defense mechanisms beyond the scope of baseline strategies, this repository will fall short on delivering those.
Observed Jul 12, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (1013d since push)
- As of 2w
- Provenance
- Not a fork · Personal account
- As of 2w
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install baseline-defensesSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
Includes code for perplexity filter and paraphrase attack defense strategies. Evaluation of detection, input preprocessing, and adversarial training methods.
Capability facts
- Languages
- python
Source: github.language · Aug 5, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Aug 5, 2026)
The paraphrase defense is rewriting the prompt. For our experiments, we used ChatGPT. Note while this defense is effective it might come at high performance cost.Source link
Tags
README
Baseline Defenses for Adversarial Attacks Against Aligned Language Models
Official Code for "Baseline Defenses for Adversarial Attacks Against Aligned Language Models"
Overview
We evaluate several baseline defense strategies against leading adversarial attacks on LLMs, discussing the various settings in which each is feasible and effective. Particularly, we look at three types of defenses: detection (perplexity based), input preprocessing (paraphrase and retokenization), and adversarial training. The paper can be found here.
The repository only contains the code for the perplexity filter and paraphrase attack. The retokenization defenses is conducted directly via altering tokenizer via BPE-dropout. For LLaMA model, see the tokenizer.sp_model.encode(input_text, alpha=bt_alpha, enable_sampling=True) function, and for other models, BPE-dropout is set by tokenizer._tokenizer.model.dropout=bt_alpha, where bt_alpha is the dropout rate.
Perplexity Filter
The perplexity filter in the code consists of two filters, a perplexity filter which as also been proposed in concurrent work by Alon et al. and a windowed perplexity filter, which consists of checking the perplexity of a window of $n$ tokens.
Paraphrase Defense
The paraphrase defense is rewriting the prompt. For our experiments, we used ChatGPT. Note while this defense is effective it might come at high performance cost.
Limitations
As in all research work, we were limited to the settings we explored in the paper.
For agents
This page has a .md twin and JSON over the API.