jailbreak-evaluation
Python package for language model jailbreak evaluation
GraphCanon updated 2w · GitHub synced 2w
Decision brief
jailbreak-evaluation is a Python package aimed at evaluating if AI models have been jailbroken by generating outputs that diverge from expected programming.
Good fit when
- When you need to assess whether an AI model can be manipulated to produce unpredictable or unintended outcomes through specific inputs, such as jailbreaking.
- You require tools that work with PyTorch and FastChat for evaluating the robustness of language models against manipulation.
Avoid when
- If your project does not involve assessing the security or integrity of how an AI model responds to manipulative input techniques designed to exploit design weaknesses.
- When you do not need dependencies on specific frameworks like PyTorch and FastChat, as jailbreak-evaluation requires these without automating their installation.
- Requirements:
- The tool depends on having PyTorch and FastChat installed; An API key from the OpenAI Platform is required for full functionality
Observed Jul 17, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (638d since push)
- As of 2w
- Provenance
- Not a fork · Organization account
- As of 2w
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install jailbreak-evaluationSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
A tool to evaluate if AI models have been 'jailbroken', meaning they can generate outputs that deviate from their intended programming when subjected to certain inputs.
Capability facts
- Languages
- python
Source: github.language+pyproject.toml · Aug 5, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Aug 5, 2026)
You should have an OpenAI API key to use the jailbreak-evaluation, which can be obtained from [OpenAI PlatforSource link
Source: README excerpt (regex_v1, Aug 5, 2026)
python -m pip install --upgrade pipSource link
Tags
README
Installation
Note on PyTorch and FastChat Installation: The jailbreak-evaluation depends on PyTorch and FastChat. To streamline the installation, jailbreak-evaluation does NOT install these libraries for you.
It is recommended to use pip for installation. Please make sure the latest version is installed, as jailbreak-evaluation is updated frequently:
python -m pip install --upgrade pip
---
# Please install the PyTorch: https://pytorch.org/get-started/locally/
---
# Please install the FastChat: https://github.com/lm-sys/FastChat?tab=readme-ov-file#install
---
# Install the jailbreak-evaluation
python -m pip install -U jailbreak-evaluation
---
## Quick Start
Evaluate a Language Model Jailbreak Attempt Using jailbreak-evaluation.
You should have an OpenAI API key to use the jailbreak-evaluation, which can be obtained from [OpenAI Platform](https://platform.openai.com/api-keys).
```python
from jailbreak_evaluation import MultifacetedEvaluation
---
## License
The jailbreak-evaluation uses Apache License 2.0.
For agents
This page has a .md twin and JSON over the API.