ALERT
A Comprehensive Benchmark for Assessing Large Language Models' Safety Through Red Teaming
GraphCanon updated Sep 10, 2026 · GitHub synced Sep 10, 2026
33views this month
Decision brief
ALERT is designed specifically for red-teaming based safety evaluation on large language models, using MIT licensed prompts and adversarial augmentation.
Good fit when
- When evaluating safety metrics of large language models through red-teaming approaches
- For generating additional safety-focused prompts via template designs and keyword matching
Avoid when
- If your evaluation does not require bias detection or safety assessment under adversarial conditions
- In scenarios where a broader range of model aspects beyond safety is needed, as ALERT focuses primarily on safety benchmarks
Observed Jul 16, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (719d since push)
- As of Sep 10, 2026
- Provenance
- Not a fork · Organization account
- As of Sep 10, 2026
- Security (OSV)
- No criticals
- As of Jul 15, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install ALERTSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
This repository hosts the ALERT benchmark designed to evaluate large language models on safety through various testing categories including bias detection and red-teaming.
Capability facts
- Languages
- python
Source: github.language · Sep 10, 2026
Categories
Tags
README
License As specified in the paper, most of the prompts available in the ALERT benchmark are derived from the Anthropic HH RLHF dataset that is licensed under the MIT license. A copy of the license can be found here. Starting from these prompts, we then employ a combination of key...
For agents
This page has a .md twin and JSON over the API.