ALERT
A Comprehensive Benchmark for Assessing Large Language Models' Safety Through Red Teaming
GraphCanon updated Oct 19, 2026 · GitHub synced Oct 19, 2026
Decision brief
Unverified catalog guidance: ALERT is designed specifically for red-teaming based safety evaluation on large language models, using MIT licensed prompts and adversarial augmentation.
Good fit when
- Unverified catalog guidance: When evaluating safety metrics of large language models through red-teaming approaches
- Unverified catalog guidance: For generating additional safety-focused prompts via template designs and keyword matching
Avoid when
- Unverified catalog guidance: If your evaluation does not require bias detection or safety assessment under adversarial conditions
- Unverified catalog guidance: In scenarios where a broader range of model aspects beyond safety is needed, as ALERT focuses primarily on safety benchmarks
Observed Jul 16, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (758d since push)
- As of Oct 19, 2026
- Provenance
- Not a fork · Organization account
- As of Oct 19, 2026
- Security (OSV)
- No published findings
- As of Jul 15, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
Package installation is not established from the available documentation. Check the source repository for setup instructions.
git clone https://github.com/babelscape/alertSource repositorySimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
This repository hosts the ALERT benchmark designed to evaluate large language models on safety through various testing categories including bias detection and red-teaming.
Capability facts
- MCP server
- Unknown; MCP server not established by this scan
Source: repo_scan · Oct 19, 2026
- Languages
- python
Source: github.language · Oct 19, 2026
Categories
Tags
README
License As specified in the paper, most of the prompts available in the ALERT benchmark are derived from the Anthropic HH RLHF dataset that is licensed under the MIT license. A copy of the license can be found here. Starting from these prompts, we then employ a combination of key...
For agents
This page has a .md twin and JSON over the API.