weak-to-strong
Novel Inference-Time Attack Leveraging Small Models to Guide Larger LLMs into Generating Harmful Outputs
GraphCanon updated Oct 4, 2026 · GitHub synced Oct 4, 2026
20views this month
Decision brief
Weak-to-Strong is an inference-time attack exploiting smaller models to guide larger LLMs towards harmful output generation.
Good fit when
- Use it for research purposes specifically geared at understanding the vulnerabilities in large language models and improving their robustness against adversarial attacks.
- Leverage if your project involves red-teaming efforts that aim to assess how well alignment strategies can be bypassed with minimum computational overhead by manipulating inference steps.
Avoid when
- Do not use it for applications requiring ethical guidelines adherence as it is designed to navigate around the safety mechanisms in large language models.
- Avoid using this tool if you are developing systems that must ensure consistent alignment and prevent any form of harmful output generation, such as public communication platforms or education tools.
- Requirements:
- Min 8 GB RAM; The smaller models guiding the large LLM must be available.; A high-performance computing environment might be necessary if running on very large datasets or models.
Observed Jul 16, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (520d since push)
- As of Oct 4, 2026
- Provenance
- Not a fork · Personal account
- As of Oct 4, 2026
- Security (OSV)
- No lockfile
- As of Jul 11, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install weak-to-strongSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
Implements Weak-to-Strong Jailbreaking on Large Language Models (LLMs), which uses small unsafe/aligned models to guide larger aligned models into producing harmful outputs with a high attack success rate.
Capability facts
- Languages
- python
Source: github.language · Oct 4, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Oct 4, 2026)
python run.py --beta 1.50 --batch_size 16 --output_file "[OUTPUT FILE NAME]" --att_fileSource link
Tags
README
Weak to Strong Jailbreaking on Large Language Models 📣 Update : Our paper has been accepted to ICML 2025 ! 📄 arXiv 🤗 HuggingFace Paper Page Overview Despite major advances in aligning large language models (LLMs), red teaming efforts consistently reveal vulnerabilities: even w...
For agents
This page has a .md twin and JSON over the API.