--- title: "heretic" type: "tool" slug: "p-e-w-heretic" canonical_url: "https://www.graphcanon.com/tools/p-e-w-heretic" github_url: "https://github.com/p-e-w/heretic" homepage_url: "https://heretic-project.org" stars: 25895 forks: 2806 primary_language: "Python" license: "AGPL-3.0" categories: ["model-training"] tags: ["llm", "transformer", "abliteration"] updated_at: "2026-07-07T19:42:48.226814+00:00" --- # heretic > Fully automatic censorship removal for language models Heretic is a Python-based tool designed to remove safety alignment (censorship) from transformer-based language models using directional ablation techniques and an Optuna-powered parameter optimizer. It supports various dense, multimodal, and MoE models. ## Facts - Repository: https://github.com/p-e-w/heretic - Homepage: https://heretic-project.org - Stars: 25,895 · Forks: 2,806 · Open issues: 68 · Watchers: 107 - Primary language: Python - License: AGPL-3.0 - Last pushed: 2026-07-07T09:04:34+00:00 ## Categories - [Model Training](/categories/model-training.md) ## Tags llm, transformer, abliteration ## Related tools - [transformers](/tools/huggingface-transformers.md) - 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models (★ 162,350) - [LLMs-from-scratch](/tools/rasbt-llms-from-scratch.md) - Implement a ChatGPT-like LLM in PyTorch from scratch (★ 98,715) - [vllm](/tools/vllm-project-vllm.md) - A high-throughput and memory-efficient inference and serving engine for LLMs (★ 85,621) - [ray](/tools/ray-project-ray.md) - Unified framework for scaling AI and Python applications (★ 43,154) - [ai-engineering-from-scratch](/tools/rohitg00-ai-engineering-from-scratch.md) - Curriculum for building AI systems from scratch (★ 37,583) - [khoj](/tools/khoj-ai-khoj.md) - Your AI second brain. Self-hostable. (★ 35,518) - [self-llm](/tools/datawhalechina-self-llm.md) - 《开源大模型食用指南》 (★ 31,199) - [llmfit](/tools/alexsjones-llmfit.md) - Hundreds of models & providers. One command to find what runs on your hardware. (★ 29,185) ## README (excerpt) ```text Logo # Heretic: Fully automatic censorship removal for language models

Heretic is a tool that removes censorship (aka "safety alignment") from transformer-based language models without expensive post-training. It combines an advanced implementation of directional ablation, also known as "abliteration" ([Arditi et al. 2024](https://arxiv.org/abs/2406.11717), Lai 2025 ([1](https://huggingface.co/blog/grimjim/projected-abliteration), [2](https://huggingface.co/blog/grimjim/norm-preserving-biprojected-abliteration))), with a TPE-based parameter optimizer powered by [Optuna](https://optuna.org/). This approach enables Heretic to work **completely automatically.** Heretic finds high-quality abliteration parameters by co-minimizing the number of refusals and the KL divergence from the original model. This results in a decensored model that retains as much of the original model's intelligence as possible. Using Heretic does not require an understanding of transformer internals. In fact, anyone who knows how to run a command-line program can use Heretic to decensor language models. Heretic supports most dense models, including many multimodal models, several different MoE architectures, and even some hybrid models like Qwen3.5. Pure state-space models and certain other research architectures are not yet supported out of the box. Screenshot   Running unsupervised with the default configuration, Heretic can produce decensored models that rival the quality of abliterations created manually by human experts: | Model | Refusals for "harmful" prompts | KL divergence from original model for "harmless" prompts | | :--- | ---: | ---: | | [google/gemma-3-12b-it](https://huggingface.co/google/gemma-3-12b-it) (original) | 97/100 | 0 *(by definition)* | | [mlabonne/gemma-3-12b-it-abliterated-v2](https://huggingface.co/mlabonne/gemma-3-12b-it-abliterated-v2) | 3/100 | 1.04 | | [huihui-ai/gemma-3-12b-it-abliterated](https://huggingface.co/huihui-ai/gemma-3-12b-it-abliterated) | 3/100 | 0.45 | | **[p-e-w/gemma-3-12b-it-heretic](https://huggingface.co/p-e-w/gemma-3-12b-it-heretic) (ours)** | **3/100** | **0.16** | The Heretic version, generated without any human effort, achieves the same level of refusal suppression as other abliterations, but at a much lower KL divergence, indicating less damage to the original model's capabilities. *(You can reproduce those numbers using Heretic's built-in evaluation functionality, e.g. `heretic --model google/gemma-3-12b-it --evaluate-model p-e-w/gemma-3-12b-it-heretic`. Note that the exact values might be platform- and hardware-dependent. The table above was compiled using PyTorch 2.8 on an RTX 5090.)* Of course, mathematical metrics and automated benchmarks never tell the whole story, and are no substitute for human evaluation. Models generated with Heretic have been well-received by users (links and emphasis added): > "I was skeptical before, but I just downloaded > [**GPT-OSS 20B Heretic**](https://huggingface.co/p-e-w/gpt-oss-20b-heretic) > model and holy shit. It gives properly formatted long responses to sensitive topics, > using the exact uncensored words that you would expect from an uncensored model, > produces markdown format tables with details and whatnot. Looks like this is > the best abliterated version of this model so far..." > [*(Link to comment)*](https://old.reddit.com/r/LocalLLaMA/comments/1oymku1/heretic_fully_automatic_censorship_removal_for/np6tba6/) > "[**Heretic GPT 20b**](https://huggingface.co/p-e-w/gpt-oss-20b-heretic) > seems to be the best uncensored model I have tried yet. It doesn't destroy a > the model's intelligence and it is answering prompts normally would be > rejected by the base model." > [*(Link to ``` --- **Machine-readable endpoints** - JSON: [`/api/graphcanon/tools/p-e-w-heretic`](/api/graphcanon/tools/p-e-w-heretic) - LLM index: [/llms.txt](/llms.txt) - Full corpus: [/llms-full.txt](/llms-full.txt) _GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_