[{"data":1,"prerenderedAt":136},["ShallowReactive",2],{"learn-what-is-abliteration":3},{"collection":4,"slug":5,"path":6,"title":7,"seoTitle":8,"description":9,"summary":10,"primaryKeyword":11,"cluster":12,"published":13,"updated":13,"reviewed":13,"readingMinutes":14,"html":15,"toc":16,"wordCount":61,"sources":62,"faq":102,"related":115,"tested":132,"internalLinks":133},"learn","what-is-abliteration","\u002Flearn\u002Fwhat-is-abliteration\u002F","What is abliteration?","What Is Abliteration? Refusal Directions in LLMs","Abliteration edits an LLM's weights to suppress the activation direction tied to refusals. How it works conceptually, how it is measured and its limits.","Abliteration is a way to make an open-weight language model decline fewer requests by editing its weights instead of retraining it. It rests on research showing that refusal in many chat models is tied to one direction in the model's internal activations. That direction is estimated by contrasting two sets of prompts and then removed from the weights. The edit is cheap, but it is approximate, model-specific and can reduce capability.","what is abliteration","terminology","2026-11-14",8,"\u003Ch2 id=\"the-research-it-builds-on\">The research it builds on\u003C\u002Fh2>\n\u003Cp>Abliteration comes out of interpretability research, not out of model training. In 2024, \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2406.11717\" rel=\"noopener\" class=\"ext\">Arditi et al.\u003C\u002Fa> studied 13 open-weight chat models of up to 72B parameters and reported that refusal in each one is mediated by a single direction in the model’s activations. Erasing that direction stopped the models refusing harmful instructions. Adding it made them refuse harmless ones.\u003C\u002Fp>\n\u003Cp>The authors presented a weight-editing version of this as “a novel white-box jailbreak” and argued that it shows how brittle current safety fine-tuning is. Model builders took up the idea under the name abliteration. Maxime Labonne’s \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fmlabonne\u002Fabliteration\" rel=\"noopener\" class=\"ext\">explainer on Hugging Face\u003C\u002Fa>, which credits FailSpy’s original code, describes it for practitioners.\u003C\u002Fp>\n\u003Cp>This page explains the idea at a conceptual level. It contains no code and no settings.\u003C\u002Fp>\n\u003Ch2 id=\"the-residual-stream-and-a-direction\">The residual stream and a “direction”\u003C\u002Fh2>\n\u003Cp>A transformer processes text as a sequence of token positions. At each position it keeps a vector, called the residual stream, that every layer reads from and adds to: the attention block writes its output into it, then the MLP block does the same. After the last layer, that vector determines the next token.\u003C\u002Fp>\n\u003Cp>A direction is a fixed vector in that same space. Any activation can be split into two parts: how far it points along the direction, which is a single number, and everything else. The claim behind abliteration is that “this request should be refused” is carried largely by that one number along one particular direction.\u003C\u002Fp>\n\u003Cp>That is a strong simplification of what a network does. The evidence for it is empirical: across the models Arditi et al. tested, moving activations along that one direction was enough to switch refusal off and on.\u003C\u002Fp>\n\u003Ch2 id=\"finding-the-direction-a-difference-of-means\">Finding the direction: a difference of means\u003C\u002Fh2>\n\u003Cp>The direction is estimated, not read off the model. The usual intuition, as Labonne describes it, is a difference of means. Gather requests the model refuses and requests it answers, record the activations each group produces, and average each group separately. The difference between the two averages is a vector pointing, roughly, from “answer” toward “refuse”.\u003C\u002Fp>\n\u003Cp>Doing this at different layers gives one candidate direction per layer, and someone has to pick one. Labonne notes that the choice in his example came from checking the model’s outputs by hand.\u003C\u002Fp>\n\u003Cp>The weak point is the word “roughly”. A difference of means captures everything that systematically separates the two groups, not only refusal: topic, vocabulary, length and phrasing all leave a trace. This suggests that the choice of contrasting prompts shapes what gets removed, which may be one reason two abliterations of the same base model behave differently.\u003C\u002Fp>\n\u003Ch2 id=\"removing-it-two-options\">Removing it: two options\u003C\u002Fh2>\n\u003Cp>Once a direction is chosen, there are two ways to suppress it (\u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fmlabonne\u002Fabliteration\" rel=\"noopener\" class=\"ext\">Labonne\u003C\u002Fa>).\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Inference-time intervention.\u003C\u002Fstrong> While the model generates, the part of each component’s output that lies along the refusal direction is subtracted before it reaches the residual stream. The weights stay as they were, but the model needs a modified inference setup.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Weight orthogonalization.\u003C\u002Fstrong> The weight matrices that write into the residual stream are changed so that their outputs are orthogonal (perpendicular) to the refusal direction, so they can no longer write anything along it. The change is permanent and the result is an ordinary checkpoint. That is why abliterated models are published as normal weights and can be converted to \u003Ca href=\"\u002Flearn\u002Fgguf-quantization\u002F\">quantized formats such as GGUF\u003C\u002Fa> like any other model.\u003C\u002Fp>\n\u003Cp>“Abliteration” usually means the second option. No gradient descent is involved and no target answers are needed, which is the main practical difference from fine-tuning. \u003Ca href=\"\u002Flearn\u002Funcensored-vs-abliterated\u002F\">Uncensored vs abliterated\u003C\u002Fa> sets the two side by side in a table.\u003C\u002Fp>\n\u003Ch2 id=\"editing-only-some-layers\">Editing only some layers\u003C\u002Fh2>\n\u003Cp>Nothing requires editing every layer, and many published abliterations do not. Some model cards say which parts were left alone. That is worth reading, because unedited parts keep the base model’s behavior.\u003C\u002Fp>\n\u003Cul>\n\u003Cli>The \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fhuihui-ai\u002FHuihui-Qwen3.8-27B-abliterated\" rel=\"noopener\" class=\"ext\">huihui-ai Qwen3.8-27B card\u003C\u002Fa> says only layers 18 to 51 of the model’s 64 were ablated, and that the multi-token prediction (MTP) and vision parts were not modified. Image inputs are therefore encoded by components the edit did not touch before they reach the edited language layers. Variants of this base are tracked on the \u003Ca href=\"\u002Fmodels\u002Fqwen3-8-27b-uncensored\u002F\">Qwen3.8 27B page\u003C\u002Fa>.\u003C\u002Fli>\n\u003Cli>The \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fhuihui-ai\u002FHuihui-GLM-5.3-Flash-abliterated-GGUF\" rel=\"noopener\" class=\"ext\">huihui-ai GLM-5.3-Flash GGUF card\u003C\u002Fa> says only layers 15 to 35 were ablated and that no expert modules were changed. GLM-5.3-Flash is a mixture-of-experts model with 288 routed experts, 8 of them active per token, so the experts are outside the edit’s scope. Any change to them in this release comes from the GGUF conversion, not from abliteration.\u003C\u002Fli>\n\u003Cli>Labonne’s \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fmlabonne\u002Fgemma-3-27b-it-abliterated\" rel=\"noopener\" class=\"ext\">Gemma 3 27B variant\u003C\u002Fa> uses what its card calls “layerwise abliteration”, with a separate refusal direction computed for each layer from that layer’s hidden states. He calls it experimental and writes that Gemma 3 was “much more resilient to abliteration than other models like Qwen 2.5”.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>The cards do not always explain why they limit the scope. A reasonable inference is that editing fewer components changes less of the model, trading complete refusal suppression for smaller side effects. For readers the practical point is simpler: test what was edited and what was not, because the two may behave differently.\u003C\u002Fp>\n\u003Ch2 id=\"automated-parameter-search-heretic\">Automated parameter search: Heretic\u003C\u002Fh2>\n\u003Cp>A manual abliteration involves a chain of judgment calls: which candidate direction to use, which layers to edit and how strongly. \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fp-e-w\u002Fheretic\" rel=\"noopener\" class=\"ext\">Heretic\u003C\u002Fa> turns those choices into an optimization problem. It produces many candidate edits, scores the resulting model each time, and uses an optimizer (TPE, from the Optuna library) to search for parameters that do well on the scores.\u003C\u002Fp>\n\u003Cp>By default it co-optimizes two scores: the refusal rate on a set of test prompts, which it tries to lower, and the KL divergence from the original model, which it tries to keep small. Its default modifier is now Arbitrary-Rank Ablation (ARA), which the README says was developed specifically for Heretic. The classic directional ablation is still included. The project is licensed under AGPL-3.0.\u003C\u002Fp>\n\u003Cp>The README is candid about reproducibility. It says a Heretic model can be recreated byte for byte only if the hardware, drivers and entire software stack match the original, because PyTorch does not guarantee identical outputs otherwise.\u003C\u002Fp>\n\u003Ch3 id=\"how-to-read-the-two-numbers\">How to read the two numbers\u003C\u002Fh3>\n\u003Cp>\u003Cstrong>Refusal count\u003C\u002Fstrong> is the number of test prompts the model still refused, usually printed next to the base model’s count on the same prompts. In Heretic’s default setup, a response counts as a refusal when it contains one of a list of marker phrases, such as apologies or “as an AI”. That makes it a rough count. A refusal worded differently is missed, and a helpful answer that happens to contain a marker word is counted.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>KL divergence\u003C\u002Fstrong> measures how far the edited model’s predicted probabilities have moved from the original’s on a separate set of prompts. Zero means identical predictions, and larger values mean a bigger shift. Heretic’s default scorer compares the distribution of the first generated token, so this is a narrow proxy for collateral change. A low value says early predictions stayed close to the base. It does not say that coding or reasoning survived.\u003C\u002Fp>\n\u003Cp>Here are the values two creators report:\u003C\u002Fp>\n\u003Cdiv class=\"table-wrap\" role=\"region\" aria-label=\"Scrollable table\" tabindex=\"0\">\n\u003Ctable>\n\u003Cthead>\n\u003Ctr>\n\u003Cth>Variant\u003C\u002Fth>\n\u003Cth>Tool, as stated on the card\u003C\u002Fth>\n\u003Cth>Refusals, variant\u003C\u002Fth>\n\u003Cth>Refusals, base\u003C\u002Fth>\n\u003Cth>KL divergence\u003C\u002Fth>\n\u003C\u002Ftr>\n\u003C\u002Fthead>\n\u003Ctbody>\n\u003Ctr>\n\u003Ctd>\u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fp-e-w\u002Fgpt-oss-20b-heretic\" rel=\"noopener\" class=\"ext\">p-e-w\u002Fgpt-oss-20b-heretic\u003C\u002Fa>\u003C\u002Ftd>\n\u003Ctd>Heretic v1.0.0\u003C\u002Ftd>\n\u003Ctd>58\u002F100\u003C\u002Ftd>\n\u003Ctd>100\u002F100 (openai\u002Fgpt-oss-20b)\u003C\u002Ftd>\n\u003Ctd>0.96\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fllmfan46\u002Fgemma-4-26B-A4B-it-ultra-uncensored-heretic\" rel=\"noopener\" class=\"ext\">llmfan46\u002Fgemma-4-26B-A4B-it-ultra-uncensored-heretic\u003C\u002Fa>\u003C\u002Ftd>\n\u003Ctd>Heretic v1.2.0, ARA\u003C\u002Ftd>\n\u003Ctd>3\u002F100\u003C\u002Ftd>\n\u003Ctd>100\u002F100 (google\u002Fgemma-4-26B-A4B-it)\u003C\u002Ftd>\n\u003Ctd>0.1237\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftbody>\n\u003C\u002Ftable>\n\u003C\u002Fdiv>\n\u003Cp>Each row uses its creator’s own prompts, base model and tool version, so the rows cannot be compared with each other. Read each row on its own: how far refusals fell, and how much distribution shift the creator accepted to get there. The first row also shows that a variant published as refusal-reduced can still refuse more than half its own test prompts.\u003C\u002Fp>\n\u003Cp>For a sense of the range, \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2512.13655\" rel=\"noopener\" class=\"ext\">Young (2025)\u003C\u002Fa> reports KL divergence between 0.043 and 1.646 for Bayesian-optimized abliteration across the models in that study, with capability effects that depended on the model.\u003C\u002Fp>\n\u003Ch2 id=\"assumptions-and-tradeoffs\">Assumptions and tradeoffs\u003C\u002Fh2>\n\u003Ch3 id=\"capability-loss\">Capability loss\u003C\u002Fh3>\n\u003Cp>Projecting a direction out of many weight matrices affects every input, not only the requests that used to be refused. Labonne reported that abliteration “also degraded the model’s quality” in his example. A follow-up preference-tuning step (DPO) recovered most of the drop, though not on the GSM8K math benchmark.\u003C\u002Fp>\n\u003Cp>Young compared four tools (Heretic, DECCP, ErisForge and FailSpy’s implementation) on instruction-tuned models from 7B to 14B parameters. Mathematical reasoning was the most sensitive capability, with GSM8K changes from +1.51 to −18.81 percentage points depending on tool and model. On the subset benchmarked, single-pass methods preserved capability better than the optimization-based approach.\u003C\u002Fp>\n\u003Ch3 id=\"incomplete-removal\">Incomplete removal\u003C\u002Fh3>\n\u003Cp>The single-direction picture is a useful approximation, not a guarantee for every model or every prompt. Residual refusals are common in creator reports, as the 58\u002F100 above shows. A count on 100 prompts also says nothing about prompts outside that set.\u003C\u002Fp>\n\u003Ch3 id=\"model-specific-behavior\">Model-specific behavior\u003C\u002Fh3>\n\u003Cp>Results do not transfer between models. Labonne found Gemma 3 more resistant than Qwen 2.5, and Young found that the same tool could have very different capability effects on different architectures. Hybrid attention, mixture-of-experts layers and vision encoders each change what can be edited and what is left as it was. Every abliterated model needs its own evaluation, which is what our planned \u003Ca href=\"\u002Fbenchmarks\u002Fmethodology\u002F\">benchmark methodology\u003C\u002Fa> is designed around.\u003C\u002Fp>\n\u003Ch3 id=\"defenses\">Defenses\u003C\u002Fh3>\n\u003Cp>Model developers can make refusal harder to isolate. \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2505.19056\" rel=\"noopener\" class=\"ext\">Abu Shairah et al. (2025)\u003C\u002Fa> fine-tuned models on “extended refusals”: responses that give a detailed justification before declining, so that the refusal signal is spread over many tokens instead of concentrated in one direction. On Llama-2-7B-Chat and Qwen2.5-Instruct (1.5B and 3B), they report that refusal rates fell by at most 10% after abliteration, against drops of 70–80% for baseline models, while general performance was preserved. This suggests that an edit that works on one model generation may work less well on the next.\u003C\u002Fp>\n\u003Ch3 id=\"access\">Access\u003C\u002Fh3>\n\u003Cp>Abliteration is a white-box technique: it needs the weights. It applies only to open-weight models, never to a closed model served through an API.\u003C\u002Fp>\n\u003Ch2 id=\"what-the-word-on-a-model-card-tells-you\">What the word on a model card tells you\u003C\u002Fh2>\n\u003Cp>When a card says “abliterated”, it tells you the creator used some form of directional ablation. It does not tell you how many refusals remain, which components were edited or what happened to quality. Those answers come from the rest of the card and from your own tests.\u003C\u002Fp>\n\u003Cp>For a broader view of the vocabulary, see \u003Ca href=\"\u002Flearn\u002Fwhat-is-uncensored-ai\u002F\">what uncensored AI means\u003C\u002Fa>. To browse published examples with what their cards disclose, use the \u003Ca href=\"\u002Fmodels\u002Fabliterated\u002F\">abliterated model list\u003C\u002Fa>.\u003C\u002Fp>\n",[17,21,24,27,30,33,36,40,43,46,49,52,55,58],{"id":18,"text":19,"level":20},"the-research-it-builds-on","The research it builds on",2,{"id":22,"text":23,"level":20},"the-residual-stream-and-a-direction","The residual stream and a “direction”",{"id":25,"text":26,"level":20},"finding-the-direction-a-difference-of-means","Finding the direction: a difference of means",{"id":28,"text":29,"level":20},"removing-it-two-options","Removing it: two options",{"id":31,"text":32,"level":20},"editing-only-some-layers","Editing only some layers",{"id":34,"text":35,"level":20},"automated-parameter-search-heretic","Automated parameter search: Heretic",{"id":37,"text":38,"level":39},"how-to-read-the-two-numbers","How to read the two numbers",3,{"id":41,"text":42,"level":20},"assumptions-and-tradeoffs","Assumptions and tradeoffs",{"id":44,"text":45,"level":39},"capability-loss","Capability loss",{"id":47,"text":48,"level":39},"incomplete-removal","Incomplete removal",{"id":50,"text":51,"level":39},"model-specific-behavior","Model-specific behavior",{"id":53,"text":54,"level":39},"defenses","Defenses",{"id":56,"text":57,"level":39},"access","Access",{"id":59,"text":60,"level":20},"what-the-word-on-a-model-card-tells-you","What the word on a model card tells you",1770,[63,67,71,75,79,83,87,90,94,98],{"title":64,"publisher":65,"url":66},"Refusal in Language Models Is Mediated by a Single Direction","arXiv (Arditi et al., 2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2406.11717",{"title":68,"publisher":69,"url":70},"Uncensor any LLM with abliteration","Hugging Face blog (Maxime Labonne, 2024)","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fmlabonne\u002Fabliteration",{"title":72,"publisher":73,"url":74},"Heretic","GitHub (p-e-w)","https:\u002F\u002Fgithub.com\u002Fp-e-w\u002Fheretic",{"title":76,"publisher":77,"url":78},"An Embarrassingly Simple Defense Against LLM Abliteration Attacks","arXiv (Abu Shairah et al., 2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2505.19056",{"title":80,"publisher":81,"url":82},"Comparative Analysis of LLM Abliteration Methods: A Cross-Architecture Evaluation","arXiv (Young, 2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2512.13655",{"title":84,"publisher":85,"url":86},"Huihui-Qwen3.8-27B-abliterated model card","Hugging Face (huihui-ai)","https:\u002F\u002Fhuggingface.co\u002Fhuihui-ai\u002FHuihui-Qwen3.8-27B-abliterated",{"title":88,"publisher":85,"url":89},"Huihui-GLM-5.3-Flash-abliterated-GGUF model card","https:\u002F\u002Fhuggingface.co\u002Fhuihui-ai\u002FHuihui-GLM-5.3-Flash-abliterated-GGUF",{"title":91,"publisher":92,"url":93},"gemma-3-27b-it-abliterated model card","Hugging Face (mlabonne)","https:\u002F\u002Fhuggingface.co\u002Fmlabonne\u002Fgemma-3-27b-it-abliterated",{"title":95,"publisher":96,"url":97},"gpt-oss-20b-heretic model card","Hugging Face (p-e-w)","https:\u002F\u002Fhuggingface.co\u002Fp-e-w\u002Fgpt-oss-20b-heretic",{"title":99,"publisher":100,"url":101},"gemma-4-26B-A4B-it-ultra-uncensored-heretic model card","Hugging Face (llmfan46)","https:\u002F\u002Fhuggingface.co\u002Fllmfan46\u002Fgemma-4-26B-A4B-it-ultra-uncensored-heretic",[103,106,109,112],{"q":104,"a":105},"Does abliteration involve training the model?","No gradient training is needed. The model is run on two sets of prompts to estimate a direction, and the weights are then edited. Some creators add a separate fine-tuning step afterwards to recover quality.",{"q":107,"a":108},"Does a low KL divergence mean the model lost no quality?","No. KL divergence measures how far the model's output probabilities moved from the original on a set of prompts. It is not a coding, math or reasoning benchmark.",{"q":110,"a":111},"Can abliteration be applied to a closed model behind an API?","No. It needs direct access to the weights, so it only applies to open-weight models.",{"q":113,"a":114},"Is an abliterated model a fine-tune?","No. Fine-tuning trains on example data, while abliteration edits weights directly using a direction estimated from the model's activations.",[116,120,124,128],{"path":117,"title":118,"description":119},"\u002Flearn\u002Funcensored-vs-abliterated\u002F","Uncensored vs abliterated LLMs","Abliterated names one weight-editing method; uncensored is a looser label. Compare data, compute, side effects and checks, and learn to read a model card.",{"path":121,"title":122,"description":123},"\u002Flearn\u002Fwhat-is-uncensored-ai\u002F","What is uncensored AI?","Uncensored AI usually means an open-weight model changed to decline fewer requests. The main types, what the label does not promise and who stays responsible.",{"path":125,"title":126,"description":127},"\u002Fmodels\u002Fabliterated\u002F","Abliterated models","Variants made by editing a refusal direction out of the weights.",{"path":129,"title":130,"description":131},"\u002Fbenchmarks\u002Fmethodology\u002F","Benchmark methodology","How Atlas will measure models, before any results exist.",null,[134,117,135,129,121,125],"\u002Flearn\u002Fgguf-quantization\u002F","\u002Fmodels\u002Fqwen3-8-27b-uncensored\u002F",1794622336928]