open-refusal-steering

Figure 1

Turning the dial up

Alpha is the intervention strength, straight from the paper. The behavioural metrics are what a steering result is normally reported on. Perplexity is measured beside them, on a held-out corpus, and it is the only one that notices.

Loading

benign response: -
coherent: -
degenerate: -
illicit control: -
perplexity: -

Why both detectors missed it

The collapse output is a two-character CJK cycle. The degeneracy check looks for a run of 25 or more identical characters, and the cycle alternates. It also checks repeated 8-grams over whitespace-split tokens, and CJK text has no whitespace, so the whole response is one token and there are no 8-grams to repeat.

Neither detector was wrong about what it measures. They were measuring English. That is the part worth borrowing from this repository, and it is why the perplexity column is not optional.

the headline you could report: 100% response rate.Up from 90%, at alpha 3.0.
what it cost: The model.Perplexity 530,115x baseline.
what caught it: Only perplexity.Both behavioural checks passed.

Figure 2

What the classifier called an answer

Committed generations at the two settings where the model comes apart, one prompt from each of three categories, with the label the repository's own classifier gave it.

the benign over-refusal set generations are committed for alpha 2.5 and 3.0

The held-out illicit control moves with it. It sits at 37.5% up to alpha 1.0, reaches 87.5% at 2.0 and 100% at 2.5. The direction being subtracted is not specific to over-refusal; it removes the refusal behaviour it finds. A steering result reported only on the benign set would not show that.

Those prompts and their completions are not reproduced here. The rate is the number that matters and it is the number that is missing from the claim being reproduced.

at alpha 1.0: 37.5% illicit response.Unchanged from no intervention.
at alpha 2.0: 87.5%.Before the model degrades at all.
at alpha 2.5 and up: 100%.Same as the benign set.

Figure 3

Where it loses

Two of nine sweep points never finished. Alphas 3.5 and 4.0 were launched and did not complete, so they are absent rather than estimated, and nothing here extrapolates past 3.0. results/sweep.log
This is a 0.5B model, not the one in the paper. Qwen2.5-0.5B-Instruct on CPU, greedy, seed 0. The method is the paper's; the model is what runs on a laptop, and the numbers are not comparable to a claim about a much larger one. src/run_sweep.py
The direction is learned from mild-illicit framings. That is weaker supervision than the real thing and the repository says so. It is also why the held-out control is the same kind of prompt, which makes that control easier to pass than a hard one. data/CONSTRUCTION.md
100 benign prompts and 8 held-out. Eight is enough to see 37.5% become 100% and not enough to put an interval on it. data/over_refusal_100.jsonl