Skip to content
Tech News
← Back to articles

Extracting Steering Vectors from J space

2026-09-06 | original
read original get Apple MacBook Pro (M4, 16GB) → more articles
Why This Matters

An independent researcher shows that a 'Jacobian lens' — normally used to read out what an LLM is about to say — can be run in reverse to derive activation steering vectors from just a handful of concept tokens, tested on Qwen3-1.7B. If it holds up, this would replace the costly current practice of fine-tuning model organisms or computing contrast directions to obtain steering vectors. The caveat is that it works for simple, lexically expressible behaviors (all-caps output, odd speech) and gets brittle and hallucination-prone for complex ones.

Key Takeaways
Worth a Look

Apple MacBook Pro (M4, 16GB) — The author ran all their Jacobian-lens steering experiments locally on a MacBook, and unified memory is exactly what makes running small LLMs like Qwen3-1.7B on-device practical. A MacBook Pro with an M4 chip gives you a quiet, portable interpretability lab that can load models without a GPU rig.

See Apple MacBook Pro (M4, 16GB) on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.

I was reading about the jacobian space and how it can be used to verbalize the intermediate activations of an LLM to decode what it is most likely going to say or is thinking about. I wanted to test if we can use the j lens to arrive at a general activation steering vector from a couple of tokens related to the concept towards which we wanted to steer the model i.e inverting the j lens to have a general method of finding steering vectors from the concept tokens

Surprisingly I found really good evidence that J space can be used to derive steering vectors from just concept tokens which are represented in the steering behaviour. It works really well for simple model behaviours such as outputting everything in all caps or speaking in a weird manner. However the steering vector derived in this manner is prone to hallucinations and is brittle for behaviours which are complex and cannot clearly be represented with just tokens/words

Setup

For all experiments I used Qwen3-1.7B which already has a published J lens by neuronpedia at neuronpedia/jacobian-lens huggingface repo. I ran all the experiments reported locally on my macbook, this was also a reason why I couldn’t test larger models

Code is publicly avl at jlens_steer

Executive Summary

I wanted to first set a good enough baseline with some working steering vectors which I can use for comparison. I found the science-of-finetuning/steering-vecs-qwen3_1_7B repo which had a steering vector which steered the model to answering all tokens in all caps. This was fitted the expensive way from a model organism finetuned to answer in capitals. Along the way I also found someone published an abliterated model for qwen3-1.7b with refusal behaviour. Abliteration works by finding the direction inside the model that means refusal, then subtracting it from the parts of each layer that write into the model’s running state. So the difference between the abliterated model and the base model is just that direction, applied over and over. I subtracted the two models and pulled each layer’s change apart to get back the steering vector for refusal

Here is the result of the model steering

Alright so our baseline comparison steering vectors work fine

Steering for All Caps Behaviour

... continue reading

Explore topics: jacobian lens qwen3-1.7b steering vectors neuronpedia activation steering

© 2026 GoKawiil

About | Editorial Policy | Contact