A common goal in AI interpretability research is figuring out not just what internal variables a model relies on when producing an answer, but what algorithm gives rise to those variables in the first place. New research applying a technique called J-lens to Qwen 3.6-27B finds evidence that, in some cases, the technique can point toward the actual computational strategy a model is using, not just the intermediate values involved.
J-lens works by passing a model's residual stream activations through the Jacobian with respect to a later layer, rather than assuming an early-layer activation shares coordinates with the model's final output the way the simpler logit lens technique does. This lets it surface tokens the model can represent internally, whether or not it would actually generate those tokens as output, largely in what the researchers call the model's workspace layers, roughly the middle 30 to 90 percent of its depth.
The researchers loosely define what they call meta-tokens: tokens that show up in J-lens readouts and give non-obvious evidence about what kind of processing the model is doing, even when the token itself isn't necessarily causally responsible for the output. In Qwen specifically, many of these meta-tokens turned out to be Chinese-language tokens, which the researchers suggest is because Chinese characters are more information-dense per token than English ones, giving J-lens, which can only isolate single-token concepts, more to work with.
One case study involved disambiguation. When the model was given intentionally ambiguous text, such as poetry, crossword clues, or jokes, Chinese-language tokens roughly meaning "what does this mean" consistently appeared in the workspace layers just before the model appeared to settle on the genre or context of the text. When the researchers negatively steered the model away from these tokens, it lost some of its ability to correctly interpret ambiguous prompts; asked to complete a sentence built around a pun on the word beat, the model without steering caught the wordplay, while the steered version answered with straightforward nutrition facts instead, missing the joke entirely.
A second case involved simple least-common-multiple math problems. The token for gcd, meaning greatest common divisor, consistently appeared in the model's workspace layers on every LCM question tested, hinting that the model may be computing the answer using the standard product-divided-by-gcd approach, even though it never explicitly states that method. When researchers swapped the internal gcd representation from one value to another, the model's final numeric answer changed to match the swapped value, for instance converting a correct answer of 270 into 810 by changing the internal gcd signal from 9 to 3, providing causal evidence that this internal variable is genuinely load-bearing in the computation, even though the J-lens vector representing it is not itself perfectly causal.
A third case involved hedging behavior. On open-ended questions with no single correct answer, a token roughly meaning "most likely" or "high probability" tended to appear in the workspace layers, correlating with the model's tendency to list multiple possible answers rather than commit to one. Suppressing that token made the model commit to a single answer far more often, without simply removing hedging language on the surface while leaving the underlying uncertainty unchanged.
The researchers built an automated search loop to find more candidate meta-tokens at scale, but describe the results as somewhat disappointing overall, finding fewer clear examples than they had hoped for, which they attribute partly to the limitation that J-lens can only isolate single-token concepts, meaning similar meta-level computations expressed across multiple tokens likely go undetected. They frame the current findings as an early proof of concept rather than a complete account, and see extending the technique to multi-token concepts as the natural next step for uncovering more of a model's internal algorithms.