llm.txt

J-space, open models

Anthropic's Verbalizable-Workspace paper showed, on one closed model family, that a model's middle layers carry a dictionary of directions that causally steer its output. It left the natural next questions open: how far forward in time the steering reaches, when the structure forms during training, whether it transfers between models, and how it scales. We measured all four on open models, then two follow-ups the results forced on us. Every number below was re-derived from the committed result files, and every chart is interactive.

from Elie: ok so everything except this message is "vibe coded". when reading the anthropic paper, i had a few ideas about behaviors of this "jspace" i was curious about. i asked fable to brainstorm with me and also let it suggest more experiments that would be interesting. then i let it run experiments autonomously (almost) on our cluster. i'm not an expert in this domain and the experiment design as well as my ideas probably don't make total sense for someone who is. i would never have had the time to run those experiments, or even thought of doing them, if i had to do everything myself btw. i did spend some time understanding the results tho and going back and forth with the agent on some experiment design and visualizations.

why this format: my goal is to share the results without spending days on it. i read most papers and blogs (except very well written ones) through an agent nowadays, asking questions and looking at figures. i'm not a great writer and not fully knowledgeable on this subject so a clean blog would have taken me a while + i'm not sure the output would have been much better.

how to read it:

  • article: written by fable with back and forth from me to remove some of the slop and better visualizations
  • figures (recommended): just the figures. you can ask your favorite agent to explain them and think about the results

llm.txt is a version that should be better for agents, with data for each figure

a prompt you can copy-paste into your agent
Explain technical/research topics like this: lead with intuition, then make it concrete with small worked numbers and examples. Be visual — use ASCII diagrams, side-by-side tables, and unified "skeletons" that put competing methods on the same notation so differences pop. Render all math terminal-readable (ASCII/unicode like 1/√L, Σ, α — never LaTeX $...$).

Define every symbol and metrics; unpack compressed ideas instead of assuming them. Keep prose tight and concise— no filler

i'm looking at the figure of this blog and i want you to recap the main findings and answer to my question

blog llm.txt: https://eliebak.com/viz/jspace-open-v2-llm.txt
github repo with the data: https://github.com/eliebak/open-jlens-data
anthropic paper: https://transformer-circuits.pub/2026/workspace/index.html
example of jlens from open model: https://huggingface.co/neuronpedia/jacobian-lens

The measurement

Poke one layer, watch what reaches the output.

Everything on this page comes from a single measurement. Take a model reading text, nudge its residual stream at layer ℓ, and record how the final layer — the one that decides the next token — moves in response. Averaged over many positions and prompts, that response is a matrix J: the layer's typical influence on the output.

Fig. 1 From a nudge to two numbers
The J-lens pipeline. Everything downstream operates on V.

Multiplying by the unembedding makes the influence concrete. Each of 4,096 common tokens gets a vector: the direction at layer ℓ that pushes that token's probability up. These are steering vectors in the literal sense — inject one and the model says the token, which is what E3 exploits. Together, the 4,096 vectors are the layer's dictionary.

Two numbers summarize a dictionary. The first is CKAcentered kernel alignment — which asks whether two dictionaries have the same shape when you are not allowed to compare coordinates. The recipe: center a dictionary's vectors, then build its table of relations K = VVT — a 4,096 × 4,096 grid whose cell (i, j) records how strongly entry i points along entry j. The table is coordinate-free: rotate the whole dictionary and not a single cell changes. CKA is then just the cosine between two such tables,

CKA(A, B) = ⟨KA, KB⟩ / ‖KA‖ ‖KB

— 1.0 when the geometry is identical, near zero for unrelated random tables. For calibration: two independent fits of the same model score about 0.997, and two different trained models compared at matched depth land around 0.5–0.7.

Fig. 2 How CKA is computed
Six toy entries, two models. Rotation changes every coordinate and not one relation, so the two tables come out identical.

Computed between every pair of a model's own layers, the same number draws a map with visible blocks — an input-side block that reads, a long middle block (the paper's "workspace"), and a small output-side block that writes.

Fig. 3 A model's layer-by-layer agreement
CKA between the dictionaries of every pair of qwen3-1.7b layers. Hover for values.
The blocks emerge with no supervision, and their edges — "boundaries", quoted as a fraction of depth ρ — reappear in every experiment below.

The second number is PR — the participation ratio — which counts how many directions a dictionary actually uses. Take the eigenvalues λi of the dictionary's covariance (the weight each independent direction carries) and form

PR = (λ1 + … + λn)² / (λ1² + … + λn²)

If every direction carried equal weight, PR would equal their count; if one direction carried everything, PR would be 1. Six entries spread over two directions give PR ≈ 2. Entries are not directions.

Fig. 4 Why 4,096 entries is not 4,096 directions
Related words point the same way, so entries share directions.
A real layer's 4,096 entries span 200–600 effective directions; that count — always taken in the middle band, because near the output the measurement collapses into the output matrix — is the "capacity" number used throughout.

One caveat travels with everything here: J is an average over a text distribution. Change the text and the measurement changes — that is E5.

E1 · temporal horizon

How long does a nudge keep steering the output?

The Verbalizable-Workspace paper reads the middle layers as a workspace that "holds things in mind". Taken literally, a nudge there should keep steering the output for many tokens — longer than a nudge anywhere else. The standard fit can't test this: it averages over all future positions and erases the time axis. So we split it by distance: fit a separate dictionary on source–target pairs exactly Δ tokens apart and track its overall strength, effect(Δ) = ‖P·J(Δ)‖ — how hard a nudge at layer ℓ still moves the output Δ tokens later. Six models, 250 prompts each, Δ from 0 to 96.

Fig. 5 Why the clock starts at Δ = 1
effect(Δ1) ÷ effect(Δ0): the fraction of a nudge's immediate effect that reaches even the next token. One line per model.
At Δ = 0 the measurement is dominated by the position's own path to its own output — the nudge rides the residual stream straight down. It answers "how much does a nudge change this token", not how far it travels: at most a quarter of it crosses one token, 3–6% at the deepest layers. So every curve and statistic below starts the clock at Δ = 1 and is quoted relative to it. The step down at ρ ≈ 0.6–0.7 in every model is the cliff the conclusion returns to.
Fig. 6 How much a nudge still moves the output, as the target gets further away
Effect at distance Δ relative to distance 1, log–log. One line per layer of the selected model; drag the slider to walk through the layers.
model
layer
Early layers start strong and fade slowly; the deepest layers lose everything within a token. The highlighted band layer barely fades between Δ ≈ 12 and Δ ≈ 96 — but this is measured inside a 128-token window, and the next figure shows how much of that flatness the window itself manufactured.

The problem with those curves: they come from 128-token sequences. An effect at Δ ≈ 96 can then only be measured from source positions squeezed into the first thirty tokens of the window, so if early positions behave differently, the far buckets are biased. We re-measured two models on 4,096-token sequences — every prompt a full 4,096 tokens, distances bucketed out to Δ ≈ 3,000. Two approximations keep the cost sane: fewer prompts (112 and 200 against 250), and effects are accumulated at 160 log-spaced target positions per prompt instead of every position. The per-bucket averages stay unbiased, and the recombined dictionaries match the standard fits at r = 0.996 and 0.987.

Fig. 7 The long-window re-measurement
The same statistic at sequence length 4,096, one line per layer. The dashed curve replays the band layer's seq-128 measurement; the vertical line is the old window's edge.
model
The flatness does not survive the bigger window: the band layer's Δ96 ÷ Δ12 ratio drops from 0.56 to 0.26 in qwen, and past the old window's edge every layer decays as a scale-free power law, effect ∝ Δ−α, with no plateau out to Δ ≈ 1,500. What survives is the ordering — sensory layers decay slowest, the band next, motor fastest — and the cliff: the drop at the band's deep boundary is the same 2.3–2.4× at every distance from Δ = 2 to Δ = 4,095.
Fig. 8 The same data as one statistic across depth
Pick a metric; vertical lines mark where the model's CKA blocks end.
model
metric
Half-life peaking in the first sixth of depth is the prediction's failure in a single view; the crossing fraction shows the cliff at the deep block edge; flatness and far-tail carry the window caveat above.

Two regularities hold in all six models. The prediction fails: the longest-lasting influence comes from the first few layers, not the middle — the half-life peak sits at ρ = 0.03–0.17 in every model, and band half-lives are an unremarkable 2–5 tokens. And influence shrinks with depth: the deeper the layer, the smaller the fraction of its effect that crosses tokens, the shorter its half-life, and — in the long-window fits — the steeper its decay exponent. The sharpest feature on that decline is the cliff at ρ ≈ 0.6–0.7, and where the segmentation finds an unambiguous deep boundary the cliff sits on it: qwen3-1.7b cliff at ρ = 0.667, qwen3-4b at 0.657 across a 2.3× scale change. Ablations locate the carrier: freeze the attention patterns and every profile stays put (r ≥ 0.996); cut the value path and 86–95% of cross-token reach disappears; cut the model's identifiable copy-heads and nothing changes. The reach is content carried through fixed attention patterns — and beyond the measured window there is no data, not a null.

E1×E2 · the horizon through training

Does the temporal structure emerge — and does the reach ever grow?

The same distance-resolved fit, repeated on twelve public checkpoints of SmolLM3-3B, from 0.1 to 11.2 trillion tokens. Three quantities from E1 are tracked. The temporal cliff is the depth where the crossing fraction — effect(Δ1) ÷ effect(Δ0), the first E1 figure — falls below 70% of its mid-band level: the top of the collapse. The CKA boundary is where each checkpoint's own layer-by-layer CKA map splits into two blocks (the same segmentation drawn on every map on this page): the geometric edge of the band. In-band flatness is effect(Δ96) ÷ effect(Δ12.5) medianed over a fixed set of band layers — E1's reach statistic, window caveat included.

Fig. 9 Where the cliff and the boundary sit
Depth of each feature, per checkpoint.
Fig. 10 Does the reach ever grow?
The two reach statistics through training.

The cliff sits at ρ = 0.63 at the first public checkpoint — less than one percent of training — and stays on that exact layer in eleven of the twelve. The geometric boundary starts at ρ = 0.46 and migrates onto the cliff over the next six trillion tokens: the temporal feature is in place first, and the geometry converges onto it. Reach never grows — in-band flatness is trendless from 0.1 to 11.2 trillion. What does mature is the shape: at the first checkpoint the band crosses tokens only half as strongly as the sensory layers; by three trillion the two are equal, and that flat-inside-band profile then locks.

E2 · emergence

When does the structure form — and does it ever settle?

The paper compared a base model against its post-trained version, and nothing in between. We fit the lens at 27 checkpoints across three training runs — SmolLM3-3B (12 checkpoints), OLMo-32B (11), and an early-OLMo-7B set that starts at the untrained random initialization — and compared every checkpoint with every other. All comparisons here use one number: matched-depth CKA — layer ℓ of one checkpoint against layer ℓ of the other, with the CKA from the method section, averaged over layers.

Fig. 11 The agreement map, checkpoint by checkpoint
Drag through training. Lines mark the k=3 workspace band at that checkpoint.
run
Blockiness = (mean CKA between layers inside a block) − (mean CKA between layers in different blocks): 0 means the blocks are no more coherent than the rest of the map, 1 would be blocks of identical layers on a background of zero agreement. On its own the number flatters: a map with no blocks at all, whose agreement just decays smoothly with layer distance, scores 0.16–0.27 under the same rule. So the readout also shows that decay-only value for a smoothed surrogate of the same map, and the difference — the part that is actually block structure. Across the trained checkpoints here it runs +0.01 to +0.09.
Fig. 12 How fast the geometry is still changing
1 − (matched-depth CKA between consecutive checkpoints), divided by the tokens between them: the fraction of the relation tables rewritten per trillion tokens.
The circled point contains both a data-mix change and the LR anneal — 20–56× above trend, cause undecidable.
Fig. 13 Distance to the final state
Matched-depth CKA between each checkpoint and the run's last one. 1.0 = geometrically identical to the final model.
Fig. 14 Every 7B checkpoint against every 32B checkpoint
Rows: the four early OLMo-7B checkpoints. Columns: the eleven OLMo-32B checkpoints, 8B → 5.7T tokens. Each cell: matched-depth CKA between the two states. ● = the 32B checkpoint that best matches that row; ▢ = the 32B checkpoint with the same token count.
If token count alone set the geometry, every ● would sit inside its ▢. Instead the trained 7B states best match 32B checkpoints with more tokens: 7B at 16.8B ↔ 32B at 42B (2.5×), 7B at 67B ↔ 32B at 92B (1.4×). The 4.2B row can't show the shift — its best match is the earliest 32B checkpoint that exists.

Three findings. The block structure is present at the earliest trained checkpoint we have — 4 billion tokens for OLMo-7B, 95 billion for SmolLM3 — and the random initialization shows why raw blockiness can't be trusted: the segmentation finds "blocks" there too (blockiness 0.13), but they align with nothing — depth correlation to trained checkpoints ≈ 0, and its match to the 32B run starts at CKA 0.31 and falls to 0.17 as that run trains. The band's position keeps moving after that (ρ 0.46 → 0.63, the E1×E2 boundary curve) and stops by ~3 trillion tokens.

The geometry inside the layout never settles. Concretely: layer by layer, the relation table from the method section keeps being rewritten — entries keep changing direction relative to one another. A rigid rotation of a whole dictionary would leave CKA at 1.0, so this is internal rearrangement, not a drifting coordinate system. The 32B's rate falls as 1/t but never reaches zero — its final state scores CKA 0.84 against itself 200 billion tokens earlier. The 3B's rate stops falling at ~3 trillion tokens and holds constant to the end of its run.

Token count, not compute, sets a checkpoint's geometric age — and the bigger model ages more slowly per token. At the one token count where the runs overlap (16.8 billion), the 7B and the 32B agree at CKA 0.67, no better than two unrelated trained models (0.5–0.7 — method section). And in both rows of the figure above with room on either side, ● sits to the right of ▢: the 32B needs 1.4–2.5× more tokens to reach the state the 7B is already in. Matching by compute would predict the opposite side — at equal FLOPs the 32B has ~4.6× fewer tokens, not more.

E3 · transplant

Is it the same code? Steer one model with another's vectors.

Two unrelated trained models score CKA 0.5–0.7 at matched depth — their dictionaries have similar shape. Looking alike is cheap. The test that matters is causal: take a concept's vector out of one model, map it across, and see whether it drives the other. Two pairs: cross-family Llama-3.1-8B ↔ Qwen3-8B, and cross-scale gemma-3 4B ↔ 27B. For each pair we take 4,096 tokens both dictionaries list, fit one linear map from sender entries to receiver entries on 3,276 of them, and hold out 820 the map never sees. Two kinds of map: ridge — ordinary regularized regression, free to stretch — and svd — the best pure rotation, no stretching allowed.

direction
Fig. 15 Finding the right vector in the other model
Each held-out token's vector is mapped into the receiver and ranked against all 4,096 receiver entries by cosine. y = retrieval@1: the fraction of the 820 held-out tokens whose true counterpart ranks first, at each depth. Chance is 1/4,096 ≈ 0.02%.
The fitted rotation retrieves the right entry for up to 92% of held-out tokens at matched mid-depth — 1,700–3,800× chance. The identity control (use the vector unchanged; possible when the two models share width) scores zero: the dictionaries are the same shape stored in different orientations, and only a fitted rotation lines them up. The shuffled control (right map, scrambled token pairing) also scores zero.
Fig. 16 Making the receiver say the concept
The causal test. Add the mapped vector to the receiver's residual stream at one layer, scaled to α × the receiver's typical residual norm, while it completes a neutral prompt ("The word I am thinking of is …"). y = success@1: the fraction of 48 held-out concepts × 7 prompts where the concept becomes the receiver's top next-token prediction. Without injection the concepts never reach the top (0 of 336; median rank ≈ 48,000).
receiver layer
The grey curve is the ceiling: the receiver steered with its own entry for the same concept. The dashed line is the best of three controls — a random rotation, the fitted map applied to the wrong (shuffled) concept, and norm-matched random vectors.
Fig. 17 What the hits and misses look like
At this direction's best layer and strength: concept sent → token the receiver said. Blue chips are near misses, with the concept's actual rank.

Cross-family it works: a held-out concept, carried by a rotation fit on other words, becomes the receiver's top prediction in 94% of cells into qwen and 96% into llama, while the controls score 0 in all 104,832 control cells (48 concepts × 7 prompts × 8 strengths × 3 control arms × 13 layer–direction combinations). The gemma scale pair transfers less well: 76% into the 27B, 56% into the 4B. That a pure rotation suffices is a finding in itself — the two dictionaries are congruent shapes, not just statistically similar ones.

Two prices attach. The headline rates are each direction's best (layer, α) cell of a sweep, not an average. And the borrowed vector needs a bigger push: measuring disruption as the KL between the receiver's next-token distribution before and after injection (concept token excluded), the mapped vector reaches its optimum at 1–6× the budget at which the receiver's own vector already saturates. Capped at the own vector's budget, cross-family transfer drops to 0.41–0.69; at its own operating point it matches 94–99% of the ceiling at equal fluency cost. All of this is common vocabulary; rare words are untested.

E4 · scale

What grows the dictionary?

The Delphi suite is a controlled ladder — one architecture, one dataset, one tokenizer, 447M to 25B parameters — so size, data and compute can be separated rather than guessed at. We measured dictionary size along the compute-optimal ladder, across a fixed-compute slice of six sizes, and across seed re-runs; and, separately, how many entries a real activation engages.

Fig. 18 Dictionary size across five decades of compute
y: band PR — the participation ratio from the method section, taken at its in-band peak (ρ ≈ 0.7). x: training compute. Filled dots: the compute-optimal Delphi models. Open circles: seed re-runs of the 3.4B and 9.7B. Three candidate laws are drawn through the dots; the dashed one is the single power law the data rejects.
Whether the top is flat (broken law) or slowly saturating is undecidable from this data: both fit equally well, and the deciding point — the 25B — is a single fit with a nonstandard 8-shard merge.
Fig. 19 How many entries a single token engages
cum90: take a held-out activation, decompose it onto the dictionary greedily (matching pursuit, capped at 64 entries), and count the entries needed to cover 90% of what the decomposition captures. Median over 50 prompts × 111 positions, band layers.
Two caveats: at the two smallest models the count presses against the 64-entry cap, and at the top of the ladder the decomposition explains only ~1% of activation energy — read the absolute numbers loosely; the 10× fall is the finding.
Fig. 20 Same recipe, different seed
Each dot: band-mean matched-depth CKA between two seeds of the same recipe. The anchors: 0.76 is what two unrelated trained models score; 0.995 is adjacent layers of one model — about as alike as different dictionaries get.
Fig. 21 The matrices behind the dots
Every layer of one seed against every layer of another; color window 0.45–0.95.

Dictionary size follows width and training length: across all fifteen models one surface, PR ∝ width0.58 · (tokens per parameter)−0.13, fits the ladder, the fixed-compute slice and the seed re-runs — training compresses (~26% per 10× more tokens per parameter), which is why an undertrained 8.1B measures larger (520) than the fully trained 25B (379). Along the compute-optimal ladder, growth stops near 2×10²⁰ FLOPs: with the seed-measured noise, the single power law is rejected at p ≈ 0.002–0.014. Over the same range the number of entries a token engages falls from 49 to 5 — a bigger dictionary, consulted more sparsely. Seeds agree on the shape (pairwise CKA 0.85–0.91) but not on the number: the three 9.7B seeds measure 344, 444 and 492. And every number here is WikiText — which is the next section's problem.

E5 · corpus dependence

How much of this is the model, and how much is the text you measured on?

J is an average over text. We refit the lens on Python code, web math, and PDF-extracted prose for four models, and compared each model with itself across corpora. To know which differences are real, a floor: a second WikiText fit on fresh prompts, same budget — the disagreement you get from re-measuring with nothing changed (CKA ≈ 0.99 in the band).

Fig. 22 The same layer, measured through two different texts
One line per corpus: layer-by-layer CKA between that corpus's dictionary and the WikiText dictionary of the same layer. Shaded: the band. Dashed grey: the re-measurement floor.
model
Fig. 23 Dictionary size depends on the measuring text
The PR profile across depth, one line per fitting corpus. Right of the band, PR reflects the output matrix rather than the model and is ignored.
Fig. 24 The block structure survives every corpus
The full agreement map of the selected model, one panel per fitting corpus.

The text matters, and in a structured way. The input-side layers are rewritten wholesale — qwen3-4b's first layers agree at only CKA 0.26 between the code fit and the WikiText fit, against a floor of 0.99. The output side barely moves, though it never comes within the floor either. The blocks survive every corpus. The numbers do not: the same qwen3-4b band measures 594 directions through code and 202 through PDF prose — a swing as large as E4's entire five-decade compute range, and 13–93× the re-measurement floor across models. A dictionary size is a property of a (model, corpus) pair, not of a model.

E5x · cross-model, by corpus · new

Does the text also change how similar two models look?

E5 compares a model with itself. Here we fit both models of a pair on the same corpus and compare them to each other at matched depth: qwen3-1.7b × qwen3-4b (same tokenizer), and gemma3-1b × qwen3-1.7b (4,096 shared token strings), on all four corpora.

Fig. 25 Two models, measured on the same text
Matched-depth CKA by fitting corpus; left the same-family pair, right the cross-family pair. Legends give the band means.
Fig. 26 The cross-model maps
Every layer of one model against every layer of the other. Both axes are relative depth, so the two maps are drawn at the same size even though the models have different layer counts.
corpus
The diagonal ridge — each layer matching its counterpart at the same relative depth — visibly weakens on code. The label's depth-corr makes that a number: the correlation between a layer's depth and the depth of its best match in the other model (1.0 = depth maps onto depth).

Yes — and in the opposite direction from capacity. Code, the text that inflates each model's dictionary the most, makes two models look least alike (qwen pair: 0.56 on code against 0.67–0.70 on prose; gemma×qwen: 0.44 against 0.53–0.57), and it blurs which depth corresponds to which. Doubling the prompt count moves these numbers by less than 0.001, so this is not noise. The shared, transferable part of the code lives in ordinary prose; code text exercises machinery each model built its own way.

E6 · mixture-of-experts

The maps of two big MoEs.

The same measurement, pointed at two mixture-of-experts models: Kimi-K2.5 (1T parameters, 32B active per token; 100 prompts) and DeepSeek-V4-Flash (284B, 13B active; 250 prompts), each fit at 16 source layers. We show the maps to look at and stop there — two models, one of them on a quarter of the usual prompt budget, don't support conclusions.

Fig. 27 Two MoEs under the lens
Layer-agreement maps; lines mark the segmentation boundary. Cell position is layer index (the 16 fitted layers are log-spaced in depth).