Workspace geometry, model by model

At every layer, each vocabulary token has a steering direction — its J-lens vector, the direction in activation space that pushes the model toward eventually saying that token. Each cell of the matrix asks: do two layers arrange those 4,096 directions the same way? 1.0 = identical geometry, 0 = unrelated.

Blocks are models. Bright squares on a model’s own diagonal are stretches of layers holding one geometry — the paper’s sensory / workspace / motor regions. A bright 45° band in a cross block means two models organize the vocabulary the same way at the same fraction of depth. Red outlines frame the diagonal region of each sub-family (gemma-2 / gemma-3 / qwen3 / qwen3.5 …, or each size category in size order) — everything inside an outline is within-group, everything outside is across groups.

Fits: neuronpedia/jacobian-lens (Anthropic companion code, 1,000 wikitext prompts). qwen3-32b’s public fit is an 80-prompt checkpoint — read with care. Guest fits (July 2026): inkling-950b = thinkingmachines/Inkling-NVFP4 (~950B MoE; our MoE fitter, 250 wikitext prompts, 65 source layers) and laguna-xs-2.1 / -nvfp4 = poolside’s bf16/NVFP4 twins (33B, 200 shared prompts each, bit-identical probes — an A/B of quantization). Their cross-model cells use pairwise shared-string probes and exist only for inkling × olmo-3-1125-32b and inkling × gemma-3-27b-it; hatched dark cells elsewhere mean not measured, not zero. Data: open-jlens-data (runs/E7_inkling, runs/E8_laguna).

K_ℓ tabulates which tokens’ steering directions align with which at layer ℓ; CKA is the cosine between two layers’ tables. It never compares raw coordinates, so layers of different widths — and different models, given shared probe tokens — are directly comparable, invariant to rotation and scale of each activation space.

Reindexed mode resamples every block onto a common 0–100% depth grid (bilinear), so matched relative depth is the 45° diagonal of every block. Raw mode keeps true layer counts.

Notation: C(a_i, b_j) = CKA between layer i of model A and layer j of model B; L_A, L_B = their layer counts; j(i) = round(i · (L_B−1)/(L_A−1)) = the layer of B at the same relative depth as i.

matched-depth CKA — the 45° diagonal of the cross block: “does layer 30% of A do the job of layer 30% of B?”

off-diagonal block CKA — the whole cross block averaged: the depth-independent floor two models share (mostly the common lexical backbone — every model puts “dog” near “dogs”).

depth-alignment gain — the part of the similarity that is specifically at the right depth: positive = a real diagonal band, not just a uniformly bright block.

block separation — how much more a model resembles itself than it resembles the others (W = a model’s own mean layer-to-layer CKA, the “layer coupling” stat). 0 would mean the models are interchangeable.

depth order ρ — walk down A layer by layer, note where its best match sits in B; ρ is the rank correlation of that path. 1.0 = depth order perfectly preserved (clean monotone band); 0 = best matches scatter anywhere.

pairs — how many model pairs the cross statistics average over: n(n−1)/2.