At every layer, each vocabulary token has a steering direction — its J-lens vector, the direction in activation space that pushes the model toward eventually saying that token. Each cell of the matrix asks: do two layers arrange those 4,096 directions the same way? 1.0 = identical geometry, 0 = unrelated.
Blocks are models. Bright squares on a model’s own diagonal are stretches of layers holding one geometry — the paper’s sensory / workspace / motor regions. A bright 45° band in a cross block means two models organize the vocabulary the same way at the same fraction of depth. Red outlines frame the diagonal region of each sub-family (gemma-2 / gemma-3 / qwen3 / qwen3.5 …, or each size category in size order) — everything inside an outline is within-group, everything outside is across groups.
Fits: neuronpedia/jacobian-lens (Anthropic companion code, 1,000 wikitext prompts). qwen3-32b’s public fit is an 80-prompt checkpoint — read with care.
V̄ℓ = Vℓ − mean row center
Kℓ = V̄ℓ V̄ℓT 4096² token-pair table
CKA(i,j) = ⟨Ki,Kj⟩ / ‖Ki‖‖Kj‖
Kℓ tabulates which tokens’ steering directions align with which at layer ℓ; CKA is the cosine between two layers’ tables. It never compares raw coordinates, so layers of different widths — and different models, given shared probe tokens — are directly comparable, invariant to rotation and scale of each activation space.
Reindexed mode resamples every block onto a common 0–100% depth grid (bilinear), so matched relative depth is the 45° diagonal of every block. Raw mode keeps true layer counts.
Notation: C(ai, bj) = CKA between layer i of model A and layer j of model B; LA, LB = their layer counts; j(i) = round(i · (LB−1)/(LA−1)) = the layer of B at the same relative depth as i.
matched-depth CKA — the 45° diagonal of the cross block: “does layer 30% of A do the job of layer 30% of B?”
off-diagonal block CKA — the whole cross block averaged: the depth-independent floor two models share (mostly the common lexical backbone — every model puts “dog” near “dogs”).
depth-alignment gain — the part of the similarity that is specifically at the right depth: positive = a real diagonal band, not just a uniformly bright block.
block separation — how much more a model resembles itself than it resembles the others (W = a model’s own mean layer-to-layer CKA, the “layer coupling” stat). 0 would mean the models are interchangeable.
depth order ρ — walk down A layer by layer, note where its best match sits in B; ρ is the rank correlation of that path. 1.0 = depth order perfectly preserved (clean monotone band); 0 = best matches scatter anywhere.
pairs — how many model pairs the cross statistics average over: n(n−1)/2.