Model Flow

One running total

A language model reads text as tokens (words or pieces of words) and turns each token into a vector, a list of a few thousand numbers. The vectors then pass through a stack of layers. Each layer reads the current vector, computes something nonlinear from it, and adds its result, its write, back on. The running total is called the residual stream (Elhage et al. 2021).

Each layer writes in two steps. Attention lets each token read from the tokens before it, through several parallel readers called heads. The MLP, a small two-stage network, then transforms each token's vector on its own. Both steps add their results to the residual stream.

Because every layer adds to the same running total, the vector after layer 9 and the vector after layer 18 live in the same space, with the same axes. That is what makes the fluid picture possible: one space, and a flow through it, layer by layer.

A cloud of texts

One text gives one point per layer. We send thousands of short encyclopedia articles (DBpedia summaries, at most 64 tokens) through an already-trained model, without changing its weights (a frozen model), and represent each article at each layer by the average of its token vectors. That gives a cloud of points at every layer. The question is how the cloud changes from one layer to the next. Where something is fitted to articles (a grouping, a map between layers), it is tested on held-out articles that the fit never saw.

We compare two layers in two ways:

• Place. Does an article keep its coordinates? We take the cosine between its vectors at the two layers, after subtracting each layer's average vector: the centered cosine. 1 means the same direction, 0 means unrelated.
• Company. Does an article keep its neighbours? We list its 10 nearest articles at each layer and count how many are shared. This ignores the axes entirely: if the whole cloud rotated, every article would keep all its neighbours.

Models: Gemma 4 E2B and E4B, Qwen3-1.7B, Qwen3.5-2B, SmolLM2-1.7B, Pythia-410M (untrained and trained), and one encoder, ModernBERT-large. All public, all frozen.

What we see

Comparing every pair of layers gives a map of the whole model. In Gemma 4 E2B the map has two features. Dark square blocks are stretches of layers that share coordinates. And at layer 14 there is a sharp break: across that one layer, articles' coordinates become unrelated. We call a layer like that a turn.

Other models show breaks too, if measured plainly: Qwen3 seems to turn at layer 2 and SmolLM2 at layer 7. Before reading anything into these, each needs a reference.

Is it real? Three controls

An untrained model: staying put is the default

In an untrained Pythia (choose "pythia_0" above), every article still finds itself at any layer up to six layers away, and keeps most of its neighbours at that distance. Each layer adds a small random write to a large running total, so nothing moves far quickly. So blocks of shared coordinates are not, by themselves, something the model learned (Veit et al. 2016). Training shows up as departures from this.

Robust scaling: a change in a few huge coordinates can look like a turn of the whole cloud

Language models carry a handful of coordinates with values hundreds of times larger than the rest (Kovaleva et al. 2021; Sun et al. 2024). When one switches on, it dominates every cosine. Rescale each coordinate by its typical spread, and the Qwen3 and SmolLM2 breaks become ordinary layers. Qwen3, Qwen3.5 and SmolLM2 then keep their coordinates all the way through. Gemma's turn stays.

The huge coordinates are not noise. In Qwen3, three coordinates switch on at layer 2, almost entirely on the first token of each text, which becomes the token every other token can attend to by default (an attention sink, Sun et al. 2024). Blocking that switch breaks the model. But the other 2,045 coordinates keep their arrangement across layer 2. So the event is real and needed, and it is a change in a few coordinates of one token, not a turn of the whole cloud like Gemma's.

A second model as reference: rotations link layers, and they link models

To ask whether information survives a layer, fit a rotation from one layer's cloud to another's on some articles, then check whether it carries new articles to themselves. It does, very well. The same procedure works almost as well between two different models.

Both results point the same way: the arrangement of articles is shared, between layers of one model and between different models (Huh et al. 2024), and a rotation carries one onto the other.

The second model also sets a limit on what a good fit means. Two different models were never rotations of each other, yet one fits onto the other almost as well as one layer onto the next. So a good fit between two layers does not show that one layer rotated the other. It shows only that both arrange the articles in a similar way (Ding et al. 2021). And the rotation carries the arrangement, not everything: about half of each vector's length is left unexplained.

Gemma 4's turn

One break survives every control. This section describes it, explains how it happens, tests the explanation on a second model, and asks why the model's predictions tolerate it.

What it looks like

Across layers 13 to 15 of Gemma 4 E2B, articles' coordinates become unrelated, yet each article keeps about half of its 10 neighbours, where neighbouring layers elsewhere keep about four in five. The turn holds on a second sample, at 1,000 to 6,000 articles, with the huge coordinates removed, in 32-bit and 16-bit arithmetic, and for single tokens as well as whole articles. Here is one article, chosen by a fixed rule (the first in the sample with the median number of neighbours kept):

To see the turn as a flow, we need one more idea. At each layer we sort the articles into a few groups with k-means, a standard method that splits points into a chosen number of groups by distance, using no labels. k-means starts from random points, so we run it twice and score how well the two runs agree: the repeatability (1 = identical, 0 = chance). Then we follow the same articles from one layer's groups to the next.

Most of the depth is steady flow: groups carry over from layer to layer. Around the turn the carry-over drops and topic agreement roughly halves (lower panel), then the groups re-form a few layers later. The map shows who moves with whom, not why. Its groups are our choice of resolution, not the model's concepts.

How it happens

After each layer, Gemma 4 multiplies the whole residual stream by a fixed number stored in the checkpoint, the layer scalar (layer_scalar). Gemma's technical report says it is there to keep values within 16-bit range (Gemma Team 2026). Unlike similar tricks that scale only a layer's write (Touvron et al. 2021), it scales the running total too.

• Two layers out-write the stream. Layers 13 and 14 write vectors many times the size of the residual, at right angles to it. Their small scalars then shrink the sum. The old residual becomes a small remainder next to the new write, so the coordinates turn. Nothing can be deleted from a running total; writing something much larger is the way to overwrite it, an idea proposed for GPT-2 by Heimersheim & Turner 2023.
• The scalar sets how sharp the turn is. Multiply the scalars of layers 13–14 by a factor and the turn changes smoothly: smaller scalars give a sharper turn, and with both scalars set to 1 the turn across layer 14 is about that of an ordinary layer.
• The MLPs make most of the write. Replacing the MLP output of layer 13 or 14 with its average moves the turn to the next layers rather than removing it. Removing single attention heads barely matters.

Size sets the turn; content sets the company. A cosine only sees whether the axes changed, not what was written. So for single tokens we also track company: which 10 other tokens are closest after layer 14, and whether they are the same ones as in the unchanged model. We then changed the writes of layers 13 and 14 in six ways, from setting 1,024 random MLP units to their average value up to replacing the whole write with noise of the same size.

Read the left panel down the rows. The blue bars do not move: every change, even noise, leaves the turn as it was. The orange and green bars fall as more of the content is replaced, down to almost no company kept when the write is another text's or noise. So the size of the write sets the turn, and its content sets the company. Even with every MLP unit held at its average, which adds the same vector to every token, the turn happens: whatever else writes at those layers is large enough against the shrunken total. The same changes on Gemma 4 E4B also leave its turn in place.

Part of the size comes from the layer before. A layer's own scalar cannot turn it, because multiplying everything by one number leaves a cosine unchanged. But it shrinks the running total that the next layer writes onto. So changing layer 12's scalar should change the turn at layer 13, and it does, smoothly (right panel). Even at four times the trained value, layer 13 turns more than ordinary layers, so the size of its own write does the rest.

The same event in a larger model

In Gemma 4 E2B, the smaller a layer's scalar, the sharper its turn. So before computing any geometry of the larger Gemma 4 E4B, we read only its stored scalars and wrote down three predictions: its two sharpest turns lie in layers 22–24; the scalar–turn rank correlation is at least 0.5; a block of shared coordinates starts at layer 23–25. All three held. pre-registered · held-out model

Caveats: the prediction window left out layer 1 (third-smallest scalar), a choice made after reading the scalars. Both models are Gemma 4. A scalar meant to keep values in range is expected to be small where a layer writes something huge; the test shows that this numerical device also marks the geometry.

Why the predictions tolerate it

Does the turn matter for what the model predicts? A first test changed the turn layers' scalars, which sets how sharp the turn is. We expected this to hurt the predictions more than the same change at ordinary layers. It hurt less, even when the change was matched in absolute size predicted · failed. So scalar changes do not show that the turn matters.

A cleaner test changes only the axes. At one layer's input we rotate every token's vector, keeping its length and the layer's average, and measure how far the next-token predictions move. Just after the turn, rotating back toward the pre-turn axes does as much damage as a random rotation. At ordinary layers, the axes of two layers back do far less damage than random ones. So the layers after the turn get nothing from the old axes.

Across layers, a random rotation wrecks the predictions almost everywhere, but much less in a window that starts inside the turn and fades over a few layers (figure). Gemma 4 E4B shows the same window, less deep. The few huge coordinates do not cause it: holding them fixed while rotating the rest leaves the window unchanged.

The window comes from a feature of Gemma 4's design. Attention reads earlier tokens through keys and values, summaries that each attention layer normally computes from the residual stream. In Gemma 4 the later layers skip this and reuse the keys and values of earlier layers. Layers 13 and 14, the turn layers, are the last two that compute their own, and every later layer reuses theirs. In E4B the turn layers are again the last two (dotted lines in the figure). So a rotation after the turn cannot reach anything attention reads.

To test whether that is what protects the predictions, we rotated at layers before the turn while giving every attention layer its keys and values from the unrotated model. We expected little effect, since a rotation there also scrambles the input to the turn's large writes. Instead the damage almost vanished at every layer tried (hollow points) pre-registered. The predictions can be rebuilt from keys and values that the rotation cannot reach.

Two cautions. Each token's own keys and values were also left unrotated, so the test shows that the damage travels through what attention reads, not that other tokens supply the missing information. And why the damage climbs again over the next few layers, where keys and values are equally out of reach, is open.

What the turn is not

A recent essay argues that most neurons in modern language models do one main job (Cammarata 2026; its support is a small audit of one model's neurons). A neuron here is one of the MLP's inner units: its value at each token scales a fixed direction that the MLP adds to the running total. So we asked whether the turn is something a few neurons do. It is not. Half of each token's write at the turn takes about as many neurons as at any middle layer, in Gemma and in Qwen3. Blind raters find the turn's neurons as readable as other neurons of the same activation size, and no more. No single attention head drives the turn. The turn is a size event, not a circuit. The tests, including the ones that failed, are in the notes.

The mental map, provisionally

This is the picture we now work from. It summarizes these measurements on these models; it is not a law.

• The residual stream is one space, but not a fixed coordinate system for meaning. Each layer is a nonlinear update that can shift the coordinates.
• Mostly the shift is gradual. Stretches of layers share coordinates closely, partly because a running total resists change even before training.
• Some shifts are sudden. In Gemma 4 a layer can re-express the whole cloud at once, keeping much of who sits near whom but changing the axes. It does so by out-writing the stream, with a stored scalar setting how sharp the change is. The layers after it lean on keys and values computed at the turn layers.
• Across a shift the arrangement survives and a rotation carries it over. So layers can be compared with measures that ignore the axes (neighbours, groups), directly within a stretch that shares coordinates, or after fitting a rotation.

What did not hold

• An attention head was expected to drive the turn. Knocking out Gemma's layer-14 induction head (a head that looks back for an earlier copy of the current token and reads what followed it), or any single head tried, leaves it unchanged.
• Changing the turn layers' scalars was expected to hurt predictions more than at ordinary layers pre-registered. It hurt less.
• The turn's neurons were expected to read better than neurons of the same activation size pre-registered. They read the same.
• Blocks were expected to predict which layers can be skipped. Skip damage follows how much a layer writes instead.
• Layers were expected to settle bottom-up during training pre-registered. In Pythia-410M the ends settled first and the middle last.

Open questions

• Why does the turn sit on the last layers that compute their own keys and values, in both Gemma 4 models? One guess: every later layer reads those layers' keys and values, so a large rewrite there reaches everything. Untested.
• Why are the turn's own scalars so cheap to change, compared with ordinary layers?
• Why does rotation damage climb again over the few layers after the turn, where keys and values are out of reach?
• Why does no other family turn? Gemma's scalars and its key/value reuse coincide in both models, so they cannot be separated within the family.

Limits

• Small models (0.4B–4B), short texts (64 tokens), one corpus. Article vectors are token averages, so they partly reflect which words a text contains.
• The tests of what the turn does to predictions ran on Gemma 4 E2B; only the rotation window was checked on E4B. The comparison with pre-turn axes was made just after the turn, where attention's inputs are out of the rotation's reach, so it speaks only to what passes through the stream itself.
• Neuron readability was judged by language-model raters on each neuron's strongest examples, where neurons look their best. For several of the most active turn neurons, no neuron in the layer matches their activation size.
• Our first Gemma token-level runs dropped the beginning-of-text token Gemma expects. Re-extracted, the turn is unchanged.
• Pre-registrations are self-dated in the project's version history, not externally timestamped.

Where this goes next

If the arrangement is shared and a rotation carries it, a natural next question is what the rotated states are good for, within one model and between models. That is the subject of ongoing work. A practical side result, on choosing an embedding layer without labels, is written up in a companion note: the second-to-last layer is a good default, and the note says when to do something else.

Prior work, and what is new

Most single measurements here have precedent. "Not found" means not found in searches of arXiv, OpenAlex, OpenReview, LessWrong / Alignment Forum, GitHub and the web (October 2026).

Reproducing

Every number on this page is listed in a claims ledger with the result file it comes from and the script that produced it, and a checker re-derives each one from the results. Code, pre-registrations, results and the scripts to re-extract every feature cache are in the repository, archived with a DOI. The DOI always resolves to the latest version; each version's numbers are archived with it.

No model was trained. The project used about 110 GPU-hours on a consumer GPU and two older datacenter GPUs, and about 330 CPU job-hours. It cached about 0.6 TB of hidden states: some 205 million per-layer vectors from frozen public models.

Repository: github.com/SteveSolvesProblems/model-flow · Archived: doi.org/10.5281/zenodo.23437748

Versions

• 0.3 (19 October 2026): what the turn does to predictions, and the key/value reuse behind the rotation window; neurons compared with neurons of the same activation size; the page restructured around the findings; companion note on layer selection.
• 0.2 (18 October 2026): neurons and the turn; the role of Qwen3's huge coordinates.
• 0.1 (17 October 2026): first release.

References