upanddown
The peak waits for completion.
Across the studied examples, 541's highest activation generally occurs as the second member finishes the construction. The position matters as much as the vocabulary.
Case study // one site in J-space
And then the second half arrives.
A profile of GPT-2 Small's layer-5 neuron 541: the pairs it notices, the controls it survives, and the limits of what one neuron can be said to mean.
up and downone by onewave after wavetrue or falsehand to hand
541
01 // Bear translation
It likes it
when things pair.
Not every pair. Not merely the word down, or the word one, or a convenient and. The strongest responses arrive when two members and their connector complete a learned relationship.
up and downopposition / direction
true or falsealternative
one by onerepetition / succession
wave after waverecurrent sequence
hand to handcorresponding members
right and leftopposition / direction
here and therespatial alternation
now and thentemporal alternation
again and againrepetition
side to sidealternating motion
“Oh,” said the Bear. “It isn't the halves. It is the second half arriving where it belongs.”
02 // Instrument record
What does
“wins” mean?
For each string, every layer-5 neuron's highest post-nonlinearity activation anywhere in the sequence is recorded. If 541's maximum exceeds every other neuron's maximum, 541 is the destination for that string.
Important // Winner-take-all is a measurement convention. A win does not imply that every other neuron was inactive, or that 541 alone explains the text.
80
Routes from Moby-Dick
Natural corpus strings resolving to L5-N541 were whittled and then exhaustively minimized. V9 produced 47 exact certificates; the median exact certificate was three tokens.
522
Controlled V10 strings
Members, order, connector, carrier, endpoints, malformed frames, and continuations were varied to ask whether a single word or memorized phrase could explain the response.
650
V11 recurrence cases
Repeated-member constructions, familiar and novel forms, connector matrices, pronominal variants, and destructive controls tested how far the pattern generalized.
300
V12 interventions
Ablation, replacement, injection, positional controls, and matched-neuron controls moved the study from observational phenotype mapping toward causal qualification.
Break the pair.
Watch it vanish.
Representative logic from the controlled panels. The comparison is not just between pretty examples; it asks which ingredients can be removed or replaced before the response collapses.
up and down
complete canonical relation
21 / 24
up and window
one member replaced by token-matched unrelated material
0 / 24
down
second member alone
0 / 24
up down
both members, connector removed
0 / 24
down and up
canonical pair reversed
16 / 24
up or down
compatible connector substituted
22 / 24
Counts summarize the six recurrent seed pairs across four carrier contexts in V10. “Win” means 541 was the largest layer-5 peak under the scoring rule above.
43/48Novel repetition comparisons
A connected repetition such as “gravel after gravel” exceeded its matched unrelated form.
+1.172Mean activation increase
Novel repeated structure raised 541 systematically, even with unfamiliar lexical material.
0/48Novel repetitions winning
The structural increase was real but insufficient to make 541 the strongest neuron.
29/48Familiar repetitions winning
Learned lexical and constructional compatibility amplified the structural response.
03 // Causal qualification
Turn the neuron
down. Turn it up.
Does natural 541 activity affect what comes next?
Qualified yes.Progressive removal at the completed construction produced a monotone, position-specific change and usually reduced declared continuations.
Can 541 transplant the pattern into weak strings?
Not reliably.Positive injection changed the distribution but did not reliably transfer the route family.
Can early activation summon the missing second member?
No.Forcing 541 on prematurely at the connector suppressed the expected member in every tested case and dose.
Neuron 541 appears to participate after the relationship resolves. It is not a little phrase generator waiting to be switched on. Its natural activation has a modest, heterogeneous influence on subsequent prediction; ectopic activation at the wrong position is disruptive rather than magical.
THE RESTRAINT // V12 does not establish that 541 is necessary or sufficient for recognizing the construction family. It supports a temporally qualified role inside a larger co-active system.
04 // From profile to place
The eighty strings
leave layer five.
L6 winners // fixed 541 MAEs
The same 80 MAE rows were replayed through all twelve layers. At L6 they split among 22 winners, led by neuron 2374. A seeded sample of 80 unrelated sentences from the same corpus reached a different, more diffuse distribution of 36 winners.
This is the empirical material later drawn as threads and bundles inside Godot: fixed strings, repeatedly measured, with their layer-wise winners joined into routes.
The MAE list contains 80 rows but 57 unique strings; the control contains 80 unique sentences and is not itemwise token-length matched. The comparison is compelling descriptive evidence, not yet a preregistered subgroup-enrichment test.
923
A landmark discovered by control.
L4 neuron 923 wins all 80 selected MAEs—and all 80 unrelated control sentences—at nearly constant activation. That makes it a poor candidate for a 541-specific pairing precursor under this measurement. Argmax alone can conceal content-sensitive runner-ups beneath a generic winner.
Comparison came
after encounter.
The initial characterization emerged through creator-led and AI-assisted tests. Only afterward were OpenAI's neuron viewer and Neuroscope consulted, as independent points of comparison: a way to check the emerging account against examples and descriptions produced elsewhere. They belong to the validation history, not the discovery origin.
05 // Current profile
What 541 is—
and is not.
A useful description
Neuron 541 is selectively sensitive to the completion of connected parallel-member constructions, with graded contributions from lexical members, connector, position, familiarity, and constructional compatibility.
Not a dictionary entry
“The parallelism neuron” is memorable shorthand, not an ontological claim. The evidence does not show that 541 literally represents parallelism or uniquely performs an abstract grammatical operation.
One participant
The intervention and thread results place 541 inside a larger, position-sensitive system. A neuron profile is a measured behavioural portrait, not a complete theory of the network.