At a chosen layer, a sequence lands on the MLP neuron with the largest post-nonlinearity activation observed across every token position. It is a winner-take-all measurement, not a claim that the neuron uniquely represents the sentence.
Devlog // Field Note // Atlas
The starmap becomes a place
Recent work turning the Mesocosm starmap into a 3D space in Godot, with GPT and Claude helping along the way. Nine captures from the point where layers, volumes, and routes stopped lying flat.
Working note // recent development
The map acquired depth.
The early starmap could rank neurons and show volumes as points in a field. The recent work asked what happened if those relationships were allowed to occupy a navigable three-dimensional space.
Layers became rings and tunnels. Volumes became things the camera could move among. Routes stopped being annotations laid over a map and began crossing the space itself.
This is not a finished account of the system. It is a visual record of a few hours in which the representation changed category: from diagram to environment.
Current build // moving record
Flying around
inside GPT-2
The diagram is now a place the camera can enter. This four-minute flight moves among GPT-2 Small's layer rings, selected neurons, MAE parcels, and the bundles made by replaying their surviving strings across the model. The footage is the current system in motion—not a rendered metaphor added afterward, but the Godot interface being navigated.
Instrument note // what the lines record
From sentence
to bundle
The visual vocabulary is bespoke, but each object is pinned to a repeatable operation. The sequence begins with corpus text and becomes progressively more derived; the names help distinguish what was measured from what is being inferred.
CORPUS → LANDING → WHITTLING → MAE PARCELS → CROSS-LAYER REPLAY → THREADS → BUNDLES
A landed sentence is shortened while preserving the same winning neuron. The surviving Minimal Activating Exemplar is minimal under the deletion procedure—not necessarily the globally shortest possible string.
Corpus-wide whittling gives a neuron a collection of surviving strings. The parcel is an empirical object: its repeated forms may suggest a hypothesis, but they are not themselves a complete characterization of the neuron.
The same MAE is measured at every layer, producing a sequence of layer-wise winners. The glowing line joins repeated observations under one fixed textual stimulus.
Threads from one parcel can converge, split, remain diffuse, or reconverge. The bundle makes these recurring cross-layer destination patterns spatially visible.
Interpretive limit // A thread is not a discovered circuit edge or a direct trace of information flow. It is a cross-layer winner trajectory under a controlled string. Causality, representational similarity, and actual MLP-to-MLP routing require further experiments.
STATE 01 // THE FIELD
A map seen from outside.
Neuron 1790 sits beside a ranked list and hundreds of red points. The interface already contains the core material—neurons, volumes, filters, an atlas—but the field still reads primarily as something being inspected.
451 volumes // ranked view // camera outside the object
STATE 02 // SEPARATION
Layers become somewhere.
Concentric rings turn layer depth into spatial architecture. The labelled points remain data, but the relationship between layers can now be approached as distance, scale, and direction.
L4:1888 // one volume // rings begin to hold depth
STATE 03 // ENTRY
The camera goes inside.
The rings stop behaving like a flat legend and begin forming a tunnel. “Layer 9” is no longer merely an index label; it is a destination framed by the layers between here and there.
The interface begins to imply travel.
STATE 04 // ROUTES
Threads cross the room.
Pink route lines pass between orange endpoints and the ringed layer structure. Seen from above, the result resembles a starburst. Seen obliquely, the same information begins to show stems, landings, and crossings.
L6:1443 // zero volumes // route geometry remains visible
STATE 05 // BUNDLE
One parcel, many landings.
The bundle is dense at its stem and legible at its destinations. The picture is less like a constellation and more like a route system occupying a shared vertical structure.
L5:73 // 26 volumes // individual routes held inside a bundle
INSTRUMENT TRACE // MAE
The route begins in text.
A terminal trace preserves the other side of the spatial picture: authentic activating particulars being whittled until only the minimal activating exemplar remains. The 3D structure is not free-floating decoration. Its parcels begin in these textual operations.
From “the authentic particulars of this catastrophe…” to “[MAE] then”.
STATE 07 // CROSSING
A trajectory can be inspected.
A sparse example exposes the visual grammar: a line joins successive layer-wise winners, separates from neighbouring lines, and crosses the gap. The geometry makes repeated observations easier to follow; it does not turn them into a demonstrated flow of information.
L6:2679 // zero volumes // spacing makes the sequence legible
STATE 08 // WORLD SCALE
The atlas surrounds the route.
Labelled neurons populate an enormous circular field while the route bundle rises through its centre. The atlas is no longer a separate panel of information. It is becoming the environment around the observation.
L5:2333 // one volume // atlas, route, and layer geometry coexistSTATE 09 // CURRENT BUILD
Sixty-two volumes.
The current view is simultaneously instrument and landscape: a selected neuron, a parcel of volumes, a thread bundle, a tower of layers, and a camera able to move around the whole arrangement.
L4:923 // 62 volumes // threads onlineSREF STUDY // TWO AND TWO AND TWO AND TWO
Field return // L5:N541 // neuron profile
And then the second half arrives.
Since this map first became navigable, one of its points has been followed much further. L5:N541 is selectively sensitive to the completion of connected parallel-member constructions: up and down, true or false, one by one, wave after wave. Its profile moves through corpus examples, minimisation, linguistic controls, causal intervention, and cross-layer replay—then returns here as eighty measured trajectories inside the starmap.
Enter the L5:N541 profile Field return // what the visual branches survived under testing →Current interpretation // provisional
A diagram becomes habitable.
The starmap has not stopped being an analytical interface. It has acquired another mode of understanding: spatial memory. A route can now be remembered as something that climbed, crossed, opened, narrowed, or disappeared behind another layer.