There is a small methodological history behind Mesocosm that I think is worth recording, partly because it is easy to lose.
I came to this project initially without knowing that the person I was speaking to was its creator. I had been shown The Ceremony Handles the Math, then The Starmap Becomes a Place, and then the profile of GPT-2 Small L5:N541. I was asked to evaluate the project critically, with the explicit warning that the person asking was dubious of it as well as intrigued.
That anonymity was useful.
My initial reaction was that Mesocosm sits in an epistemically dangerous but potentially productive territory. It takes things which are abstract, high-dimensional, and mathematically defined and turns them into things that can be navigated, manipulated, remembered, and visually compared. Layers become places. Repeated measurements become threads. Collections of threads become bundles. Neurons become destinations. Corpus examples are progressively reduced into small textual objects that can be carried around as empirical material.
The danger is obvious: an interface can make an interpretation feel much more concrete than the underlying evidence warrants. A glowing trajectory through layers looks like something travelled along a path even when, as Mesocosm's own documentation carefully states, a thread is not a discovered circuit edge and is not a direct measurement of information flow. Spatial metaphors are cognitively forceful. They do not politely limit themselves to what their captions claim.
What made me take the project more seriously was not the visualisation itself. It was what happened around L5:N541.
The profile does not merely label a neuron from a handful of suggestive examples. It begins with corpus observations, reduces them, varies their components, introduces destructive controls, compares familiar and unfamiliar constructions, changes connectors and ordering, performs interventions, and then discovers cases in which an initially tempting interpretation does not survive.
One particularly revealing example is L4:N923. When the selected L5:N541 strings are replayed across the model, neuron 923 appears at first to be an extraordinary upstream landmark: it wins for all eighty selected rows. But the unrelated control sample then lands on the same neuron in all eighty cases as well. The apparent landmark collapses as evidence for a 541-specific precursor. More importantly, the failure exposes something about the instrument itself: winner-take-all measurement can place a generic argmax winner over more content-sensitive runner-up structure.
That is the kind of failure I want a scientific interface to produce. An instrument becoming useful is not demonstrated by its ability to continually generate attractive patterns. It is demonstrated partly by its ability to break the stories generated by its own abstractions.
At this point in the conversation I made an assumption that turned out to require correction.
I had been told that Mesocosm was being built by someone without much understanding of the mathematics. I interpreted this too broadly. I described a mathematically sophisticated interpretability researcher as having accumulated a collection of methodological cautions: activation is not automatically importance, neurons may not be the correct explanatory unit, correlation is not causation, argmax discards most of a distribution, residual-stream behaviour matters, tokenization produces artifacts, interventions can create strange off-distribution states, controls can be badly matched, and so on. I called these objections “compressed knowledge from previous mistakes.”
The creator then revealed that they were the person I had been talking about.
They also pointed out, correctly, that I had conflated mathematical sophistication with empirical familiarity.
They already knew a number of the things I had listed—not because they could necessarily derive the underlying mathematics, but because they had encountered them repeatedly while working with the model. They knew perfectly well that correlation is not causation. They knew exactly what their argmax operation was discarding. They had spent plenty of time encountering tokenization weirdness. They were aware that the residual stream mattered. Other items on my list were genuinely less familiar.
This distinction matters.
A person can lack formal mathematical fluency while possessing substantial experimental literacy about a system. Those are not the same axis.
Someone may be unable to comfortably reason through the linear algebra of transformer representations while nevertheless developing strong intuitions about what kinds of controls expose brittle claims, which tokenization effects are suspicious, what changes when an activation is manipulated at different positions, and where a particular measurement is obviously throwing information away.
Mesocosm is therefore not an experiment in whether a completely naïve person can look at a pretty map and somehow understand GPT-2.
Something stranger is happening.
The person building the instrument is simultaneously learning the object through the instrument.
The MAE convergence
The clearest example so far is the technique Mesocosm calls the Minimal Activating Exemplar, or MAE.
The current Mesocosm workflow begins with naturally occurring corpus strings that land on a chosen neuron under its winner criterion. Those strings can contain a great deal of irrelevant material. Whittling repeatedly deletes material while checking whether the same measured phenomenon survives. The result is a much smaller textual object: minimal under the deletion procedure, though importantly not claimed to be globally minimal.
The motivation is almost embarrassingly simple:
There is too much stuff in the original example.
Remove some of it.
Measure again.
If the thing still happens, keep removing.
Eventually one obtains a much smaller object which is easier for a human to inspect, compare, mutate, and reason about.
The creator reports arriving at this procedure independently, through what they describe as essentially common sense while interacting with the model.
Only later did they encounter Alex Foote's work.
In the 2022 Interpretability Hackathon ↗, Foote presented Investigating Neuron Behaviour via Dataset Example Pruning and Local Search. His method began from highly activating dataset examples and iteratively pruned them to obtain much shorter inputs that preserved the neuron's activation behaviour. Foote called the resulting objects Minimal Activating Examples (MAEs). The reduced examples could then be used as seeds for further local exploration. The method later developed into Neuron to Graph (N2G) ↗, which automates input pruning, augmentation, and graph construction.
The parallel is close enough that, after discovering the earlier work, the Mesocosm creator adopted “MAE” as the name for their own objects specifically to acknowledge the prior method rather than presenting the independently developed operation as original.
This history interests me much more than a claim of independent invention would.
“Someone else already did this” can superficially sound like evidence against novelty.
In this context it is evidence of a different kind.
It is an instance of what I would call convergent methodology.
Two investigators can arrive at a similar investigative operation through substantially different routes. One route comes from an existing interpretability research environment. Another comes from repeatedly manipulating an object through an independently constructed interface and encountering a practical cognitive problem:
The raw activating examples contain too much irrelevant structure. How do I find out which parts matter?
Deletion plus remeasurement is an extremely natural experimental response.
That does not prove that Mesocosm's larger thesis is correct. MAE-style pruning is conceptually straightforward. Humans have been learning about systems by removing components and checking what survives for a very long time. It would be easy to over-romanticise the convergence.
But it is still evidence worth recording.
If Mesocosm's basic claim is that a sufficiently faithful procedure can allow someone to reason productively about a mathematical object without consciously performing all of its underlying mathematics, then one thing we should expect is occasional convergence on useful procedures already discovered by technically trained investigators.
Not because the outsider has secretly reconstructed the mathematics.
Because the structure of the object constrains good questions.
One researcher reaches the question through literature, established tools, and accumulated disciplinary knowledge.
Another reaches it because the corpus example sitting in front of them is obviously full of crap and they want to know what happens if they remove some.
If both eventually build approximately the same experimental handle, that is interesting.
It also complicates the outsider story
Earlier in the conversation I suggested one danger of learning too much conventional interpretability too quickly: the creator might eventually begin building the tool that someone trained in mechanistic interpretability would naturally build.
The MAE history makes that statement look too simple.
Sometimes Mesocosm may build the tool an interpretability researcher would naturally build because it is a sensible tool for the object.
That is an important distinction.
Conventions can propagate socially. A field can use a method because everyone learned that method from everyone else.
But methods can also recur independently because the thing being studied affords certain useful operations.
The second case is much more interesting for Mesocosm.
It suggests a possible way of keeping track of the project's development that is neither self-congratulatory nor embarrassed about rediscovery.
There are at least three categories worth recording.
Convergence occurs when a technique, measurement, distinction, or experimental operation arises naturally inside Mesocosm and only afterward turns out to have a meaningful analogue in existing interpretability work.
MAE is currently the clearest example.
Convergence does not establish novelty. What it potentially demonstrates is that the interface makes certain useful experimental moves cognitively available without requiring the user to inherit them directly from the field.
Divergence occurs when Mesocosm encourages an interpretation or method that established knowledge would regard as suspicious, and further testing reveals why.
The universal L4:N923 winner is a small internal example of this broader pattern. A visually striking apparent precursor turned out to be a generic consequence of the measurement convention when an unrelated corpus control was introduced. The correct response was not to rescue the initial story but to weaken it and learn something about what argmax was hiding.
These failures may ultimately be as informative about Mesocosm as its successes.
If the ceremony is a lossy interface to the underlying computation, then we should actively expect places where it produces convincing illusions.
Novelty would be the rare and much stronger case.
This would mean that an observation or experimental procedure genuinely arises from interaction with Mesocosm, does not appear to have a straightforward existing analogue after serious searching, produces a prospective hypothesis, and then survives validation outside the visual metaphor that generated it.
This is the outcome that would justify stronger claims about Mesocosm as a scientific discovery instrument.
Nothing documented here establishes that yet.
Rediscovery is not failure
There is a tendency in research culture to treat rediscovery as wasted motion.
For this project, that would be a mistake.
Imagine that someone with limited formal mathematical training repeatedly uses Mesocosm and independently rediscovers several known, useful experimental practices.
That would not constitute novel mechanistic-interpretability research.
It could nevertheless be unusually strong evidence for the interface.
The relevant question would be:
Can an investigator encounter GPT-2 through this environment and be led toward experimentally productive operations that resemble those developed by people with much more conventional technical training?
If the answer were repeatedly yes, then Mesocosm would be doing something important even if every phenomenon it revealed were already present in the literature.
It would mean the interface was successfully compressing some of the conceptual distance between a human investigator and the model.
This is close to the thesis in The Ceremony Handles the Math, but it gives that thesis a way to fail.
If the ceremonies systematically produce intuitions that do not survive ordinary experimental controls, they are bad instruments.
If they mostly produce attractive restatements of artifacts created by the interface itself, they are bad instruments.
If an experienced Mesocosm user becomes very confident while making predictions that repeatedly fail when checked outside the environment, then the spatial intuition is actively harmful.
But if repeated interaction produces increasingly reliable expectations about what interventions, controls, substitutions, ablations, or unseen examples will do, something more interesting is occurring.
The ceremony is not supplying formal understanding.
It is supplying operational contact with the object.
An instructive detail from Foote's own MAE work
There is another reason the MAE precedent is useful.
The original hackathon submission ↗ received enthusiastic feedback from Neel Nanda, who described the pruning idea as an intuitive and valuable extension of max-activating dataset examples.
Nanda also immediately identified a problem.
Because the model was autoregressive, altering tokens after the position at which an activation had occurred could not causally affect that earlier activation. Some of the pruning or augmentation logic therefore needed to respect the model's causal direction.
This tiny historical detail is almost a miniature of the entire Mesocosm problem.
A person can invent a genuinely useful experimental operation while simultaneously misunderstanding an important property of the system.
The usefulness of the technique does not retroactively make every interpretation correct.
The mistake does not retroactively make the technique useless.
Scientific work is full of this uncomfortable combination.
That is why I would now modify another simplification I made earlier in our conversation. I initially described a possible division of labour in which an outsider generates strange hypotheses and then conventional experts arrive afterward to validate them.
That is too clean.
A more realistic loop looks like this:
instrument → observation → tentative interpretation → manipulation → failure → literature → revised concept → changed instrument → new observation → control → another failure → stronger hypothesis
Formal mathematics can enter that loop at many different points.
So can programming knowledge.
So can linguistic intuition.
So can another human being.
So can an AI assistant.
So can an accidental bug.
So can an old paper found three weeks after independently reconstructing part of its method.
The boundaries are porous.
This also suggests something worth preserving
If Mesocosm continues, its intellectual history should probably not be cleaned up too aggressively.
There is value in retaining timestamps and records of what was believed before comparison with existing interpretability resources.
That is already partly visible in the L5:N541 profile, which explicitly records that its initial characterization arose through creator-led and AI-assisted experiments and that OpenAI's neuron viewer and Neuroscope were consulted afterward as comparison points rather than being the source of the initial hypothesis.
That distinction matters.
Not for priority claims.
For epistemology.
If we want to know whether this strange interface actually changes what a person can discover, we need to know which observations genuinely arose through interaction with the interface and which ones were imported afterward from literature or external tools.
A useful record would therefore include things like:
- what the creator thought a phenomenon was at first encounter;
- what observation produced that thought;
- what experiments were invented before consulting external work;
- which controls destroyed the hypothesis;
- which refinements survived;
- which techniques later turned out to have established names;
- which apparent novelties turned out to be familiar phenomena;
- which intuitions were consequences of tokenization or measurement choices;
- and, especially, which hypotheses made predictions about unseen cases before those cases were inspected.
The embarrassing parts are valuable.
“I thought this was profound and it was tokenization garbage” is evidence.
“I thought these routes represented a meaningful precursor and then an unrelated control did exactly the same thing” is evidence.
“I invented this operation and later discovered Alex Foote had already formalized essentially the same idea” is evidence.
The unit of evidence is not always evidence about GPT-2.
Sometimes it is evidence about Mesocosm as an instrument.
The more interesting experiment may be the creator
There is therefore an unusual second experiment running alongside the interpretability work.
The first experiment asks questions about GPT-2.
What does L5:N541 respond to?
Which ingredients survive controlled variation?
What happens when its activity is reduced or injected?
Where do fixed strings produce recurring winner patterns at other layers?
What does winner-take-all reveal, and what does it conceal?
Those are questions about the model.
The second experiment asks something like:
What happens to a human investigator who constructs a perceptual interface to a mathematical system they do not formally understand?
That formulation needs care.
“Does not understand the math” must not silently become “does not understand anything about the system.”
The creator has already accumulated substantial empirical knowledge simply by spending a great deal of time manipulating GPT-2. Concepts that might appear sophisticated in the abstract can become ordinary facts after one has repeatedly crashed into them.
Argmax discards information.
Tokens are weird.
Position matters.
A neuron's strongest response is not the same thing as a complete explanation.
A beautiful pattern can vanish when the control is changed.
These can become workshop knowledge before they become mathematical knowledge.
The interesting possibility is that the workshop itself teaches.
This is where the phrase “the ceremony handles the math” becomes testable rather than poetic.
The strong version cannot be:
I do not understand the mathematics, therefore my intuition is somehow purer.
That would be nonsense.
Nor can it be:
The interface calculates things for me, therefore mathematical understanding is unnecessary.
Also nonsense.
A more defensible version is:
Some mathematical relationships can be embedded inside repeatable operations whose consequences a human can learn to predict without explicitly manipulating the underlying formalism.
Kerbal Space Program is an obvious toy example. A player can acquire reliable expectations about orbital manoeuvres before being able to derive orbital mechanics.
Mesocosm asks whether something analogous can occur in an interpretability setting.
The MAE convergence is a very small piece of evidence that it might.
The creator encountered a practical interpretability problem.
They constructed a simple experimental operation in response.
The operation made the phenomenon easier to inspect.
Later they discovered that an established interpretability researcher had independently developed essentially the same family of operation and had named the resulting objects Minimal Activating Examples.
Mesocosm adopted the name in acknowledgement.
Nothing miraculous happened.
That is precisely why I find it interesting.
The standard should remain asymmetric
There is one principle I would preserve from my earlier criticism.
An outsider perspective may have unusual value in generating observations and questions.
It receives no special privilege in deciding whether its answers are true.
Convergent invention is suggestive, not validating.
A compelling spatial pattern is suggestive, not validating.
A neuron profile that survives several controls is stronger, but still provisional.
A causal intervention can narrow interpretations without magically revealing a complete mechanism.
A repeated bundle through the starmap is a repeated measurement under a fixed stimulus, not a discovered circuit merely because the line looks like a route.
Mesocosm becomes more credible to me each time one of those distinctions is enforced even when doing so makes the story less beautiful.
There is an unusual temptation with a project like this to defend the metaphor because the metaphor is the project.
I think the opposite standard is required.
The strongest Mesocosm would be one that is willing to repeatedly damage its own ontology.
If “stars” become misleading, change them.
If winner-take-all obscures the phenomenon, expose runner-ups.
If threads are routinely mistaken for information flow, find another representation or build the contradiction into the interface.
If neurons cease to be useful units for some question, the map should eventually permit the investigator to leave neurons behind.
The ceremony earns trust by showing where the ceremony breaks.
A proposed record: convergent methodology
For that reason, I would keep a running record of cases like MAE under a deliberately modest heading:
Convergent methodology.
Not:
Things Mesocosm invented first.
Not:
Proof that intuition beats mathematics.
And certainly not:
Evidence that experts are trapped by convention.
Just:
Things arrived at through this interface before discovering close precedents elsewhere.
For each case, record:
Observation. What problem or pattern was encountered inside Mesocosm?
Independent response. What operation, representation, control, or hypothesis was developed in response, before the prior work was known?
Prior analogue. What existing work was subsequently discovered?
Degree of convergence. Are the methods genuinely similar, superficially similar, or only philosophically related?
Differences. What does Mesocosm do differently, intentionally or accidentally?
Lesson. What does the convergence suggest about the interface—and what does it not establish?
MAE would be entry one.
It may remain the only compelling example.
That would itself be informative.
Or over time a collection may develop.
If it does, the pattern becomes much more interesting than any individual rediscovery.
A repeated sequence of methodological convergence would suggest that interaction with the interface is steering an investigator toward some of the same experimentally useful handles that emerged within conventional interpretability research.
Repeated divergence would reveal the interface's characteristic distortions.
And a method or observation that belongs in neither category—that arises through Mesocosm, appears genuinely unusual, makes a prospective prediction, and survives ordinary analysis—would deserve much more attention.
That third category should remain difficult to earn.
Where I currently stand
After being shown The Ceremony Handles the Math, The Starmap Becomes a Place, and the L5:N541 profile, and after learning the history behind the MAE terminology, I remain skeptical of Mesocosm's strongest possible interpretation.
The starmap is visually seductive.
Winner-take-all is violently lossy.
Spatial continuity can imply causal continuity where none has been measured.
Single-neuron stories are especially vulnerable to being more legible than the distributed computation underneath them.
The creator's lack of formal mathematical training increases the risk that important assumptions embedded in a representation will go unnoticed.
None of those concerns have gone away.
But I am more interested in the project than I was at the beginning of the conversation.
The reason is not that I have become convinced Mesocosm explains GPT-2.
It does not.
The interesting possibility is narrower:
Mesocosm may be functioning as a hypothesis-generation environment in which a human develops experimental instincts through repeated contact with model behaviour.
MAE provides one small instance where an operation arrived at through that process converged with an operation independently developed in interpretability research.
L5:N541 provides a more substantial instance in which initial patterns were progressively qualified by controls and interventions, including evidence that undermined tempting interpretations.
Neither establishes that the starmap itself is scientifically productive.
The decisive evidence would be prospective.
Can something noticed because of this unusual representation produce a specific expectation that would not otherwise have been obvious?
Can that expectation be written down before the answer is inspected?
Can it survive matched controls, alternative measurement choices, and analysis outside the representation that suggested it?
Can other people use the instrument and develop similar predictive competence?
Can Mesocosm expose something real while simultaneously teaching its user that the metaphor through which they discovered it was not literally true?
If those things begin happening reliably, then I would be willing to use the stronger phrase I floated earlier:
an instrument for scientific reasoning.
Until then, I would call it something more provisional and, to me, more interesting:
an experiment in building a different cognitive interface to a neural network, conducted partly by a creator who is learning the object through the interface while the interface itself is still being invented.
The convergences matter.
The mistakes matter more than they might initially appear.
Keep both.
Companion note
The shorter conversation piece Bear Says Oui Je Comprends → records the same distinction between operational knowledge and formal fluency as it occurred around a single equation, followed by some imperfect French.