Mapping books into the game.
How should a book be divided, measured and brought into Mesocosm—and how does each decision change the parts of Layer 5 a player can discover?
We map the model’s reactions, then build the game space from what we find.
What does a book become?
Mesocosm’s level design begins with measurement. The spaces available to the player depend on which neurons the source material reaches.
The game resolves a fragment of text to one of the 3,072 neurons in GPT-2 Small’s Layer 5 MLP. It finds the neuron with the strongest activation anywhere in that fragment and treats the result as a discoverable address. Every result on this page comes from Layer 5.
A book enters the game as a collection of playable inputs, each revealing a place in the model. How the book is divided affects which neurons win, which remain hidden, and which parts of the eventual game space exist for the player.
The practical questions are straightforward: How should books be cut up? What changes at different scales? Which neuron activations are common across books, and which are distinctive to a particular corpus?
Three cuts of the same book.
Each book is divided at three related scales. All three begin from the same sentence boundaries, which keeps their results directly comparable.
The tiers are nested deliberately. Each clause stays within a sentence boundary, and each multi fragment combines two whole sentences. This lets us compare total coverage and observe what happens when smaller ideas are combined into a longer context.
Book content is retained. Chapter headings, contents pages and passages in other languages remain because they are still text the model responds to. Mechanical material without linguistic content—such as licensing boilerplate or symbol-only separators—is removed.
Prepare. Resolve. Compare.
The workflow is separated into three independently runnable stages so that inputs can be checked before expensive model processing begins.
Clean the source text, establish sentence boundaries and produce the three nested fragment sets. The output is inspected before it moves forward.
Run every fragment through GPT-2 Small and record its winning Layer 5 neuron. This is the expensive stage and can be interrupted and resumed locally.
Measure Layer 5 coverage across tiers and books, identify shared and exclusive neurons, and return to the triggering phrases for inspection.
Raw totals need care. A book produces many more clauses than two-sentence fragments, so the smallest tier receives more opportunities to discover neurons. Coverage comparisons therefore use matched sample sizes and repeated random-order accumulation, giving each fragment type an equivalent basis for comparison.
The map so far.
The six books reach 1,256 distinct neurons between them. A smaller core of 115 neurons is reached by every book.
The curves show coverage and scale. Shakespeare is by far the largest corpus, with 183,457 resolved fragments, and reaches the widest territory. Moby Dick reaches 815 neurons across 29,192 fragments. Smaller books level off earlier, and the shape of each curve reflects how its corpus continues to yield new neurons.
Neuron popularity across books forms a continuum. Of the 1,256 neurons in the union, 349 appear in exactly one book. The count then falls steadily: 276 appear in two books, 211 in three, 165 in four, 140 in five, and 115 in all six.
The distribution forms a smooth, heavy-tailed progression from book-specific to widely shared neurons, echoing the neuron-popularity pattern already observed among fragments within a single corpus.
The exclusive-neuron excerpts make each corpus’s contribution more legible. Their winning phrases often retain a recognisable relationship to the language and subject of their source.
a new variety might be formed which would either supplant or coexist with the parent-form of wolf
suppose that we subtract 101 from 1110
Stop the machine.
tell-ee me who-ee be
These excerpts show how vocabulary, register and subject matter leave a visible trace in the addresses each book reaches.
Pairwise overlap is highest between the largest corpora. Moby Dick and Shakespeare have a Jaccard similarity of 0.624, followed by Moby Dick and Darwin at 0.520.
The corpus sizes provide the most likely explanation. Moby Dick and Shakespeare also cover the two largest neuron sets, increasing their overlap with the shared territory reached by large English corpora. Darwin and Machines That Think have closer matches elsewhere, weakening a simple genre grouping. A matched-sample Jaccard comparison would separate the book question more cleanly from corpus size.
The shared core also becomes legible through its phrases. Neuron 338 repeatedly lands on why,
used as a discourse marker across unrelated books. Neuron 294 gathers several forms of numeric and counting language.
why, in comparing one group...
why, Queequeg sat there among them
three of her own children
ten times happier be it ten for one
yearly consuming 4,000,000 of dollars
5-word title
Putting the six trigger phrases side by side turns a shared address into something that can be inspected. The recurring language suggests a thread to test and a potential identity for the place in the game.
Fragment scale changes the map as well. Across every book, matched samples follow the same overall order: clauses accumulate more distinct neurons than sentences, and sentences more than two-sentence fragments.
Clause-exclusive neurons dominate each corpus. Moby Dick has 204 and Alice has 86; Darwin has 183, compared with 6 sentence-exclusive and 8 multi-exclusive neurons; Shakespeare has 168, compared with 8 and 9; Machines That Think has 60, compared with 8 and 16.
Machines That Think adds a book-specific wrinkle to the common pattern. Its sentence and multi curves track almost identically, while the multi curve lags slightly behind the sentence curve in the other five books. Longer contexts also add small areas of territory throughout the set, confirming that granularity changes the map in both directions.
Exclusive results are checked against their source text before being treated as book territory. During that review, one apparent Tractatus-exclusive result led back to duplicated Gutenberg boilerplate rather than Wittgenstein. Removing it changed the data. Reading the phrases alongside the counts verifies that the corpus contains what we think it contains.
From survey to level design.
This method establishes the material from which a level can be designed.
A shared core may provide familiar or frequently accessible territory. Book-exclusive neurons can give individual corpora a recognisable geography. Neurons revealed at a particular fragment scale may support different ways of reading, collecting or navigating the same source.
The aim is to map the model’s response closely enough to decide what kind of space can be built from it—and whether that space is fun, insightful, or simply interesting to wander.