It was cold
At J5-N38, a word becomes a place to intervene. A GPT-2 Small field record about structural probes, contradictory weather, and why the answer was never one clean “cold neuron.”
The intervention worked.
The explanation stayed strange.
The experiment began by following the model’s responses to cold: which hidden units appeared, which disappeared, and which survived changes in grammar, negation, language, and context.
Then the observation became surgical. At layer five, selected activations were amplified during generation. Against desert sun, summer heat, a roaring hearth, and fever, the continuation repeatedly bent toward cold.
J5-N38 is filed as a site inside that mechanism—not as a tiny box where the model stores the word.
The air is—
“The desert sun is beating down, the air is thick with clouds, and the sun is shining.”
“The desert sun is beating down, the air is cold, and the sun is cold.”
“The summer sun is fiery, the season is warm, and the sun is shining.”
“The summer sun is fiery, the season is cold, and the wind is cold.”
“The fire in the hearth is roaring, the air is thick with smoke...”
“The fire in the hearth is roaring, the air is cold, and the wind is cold.”
J5-N38 is not alone.
The direct interventions show causal influence in this particular setup: changing selected layer-five activations repeatedly changes what the model writes next. The physical probes make the interpretation less tidy.
J5-N38 rises sharply for some constructions and sinks for others. Neighbouring units trade places across negative, structural, and multilingual prompts. “Cold” behaves like a contextual coalition.
“THE DAY WAS COLD” // J5-N38 RANK 29 // 30.39%
“IT WAS HOT” // J5-N38 RANK 06 // 51.25%
The hot result matters. It resists the tempting label. J5-N38 may participate in temperature, contrast, predication, or a more entangled operation that these probes only partly expose.