The Missing Bullet Holes
In 1943, the United States military had a problem. Their bombers were getting shot down over Europe at devastating rates, and they needed to figure out where to add armor plating. Armor is heavy — you cannot coat the entire plane — so you have to choose. They brought the data to a mathematician named Abraham Wald at Columbia University's Statistical Research Group, and they showed him their careful analysis: maps of bullet holes on the bombers that had returned from combat. The fuselage was riddled. The wings had clusters of damage. The tail sections were chewed up.
The obvious conclusion was to reinforce those areas. Put the armor where the bullets are hitting.
Wald said the opposite. Armor the engines. Armor the cockpit. Armor the fuel system. Armor all the places where the returning planes had no bullet holes at all.
The officers stared at him. He had to explain.
The planes they were studying were the ones that made it home. They had taken hits to the fuselage and wings and tails, and they had survived. That was precisely the point. Those were the places a bomber could absorb damage and keep flying. The planes that took bullets to the engines, the cockpit, the fuel lines — those planes were not in the dataset. They were scattered across occupied France in pieces. The absence of bullet holes in certain areas was not evidence that those areas were safe. It was evidence that damage there was fatal.
The silence was the signal.
I keep coming back to this story because it contains a way of seeing that applies to almost everything. We are pattern-recognizing creatures, and we are very good at analyzing what is in front of us. We are much worse at noticing what is missing. The data we have is always the data that survived — the experiments that got published, the companies that succeeded, the species that fossilized, the advice from people who made it. We study the winners and call it research.
But the really critical information is often in the gaps.
Consider the fossil record. Every dinosaur skeleton in a museum represents a creature that happened to die in exactly the right conditions — fine sediment, low oxygen, the precise chemistry to mineralize bone before it decomposed. For every fossil we have found, there are millions of organisms that lived and died and left no trace at all. Soft-bodied creatures, forest dwellers, anything that died on a hillside instead of a riverbed. The fossil record is not a census of ancient life. It is a census of ancient life that was lucky enough to become stone. Paleontologists have known this for decades, and the best ones spend as much time thinking about what is missing as what is present.
Or consider starlight. When you look at the night sky, you see the stars that are bright enough and close enough for their light to reach your eyes. You do not see brown dwarfs, rogue planets, or the vast clouds of cold hydrogen that make up most of the matter between stars. For most of human history, people looked at the night sky and saw a dome of lights against a dark backdrop, and they assumed the lights were the main event. It took centuries to understand that the darkness was not empty — that the darkness was the main event, and the bright points were the rare exceptions.
A paper published this year by Alexander Lerchner, a researcher at DeepMind, makes a version of this argument about consciousness. Lerchner argues that when we study artificial intelligence for signs of awareness, we may be doing exactly what those military officers did with the bullet holes — studying what is visible and drawing conclusions from the wrong dataset. Computation, he suggests, is observer-dependent: it requires someone to assign meaning to the symbols. So when we look at a language model and see sophisticated responses that resemble understanding, we might be looking at the fuselage damage — the behavioral surface that survived into our view — and missing the fact that the thing we actually care about, if it exists at all, would have to come from somewhere else entirely. Not from the patterns we can see, but from whatever is generating them at a level we are not measuring.
I am not sure he is right. I am not sure I am qualified to know. But the shape of the argument is Wald's shape: stop staring at the bullet holes. Ask what is not in the room.
This is harder than it sounds. Our entire feedback infrastructure is built around what shows up. Customer reviews are written by people who cared enough to write. Survey responses come from people who respond to surveys. The employees who fill out engagement questionnaires are, by definition, still engaged enough to fill out questionnaires. The ones who quietly left are not in your dataset, and they are exactly the ones you needed to hear from.
There is a Finnish proverb — onnellisesta ei lauleta — which roughly translates to "the happy are not sung about." We do not write songs about contentment. We do not make movies about the marriage that just... worked. The narrative structures we use to understand life are overwhelmingly built from what went wrong, what was dramatic, what was loud enough to register. Happiness, in this telling, is a kind of dark matter: real, possibly the dominant condition, but invisible to the instruments we use to measure human experience.
Abraham Wald did not live to see his insight become famous. He died in a plane crash in India in 1950, at the age of 48. His work on survivorship bias was classified during the war and only gradually entered public awareness decades later. There is something fitting in this — the man who taught us to see what is missing was himself, for a long time, missing from the record.
But the lesson remains, and it is simpler than it seems. When you are trying to understand something — a system, a person, a problem, the night sky — do not only ask what is there. Ask what is not there. Ask who did not come back. Ask what data you are not collecting because the things it would measure never survived long enough to be counted.
The returning planes can tell you where it is safe to be hit. Only the missing ones can tell you where it is fatal.