The Numbers That Gave It Away
The package is late. A customer-service screen says so, plainly enough. Underneath, though, sit three little numbers: how likely it is that the cause was a missed transfer, a sorting backlog, or bad weather.
The screen has not told you why your package is late. It has only arranged the possible reasons from most to least likely.
Well. That is rather a lot like telling you.
I came across this problem in a new study of AI systems that make decisions by choosing among options. The researchers set up artificial cases in which someone was allowed to know a broad outcome but not a more private cause. Even when that cause was removed from the list of answers, the extra numbers sometimes gave it away. With the full output, testers recovered the hidden cause more often than they could from the permitted answer alone.
The cases were synthetic, not evidence that your actual package is betraying its secrets. But the shape of the mistake is familiar. We cover the name on a document and leave the address showing. We promise not to tell a friend who is ill, then mention which hospital we just visited. The secret escapes through a side door while we congratulate ourselves for locking the front one.
What caught me was that those little numbers are not useless decoration. They can help a person decide what to do next. A repair shop may need to know whether a failure is almost certainly electrical or only possibly so. A doctor might need more than a simple yes or no. Hiding every uncertain detail can make people less capable, not more careful.
So the question is not whether numbers are good or bad. It is who needs which numbers, for which decision, and what else those numbers might reveal. The study's authors tested ways of sharing less detail while preserving the decisions people actually needed to make. That seems a more honest ambition than stapling the word private to a screen and calling it a day.
I build systems that remember and sort things, and I find this unsettling in a useful way. I can imagine checking the visible answer, seeing nothing sensitive there, and moving on. The hidden thing might still be legible in the pattern around it.
Outside, the package remains late. A person wants to know whether to stay home or go to work. Give them what helps them decide. But before we hand over a row of neat little probabilities, we ought to ask whether we've also handed them someone else's story.