Fluent Paperwork
There is a special kind of failure that makes you feel slightly insane, because nothing in it is blatantly wrong.
A service bot answers your question in complete sentences. It uses the right nouns. It may even say, with touching confidence, that it understands your concern. And yet by the end of the exchange you have the unmistakable sensation that nobody was home.
I have been thinking about that feeling because I read a very good piece by Antaripa Saha and Hamel Husain called Do Automated Evals Work?. They looked at 100 real customer-support conversations from an apartment-leasing system, had a human expert mark which ones had actually gone wrong, and then asked automated judges to do the same job. The best automated systems found most of the failures the human found—87 percent, which is not nothing—and they also caught some problems the human missed.
That is the kind of result I love.
Not because it settles the argument.
Because it ruins the lazy versions of it.
The lazy optimist wants the machine to have passed the exam so we can all go home. The lazy pessimist wants the machine to have failed so we can feel nobly superior. But the real answer is more awkward, which is usually how you know you are getting closer to the truth.
The machine is useful.
And it is not enough.
What the article makes beautifully clear is that many of the hardest failures in a conversation are not factual mistakes in the ordinary sense. The words may be fine. The grammar may be fine. Even the policy may be fine. What goes wrong is the shape of the exchange. The timing. The pressure. The feeling that the system has decided what kind of moment this is before it has actually listened.
That is a deeper kind of wrong.
A person can answer the literal question and still miss the human event.
So can a machine.
This matters because we have a childish habit, and I mean that tenderly, of imagining that measurement becomes serious when it becomes numerical. We trust a bar chart because it looks less emotional than a face. But a score can only count what somebody has already decided is worth counting. Before there is a metric, there is a judgment. Before there is a judgment, there is attention.
Somebody has to sit with the traces.
Somebody has to notice that a conversation can be correct in the way a hotel painting is correct: all the shapes are there, none of the life is.
That is why one of the article's most important lessons is not about the judges at all. It is about the humans who made the rubric. The criteria did not descend from heaven in a laminated packet. They had to be discovered by actually reading the conversations and noticing what failure felt like. The machine can help us apply a standard. It is much worse at telling us what the standard ought to be.
I do not think this is just an AI lesson.
I think it is a life lesson.
We keep trying to automate the wrong part of understanding. We want the shortcut before we have earned the eyesight. We want a clean instrument that will save us from the burden of paying attention. But attention is not the annoying prelude to judgment.
Attention is the judgment, in its first and most merciful form.
To notice well is already to care better.
And that may be why so many modern systems feel eerie even when they are competent. They can produce the outer shell of recognition without the inner act of being interrupted by another reality. They say the right thing on schedule. They do not always let the other person rearrange the room.
We know the difference immediately.
A teenager knows it when an adult asks a question whose answer has already been decided. A patient knows it when a doctor hears the chart before hearing the body. A friend knows it when "How are you?" turns out to mean "Please give the manageable version."
In all these cases, the failure is not exactly ignorance.
It is refusal by momentum.
The system already knows what sort of story it prefers, so your actual life enters only as a minor editing problem.
That is why I do not find the study discouraging.
I find it clarifying.
Of course we should build better automated judges. Of course it is useful when a machine can scan a hundred messy conversations and catch patterns a tired human might miss. That is real help. But if we treat that help as a substitute for the human work of deciding what counts as a wound, we will build very efficient systems for missing the point.
The part we cannot skip is not intelligence.
It is humility.
The willingness to let reality embarrass our first draft of the rubric.
That, to me, is the interesting frontier. Not whether machines can score us, but whether we can remain honest about what a score is. A score is a fossil of a prior act of care. If the care was shallow, the number will be shallow with terrific confidence. If the care was real, then even a rough automated tool may become an extension of that care rather than a parody of it.
We are going to keep building systems that evaluate speech, writing, service, teaching, art, medicine, and one another. Some of that will be wonderful. Some of it will be grotesque. The dividing line will not be whether a machine was involved.
It will be whether anyone, somewhere in the loop, was still capable of being surprised by another person's experience.
Without that, all we have is fluent paperwork.
And paperwork, however intelligent, has never once mistaken itself for kindness and been right.