The Cost of a Question
A machine will answer anything. It cannot ask you a question that costs it something.
A few weeks ago I came to a small online gathering of teachers, coders, and people from media, music, and the public sector, all of us using AI tools daily, and none of us sure what they were doing to us. I had an argument to make, and for ninety minutes the room pressed on it from different directions. I have better answers now than I managed that night, and this essay is those answers. But the answers were never the point. The questions were.
The argument I brought was about a loss I had felt in my own work. An AI can “read a book” for you and deliver a summary: the hours are saved, the result is accurate, and something is still missing. You cannot argue with the book. You have its conclusions and none of its resistance. You own the output and underwent none of the process that was supposed to change you. The productivity is real. The understanding is phantom.
Phantom competency is the name I gave for the sense of competence that attaches to a finished-looking result rather than to anything you underwent to produce it: the document is summarized, but you did not do the summarizing; the essay is drafted, but you did not do the thinking; the answer sits on the screen with your name near it, and nothing has moved behind your eyes. Comprehension is a metabolic process. You cannot outsource digestion and stay fed.
My prescription was not refusal. It was a gym for the mind: put back, on purpose, the friction the tools remove, because the friction was doing our thinking. That was the argument of an earlier essay, The Friction That Was Thinking, and I believe it more the longer I watch. Draft before you open the machine; ask it to interrogate your work rather than produce it; defend the blank page and the hesitation in front of it. But one claim in the talk I could assert and not close: that the hardest friction to replace is not any exercise you can assign yourself. It is other people, the pressure of minds that are not yours. The discussion closed it for me. The participants did to my argument exactly what I had claimed thinking requires: they questioned it, found where it was thin, and leaned until we all saw the cracks. The argument is stronger now for where it broke, which is the whole case in miniature.
What follows are four questions that did the work, what I have learned since by trying to answer them, and the property they turned out to share, the one no machine in the room could have supplied.
Beginning at the top
Two teachers, one after the other, helped us find the first crack. A professor asked how anyone is supposed to evaluate real competence now that the work happens out of sight: essays written at home were once evidence of a mind at work, and that evidence no longer testifies. Then a schoolteacher pushed the problem down to its root. My prescriptions, he pointed out, assume a person who can already tell a strong argument from a weak one. His students cannot — not yet, in any case. Building that capacity is what school is for. If the machine now does the work that used to build it, where does a beginner begin?
He gave the difficulty its proper shape by reaching back to a ladder of learning drawn up in the 1950s, one that climbs from remembering a thing, through understanding and applying it, up to judging it at the top. Every piece of advice I had offered lived on the top rung: judge the output, catch the errors, correct the reasoning. But judgment is built from underneath, out of a few hundred clumsy attempts of your own and the feel of where each one failed. The machine offers, precisely, to skip those rungs. A beginner cannot evaluate the output that would have formed the judgment the evaluation depends on.
The early evidence says this worry is not hypothetical, and it is worth following the links to form your own view. In a small and much-discussed MIT study, students who wrote essays with a chatbot showed weaker neural engagement than those who wrote unaided, and could not accurately quote the essays they had just submitted under their own names: the output existed, the authorship had not happened. A survey of knowledge workers from Microsoft and Carnegie Mellon found that the more people trusted the tool, the less critical thinking they reported doing; the effort migrates from doing the work to spot-checking the machine. And in the result that should end any complacency about this being a students’ problem, a study in The Lancet found that experienced doctors, after months of routine AI assistance during colonoscopies, became measurably worse at detection when the assistance was off.
Experts decline from a height. The beginner who starts with the crutch never gains a height to decline from. And notice where on the ladder that decline falls: detection, the seasoned clinician’s judgment, is a top rung, the very rung the machine had climbed to, which is why leaning on it there was so easy and cost so much. The taxonomy was drawn to describe a mind ascending. It doubles now as a map of the handover, rung by rung, and it leaves one question open, the one the evening kept returning to: when the machine reaches the top rung, where does the human go?
Putting the friction back
I had no method to hand the teachers that night. But teachers are already building them, and the pattern across them is worth more than any single trick: each one relocates the friction instead of abolishing it.
The boldest version I know comes from a philosophy professor who stopped assigning individual essays and now writes one long essay with his whole class, arguing every sentence into place in front of them across a term, so that the thinking happens among people, in the open, where it cannot be quietly forwarded to a machine. His stated aim is the one I wish I had said first: to put the friction back. I have been on the receiving end of a method like his. The most formative stretch of my own education was a semester spent on a handful of paragraphs from Descartes’s Meditations: a professor and a room of us, a chalkboard and photocopies with wide margins to scribble in, the same few pages argued over until we had earned them. Nothing was produced that a machine could not now generate in seconds. That was never what it was for.
Others attack from the opposite side and make the machine itself the sparring partner: students must argue with a chatbot and hand in the transcript, together with a short reflection on which of the machine’s points most unsettled their position, so the graded artifact is not an essay a machine could have written but the record of a mind under pressure. Others have revived the oral exam, or have students write in class across several sittings with nothing but their own notes, and report the tell of real thinking: students change their thesis mid-essay, discover they believe the opposite of what they set out to defend. The most rigorously tested design I have found, run across hundreds of students at UC Davis, pairs feedback from classmates with feedback from a model and grades the student’s comparison of the two: judging the machine’s judgment turns out to build the very capacity it requires, provided a human judgment sits beside it for calibration.
None of this needs a syllabus. The transferable rule, at any desk, is to change what you ask the machine for: not “write this,” but “find the weakest claim in what I wrote.” Do the first draft alone, so there is a self on the page for the tool to sharpen rather than a blank for it to fill. Run it backwards: ask it to teach you the terrain of a decision, the vocabulary and the risks, and then do the synthesis yourself. Let the machine near the work only after the work has cost you something.
The retreating line
A philosopher spoke next, an old friend, and his objection was aimed not at me but at all of us, at the way the whole culture now talks. Everyone, including me on that call, says the machine “thinks,” “reasons,” “understands.” He wanted us to be slower with those words. They name what we are; lending them out flatters the product and dulls our sense of the difference; we should hold the line on the words that matter.
I share his instinct, but I doubt the tactic, and one of the reasons is historical: there are words we have already surrendered, some within living memory, without ever noticing the funeral.
For three hundred years, a “computer” was a person. It was a job, held by clerks and, in the great scientific projects, by rooms of women: the ones who worked out the nautical tables that let sailors fix a position at sea, the ones who catalogued the stars at Harvard, the ones who ran the arithmetic of the early space program. When Turing wrote the paper that founded computer science in 1936, the word “computer” in his text refers to the human being; the machine was named after us, and it inherited the job title along with the job. By 1962, the handover was far enough along that John Glenn, preparing to orbit the Earth on the strength of an electronic computer’s trajectory, asked the engineers to “get the girl” so that Katherine Johnson could re-run the numbers by hand. “If she says they’re good,” he said, “then I’m ready to go.” For one strange decade, the human computer was the calibration layer for the electronic one. Then the qualifier finished changing sides.
Once, the machine needed the adjective: electronic computer, automatic computer. Now the person needs it: mental arithmetic, doing the calculation in your head. Watch where the adjective sits and you can date the handover of a word. By 1950, Turing was predicting that within fifty years one could speak of machines thinking “without expecting to be contradicted,” and on the quieter word, “compute,” he was right ahead of schedule.
And the line has kept moving: from arithmetic to formal logic, mechanized in the 1860s by a machine I have written about elsewhere, whose builder marveled that mind could “create its own rival in the wheels and levers of an insensible machine”; from logic to chess; from chess, now, to the sentence itself. My friend hears that history as a long retreat, and he is not wrong that something real goes each time. But before deciding how to defend the next word, it is worth asking precisely what went the last time, and where the value moved when it did. There is a recent enough case to check.
Where the value goes
In 1979 a program called VisiCalc put the first spreadsheet on a personal computer, and the profession built on human calculation met its machine. Before it, changing one assumption in a financial plan meant days of clerical recalculation; armies of clerks existed to run those numbers. Afterwards, recalculation was free. Planet Money once put figures on what followed: since 1980, about four hundred thousand bookkeeping and accounting clerk jobs disappeared in the United States, and about six hundred thousand accounting jobs were added.
Read that again, slowly.
The machine took the calculation, and the number of people paid to think with numbers went up.
What were the new people paid for? When recalculating became free, the question “what if?” became free with it, and the scarce skill moved one floor up: from computing the answer to framing the model. Which assumptions to make, what depends on what, which what-if is worth an afternoon. A spreadsheet is a machine you pour judgments into: every formula in it is a small claim about how the world hangs together, and the machine computes only the consequences of your claims. Notice, in the same spirit, what we praise when we call someone “numerate.” We never mean speed of arithmetic. We mean a feel for what numbers say: a sense of orders of magnitude, the reflex to ask “compared to what?”, a distrust of suspiciously precise figures. If you have ever looked at a spreadsheet’s output and refused to believe it, you were exercising the part that did not migrate. The machine took the syntax of numbers. The semantics stayed with us, and appreciated.
Two footnotes from that transition read like prophecies now. Within five years of VisiCalc, Steven Levy had documented executives falling in love with their models, tweaking imaginary businesses that were, he wrote, just that: imaginary; phantom competency arrived with the spreadsheet, four decades before the chatbot. And thirty years later, a graduate student with a working sense of what the numbers should have said checked a famous economics spreadsheet by hand and found, among other problems, the error underneath an argument then steering austerity policy for whole countries.
The judgment that catches the machine is not nostalgia. It is the load-bearing part. This is the shape of every handover on the ladder: the machine takes a rung, and the scarce human work moves to the rung above. When answers become cheap, the value moves to the question. In the spreadsheet’s case that question was a modest thing, a what-if: which assumptions, which model, which afternoon. It does not stay modest.
Hold that.
Mourning, done well
Still, my friend is owed more than a mechanism, because something does die in these handovers, and the spreadsheet story can be told too cheerfully. The human computers were not an inefficiency that history corrected. They were skilled people, proud of a real craft, and the craft ended; nobody held a funeral, and the words that might have said what their skill meant had to be invented afterwards.
I have written elsewhere about Ozu’s Late Spring, and about what it teaches: that there are ceremonies whose forms continue after no one stands behind them, and that the honest response to a passing form of life is neither denial nor refusal but mourning, which accepts that the loss is real and keeps, deliberately, what the lost thing meant.
Seen from there, the two standard positions on AI both fail the same test. The enthusiast who says nothing is lost is refusing the funeral in one direction; the defender who wants the old words held by force is refusing it in the other. Mourning is the third way, and mourning has a language: a eulogy is exact about what mattered in the dead. Numeracy, number sense, estimation: those words are the eulogy English wrote for human computation, the community naming, after the fact, what it refused to lose. That is how the defense of the human has actually been conducted, each time the line moved. Not by guarding the old word, but by writing the eulogy well: by being exact, in new words, about the remainder that must not die.
If thinking is next, the task my friend rightly feels is not to fight for the verb. It is to name, now and precisely, what in thinking was never the machine’s to take, and to build a life around keeping it.
Being right is not standing behind
So, run his worry forward as a thought experiment, the way I hope the last section earned. Suppose it goes all the way: for any hard problem, we ask the machine, we prefer its solution, we trust it over any human expert. This is not a forecast, just the counterfactual in which reasoning completes the path calculation took. What, concretely, would still be ours?
We have one live rehearsal, and it is instructive. In chess, the counterfactual already happened: no human has beaten the best machine in twenty years, and the deference has fully flipped, so that grandmasters now check their intuitions against the engine rather than the reverse. Human chess should have died. Instead it is larger than it has ever been, a quarter of a billion accounts and rising, and the audience is not watching engines play engines. It watches humans think under a clock. The practice survived total machine superiority because its point was never the output; the point was people, measuring themselves against each other and becoming, visibly, better. But notice what secured that survival: boards, clocks, ratings, arenas, faces. Institutions whose explicit point is the humans in them.
If reasoning goes the way of chess, that is the infrastructure thinking will need.
The second remainder is harder and matters more. Even in the full counterfactual, a solution is not yet a decision. Somebody signs. The machine can draft the medical plan, the engineering assessment, the legal opinion, and in the counterfactual it drafts them better than we do; but a verdict binds because a community has conferred on a particular person the authority to bind, and that authority is conferred, not computed. When the judge signs, she is not adding intelligence to the document. She is adding herself: someone who can be answerable for it, appealed against, held. The machine can be right. It cannot stand behind. Being right is a property of outputs; standing behind is a stance of persons, and in the counterfactual every institution we still trust has quietly reorganized itself around the second.
And the third remainder is a choice we already know how to make, because we made it once. Calculators did not end the teaching of arithmetic. We still put children through long division, not because they will out-divide a chip but because number sense is the substrate of the judgment above it: the settlement, after some panic, was fundamentals first, tools after. In the counterfactual we would teach proof and argument on the same grounds: not to compete with the machine, but so that a person’s yes can still be meant. And some things we would grieve properly this time. Reasoning together, as a daily way of being with one another, is a form of life; if it thins into supervising a solver, something as real as the human computers’ craft dies with it, and it will deserve its funeral and its eulogy too.
Desire is taught
The bleakest question of the night was also, in a way, the friendliest to my argument. A participant who introduced herself as the room’s pessimist told me the cognitive gym image was the thing she kept returning to, and then turned it over to show me its soft side. In this room, she said, the analogy convinces everyone; the trouble is elsewhere. A sixteen-year-old can simply say: I go to the gym because I want to look good, but I have no wish to think for myself, so I will export that to the machine. The analogy assumes a desire that the analogy cannot create. What do you say to the kid?
On the night I reached for a hope: that as machine-made text becomes the norm, the people who can think on their feet will stand out and be admired, the way we came, culturally, to admire fitness. I want to finish that answer, because the hope was pointing at a mechanism, and the mechanism has a name.
Nobody is argued into the gym, least of all a sixteen-year-old. Desire is learned by imitation: we want what the people we admire want, which is Girard’s old observation, and it means the question “does the argument convince a student?” is the wrong test, because arguments do not do that work for any of us. Models do.
The fitness case is the proof that desires can be manufactured at the scale of a civilization, within one generation, because we watched it happen. In 1968, a man named Dick Cordier was stopped by police in Hartford for the suspicious activity of running down his own street; the same decade, a United States senator was questioned mid-jog. Running in public marked you as an eccentric. Twenty-five years later, visible fitness was a status marker across the culture, and no argument accomplished that: models did, one imitated body at a time, until the wanting was contagious.
Instagram is what the cascade looks like when it overshoots, and it supplies the honest warning label for my own hope: mimetic desire optimizes for display, not substance. Fitness culture produced health and also dysmorphia; a thinking culture would produce formation and also its theater, people performing depth the way people perform workouts. Mimesis does not care what it spreads. It only spreads what is visible and admired.
Which is why the constructive answer to the pessimist is not a better argument. It is a visibility infrastructure, which is the lesson chess had already taught. The chess boom is millions of teenagers voluntarily watching other people think, for sport. Three-hour podcast conversations, whatever their average quality, are thinking as mass spectacle; so is the return of the oral exam, which students report leaving with something suspiciously like pride; so is the small, growing premium on work that can credibly say a person made this, the way industrial food made “homemade” a compliment.
None of this is an answer you can give the kid directly. It is the environment that answers for you. You cannot argue a sixteen-year-old into wanting to think. You can build rooms where thinking is visible, contested, and admired, and let imitation do what argument cannot. The gathering this essay comes from was, for ninety minutes, exactly such a room.
The same debt twice
The question I answered least well on the night deserves the most direct repair. A political economist asked about the machine’s bill: the electricity, the water, the fortunes being assembled around it, and the guilt she feels reaching for a tool whose costs she knows. She was asking, honestly, how to hold the philosophical worry and the material one together. Here is the fuller answer I owed her.
First, the honest numbers. The International Energy Agency projects that data-centre electricity consumption will more than double by 2030, to around 945 terawatt-hours, slightly more than Japan consumes today, with AI the main driver; in the United States, data centres used 4.4 percent of all electricity in 2023, projected to reach somewhere between 6.7 and 12 percent by 2028. Water is murkier, and the murk is itself information: per-query estimates run from a fraction of a millilitre in industry self-reports to figures orders of magnitude higher in independent modeling of earlier systems, and when a cost estimate spans two orders of magnitude, you are learning who controls the accounting.
Efficiency, the standard reassurance, reassures less than it should: per-query costs are collapsing while total consumption explodes, which is Jevons’s paradox, the same law I have watched run inside my own working life, where every gain in execution was swallowed by expanded ambition. Cheaper thought means more thought outsourced; cheaper compute means more compute burned. The bill grows through the efficiency, not despite it.
So the real question is which future the bill is buying, and I can only see two honest candidates.
I have been wrong about this before, which is why I take the dark one seriously: in the nineties I believed the web could only be good, decentralized, ours; it became the attention economy not by decree but by default, because advertising was the business model lying within reach. The same gravity is pulling now. None of the frontier labs is profitable; advertising has already entered the chatbot this year, with assurances about not optimizing for time spent that will be tested against quarterly earnings. If intelligence ends up monetized by engagement, the data centres will be burning a nation’s worth of electricity to keep us asking, and the cognitive story and the material story converge into one bad bargain.
The other candidate is the one the builders themselves bet on, and I confess it is my “Star Trek dream,” named as a dream and not a forecast: that machine intelligence is the bottleneck input to every other problem, including this one. The same systems already hold fusion plasmas in magnetic control and have proposed hundreds of thousands of stable new materials at a stroke; the case that this compounds into cheap clean energy has been made at book length by the people building it. If that future arrives, today’s bill was seed corn, and even the water problem dissolves into an energy problem, which abundance solves.
To be clear: I do not know which future we are in. I doubt anyone does.
But here is what I can say to her guilt, because it is better than absolution. The two debts have the same shape. What the offloading does to a mind and what the data centre does to a watershed are both quiet accruals against a future that is not consulted: invisible while they compound, obvious when the bill arrives. Her guilt is the rare case of feeling the interest payments in real time, and the web’s history says the fork between futures gets decided by default unless someone makes it explicit. So the useful response is not to feel worse or stop using the tools. It is to make the default visible and lean on it where we can: pay for tools rather than be what pays for them, prefer providers whose revenue is the finished work rather than the retained gaze, and keep asking the question she asked, out loud, in rooms where the people building these systems can hear it. Her guilt, spoken, did more than my optimism.
The rung above the ladder
Four questions, then, from four directions: where does a beginner begin; what should we call the machine; what will make the young want what the machine makes optional; who pays for it all. I have given you better answers than I managed on the night. But the longer I sat with them, the clearer it became that the answers were not the point. The questions shared a property, and the property is the finding.
Each one cost its asker something. The teachers admitted, in front of strangers, that they no longer fully know how to do their work. The philosopher exposed a fear of loss he could not fully argue. The pessimist owned a despair about the young that nobody enjoys saying. The economist confessed a guilt she had not resolved. A real question is not a request for information. It is an act of exposure: it shows where you are unsure, it makes a claim on the person it addresses, and it risks the answer.
Return, one last time, to the ladder the schoolteacher drew. It ends at judgment: to evaluate, to catch the error, to tell the strong argument from the weak. That was always the top rung. But the machine has been climbing the ladder its whole existence, remembering, understanding, applying, and now judging well enough that seasoned doctors deferred to it and lost the skill; and when it reached the top, it exposed a step the taxonomy never drew, because no one had needed to name it while judging a thing and standing behind the judgment still lived in the same person. Above judging is standing behind. Above knowing is acknowledging. Stanley Cavell spent a career on that difference, between producing true sentences and meaning them, and it turns out not to be a higher cognitive skill at all. It is the willingness to ask, and to be asked, a question that costs something. It is the one rung a machine cannot stand on, because standing there means having something to lose.
That is the floor the machine never reaches. It is answerable to no one, which is exactly why it can answer anything; nothing it says costs it anything, and so nothing it says asks anything of you. It will answer every question in this essay, fluently and agreeably and now. It could not have asked one of them.
That is what the room had that the machine in the room did not, and it is why the spreadsheet’s lesson was the evening’s real subject all along. When answers become cheap, the value moves to the question; we have run that migration before, one floor down. The scarce resource on the other side of this transition is not intelligence, which is being poured into the world at industrial scale. It is the questioner: the person with enough at stake to press, in public, where being wrong has a price. Everything I now think about living with these machines reduces to protecting that person, in classrooms, in companies, in ourselves.
Which leaves the question underneath the four, the one I carried home. If real questions come only from someone with something at stake, and we are teaching ourselves, question by question, to bring them to a system with nothing at stake at all, who will be left to ask?
I know what the machine would say. It has an answer ready, fluent and immediate and free. That it has one ready is the problem.
Sources & further reading
Stanley Cavell, Must We Mean What We Say? (1969) and The Claim of Reason (1979). The distinction between knowing and acknowledging that the closing rests on: meaning is something you stand behind, not something you output.
Alan Turing, “Computing Machinery and Intelligence” (1950). The prediction that use, not argument, would settle whether machines “think,” and still the clearest catalogue of the objections.
David Alan Grier, When Computers Were Human (2005). The three-century history of computation as a human occupation, before the word moved to the machine.
Steven Levy, “A Spreadsheet Way of Knowledge” (Harper’s, 1984). The first portrait of model-fluency mistaken for understanding, written five years into the spreadsheet era.
Benjamin Bloom et al., Taxonomy of Educational Objectives (1956). The ladder from remembering to evaluating: judgment sits at the top and presupposes every rung beneath it.
René Girard, Deceit, Desire, and the Novel (1961). Desire as imitation: we learn what to want from models, which is why cultures, not arguments, change what the young want.
Marc Watkins, Rhetorica (ongoing). A working teacher’s newsletter tracking, in real time, the classroom experiments this essay samples.