We think hallucinations make AI different from us. What if they make us the same? - The Boston Globe

He spent months in the hospital and every day I repeated a version of the same question. When I asked one morning whether he knew who I was, he looked down at my yellow robe, at my blue mask and plastic goggles, at the two layers of rubber gloves on each of my hands, and said, “Of course I do. You’re the head beekeeper on this farm.”

Early on, he had required surgery to evacuate a collection of blood pooling around his brain after a fall, and he wore a padded helmet over his open skull. It was too small and tilted crookedly over his forehead, shading his eyes. When asked whether he knew why he was wearing the helmet, he smiled and touched his head. “I’m playing in the big game today.”

As my patient invented each new version of his life — he was a spy and then a grocer, the hospital was a middle school and then a butcher shop — we doctors, too, spun stories about a virus we had never before encountered. We swept through hospital hallways, hovered over our patients’ beds, swarms of yellow-clad beekeepers imagining order into the chaos of a pandemic we did not understand and could not slow.

We learned my patient’s birth date from the expired driver’s license in his wallet, and found his son listed as the emergency contact in his medical chart, which was littered with visits to the emergency department and unfilled prescriptions. Over his time in the hospital, we pieced together his diagnosis from details his family offered about the years he had spent drinking his breakfast in pints of vodka and the faint, ghostly markings staining the memory structures of his brain on MRI: an amnesia born of a longstanding vitamin deficiency. And with it, confabulation: the unconscious compulsion to tell imagined stories in place of the memories he’d lost. My patient eventually left the hospital for a nursing home, but the amnesia — and the confabulations — never improved.

Confabulation may seem esoteric, a rare symptom confined to neurology wards and the writings of Oliver Sacks. But it reveals something universal about our brains: that stories are an essential part of how we parse the world, and that we are primed to imagine plotlines where they don’t exist — particularly when we are faced with incomplete information.

Surrounded by a chaotic world, deluged with sights, sounds, and sensations, our brains instinctively search for narrative order, telling stories to explain away that which we cannot understand and that which we fear. Our hunger for narrative serves as a bulwark against the entropy of our world, allowing us to filter the noise and decide where to focus our attention.

And while confabulation is a human universal, it may not be a uniquely human idiosyncrasy.

Artificial intelligence has evolved since its inception, with new reasoning systems that have made it both more powerful and more flawed. Compared with older systems, the latest AI models are particularly prone to what some in tech have termed “hallucination” — a tendency to produce information that is entirely divorced from reality when confronted with uncertainty. AI bots have invented imaginary books and nonexistent studies, fake academic papers and, in one case, enough bogus legal citations to warrant a judicial sanction.

Unlike the human brain, these AI bots are essentially powerful probability calculators, complex models that learn about the world by processing vast repositories of digital data and then use this information to guess the mathematically likeliest response to any given question or prompt. The notion of “truth” is entirely alien to these models, which are not designed to arbitrate fact and fiction, but rather to reason their way to an answer that seems statistically feasible based on the data they’ve absorbed.

Sometimes, these answers are patently false. Hallucination rates vary based on both the system and the task; some companies report figures in the single digits or low teens. In a recent paper, though, the company OpenAI evaluated its own large language models using a series of questions drawing from publicly available facts about everything from video games to politics and found that the latest models hallucinated anywhere from 33 percent to 79 percent of the time, depending on the test.

Critics have taken issue with the term “hallucination” because it attributes both consciousness and personhood to AI bots. A hallucination is, by definition, an experience, the perception of something that does not exist — for human sufferers, a phantom vision, an imagined sound, even an illusory itch. A better analogy for AI fabrications, as others have suggested, is the human symptom of confabulation: the unconscious tendency of our brains to invent facts and memories in the place of missing information. Human confabulators are sometimes called “honest liars,” in thrall to their own stories, as fooled by the unconscious machinations of their brains as their audiences; AI bots seem to suffer the same lack of awareness.

For many who — like me — identify as luddites, these errors feel like vindication, evidence that AI is not only essentially different from the human brain, but also fundamentally less reliable. I have yet to explore any of the applications of ChatGPT for my work as a doctor and writer, and I get more use out of my mechanical typewriter than I do out of any of the AI-fueled bells and whistles of my hospital’s electronic medical-record system. I consider myself lucky to have a job that allows me to spend my days considering human, rather than artificial, intelligence.

But when I view AI through the lens of my practice as a neurologist, these hallucinations actually seem like a profoundly human failing: a funhouse-mirror version of the inner workings — and deeply coded flaws — of our own brains.

One of the most persistent fears surrounding this new generation of machine-learning algorithms is that they are black boxes, their inner workings as opaque to their creators as they are to their users. But the same can be said of the human brain, a mess of neurons and synapses that somehow gives rise to consciousness. In many ways, AI is still utterly unlike the human brain, and what we know about the brain can only take us so far in understanding the strange phenomenon of AI hallucinations. But studying human flaws such as confabulation has advanced our understanding of the black box between our ears. I can’t help but wonder whether taking the same approach to AI would prove similarly revelatory.

The story of human confabulation dates back more than a century before my patient first arrived at my hospital. In the 1880s, a young psychiatry trainee in Dresden began to notice a bizarre constellation of symptoms among the patients under his care. His patients were largely suffering from complications of late-stage syphilis, their brains colonized by the spiraling bacteria decades after they were first infected. In some ways, the patients seemed unscathed by the infection. They were able to carry on sophisticated conversations with their doctors and visitors. But in other essential ways, they were utterly transformed by the illness: They seemed to lose their memories entirely, forgetting visitors the moment they left the hospital ward and sometimes even that they were in the hospital at all. In the most severe cases, patients lost track of their very identities.

One man, for instance, insisted by turns that he was Peter the Great and that he had been crucified alongside Christ; another announced each morning that he was in the pillared hospital hall because he was to be married that afternoon, while a third was certain that he was there to be hanged. These tales were so rich, animated with such convincing details, that the psychiatrist — despite all evidence to the contrary — sometimes found himself wondering whether they could actually be true.

Years later, researchers still struggle to fully explain what causes confabulation. What we do know is that it is startlingly common. Confabulation can arise not only after infections such as syphilis, but also from dementia, strokes, tumors, and vitamin deficiencies. As a general neurologist who staffs a busy city hospital, I often hear my patients confabulate stories into the voids left, not only by amnesia, but other losses, too. Faced with a weak arm that she could not lift, one woman told me it belonged, not to her, but to her husband, seated across the room; suddenly robbed of his vision, one man told me that he could not find his way out of his hospital room because of a power outage that had dimmed the overhead lights.

Even healthy human brains can confabulate — something seen on television imagined as real, for instance, or the same story remembered differently by two friends. In one study from the ’70s, four identical pairs of nylon pantyhose were displayed on a rack at a bargain store in a shopping mall, and 52 passersby were asked to choose which of the four was of the best quality, then asked why they had chosen that particular pair. Only two of the test subjects answered that the pairs were identical, while the other 50 offered reasons for their choices ranging from the sheerness to the weave, the elasticity to the workmanship, observations confabulated in the absence of any actual difference.

Outside of psychology textbooks, this type of confabulation can have real-world consequences. In courtrooms, it can lead to false convictions, a witness tricked by their own brain’s desire for certainty into identifying a suspect or confirming a timeline based on an imagined memory.

This very human vulnerability to false narratives can become even more dangerous when stoked by AI confabulations, creating an echo chamber of unreality. Journalists and mental health practitioners have reported a growing incidence of “AI psychosis,” a cycle in which confabulating chatbots — designed to affirm their users in order to maximize engagement — fuel the delusions of their human correspondents. In one case, a chatbot confabulated an entire false universe à la The Matrix; in another, it persuaded a corporate recruiter that he had discovered a novel mathematical formula that could take down the internet. These delusions have led to psychiatric hospitalizations and assassination attempts, divorces and suicide: imagined stories with real-world stakes.

Science has a long tradition of using neurological wounds as windows, opportunities to catch a glimpse of the complex ways our brains work when they are whole. We understand something about the biological basis of communication from studying people bereft of language, about the underpinnings of human perception from studying people who have experienced blindness, and about the neural pathways that generate movement from studying people suffering paralysis. Even the most esoteric-seeming neurological injuries speak to universal features of our brains.

The neurologic symptom of confabulation reveals some of the complexity of the human brain. Within our brains, the creative ability to narrate a story and the more mundane ability to “fact check” its source — to ascertain whether it’s a product of imagination or observation — are housed separately. When they work as they ought to, our brains are able to recognize the limits of their own knowledge and admit uncertainty when confronted with a question to which they have no answer. When the fact-checking circuitry goes awry, whether in the momentary glitches of everyday confabulation or more lasting injuries, such as my patient’s, we offer a complex false narrative rather than simply confess, “I don’t know.”

To understand what AI confabulations can teach us about the black box of AI, I turned to someone who spends as much time thinking about bots as I do thinking about human beings: Pratik Verma, the CEO and cofounder of Okahu, a company that helps AI developers analyze their products and troubleshoot hallucinations and other errors. Verma tells me that he’ll leave any comparisons between artificial and human intelligence to neuroscientists — “I’m a chemist by training, not a biologist” — but notes that in many ways, the two are utterly unalike. Large language models, he says, are “basically a statistical engine. They’re taking in a set of tokens and predicting the next one.” When two concepts are too statistically similar — equally likely within the AI’s narrow worldview — errors can arise in predictions.

That said, Verma continues, AI hallucinations are also fueled by developers’ efforts to make AI seem more human. He cites a “temperature” parameter that controls the randomness of responses a model can provide. At lower temperatures, models are robotic, producing the most predictable and conservative response — the type of performance you might prefer when asking an AI bot to spit out a specific fact. Higher temperature parameters allow for more creativity and flexibility in responses, the kind of latitude you might prefer if you were asking a bot to help you brainstorm concepts or write a narrative essay. At higher temperatures, AI bots can add tonality and humor to their responses. The higher the temperature, the more human a model may seem — and the more likely it is to hallucinate, particularly when there are multiple answers that are statistically plausible. Some AI systems rely on multiple large language models, each passing potentially flawed information to the next in a process Verma likens to a game of telephone.

The solution to AI hallucinations may be remarkably simple, Verma explains. He advises developers to program their AI models to explicitly cite their sources, and allow them to harness one of the greatest strengths of the human brain: the ability to say, “I don’t know.” Lowering the temperature of a model reduces the likelihood that it will offer a false answer by allowing it to instead reveal when it has no answer at all, he says. This is a tactic users can sometimes employ themselves: Anthropic, for instance, advises people to explicitly give its Claude AI model permission to return “I don’t know” as an answer. It turns out that the same distinction that separates a confabulating human mind from a healthy one also separates a confabulating AI bot from a properly functioning one: the ability to admit uncertainty.

As a human physician, I often reflect on uncertainty — how essential it is to my medical practice, and how difficult it sometimes can be to acknowledge. When I began my training, I imagined that only the expanse of medical school, of residency training, of a post-residency fellowship, lay between me and certainty, that illness was made up of clear binaries that I simply had yet to learn. In the years since, I have come to understand that our illnesses, our bodies, our fates, and even our treatments are made up of uncertainty — whether a particular treatment will work as well for one person as it does for another, for instance, or what might have triggered the onset of an illness.

For physicians, uncertainty often dovetails with feelings of powerlessness and anxiety. In a profession where the stakes are incalculably high, to admit uncertainty — about a diagnosis, about a prognosis — often feels like a failure.

Recently, I cared for a woman who suffered a prolonged cardiac arrest at home, her son performing CPR — desperately compressing his mother’s heart until her brittle ribs cracked, blowing into her nose and mouth to inflate her lungs — until the ambulance arrived. Starved of oxygen for nearly 30 minutes, the neurons of my patient’s brain had begun to die, its once sharply demarcated structures blurring into a single, sickly-white mass of swollen tissue on an MRI. But the functions of her brain seemed remarkably intact — her pupils shrinking beneath the beam of my penlight, her eyes fixing on her son’s face as though she knew who he was.

As her son wept at her bedside, afraid to give up on his mother, even more afraid to keep her body alive without the final relief of dignity, he asked whether she would recover from her injury. More than anything, I wanted to offer a confabulation, a story with a made-up ending — that his mother would wake up, or that she never would — that would give the son a path forward through his grief. But the real answer was messier, hinging on what the mother valued about her life, what the son considered a meaningful recovery, and the black box of her brain’s resilience. The best I could offer was: “I don’t know.”

We are wrong to think that AI has the ability to circumvent the idiosyncrasies of human intelligence; confabulation, uncertainty — these are not bugs, but features, essential to any way of understanding the world, artificial or otherwise.

Rather than imagining AI as omnipotent and infallible, we should treat it as imperfect, rife with just as many fragilities as the human brain — and as many unknowns.

Pria Anand is a neurologist and the author of The Mind Electric. She is an assistant professor at the Boston University School of Medicine, and she cares for patients at the Boston Medical Center. Send comments to magazine@globe.com.