The Undoing Project: A Friendship that Changed the World

Beyond that, Amos was the most terrifying mind most people had ever encountered. “People were afraid to discuss ideas in front of him,” said a friend—because they were afraid he would put his finger on the flaw that they had only dimly sensed. One of Amos’s graduate students, Ruma Falk, said she was so afraid of what Amos would think of her driving that when she drove him home, in her car, she insisted that he drive. And now here he was spending all of his time with Danny, whose susceptibility to criticism was so extreme that a single remark from a misguided student sent him down a long, dark tunnel of self-doubt. It was as if you had dropped a white mouse into a cage with a python and come back later and found the mouse talking and the python curled in the corner, rapt.

But there was another story to be told, about how much Danny and Amos had in common. Both were grandsons of Eastern European rabbis, for a start. Both were explicitly interested in how people functioned when they were in a “normal” unemotional state. Both wanted to do science. Both wanted to search for simple, powerful truths. As complicated as Danny might have been, he still longed to do “the psychology of single questions,” and as complicated as Amos’s work might have seemed, his instinct was to cut through endless bullshit to the simple nub of any matter. Both men were blessed with shockingly fertile minds. And both were Jews, in Israel, who did not believe in God. And yet all anyone saw were their differences.

The most succinct physical manifestation of the deep difference between the two men was the state of their offices. “Danny’s office was such a mess,” recalled Daniela Gordon, who had become Danny’s teaching assistant. “Scraps on which he’d scribbled a sentence or two. Paper everywhere. Books everywhere. Books opened to places he’d stopped reading. I once found my master’s thesis open on page thirteen—I think that’s where he stopped. And then you would walk down the hall three or four rooms, and you come to Amos’s office . . . and there is nothing in it. A pencil on a desk. In Danny’s office you couldn’t find anything because it was such a mess. In Amos’s office you couldn’t find anything because there was nothing there.” All around them people watched and wondered: Why were they getting along so well? “Danny was a high-maintenance person,” said one colleague. “Amos was the last one to put up with a high-maintenance person. And yet he was willing to go along. Which was amazing.”

Danny and Amos didn’t talk much about what they got up to when they were alone together, which just made everyone else more curious about what it was. In the beginning they were kicking around Danny’s proposition—that people weren’t Bayesians, or conservative Bayesians, or statisticians of any sort. Whatever human beings did when presented with a problem that had a statistically correct answer, it wasn’t statistics. But how did you sell that to an audience of professional social scientists who were more or less blinded by theory? And how did you test it? They decided, in essence, to invent an unusual statistics test and give it to the scientists, and see how they performed. Their case would be built from evidence that consisted entirely of answers to questions they’d put to some audience—in this case, an audience of people trained in statistics and probability theory. Danny dreamed up most of the questions, many of which were sophisticated versions of the questions about red and white poker chips:

The mean IQ of the population of eighth graders in a city is known to be 100. You have selected a random sample of 50 children for a study of educational achievement. The first child tested has an IQ of 150. What do you expect the mean IQ to be for the whole sample?

At the end of the summer of 1969, Amos took Danny’s questions to the annual meeting of the American Psychological Association, in Washington, DC, and then on to a conference of mathematical psychologists. There he gave the test to roomfuls of people whose careers required fluency in statistics. Two of the test takers had written statistics textbooks. Amos then collected the completed tests and flew home with them to Jerusalem.

There he and Danny sat down to write together for the first time. Their offices were tiny, so they worked in a small seminar room. Amos didn’t know how to type, and Danny didn’t particularly want to, so they sat with notepads. They went over each sentence time and again and wrote, at most, a paragraph or two each day. “I had this sense of realization: Ah, this is not going to be the usual thing, this is going to be something else,” said Danny. “Because it was funny.”

When Danny looked back on that time, what he recalled mainly was the laughter—what people outside heard from the seminar room. “I have the image of balancing precariously on the back legs of a chair and laughing so hard I nearly fell backwards.” The laughter might have sounded a bit louder when the joke had come from Amos, but that was only because Amos had a habit of laughing at his own jokes. (“He was so funny that it was okay he was laughing at his own jokes.”) In Amos’s company Danny felt funny, too—and he’d never felt that way before. In Danny’s company Amos, too, became a different person: uncritical. Or, at least, uncritical of whatever came from Danny. He didn’t even poke fun in jest. He enabled Danny to feel, in a way he hadn’t before, confident. Maybe for the first time in his life Danny was playing offense. “Amos did not write in a defensive crouch,” he said. “There was something liberating about the arrogance—it was extremely rewarding to feel like Amos, smarter than almost everyone.” The finished paper dripped with Amos’s self-assurance, beginning with the title he had put on it: “Belief in the Law of Small Numbers.” And yet the collaboration was so complete that neither of them felt comfortable taking the credit as the lead author; to decide whose name would appear first, they flipped a coin. Amos won.

“Belief in the Law of Small Numbers” teased out the implications of a single mental error that people commonly made—even when those people were trained statisticians. People mistook even a very small part of a thing for the whole. Even statisticians tended to leap to conclusions from inconclusively small amounts of evidence. They did this, Amos and Danny argued, because they believed—even if they did not acknowledge the belief—that any given sample of a large population was more representative of that population than it actually was.

The power of the belief could be seen in the way people thought of totally random patterns—like, say, those created by a flipped coin. People knew that a flipped coin was equally likely to come up heads as it was tails. But they also thought that the tendency for a coin flipped a great many times to land on heads half the time would express itself if it were flipped only a few times—an error known as “the gambler’s fallacy.” People seemed to believe that if a flipped coin landed on heads a few times in a row it was more likely, on the next flip, to land on tails—as if the coin itself could even things out. “Even the fairest coin, however, given the limitations of its memory and moral sense, cannot be as fair as the gambler expects it to be,” they wrote. In an academic journal that line counted as a splendid joke.

They then went on to show that trained scientists—experimental psychologists—were prone to the same mental error. For instance, the psychologists who were asked to guess the mean IQ of the sample of kids, in which the first kid was found to have an IQ of 150, often guessed that it was 100, or the mean of the larger population of eight graders. They assumed that the kid with the high IQ was an outlier who would be offset by an outlier with an extremely low IQ—that every heads would be followed by a tails. But the correct answer—as produced by Bayes’s theorem—was 101.

Michael Lewis's books