The Election of '36

Franklin D. Roosevelt ran for reelection against Alfred Landon in the 1936 presidential election. In all presidential races since 1916, the Literary Digest, a popular magazine, had correctly predicted the winner. However, it was wrong in 1936 when the largest sample survey in history was conducted. The magazine predicted that Landon would win, 57% to 43%. In reality, Roosevelt won, 62% to 38%. What went wrong?

There were two major problems with this study.

  1. How the Digest picked their sample: The Digest mailed questionnaires to 10 million people, but got only 2.4 million replies. The names and addresses that made up the frame came from sources such as telephone books, lists of automobile owners, club membership lists, and the Digest's own subscription lists. This sample, in effect, screened out the poor, who were unlikely to have telephones or belong to clubs. (There were 11 million residential telephones and 9 million unemployed in 1936.) In the Digest surveys conducted prior to 1936, the rich and poor voted along similar lines, but this was not the case in 1936. Therefore, the Digest did very badly at the first step in choosing the sample; selection bias created a tremendous error.
  2. How nonrespondents affected the sample survey: The 2.4 million respondents did not even represent the 10 million people who were sampled, and, even worse, they did not represent the population of all voters in the U.S. at that time. It is known that the lower-class and upper-class tend not to respond to questionnaires. Thus in the Digest poll the middle class was over-represented among respondents.
In summary, the Digest poll was spoiled by both selection bias and nonresponse bias. The sample was not representative of the population of American voters. In 1936 a young man named George Gallup was just setting up a survey sampling organization. By taking a random sample of 3,000 voters from the Digest's list of voters, and mailing all of them a postcard asking them how they would vote, he was able to predict Digest's predictions within 1% error. Using another better-designed and better-conducted sample of 50,000 he correctly forecast the Roosevelt victory.