The Election of '36
Franklin D. Roosevelt ran for reelection against Alfred Landon in the
1936 presidential election. In all presidential races since 1916, the
Literary Digest, a popular
magazine, had correctly predicted the winner. However, it was wrong
in 1936 when the largest sample survey in history was conducted. The
magazine predicted that Landon would win, 57% to 43%. In
reality, Roosevelt won, 62% to 38%. What went wrong?
There were two major problems with this study.
- How the Digest picked their sample: The Digest
mailed questionnaires to 10 million people, but got only 2.4 million
replies. The names and addresses that made up the frame came from
sources such as telephone books, lists of automobile owners, club
membership lists, and the Digest's own subscription lists. This
sample, in effect, screened out the poor, who were unlikely to have
telephones or belong to clubs. (There were 11 million
residential telephones and 9 million unemployed in 1936.) In the
Digest surveys conducted prior to 1936, the rich and poor voted along
similar lines, but this was not the case in 1936. Therefore, the
Digest did very badly at the first step in choosing the sample;
selection bias created a tremendous error.
- How nonrespondents affected the sample survey: The
2.4 million respondents did not even represent the 10 million people
who were sampled, and, even worse, they did not represent the population
of all voters in the U.S. at that time. It is known that the lower-class
and upper-class tend not to respond to questionnaires. Thus in the
Digest poll the middle class was over-represented among respondents.
In summary, the Digest poll was spoiled by both
selection bias and nonresponse bias. The sample was not
representative of the population of American voters.
In 1936 a young man named George Gallup was just setting up a survey
sampling organization. By taking a random sample of 3,000 voters from
the Digest's list of voters, and mailing all of them a postcard asking
them how they would vote, he was able to predict Digest's predictions
within 1% error. Using another better-designed and
better-conducted sample of 50,000 he correctly forecast the Roosevelt
victory.