A poll (fl. 1948- ) is a survey of public opinion. You see them in the news, especially during election campaigns where they give a rough idea of who is ahead and who is behind and by how much.
Pollsters, who carry out the polls, do not ask everyone in the country. Instead they ask a random sample of hundreds or thousands of people, people chosen by chance from across the country. By asking just 400 people in a country the size of the US there is a 95% chance you will be within 4.9 percentage points of the number you would get if you asked everyone. Pretty amazing. That margin of error can be brought down by asking more people. Asking 800 brings it down to 3.5 percentage points. Asking 10,000 brings it down to 1 point.
Because some people are harder to reach than others, like young people or people of colour, most pollsters demographically weight the answers they get, like by age and race, so that it matches the country as a whole.
The better polls tend to be:
- done by top news outlets;
- done by pollsters you have heard of;
- done using live telephone interviews instead of robocalls or online surveys.
Note that some polls are fake!
Here are some US pollsters rated by how well they have done in the past (according to FiveThirtyEight.com):
- A+ Monmouth University
- A+ ABC News/Washington Post
- A Marist
- A- CNN
- A- CBS News/New York Times
- A- NBC News/Wall Street Journal
- A- Los Angeles Times
- A- Time
- A- Quinnipiac
- A- Ipsos
- B+ Public Policy Polling
- B+ Pew
- B Fox News/Opinion Dynamics Corp
- B YouGov
- B- Gallup
- B- NPR
- B- Gravis
- B- RAND
- C+ Rasmussen
- C- Zogby
- C- SurveyMonkey
Even polls done by the best pollsters will show a consistent bias or house effect. For example, in the 2016 US election, NBC polls favour Hillary Clinton, those by the Los Angeles Times favour Donald Trump.
Some try to “unskew” a given poll. That is generally a waste of time. Instead:
An average of polls is way better than any single poll. That is because:
- The sample size of all the polls together will be greater and the margin of error therefore lower.
- The house effect of different polls will tend to cancel each other out.
For US elections, Real Clear Politics and FiveThirtyEight keep a running average of polls.
Polls done closer to election day are more likely to be right.
The Bradley Effect is where people lie to pollsters so as not to seem racist. It seems that was true in the 1980s and 1990s, but not since.
Crowd size: Some say the polls cannot be right because they do not match how many people come out to see candidates speak. In the US in early 2016, for example, way more people came out to see Bernie Sanders than Hillary Clinton. But Clinton still won – just as the polls said she would. Polls, for all their faults, are still the best measure we have before election day.
– Abagond, 2016.
Sources: mainly FiveThirtyEight (2016).
- FiveThirtyEight ratings of pollsters
- Nate Silver
- 2016 election for US president
- Jacob Bronowski: Knowledge or Certainty – has a bit on Gauss and the uncertainty of measurement.