By Linda Sutton
We are being subjected to the continual release of poll after poll. It becomes an emotional roller coaster, as we are either happy or sad with the resulting numbers. But should we pay attention to them?
Each poll is a snapshot in time. Each is controlled by the METHODOLOGY used in the sample of people who are polled, as well as the questioning techniques employed.
Online you can get an in-depth education on polling. Aapor.org comes up first. Then the New York Times and Pew. The following is a brief overview.
Each poll selects a group of people to record their opinions and views. How large the sample is matters, and the selection process is critical. In probability sampling, the gold standard until recent years, “all persons in the target population have a chance of being selected.” From these results, the poll extrapolates to the entire population AND can calculate a margin of error for the poll. Usually, the larger the sample size, the smaller the margin of error.
A main problem with this type of polling is that only those who agree to answer the questions are included. And, because the polls are by phone, specifically landlines, the population polled has skewed older and more settled. As in, how many younger people have landlines now? I’m not sure the extent to which cell phones have become part of the mix on sample selection.
Some of the most egregious polls are those done with non-probability sampling in which people are NOT selected at random, but opt in to participate. This could be by email, but more likely by a website. The results in this type of polling are inevitably biased and do NOT represent the population at large. There is also no way to calculate a margin of error. They are usually called STRAW polls and cannot be relied on at all because of the inherent bias. A good for instance would be the Daily Kos polls. Most of these polls should be ignored.
In addition to sampling techniques, the QUESTIONS asked matter and can be manipulated by wording or sequencing of questions. To get to these, you have to go to the website of the polling firm. Usually, the size of the poll is disclosed in media reports.
So, next time you are subjected to the latest poll results, look beyond the numbers to HOW the polling was done, size of the sample, and the polling firm that did it. There are some that skew conservative and others who can be more trusted to give accurate results. But again, each poll is only a snapshot of that moment in time and can flip quickly as we’ve seen throughout recent elections.