Challenges with Surveys
Thursday, August 28th, 2008Surveys are a very valuable resource for organizations whether they are businesses, non-profits, government agencies, or political parties. They help the surveyor when they assess the population of interest by identifying the interests, characteristics, perceptions, likes and dislikes, and expectations of those who fill out the surveys as long as the research process is well designed and executed. This means that a random sampling process is used in order to avoid biased results, the survey questions are well thought out with an eye on the hypothesis tests that will be conducted, and the analysis and interpretation of the data is carried out correctly. Problems do arise, though, if the researcher does not completely have access to the population they are interested in. Here is an important example from the 2008 election campaign.
I have selected two articles in the New York Times about the impact of increasing cell phone usage on the ability of political pollsters to assess the opinions of voters. The problem is that these pollsters are not able to as easily contact voters who use cell phones, and as a result this raises issues about whether or not they are clearly surveying the population of interest. One article, written by Megan Thee appeared in the December 7, 2007 issue of the New York Times (http://www.nytimes.com/2007/12/07/us/07polling.html?scp=14&sq=political%20polls%20&%20cell%20phones&st=cse). The author points out the fact that public opinion researchers have historically relied on their home land-line telephones which have the advantage of being geographically defined by the area codes. Cell phones are not as clearly geographically based, and those who use cell phones are less likely to participate in a survey. After all, you pay to use your cell phone by the minute.
The problems associated with cell phone usage go beyond these points. Few drivers on the road would want to see those around them on their cell phones completing a survey, including political polls. Those who primarily use cell phones tend to be younger and they are usually less likely to vote possibly making the issue less important, but is that true in 2008? The Pew Research Center completed a survey showing the differences in the political views between land-line users and cell phone users are not significant and as a result not adversely affecting the results of political polls. The issue is that the number of people, especially younger people who rely on cell phones, is growing. The problem is constructing a survey design to remedy these concerns. An article by the same author that appeared on July 23, 2008 (http://thecaucus.blogs.nytimes.com/2008/07/23/cellphone-only-it-holds-little-sway-in-polls/?scp=6&sq=political%20polls%20&%20cell%20phones&st=cse) follows up on these points. She reported that at that time pollsters were regularly undertaking cell phone surveys to increase the accuracy of their polls. The Pew Research Center has found that the “cell phone only” cohort differs greatly from the general public, but the “cell phone mostly” cohort does not. The “cell phone only” cohort more strongly supported Barack Obama than the general voting public, but is considered to be less likely to vote.
See the following as examples of opinion pollsters to learn more about their process; Pew Research Center (http://people-press.org/), Gallup (http://www.gallup.com/home.aspx), Harris Interactive (http://www.harrisinteractive.com/), Kaiser Family Foundation (http://www.kff.org/), Survey Research Center (http://www.src.isr.umich.edu/), and Zogby (http://www.zogby.com/).
