Essential Guidelines for Interpreting Political Polls
To analyze political polls effectively, understanding their conduct, sponsorship, and question wording is crucial. Transparency, credibility, and context play major roles in evaluating poll results. Ensure you know key details such as sample size, demographic information, and data collection methods to make informed judgments on credibility and newsworthiness.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Political Polling Political Polling EMILY SWANSON, ASSOCIATED PRESS NATALIE JACKSON, PRRI
So youre thinking of writing about a poll....
DISCLOSURE & TRANSPARENCY DISCLOSURE & TRANSPARENCY You should be able to figure out when, how and by whom a poll was conducted How many people were interviewed? When? What was the target population and how were they selected and interviewed? What questions were asked and in what order? If you can t answer these questions, you have no starting point to judge a poll s quality or newsworthiness
Who conducted the poll and Who conducted the poll and who sponsored it? who sponsored it? Is there a strategic reason for releasing this poll? Political campaigns sometimes selectively release polls that suggest their candidate is competitive Are the questions asked in a fair and unbiased way? It s always important to know a question s wording and context, but especially take care when the poll sponsor has a stake in the outcome
Do you support or oppose offshore drilling for domestic oil and natural gas resources? Poll s sponsor the American Petroleum Institute. Comes after: How important to you is producing more oil and natural gas here at home? How important is energy when it comes to the issues the federal government should be focusing on? Do you think the federal government does enough to encourage the development of oil and natural gas resources right here in the U.S.? Please indicate if you agree or disagree: Increased production of domestic oil and natural gas could lead to more jobs in the U.S. Increased production of domestic oil and natural gas could help stimulate the economy. Increased production of domestic oil and natural gas could help strengthen America s energy security. Producing more domestic oil and natural gas could help lower energy costs for consumers. Producing more domestic oil and natural gas could help strengthen America s national security by lessening the negative impacts of political instability occurring in other parts of the world Producing more domestic oil and natural gas could benefit federal and state budgets through lease payments, royalty fees and other sources of revenue.
How were people contacted How were people contacted to take part in the poll? Phone Online Mail In-person
PHONE POLLS PHONE POLLS Major considerations: Is everyone covered? 65% of American adults don t have a landline phone at home They differ from adults overall: 80-85% of those age 25-34 only have a cell phone, compared to 37% 65+ Where do phone numbers come from? Random digit dialing Voter lists IVR/ Robopolls
ONLINE POLLS ONLINE POLLS Probability based panels: Ipsos KnowledgePanel, NORC AmeriSpeak Panel, Pew American Trends Panel Opt-in panels: ongoing research and debate about ability to be representative of general population: Major players include Morning Consult, SurveyMonkey, YouGov, Ipsos (has both opt-in and probability) Web votes and email/text message campaigns
DIFFERENT ANSWERS ONLINE DIFFERENT ANSWERS ONLINE vs PHONE vs PHONE
Margin of sampling error plus or minus X percentage points The fewer people interviewed, the larger the margin of error Higher for subgroups like men and women, whites and African Americans, and likely voters What it means: For a poll with a margin of error of plus or minus 4 percentage points, we should expect the measured value to fall within 4 percentage points of the true value 95 times out of 100 BUT not the only source of error in polls: Question wording and order, refusal to participate, imperfect likely voter models, etc.
When is a candidate leading? A survey margin of sampling error applies to every candidate or poll response If the difference between two response options is more than twice the margin of error, then the poll shows one candidate is leading or one group is larger than another. If the difference is at least equal to the margin of error, but no more than twice the margin of error, then one candidate can be said to be apparently leading or slightly ahead, or one group can be said to be slightly larger than another. If the difference is less than the margin of error, the poll says a race is close or about even or that two groups are of similar size. Do not use the term statistical dead heat, which is inaccurate if there is any difference between the candidates. If the poll finds the candidates are tied, say they are tied.
TIMELINESS It s important to know when a poll was conducted! Consider if public opinions could have changed because of events occurring between when the poll was conducted and when it was released (or you re reporting on it) If so, the poll might still be usable but the context should be noted ( the poll was conducted the week before Congress passed the new health care legislation. )
PUSH FOR ANSWERS Pollsters don t always release all the details, but you re in a good position to ask The American Association for Public Opinion Research has disclosure standards that can be a useful tool to push reticent pollsters (and a good resource for you)
Poll aggregation: Benefits and drawbacks Easy to quickly get a sense of where public opinion stands Can be useful to compare to trends from individual polls: If one poll shows a change but the rest don t, the one showing a change might be an outlier Usually include polls of varying quality Important to look at multiple polls Look at the spread: If polls disagree, wording or methods may be instructive
Value of issue polling Often as important as horse race numbers. Helps to: Scientifically represent public attitudes, values, experiences Add scope, meaning to anecdotes Track society s changing attitudes Give voice to those not always heard
Where to get it Many national pollsters where you see horse race numbers are also great sources for polling on issues of the day Other good sources of issue polling include: Pew Research Center, Gallup, NORC, PRRI For great long-term trends on issues, the General Social Survey (GSS) is easy and accessible here: http://gss.norc.org/
Lessons of 2016 & 2020
What actually happened in 2016? National polls were actually pretty accurate by historical standards! Source: American Association for Public Opinion Research report on 2016 poll accuracy
But state polls were less accurate, especially in key states Trump ended up winning Source: American Association for Public Opinion Research report on 2016 poll accuracy
2020 national polls didnt fare as well Source: American Association for Public Opinion Research report on 2020 poll accuracy
Biden was overestimated in most states Source: American Association for Public Opinion Research report on 2020 poll accuracy
Why? Was there a shy Trump effect? Does Trump attract people unlikely to participate in polls? Turnout? Likely voter modeling? Expectations set by forecasting give a false sense of precision in polling Survey mode? A check-in from 2021 elections: o Virginia polling accurately showed a closing race, with Republicans favored in the final week o New Jersey polling didn t see the same trend occurring
The likely voter challenge illustrated
2022 2022
So whats going to happen in 2022? There won t be polls in every congressional district even the competitive ones National polls can give a rough indication of how many seats a party is expected to win
The generic ballot Measures support for an unnamed Democrat or Republican candidate for Congress, for example: If the election for the U.S. House of Representatives were being held today, would you vote for the Democratic candidate or the Republican candidate in your congressional district? For Democrats, it s not enough to be ahead: Forecasters and analysts estimate they need at least a 6 to 8 point advantage to win a House majority Most are now trying to measure likely voters. This can make a big difference if turnout heavily favors one party! It s still early! A sitting president s party often loses seats, but there s a lot of time between now and November
Resources AAPOR online course for journalists: http://www.aapor.org/Education- Resources/For-Media/Online-Course-for-Journalists.aspx AAPOR transparency initiative: http://www.aapor.org/Transparency_Initiative.htm Roper Center: https://ropercenter.cornell.edu/ National Council on Public Polls 20 questions : http://www.ncpp.org/?q=node/4 Polling Report pollingreport.com Pew Survey Basics: https://www.pewresearch.org/topic/methodological-research/survey- basics/
