Thursday, October 1, 2015

The Problems with Polls - Often Wrong and Not Representative.

Get out the Vote and Vote with the Courage of your Convictions

Don't let this Happen:  

UK Election 2015 - Polls completely wrong.
Who answers polling phone calls on their cell? 
I am still a relic that has a landline & a cell phone & I hang up or don't answer Polling calls.  Caller ID allows everyone to screen calls.
  • Much of the population don't have land-lines.
  • Many Canadians (on landlines, VOIP, cellphones) have call screening.
  • Cell phone only citizens don't want to waste money on minutes answering poll questions.
  • Most cell-phone-only citizens don't want to waste their battery charge on answering poll questions.
  • Most cell-phone-only citizens represent a younger demographic. 
  • 9% is considered a good response rate (by Nanos) - so that means ***91%*** of people hang up, or don't answer. You have to wonder how those 91% would have responded.
  • So, even aggregates of polls are still not contacting 91% of their target. 
  • We no longer have a Mandatory Long Form Census, so it's difficult to discern the demographic of the Cdn population. 

    In A First Past the Post System, Polls do not translate into seats won eg:  Nanos poll May 02 of 2011 put Liberals at 20.5% support but that didn't translate to the 34 seats they won which represented only 11% of the seats in the house 34/308
 Link: From Michael Harris's article: Even Nick Nanos is reported to have run biased polls.

How do pollsters even know what Canada looks like demographically? Thanks to the Conservatives, we know longer have the statistics from the Mandatory Long Form Census to answer those questions.

Link: From Laurier Prof:  The Polls are Bad -- Their Accuracy, that is. 
Polls are often designed to achieve a specific outcome and are sometimes more of a marketing strategy. Consider who owns the Mainstream Media and which political parties are in-line with their corporate agenda. 
  1. The problem with modern polling, in one chart: http://www.washingtonpost.com/blogs/govbeat/wp/2014/03/12/the-problem-with-modern-polling-in-one-chart/
  2. Link: The Future of Polling is Hard to Predict - Just look at Alberta Election in 2012
  3. Link: "Good" Polls / "Bad" Polls -- How Can You Tell?: Ten Tips for Consumers of Survey Research
    1. Know the Purpose and the Sponsor of the Study
    2. Know Who was Interviewed
    3. Know How the Survey Respondents Were Selected
    4. Know How Many People Were Interviewed
    5. Know the Exact Wording of the Questions Used
    6. Know the Order and the Context of the Questions
    7. Recognize That There Are "Opinions" and "Non-Opinions"
    8. Check the Interpretation of the Data
    9. Recognize That What Was Not Reported May Be Important
    10. Be Familiar with the Reputation of the Organization Conducting the Study 

      BC Provincial Election 2013
      BC Prov Election 2013 - Polls completely wrong.
       
  4. Cell Phones and Political Polls: http://www.factcheck.org/2008/02/cell-phones-and-political-polls/ 
  5. In a Twitter exchange with a Major Pollster this is what I found out:
    1. Cell phone numbers are gathered from RDD (random digit dialing) sample of all cell phone exchanges in Canada. A telephone exchange determines if the number is land-line or cell.
    2. 9% is considered a good response rate (by Nanos) - so that means ***91%*** of people hang up, or don't answer. You have to wonder how those 91% would have responded.
      Nanos Poll Technical Notes - pretty short on details
    3. Thanks to the current government we no longer have a mandatory long form census so many of the statistics that tell us the demographics of Canada don't exist.
    4. Even if these polls were representative (they aren't) they only try to project popular opinion, which is vastly different than seat count (because of our first past the post electoral system). Ie The leader in the polls may not get the proportional number of seats. 
    5. Even Nick Nanos has been accused of bias in polling: http://tinyurl.com/qg948ld
    6. Nanos numbers (as well as most political campaign polls) are based on a rolling average NOT a single shot in time.
      The organization that pollsters like to follow for standards is: Link: World Association for Public Opinion Research
  6. Cell Phones and Election Polls: An Update: http://www.pewresearch.org/2010/10/13/cell-phones-and-election-polls-an-update/
  7. 20 Questions A Journalist Should Ask About Poll Results: http://www.ncpp.org/?q=node/4
    1. Who did the poll?
    2. Who paid for the poll and why was it done?
    3. How many people were interviewed for the survey?
    4. How were those people chosen?
    5. What area (nation, state, or region) or what group (teachers,lawyers, Democratic voters, etc.) were these people chosen from?
    6. Are the results based on the answers of all the people interviewed?
    7. Who should have been interviewed and was not? Or do response rates matter?
    8. When was the poll done?
    9. How were the interviews conducted?
    10. What about polls on the Internet or World Wide Web?
    11. What is the sampling error for the poll results?
    12. Who’s on first?
    13. What other kinds of factors can skew poll results?
    14. What questions were asked?
    15. In what order were the questions asked?
    16. What about "push polls?"
    17. What other polls have been done on this topic? Do they say the same thing? If they are different, why are they different?
    18. What about exit polls?
    19. What else needs to be included in the report of the poll?
    20. So I've asked all the questions. The answers sound good. Should we report the results?  
Alberta Provincial Election 2012 - Polls completely wrong.





2 comments:

  1. This is Simple, "He who pays the piper calls the tune!" Doesn't this tell us the answer to many things that are happening in this election cycle. Cuts to CBC, appeals to the commercial broadcaster, who panders to business. More of the polls respond to who commissions them.

    ReplyDelete