Bloggers everywhere: 2; Ipsos Reid: 0

John Wright has issued a semi-substantive response to the corpus of data presented here on this blog.

Unfortunately for Mr. Wright, as he continued talking he revealed far more than he intended.  Now that he has been talking, he seems to have stumbled headfirst into what I believe is a contradiction against himself and I invite my readers to think critically for themselves about whether they share in this analysis.

In one of his initial comments here at Paulitics, John Wright claimed that the source for his polling data was “Elections Canada”.  The suggestion was, of course, that his data integrity was superior to mine and that therefore his numbers should also be more valid.  At the time that he wrote this, I thought it was a bit strange since I was unaware of Elections Canada publishing or even keeping a database of polls released by private polling firms.  I even did a search of the Elections Canada website, and found that in the last 12 months, Elections Canada hasn’t published a single article containing the word “Ipsos”.

However, Mr. Wright gave me enough material to respond to that I just let it be.  While I knew that it was unlikely, I also knew that it was certainly not impossible that I could be missing some polls from 2009 and thus I asked him for the first time to provide his source.  This week Mr. Wright is now claiming his source as nodice.ca and that his source this whole time has been not just 2009 polls but also some 2008 polls (presumably also from nodice.ca).

(makes Scooby-Doo “Ruh-roh” sound)

This is a big problem for Mr. Wright as nodice.ca contains nowhere even close to an exhaustive list of all polls.

The Paulitics Polling Resource and Wikipedia both documented a total 33 polls between the start of 2009 and when this whole episode blew up (two new polls have since been released, neither of which is by Ipsos Reid).  At only 12 polls, Nodice has less than half of all the polls conducted in 2009 and this is the source John Wright has been using (in addition to 2008 polls which I haven’t analysed once yet, but which I would be happy to take a look at.

Mr. Wright then states a complete and demonstrable falsehood.  He writes that

In the last of our 4 polls the NDP have been in the lower range (running 14-15 points) with EKOS and Harris Decima roughly the same and the others at the higher end. This is when Paulitics made the calculations. But just previous to that are 4 of our polls with the NDP running between 18-19 points. [emphasis added] (source)

Below is a list of the last four polls Ipsos Reid released in 2008.  See for yourself if the last four of Ipsos’s 2008 polls all have the NDP “running between 18-19 points”

Last four Ipsos Reid polls, 2008

If anybody is interested in seeing for themselves, the published sources for these four polls can be found here, here, here and here.  Furthermore, the October 9th, 2008 Ipsos Reid poll, even though it shows the NDP at 18 percent is actually still an outlier because it comes directly before a Nanos, Angus and Ekos trifecta of polls which put the NDP at an average of 20%; and it came on the same day as Segma poll which put the NDP at 21%, a Nanos poll which put the NDP at 22%, and an Ekos poll which put the NDP at 19% (source).

Finally, Mr. Wright again attempts to reiterate his claim that taking the unweighted averages of several unevenly-weighted averages is not problemmatic despite multiple, solid, irrefutable proofs to the contrary.  Mr. Wright also attempts to argue that his method of calculating averages is “perfectly legit and pretty much what the original author [i.e. me, Paul] did with his handful of polls in his charts.” (source)

I am truly sorry John, but this “new math” method calculating averages is neither legitimate nor has it been used by me.  Furthermore, since I’ve demonstrated why this technique is not “perfectly legit”, I think this argument is just a particularly sad attempt to attack the messenger when the data and the arguments presented here speak for themselves.

So, to reiterate:

  1. Mr. Wright seems to have forgotten his original source for 2009 polls (originally it was elections canada, now he says it’s nodice.ca)
  2. Mr. Wright then admits that his numbers are actually based on a database (nodice.ca) that contains less than 50% of the pertinent data
  3. (perhaps because of this) Mr. Wright has been unable to provide a single poll which I have missed in my data.
  4. Mr. Wright then demonstrates a shocking lack of knowledge about the polls that his own firm released in late 2008.
  5. Mr. Wright then proved unable to articulate a single methodological fault in any of my analyses.
  6. Mr. Wright then admits that he has supplemented his 2009 data with 2008 polls
  7. Mr. Wright then reiterates once again that taking the unweighted averages of several unevently-weighted averages is not problemmatic despite multiple instances in which I have provided mathematical proof to the contrary

I’ll tell you, Mr. Wright is the gift that keeps on giving.  Swing and a miss, Mr. Wright, swing and a miss.

About these ads

26 Responses to “Bloggers everywhere: 2; Ipsos Reid: 0”


  1. 1 john.wright@ipsos.com 26 June, 2009 at 9:30 pm

    Oh please…

    You are obsessive compulsive.

    Whatever you produce is falacy on this.

    Good luck to you…

  2. 2 janfromthebruce 26 June, 2009 at 10:36 pm

    I trust your polls and personally I am finding this all weird – him and not you.

  3. 4 marcel 27 June, 2009 at 12:24 am

    Little Elvis.

  4. 6 marcel 27 June, 2009 at 12:39 am

    Obssesive compulsive?

    Blogging on the internet is a normal thing to do. Some graduates actually learned something in school, and enjoy using it.

    You picked a fight with the wrong guy this time, Mr. Wright.

  5. 7 John Wright 27 June, 2009 at 12:41 am

    Marcel…any place, any time…

    You all make me laugh…

    Thanks so much.

  6. 8 John Wright 27 June, 2009 at 12:52 am

    “Mr. Wright then demonstrates a shocking lack of knowledge about the polls that his own firm released in late 2008.”

    Stop it please…the laughing hurts so much…oh my gawd…I’m going to split in half…I did all of these polls myself…I’m so shocked…that you continue this…thanks so much…this is the best laugh I have had in ages…

    So, what’s your e-mail? What’s your phone number? What’s your address??

    What, you aren’t a coward are you???? Show us where you are…you have to have more than that “Paul Bernardo” lookalike picture, don’t you???

    C’mon, fess up…

  7. 9 john wright's mother 27 June, 2009 at 6:48 am

    Johnny boy, you’re starting to come across like the insufferable douchbag you were when you were a child. Please, take your meds.

  8. 10 "John Wright's" IP address 27 June, 2009 at 9:58 am

    Actually, who is Paul? He seems like the kind of guy who could use a good douche vaginale himself…

    Webmaster’s note: This comment was written with the same IP address as John Wright but was originally written under the pseudonym “Condor”

  9. 11 john zogby 27 June, 2009 at 11:40 am

    “You all make me laugh…”

    Well that’s ironic, since the more you comment, the more you look like a complete joke. Adolescent.

    BTW paul, another sight threehundredeight.com also did analysis and found Ipsos favors the Cons too. Also, I think you should go back to the 2004 and 2006 election, because the bias is quite evident there, massive Con over support. Maybe it’s bad methodology. I tend to believe it’s a problem with the pollster, because if this guy is running the show, it seems a pretty low rent operation.

  10. 12 John Wright 27 June, 2009 at 12:23 pm

    “This comment was written with the same IP address as John Wright but was originally written under the pseudonym “Condor””

    It may have come from the same IP address but I didn’t write it nor would I.

    Please remove my name from it. Thanks.

  11. 13 Saskboy 28 June, 2009 at 1:24 am

    What’s the difference? It’s not much more embarrassing than the one suggesting Paul looks like a famous serial killer. Or did you not write that one too?

  12. 14 Pro bono publico 28 June, 2009 at 9:30 am

    Mr. Wright,

    I’m saddened that you feel compelled to assert your claims with so much arrogance and so little fact. You’re not only discrediting yourself and your company; you’re beginning to discredit the whole system of polling upon which your clients depend for their public relations and propaganda.

    It’s unfortunate to see your defence degrade into ad hominem attacks that include ableist slights against people with OCD (“You are obsessive compulsive./Whatever you produce is falacy [sic] on this.”). How ironic that your line of argument is fallacious.

    Look at your own obsessive compulsiveness: A VP, resting on the perch of his big polling firm, obsessing over a little mouse of blog. And the compulsion to return to this perch and squawk at this mouse, hoping that one day it will just die so that you can swoop down and silence its squealing once and for all. And what’s with splitting your identity between “John Wright” and “Condor”? A vulture of all things? Not very subtle, Mr. Wright. Considering your odd behaviour, you might want to think twice before knocking people’s mental abilities again.

    Besides, people of all kinds of abilities can conduct sound analysis. In fact, your own credibility depends on this while you project your failures on Paul and exhibit reaction-formation by laughing at everyone you feel is laughing at you.

    This is the best laugh you’ve had in ages? Really? You’d be the first human being to enjoy laughing alone when no one else agrees with them (even your own company told you to shut up). No wonder “the laughing hurts so much” that you’re “going to split in half.”

    You sure are exhibiting signs of schizophrenia.

    On one hand, there’s John “Right”; the knowledgeable and competent VP of a big polling firm who exudes confidence because he always has the stats to back him up.

    And then there’s John “Wrong”; the helpless, infantile little emperor with no clothes who resorts to name calling when someone he perceives as getting more sex than him (hey, you made the Paul Bernardo reference) points out that two plus two does not equal five. How dare Paul — the rapist of statistics — tell John Wright — the father of statistics — that two plus two does not equal five?!? How dare he?! Paul’s so funny! Look at him with all those quaint charts and figures! And everyone loves him for it! But… but… but I have my own polling company! How come everyone thinks his polls are more solid than mine? Anyone that obsessed with stats must be a rapist! That coward! Where does he live? Everyone get your axes and pitchforks and… lawyers! We’re heading to Paul’s house to end the threat he is to my polls!

    Your polls are looking a little flaccid these days, Mr. Wright. And when compared to all the other polls out there, your blue poll isn’t as big as you think it is.

    This will be very difficult to accept at first. The threat to your ego will be so immense that you’ll repress your shortcomings. All that will seemingly remain is your cocky self-assurance. But your feelings of inadequacy will continue to leak through projection, reaction-formation, and laughter that masks the pain you’re feeling deep down in that part of yourself you won’t dare acknowledge to people who are scoring points off you.

    But there’s good news!

    The denial and acting out need not continue in order to protect John Wrong.

    Instead of reacting, you can develop healthy responses that might alleviate your polling dysfunction:

    Either produce a reasonable rebuttal that John Right is capable of;

    Or, if John Wrong and John Right are really one person, accept you’re Wrong this time, incorporate this failure into your whole being, use it to make your polls more Right in the future, and go on as a reconstituted John Wright.

    Good luck, Mr. Wright.

  13. 15 Ti-Guy 28 June, 2009 at 5:02 pm

    I think the blogger here should ascertain first whether any of these “John Wright’s” is authentic.

    I’m having a hard time believing that, although seeing Mr. Wright go from his usual petulance to appalling childishness doesn’t strike me as much a stretch.

  14. 16 marcel 28 June, 2009 at 5:30 pm

    “Marcel…any place, any time…”

    Well, I sure would never want to meet a blade-carrying ghost anywhere, any time. Fortunately I don’t have to.

    Anyways, I am not your problem…and I never was. Except for maybe earlier, in previous threads, when I was outright outtaline and rude.

    But we’ve both moved on since then.

    So imagine you’ve dropped me. I say “uncle”, you are still mad so you lay the Is it good enough yet? Am I out enough? Well, you know what comes next anyways so I won’t inform you of it.

    There, I’m gone. Do you see your real problem yet?

  15. 17 marcel 28 June, 2009 at 5:32 pm

    “Marcel…any place, any time…”

    Well, I sure would never want to meet a blade-carrying ghost anywhere, any time. Fortunately I don’t have to.

    Anyways, I am not your problem…and I never was. Except for maybe earlier, in previous threads, when I was outright outtaline and rude.

    But we’ve both moved on since then.

    So imagine you’ve dropped me. I say “uncle”, you are still mad so you lay the boots to mu temples. Is it good enough yet? Am I out enough? Well, you know what comes next anyways so I won’t inform you of it.

    There, I’m gone. Do you see your real problem yet?

  16. 18 Tron 4 July, 2009 at 10:46 pm

    I am new to this blog and a little late to this conversation, but
    please allow me to raise a question or three that I feel need raising.

    First though, the reason: I’m not sure if Paulistic is implying that there is an INTENTIONAL bias in Ipsos polls (survey design, sampling, weighting), but there are certainly a number of folks in the Comments Section who are clearly in this camp.

    Legally speaking, I’d advise you to be careful of such an accusation. Logically speaking, well, here are my questions:

    1 What would be in it for Ipsos to intentionally bias the vote intent survey design and/or weighting? What would be the beneficial motive?

    2 What are the drawbacks?

    3 What is the gain-loss analysis?

    The short answer: There is little-to-no benefit or good reason. The risks far outweigh any possible benefits for a company. Remember that they are a business trying to make money.

    I think that any accusations of intentional bias to their vote intent question design are illogical. And, if you think out reasonable answers to these questions, I think you’ll come to the same conclusion.

  17. 19 paulitics 5 July, 2009 at 9:04 am

    Hi Tron, thanks for your questions.

    I should start off by saying that I want to be very clear that I’m not ascribing intentionality to my data. My personal inclination is that it is spectacularly unlikely that Ipsos Reid would do this deliberately. I suppose you could make an argument that the National Post commissions these polls and thus there will naturally be a symbiotic relationship between supplier and buyer which could make it beneficial for them to present a pro-Tory bias. However, I don’t think this argument is particularly strong in part because I can’t prove it and in part because I think it confuses causality (i.e., it would seem to me more reasonable that Ipsos Reid is the National Post’s pollster because they tend to favour the Conservatives rather than them favouring the Conservatives because they are the Post’s pollster).

    http://en.wikipedia.org/wiki/Questionable_cause

    That doesn’t mean that there aren’t structural reasons that could have a business logic which could explain a resultant party bias. For instance, maybe a polling firm doesn’t want to put the money into hiring a polyglot workforce of interviewers. Maybe they only (or almost only) staff English and French interviewers and don’t want to pay the premium it would cost to hire interviewers who speak a third and fourth language. This could have a serious bias effect if non-English and non-French speaking communities tend to support particular parties over others (which, as an aside, I think would be very likely).

    Maybe getting French-speaking interviewers is harder than getting English-speaking interviewers (people who can speak French, generally command a higher salary if they can also speak English). A polling firm having difficulty acquiring Francophones may over-hire English-speakers which would thus negatively impact on their response rate for French Quebecois and positively impact on their response rate for English Quebeckers. This could have a serious bias effect if English Quebeckers tend to vote differently than French Quebecois (which they do).

    Maybe a polling firm wants to hire high school and university students because they know they can pay them considerably less than people with careers or with university degrees. Well, if your workforce is comprised primarily of this type of interviewer, that polling firm is going to be over- or fully-staffed in evenings and on Fridays (most universities in Canada have fewer classes on Fridays). This could have a serious bias effect if people who are home during the day and who work on Fridays and in evenings tend to support particular parties over others (which I also think would be very likely).

    I hope I answered your questions. It’s always great to hear from a new reader of Paulitics.

    Cheers.

    Paul

  18. 20 Michael 6 July, 2009 at 8:55 pm

    With respect to your orginal post, Paul, I went to the nodice site and I am seeing well over 30 total polls for 2009. Perhaps they have rectified an error or glitsch in the interim.

    I guess I was one of the first (way back) to call attention to differences between polling companies, in a comment on your polling resource, and I have commented on this since in another thread.

    As a scientist, I am really interested to know what accounts for these differences. It may be very subtle differences in wording of questions, or in methodology as you suggest above. It is not a question of who is “right” as there is no way of determining that – even an exhaustive large scale poll on election day would give different results from the actual election (as exit polls have shown in some countries) because the population being sampled need not be the same, and because people spoil ballots, change their minds at the last moment, decide not to vote for various reasons, and lie to pollsters.

    What I don’t like about current polls is that they include often “leaning” votes, which are more likely to be from people unlikely to vote at all. Polls should be adjusted to give more weight to people who say they are 100 % likely to vote, and voted in the last election, if that is possible mathematically.

  19. 21 Tron 6 July, 2009 at 10:52 pm

    Paul,

    The structural items you mention are essentially identical and/or not actually issues at all polling firms: They all field English and French only surveys and employ low-wage interviewers who are everything from students and the elderly, to the underemployed and artistic set. And, all of the firms have dedicated call centres in Quebec to conduct the Quebec portion of the polls in French. And, they have no trouble staffing their call centres and fielding surveys on any night of the week. So, these items being equal across the polling firms, there is no explanation therein.

    Moreover, I would highlight for you the fact that they are a number of different methodologies being used across the firms – some of which do not rely upon telephone call centres. Some are using online exclusively. Some are using telephone on some occasions and online on others. Some are using automated interactive recordings.

    I would posit that sampling and execution approaches as well as weighting techniques and questionnaire design (although there is not a lot of difference in either across the firms; just look at their releases) are the most likely causes of any differences you are finding.

    But, strictly speaking, you cannot do a straight-up comparison of these polls because they are all done differently – method of execution, dates when conducted (remember: each poll is a snapshot), sample sizes and associated margins of error, question, weighting. That’s quite a strict, scientific, academic approach to this, I agree. But, you were taking quite a strict approach to John Wright’s average-of-averages which I read as something that he was quickly providing as a rough yardstick. And, from my perspective, over-focusing on this to such an extent that it took this whole discussion far off-topic. If you can compare polls with such varying components, then he can calculate an industry average. It’s all rough math.

    And, to be frank, I would argue it’s pointless. As Michael says above, no one can say who is right. Period. More importantly, you’re talking about a few percentage points here and there that are often accounted for by the margin of error of each poll.

    Overall, I do think this whole sordid debate is mountains of molehills. What is interesting and impressive is just how similar they all are in their results, regardless of the method used. And that, I think, is the most important point about all of this that this discussion threatens to obscure.

  20. 22 Michael 27 August, 2009 at 3:08 pm

    Just saw this quote from the globe and mail on the Nanos site today:

    “Niks polls are well known for being fair and accurate. It doesn’t appear that they are all that way.

    Source : Globe & Mail
    It’s not often you’ll find Prime Minister Stephen Harper at a media bash. But there he was the other night pumping hands at an Ottawa fete celebrating the expansion of the CanWest Global news organization.

    No surprise, therefore, that the Prime Minister showed up. The conservative presence in Canada was being further entrenched — and in no small way.

    The CanWest expansion will secure, among other things, a larger media presence for Western Canada, which has understandably chafed at Eastern bias. “It’s almost ironic,” said CanWest’s CEO Leonard Asper, “that we have a Western-based company in CanWest that has the only national newscast coming out of Ottawa.”

    The Aspers have made a big leap. After purchasing the Southam News chain and the National Post from Conrad Black, they looked out of their depth and were roundly criticized for their management practices. Now they are growing in strength and impact.

    Their continued ascendancy is a major blow — tilt the message and you gradually tilt the mind — to the left and to moderates. The Aspers make no bones about their conservative bias. It is evident every day in their national flagship paper, The National Post, which regularly runs the likes of Charles Krauthammer.

    Currently, polling is a source of some contention. CanWest has been running polls by Ipsos-Reid, which have consistently been giving the Conservatives a showing five or six points higher than other pollsters. Ipsos pollster John Wright, who says he’s been getting heat on this from bloggers, says there is no bias and anyone who makes such a suggestion will be sued. Other pollsters I talked to aren’t suggesting any bias. They just find the discrepancy “strange.”

    You alter the character of a country by changing how it sees itself. You can change how it sees itself by changing the character of its media. Led by the Aspers, the character of Canadian media is changing.

    [updated Sun Aug 23 19:05:08 EDT 2009]“

  21. 23 paulitics 27 August, 2009 at 7:10 pm

    Thanks for that heads up Michael!

    I actually hadn’t read that Globe and Mail piece until you pointed it out to me.

    I wonder if that Globe & Mail author would be interested in doing a follow-up piece.

  22. 24 Simon 17 September, 2009 at 9:57 am

    An Ekos poll released today might explain a discrepancy between polling companies:

    Contrary to some reports, the New Democrats are holding up reasonably well as they face increasing media attention, as one of the parties that will determine whether the Harper government stands or falls in the next few weeks.

    “We have found that the NDP does relatively better among people in cellphone-only households,” said Frank Graves, President of EKOS, “while the Conservatives do much worse. This may account for some of the discrepancy among recent polls with regard to the party.”

    Source:

    Note:

    In the last month, only Ipsos has polled the NDP below 14% (as low as 12% just this past Sunday). All other companies have polled the NDP between 14 and 19 percent.

    Also, all companies in the last month, except for Ipsos, have polled the Conservatives between 31 and 37.5 per cent. Only Ipsos pegged the governing party at 39 percent in both of their polls in the last month.

  23. 26 Seesveils 17 March, 2010 at 8:23 pm

    Im looking in to getting one, who has got experience of one of these:

    http://video.libero.it/app/play/?id=55780c06250821715ad8607b7758356a


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




Resources:

home page polling resource

Click below to download the

Paulitics Blog Search

Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 2.5 Canada License.

DISCLAIMER: The views and opinions expressed in the comments section beneath each post on this blog do not necessarily reflect the views and opinions of the blog's author and creator. Individual commentators on this blog accept full responsibility for any and all utterances.

Reddit

Progressive Bloggers

Blogging Canadians

Blogging Change

LeftNews.org

Paulitics Blog Stats

  • 790,643 hits since 20 November, 2006

Follow

Get every new post delivered to your Inbox.

Join 34 other followers

%d bloggers like this: