A thought about exit polls

One theme keeps emerging in the current discourse on exit polls. The media keep insisting that the public should draw no conclusions from them. They are not meant to provide an indication of who will win, or to judge the legitimacy of the election. They are not meant to be compared against the actual vote count. They are not presented as scientific surveys. In short, they have no actual meaning at all, and the public should in no way react to them. From the media's perspective, judging from all the articles I've read this week, exit polls exist solely to allow newscasters the option of "calling" a state for a particular candidate. They are for newscasters, not for the public. If this claim is true, does it not mean that exit polls are a form of fake news? If the information in them is not real, what is it? Are the media now admitting that exit polls are presented as a facade of information? Perhaps they are meant to trick voters into thinking that what they read and hear is news (facts), when in fact it is a performance solely designed to steer the public's eyes toward advertisements. Is that what an exit poll is for? Eye candy for advertisers? Maybe this is the real story that the mainstream media don't want to be exposed. Maybe the exit polls are a sham, and the bloggers have now discovered it. That is why the media must dismiss the bloggers. Bloggers are, paradoxically, nothing more than the general public. Bloggers are thus their very viewing audience. The "blogosphere rumor-mill" is the media's worst nightmare, because it demonstrates that the audience has caught on to the half-assed sham that is the associated press. "Rumor-mill". There's another issue. The blogosphere can never be a rumor mill, and here's why. A blog post repeats something seen on another site somewhere. Typically, the blog post also links to the site itself. Rumor mills delete and distort information as they propagate. But blogs actually add and organize information as they go, and every reader is directly connected to the source. The rumor-mill is an important concept to me, because I have in the past done some analysis on the editorial processes of the news media (especially newspapers). Traditional media is, in fact, a form of rumor-mill. The article is isolated from its sources through anonymity, processed through the unique perspective of the author, and then filtered by the influence of various editors and perhaps financiers. All of this filtering only distorts the information. By linking directly to the sources (as much as possible), and by combining and organizing relevant links with analysis and commentary, blogs can without a doubt provide a superior reporting of the news. For the mainstream media to call the blogosphere a "rumor-mill" is some form of neurotic projection of self-doubt.

Voting anomalies: probability is 1 in 250 million.

I found this report linked via BradBlog. Steven F. Freeman, a U. of Penn. professor, has applied a more thorough mathematical analysis to the exit poll results. Napkin calculations by myself and others had found a probability of .002% for the discrepancy (I think Freeman himself was one of the "others" who earlier reported that number). Based on a more careful analysis assuming Gaussian distributions, Freeman has arrived at a probability of one in 250 million for the exit poll discrepancies. The napkin calculations assumed a discrete Bernouli process, in which polling errors would skew toward Bush or Kerry like a coin toss (at least that's what I assumed in my estimate). This report is much better and more detailed than that straw-man piece of garbage from the Voting Technology Project. Some choice quotes from Freeman's article:

"As much as we can say in social science that something is impossible, it is impossible that the discrepancies between predicted and actual vote counts in the three critical battleground states of the 2004 election could have been due to chance or random error." "Stereotypically, Republicans are early risers." "Given that neither the pollsters nor their media clients have provided solid explanations to the public, suspicions of fraud or, among the less accusatory, 'mistabulation,' is running rampant and unchecked. That so many people suspect misplay undermines not only the legitimacy of the President, but faith in the foundations of the democracy. "Systematic fraud or mistabulation is a premature conclusion, but the election's unexplained exit poll discrepancies make it an unavoidable hypothesis, one that is the responsibility of the media, academia, polling agencies, and the public to investigate."

Meanwhile...

"What Dover has done goes much further than what's happened in Georgia," said Witold Walczak, legal director of the Pennsylvania ACLU. "As far as we can tell, Dover is the first school district that has actually mandated intelligent design."

Status Quo Quickly Maintained

Whew! That was a scary few moments of healthy skepticism and a demand for accountability, but, luckily, the Times has asserted its supreme moral authority and comprehensively dismissed any and all weblog-based notions of electoral fraud.

Fraud Theories Quickly Buried

excerpt:

Clay Shirky, an adjunct professor in the interactive telecommunications program at New York University, suggests that the online fact-finding machine has come unmoored, and that some bloggers simply "can't imagine any universe in which a fair count of the votes would result in George Bush being re-elected president."

I, for one, intend to revel in this unmooring, and I will wear my tin foil hat with pride on Inauguration Day.

Least Surprising Headline

"Libertarian Arrested for Tax Evasion"

PHILADELPHIA (AP) -- A Libertarian who lost a race for Congress on Tuesday has been arrested for evading $87,000 in federal income taxes. Arthur L. Farnsworth, 42, had vowed on his Web site that he would "never file an individual federal income-tax return again." A federal grand jury alleged the electrical engineer failed to pay taxes for three years on more than $221,000 in income and tried to conceal his earnings by transferring assets to fraudulent trusts and overseas bank accounts. Farnsworth, treasurer of the Pennsylvania Libertarian Party, drew fewer than 4,000 votes in his bid for a seat in the House of Representatives. Republican Michael Fitzpatrick won the race.

Open Voting Consortium

For those who may not be aware, there is something called the Open Voting Consortium. This looks like a pretty good organization, and I intend to join it (membership is $10). They are currently lagging in their goal of 1111 members by 11/11. I recommend that all five Tsujigiri readers should strongly consider joining this group. I would also recommend donating to any groups advocating open-source, open-design, and paper-record electronic voting systems.

Another Reprimand for the Bloggers

According to this article:

The fallacy of using exit polls as an exact predictor of outcome is that "you could be very accurate in terms of your sampling error for traditional polling," he said, "yet if you're off by a point or two, that could be the difference between a winner and a loser." "You have to understand that it's still just a survey, and it can't pick an exact number," Lenski said. "There's a few campaign workers in the Kerry campaign that know that now all too well."

Ah, yes, sampling error. Off by a few points in a close race. Yes, I get it. Bloggers clearly don't understand the nuances of statistical analysis. But what if the end-of-day tabulated exit poll results are off by 10% or more? What if the race didn't appear that close at all, but the loser some how pulled an extra 18% just out of evening voters? Do Bush's voters only come out at night? Are they uniquely adept at avoiding exit polls? Was there poor methodology in conducting the exit polls? Come on, mainstream media morons. Don't lecture bloggers about sampling error when you ignore the actual complaint. Let's talk about Pennsylvania. Give me a nice easy tutorial, so I can understand how "one or two points" of sampling error turns into a 20% after-hours explosion.

Caltech VTP releases "report"

The Caltech/MIT Voting Technology Project has released a "report" on the discrepancy between exit polls and the official vote count. The report concludes that suspicions of foul play are groundless based on the data. The report itself is, however, very lean on actual data and reads like a blog itself. It presents figures on the distribution of voting technology used in several states, which is good information. But its analysis is fundamentally flawed in several ways:

• It assumes that the early exit poll results are essentially useless, and should be pretty much ignored. This may or may not be true. The report does not give any evidence either way, so poll accuracy remains open to speculation.
• Second, they argue that if e-voting fraud is the cause, then we would expect to see more discrepancy in states with the highest prevalence of e-voting. This argument assumes that e-voting fraud is a random phenomenon with uniform distribution across states and precincts. If I were going to comit fraud, I would probably be smart enough to do it strategically, targeting only those states and precincts which would be most profitable for my candidate.
• Third, they assume that security breaches, if they occur, should be uniform across states and precincts. This sort of begs the question. If there is a security breach in one state, then we might expect to see a large anomalie in the vote tallies for that state. The probability of security breech is only loosely dependent on the prevalence of e-voting machines, because there are potentially several ways to breech security, including unauthorized access to central systems.
• Fourth, they don't actually state their assumptions clearly. They just sort of make them in a sloppy, bloggesque manner. The report is really an obnoxious read, in that it makes little substantive argument, but instead adopts a dismissive stance toward the blogosphere.

Slate agrees: publish everything.

This is from a Seattle Times article, quoting Jack Shafer:

Some critics believe the exit polls have become too proprietary. Instead of trying to keep the results secret, the poll results and methodology should be entirely above board, said Jack Shafer, the media critic for Slate, an online magazine that published leaked poll results on Tuesday. That way, he said, the public can be better informed about how to view poll results — and make a more informed decision about how to interpret them. "There is very little transparent about what they do," he said. "They are basically accountable to no one."

Black Eye for Blogging?

This article has an interesting take on the exit poll anomalies: it was all the bloggers' fault.

WASHINGTON (AFP) - Internet "blogs" ended up with egg on their faces this week after releasing early exit-poll data from Tuesday's vote suggesting John Kerry (news - web sites) was on his way to a victory against George W. Bush. Blogs, short for web logs, are online diaries, became a major phenomenon in the 2004 campaign. But the mistakes, while not of the magnitude of the 2000 election fiasco, opened a debate over the credibility of the sites and of the exit polls being used. Although the preliminary exit poll data were not widely used by television networks and other mainstream media, the misleading news spread like wildfire and even prompted a selloff late in the day Tuesday on Wall Street when it appeared Bush was in trouble.

That answers everything: the bloggers do not practice "journalistic standards" and thus they screwed up the data. There are a few things missing from the article. I'd like to see an actual explanation of normal exit poll patterns. Is it a good "journalistic standard" to monkey with the numbers at the end of the day? Is it "solid reporting" when exit poll "results" are manipulated to match the official numbers, and presented in summary form to the public? Are the bloggers (and the general public) unable to cope with the actual raw exit poll data, including time stamps? Poor stupid public. Silly bloggers. You just focus on the anchor-lady's boobs and let us do the math.

Wired Hints at an Explanation

This Wired article suggests that pollsters may be putting together a more detailed explanation of the exit poll results. The presentation is still a little paternalistic:

But the people who read these numbers -- among them, thousands of ordinary Americans with an intense interest in the election -- put too much faith into them and leaped to conclusions, said Bill Schneider, CNN's polling expert. "I think people believed them, and it's particularly the case with internet bloggers," said Kathy Frankovic, CBS News' polling director. "That's unfortunate because it sets up expectations that may or may not be met. I think it's a good exercise because it reminded people that early exit polls can be unreliable." Bloggers picked out different numbers to use for their purposes, said Joseph Lenski, who ran the poll with partner Warren Mitofsky for the NEP. As the day wore on, later waves of exit polling showed the race tightening. "Doing an early poll is like reporting the results of the game at halftime," Lenski said. "You only have about a third of the information. No other survey research is held to that level of accuracy." The NEP had enough concerns that its early exit polls were skewing too heavily toward Kerry that it held a conference call with news organizations mid-afternoon urging caution in how that information was used. Early polls in New Hampshire, Pennsylvania, Minnesota and Connecticut were then showing a heavier Kerry vote than anticipated. Pollsters anticipate a postmortem to find out why that happened. Some possibilities: Democrats were more eager to speak to pollsters than Republicans, or Kerry supporters tended to go to the polls earlier in the day than Bush voters. "The exit poll is one of several tools that AP uses to call races," said Kathleen Carroll, the news agency's senior vice president and executive editor. "After every election, we look back at how all our tools worked. We'll be doing that in the next few days with our election experts and our colleagues at the National Election Pool, and expect to be able to address any concerns in that process."

I have a suggestion: don't blame the bloggers. If you want people to "read the polls correctly" then publish the data with confidence intervals. This is something pollsters are loathe to do, because they don't like attaching explicit skepticism to their results. But I, as a small-time Z-list blogger, do not care about "journalistic standards." I care about scientific standards, which are designed to maximize caution in the interest of discovering facts. The mainstreamers publish a bunch of voodoo numbers and then complain that people "read them incorrectly." They then suggest that "maybe we shouldn't release any poll data at all." NO! Release all the data! We can handle it! Probably better than you can! Mainstream media = information miser.

Exit Poll Charts

I was a little surprised and overwhelmed by the discrepancy between exit polls and the official vote count in several states. I decided to verify the numbers myself, and produced some charts of my own. Mine are a little more detailed, showing all available poll timing information. They also have an overlaid line showing the change in votes (Bush-Kerry) throughout the day.

David Grenier's election-related Magnum Opus

Perhaps the optimist-realist-anarchist viewpoint is at least partially valid -- although he might not list the -isms in that order.