Polling Results Within the “Margin Of Error” Are Not a “Statistical Tie.”

Political Myth: If the difference between two candidates’ polling numbers is less than the “margin of error” for the poll, then it’s a “statistical tie.”

Reality: Even if the results are “within the margin of error,” the candidate with higher polling numbers is still most likely to be in the lead.


Suppose you have a bag with 100 million balls.  Some of them are blue, and some of them are red.  You want to know how many of your 100 million balls are red and how many are blue, but it would take too long to count them all.  So, you thoroughly mix up the balls in the bag, close your eyes, and pick out 500 balls at random.  By counting those 500 balls, assuming you truely did pick them randomly, you can come up with a pretty close approximation of how many of the 100 million balls in the bag are blue, and how many are red?

So the question is, how close an approximation can you get, and how reliable is it?  If 250 of the balls you chose were blue and 250 were red, then it is theoretically possible that the total number of blue balls in the bag was 250, and you just happened to randomly select all of them.  It is, similarly, theoretically possible that there were only 250 red balls in the bag.  The chances against either being the case are astronomically small, but it’s possible.

If you choose 500 balls at random from a bag with 100 million balls, there is a 95% chance that the 500 balls you picked will accurately represent the colors of the balls in the bag +/- 4.4%.  The “95%” figure is called the “Confidence Level.”  The “4.4% figure is called the “margin of error.”  A 4.4% margin of error at a 95% confidence level means 95% of the time, the result will be accurate within + or – 4.4%, but it also means that 5% of the time it will be more than 4.4% inaccurate.

So, suppose Candidate A is polling at 47% and Candidate B is polling at 50%, with a poll that has a margin of error of +/- 4.4% at a 95% confidence level.  Since the two are only 3% apart, we can’t be 95% cure that Candidate B is beating Candidate A.  But does that mean each candidate is equally likely to be winning?  Not by a long shot.  We’re 95% sure that the sample is accurate within +/- 4.4%, but we’re 90% sure that the sample is accurate within +/-3.7%, and 80% sure it’s accurate to within 2.9%.  The “margin of error” and “confidence level” of a poll go hand-in-hand.  One number is meaningless without the other.  Unfortunately, most polls published in the press do not specify what their confidence level is, but the industry standard is 95%.

Of course, the closer the two candidates’ polling numbers are, the more likely it is that the candidate who appears to be behind is actually ahead, but just because we can’t be 95% sure that Candidate B is ahead doesn’t mean that we might not be 90% sure, or 80% sure, or even 75% sure.  But, to stretch a point, even if a poll with a margin of error of +/- 10% at a 90% confidence leval has Candidate A beating Candidate B by 51%/49%, there is  always more than a 50% chance that Candidate A is winning.  Depending on the poll it may be a 60% chance, an 80% chance, or an 85% chance, but it’s always more likely than not that the candidate who appears to be ahead actually is ahead.

That said, “margin of error” is far from being the biggest issue in accurate polling.  The only thing “margin of error” accounts for is random variation — i.e. the possibility that a random sample might not be completely representative of the population the sample is taken from.  But if you’re confident that you can determine what an entire pot of soup will taste like by tasting only one spoonful, you can also be generally confident that you can determine how 100 million people will vote by asking a randomly selected 1,000 of them.

This entry was posted in Political Myths. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s