By Len Lazarick
Is it possible that a partisan Republican poll in the Maryland governor’s race has as much or even more validity than a poll done for some of the most respected names in the media?
That’s the surprising conclusion from looking at the numbers and methodology behind the two most recent polls pitting Democrat Anthony Brown against Republican Larry Hogan Jr.
A Battleground Tracker poll released Wednesday morning by CBS News, the New York Times and YouGov has Brown at 46% and Hogan at 34%, numbers consistent with other independent polls taken months ago by Rasmussen Reports and The Washington Post, according to a compilation by Real Clear Politics. And those results were just among likely voters. Adding all voters it was Brown 51%-Hogan 37%.
That’s much different from the results of a poll the Maryland Republican Party released Aug. 22 showing Brown with 45% and Hogan with 42%. This was consistent with rumors of internal polling done by the two campaigns showing the race tightening.
Of course a poll paid for by a political party is less legitimate than a poll by a news organization, right?
Problem with methods, turnout, weighting
The problem with the CBS-Times poll is methodology and weighting. The GOP poll was a standard phone interview of 500 likely voters. That’s only half the sample size in the CBS-Times poll, but that poll was done entirely on the Internet by contacting people signed up for polling.
That’s problematic, but not totally disqualifying. Such polling is a much cheaper way to reach a lot of people — much as Rasmussen’s robotic phone calls save on polling costs.
The more serious problem is how CBS-Times predicted the turnout and weighted the results.
Predicting turnout is one of the most serious problems with any kind of election polling. If elections were won just based on people’s opinion of candidates, polling would be a lot more accurate in predicting the outcome of a given race. But those voters actually have to show up at the polls or send in absentee ballots.
That is why polls try to ferret out “likely voters,” people who say they will vote. After these self-identified “likely voters” respond — however randomly they are chosen — the results have to be “weighted” to accurately reflect party affiliation, race, age and gender. Did the pollster contact too few Democrats and too many African Americans, too few millennials and too many senior citizens, and so on?
Getting predictions wrong
Here’s where the CBS-Times poll seems to have gone astray. The GOP pollster said he used the last two elections to predict voter turnout. It is not clear what CBS-Times used, but its numbers seem way off.
In rebalancing the numbers for responses, Republicans made up only 21.6% of the responses, while they make up 25% of registered voters and 25% of the people who answered the poll. They also turn out to vote in higher numbers.
For the poll, independents made up 26% of the weighted respondents, but they make up only 19% of the electorate, including third party registrants.
While only 45% of those clicking the poll choices on the Internet said they were Democrats, they actually make up 55% of Maryland’s registered voters and were weighted to make up 52% of likely voters that the poll counted.
Three-quarters of African Americans or more in both polls support Brown. The CBS-Times poll weights their turnout at 28% of voters, even though turnout among blacks has traditionally been poor in gubernatorial elections and they make up 30% of the total Maryland population.
In 2010, for instance, the turnout in the two majority black jurisdictions — Prince George’s County and Baltimore City — was 45%, 9 points below the state average. Majority Republican jurisdictions such as Carroll and Harford counties were well above average at 61% and 64% respectively.
If a poll undercounts Republicans and over-counts African Americans, it gets results that favor the Democratic candidate, especially one who is African American like Brown.
Does it help Brown or hurt?
There are any number of ways a poll can get things wrong, but inaccurately predicting turnout, then weighting responses incorrectly is a fundamental flaw.
There go the mainstream media, you might say, putting their thumbs on the scale in favor of Democrats. Whatever the reasons for the errors, the CBS-Times poll might actually have the opposite effect.
One of the principal worries of the Brown-Ulman campaign is that voters in Prince George’s and Baltimore City will not turn out in big enough numbers. A poll that says Anthony Brown has got a lock on the election could tend to discourage these black voters who have little other reason to go to the polls, since most of their local and legislative races were decided in the June Democratic primary.
In other words, a poll that unrealistically boosts Brown could potentially suppress the black votes he must have to win. Yet Maryland Democratic Party Chair Yvette Lewis was crowing about the results of the poll.
The Republicans released their poll with much fanfare in order to have the opposite effect on GOP voters.
And what if you don’t believe in polling at all? Well, for all its potential failings in predicting the outcome of an election based on a snapshot taken in the past, polls are still the most objective way to assess how a candidate is doing and how a message is playing. That’s why campaigns and candidates still pay for polling.
Hopefully we will soon get a more accurate picture of the race from some truly independent and accurate polling by the Baltimore Sun or the Washington Post, as well as by others.
I actually was called by the GOP pollsters. One issue with their poll was the questions. It was pretty close to a push poll. I actually had to laugh at a few of the questions that were very much driving respondents towards Mr. Hogan. So any analysis of a poll must include looking at how the questions are framed.
This is among the factors that can skew the poll. After the initial story, I asked for and received the poll questions and the order in which they were asked. All the head to head and favorability questions were asked before the issues questions — which they did not fully share. If that wasn’t the case, then that is a problem.
This is a very enlightening analysis, and I completely agree that most of us would like to see an accurate and unbiased poll. However, your analysis demonstrates how difficult it is to design a poll that will give an accurate result, even if you WANT one. Among the considerations often mentioned today is how difficult it is to conduct a valid telephone poll when so many people have cell phones and no landlines, for example. An online poll produces other kinds of distortions. (My gut feel is that the distortions caused by telephone polls and online polls would tend to cancel each other out, but I have no data to support that.) What’s the answer? If I knew, I could make a lot of money conducting polls for groups who want a useful product.
No, it’s doable, it’s just more costly. IMO, the easiest method is to start with the pool of all registered voters and then randomize a selection of a number (say 1,500 or 2,000). Then you proceed to poll all of those people. If you don’t have a landline, you use the big data companies to cross-reference to cell lines. You will get almost everyone this way.
From the larger list, you ask likelihood to vote and reference it against previous voting records on the person. Someone saying they’re 100% and having only voted once in a 10 yr time frame is an anomaly so you can weed them out. But, if in the aggregate you see a lot of those 100% for previous 1 of 8s, then you also have a trend.
The harder issue is dealing with no-show affect. People see voting as something they should do, therefore if you ask them if they plan to do it, they feel they should say yes even if they’re a 50/50 or don’t plan to vote. It’s why nearly every poll you take projecting turnout from RVs will indicate a turnout 10-15 points higher than the actual day.