The Baltimore Sun conducted a poll surrounding the upcoming Mayoral primary election in Baltimore, and wrote about it on Saturday, August 27th.
While it is disappointing that the Sun produced this faulty and misleading polling and analysis, we will assume that everyone involved had the best intentions.
Regardless, these flaws are clear:
- A telephone poll of “likely voters” disproportionately reaches senior citizens. Senior citizens are the most likely to appear in “likely voter” databases, have a land-line phone number, and have a stable and identifiable voting record at a Baltimore city address. As their own article points out, the incumbent mayor’s strongest voting bloc is senior citizens, and Otis Rolley’s supporters are predominantly under 35. People under 35 are the least likely to have land lines and be reachable for a poll.
- There was no discussion at all about how turnout may affect the outcome – especially turnout among “new” or younger voters. The article seems to conclude that the people polled will be the only people doing the voting. There is no reason to think this will be the case. Both low and high turnout scenarios could materially affect the outcome, disproportionately emphasizing particular voting blocks in each case.
- The tone of the analysis is sweeping and distorted in favor of the incumbent. In fact, the tone is so forceful that it may actually distort the election by lowering turnout. Using phrases like “sweeping competition” and “more than all of her rivals combined” suggests an air of inevitability. However, the article itself goes on to say that a “challenger could still win it,” and states that 18% of those polled are undecided. More balanced analysis would have emphasized the number of undecided voters, and would have spent more time discussing scenarios where challengers could succeed.
- There was no attempt made to report how many new voters there might be. New voters could make up a significant percentage of the vote. Any valid attempt to prognosticate about the outcome of the election must factor in new voters.
- The reporter did not seek comment from the challenging campaigns. Subsequent comments made by both the Pugh and Rolley camps indicate that they are seeing wildly different measurements in their own field work. Both campaigns have spent money on polls, and have reported numbers as high as 50% undecided (Sun says 18%), and job approval ratings as much as 25% lower than what the Sun reported. The challengers should have had a chance to comment, and report their own poll findings. The Sun should have identified, and tried to explain, the disparities between their poll results and the numbers reported by the campaigns.
- There was no mention made about the cost of the poll, or the specific methodology employed. Primary elections are notoriously difficult to poll correctly. Fast, inexpensive polls are likely to be very noisy and inaccurate. The campaigns know this and have likely spent considerably more than the Sun did to collect their information. The Sun should have made a statement about the cost and quality of their poll, the historic track records of similar polls in comparable, small races done by the same firm, and how this poll compared to polling which may have been done by the campaigns.
It was possible for the Sun to write an article about this poll that was fair and balanced, but they chose not to do so. Why?
UPDATE: The Sun published a subsequent op-ed which addresses some, but not all, of our concerns.