Showing posts with label public opinion. Show all posts
Showing posts with label public opinion. Show all posts
Wednesday, April 29, 2015
Tuesday, April 8, 2014
Brand Images of Cable News
According to YouGov's BrandIndex, the perceived brand image of the major cable news networks has remained fairly constant in recent weeks, despite the small jump in viewing linked to the missing aircraft story.. The BrandIndex Buzz Index compares consumer perceptions of brands, and looks at the net difference between positive and negative comments.
As the graph shows, only Fox News has generated a net positive "Buzz", and seems to have more consistent values (in the +4 to +6 range) in 2014. None of the other measured cable networks generated a positive Buzz value over the measured time frame. CNN's perception score seems to have bottomed out last June (hitting -17), before making it back to a -4 score in October 2013, with consistent fluctuations after that. Consumer perceptions of CNN Headline have largely tracked those of CNN. Consumer perception of MSNBC has been very consistent since the start of 2014, with scores in the -9 to -10 range. That suggests the audience has pretty much made up its mind about MSNBC, and the network hasn't been able to do much to shake that overall negative impression. Since many of the big swings seem linked to major news events, it might also suggest that MSNBC's specific coverage of news doesn't change viewers perceptions about the network.
Source - Despite disaster coverage CNN perception declines, YouGov BrandIndex report
As the graph shows, only Fox News has generated a net positive "Buzz", and seems to have more consistent values (in the +4 to +6 range) in 2014. None of the other measured cable networks generated a positive Buzz value over the measured time frame. CNN's perception score seems to have bottomed out last June (hitting -17), before making it back to a -4 score in October 2013, with consistent fluctuations after that. Consumer perceptions of CNN Headline have largely tracked those of CNN. Consumer perception of MSNBC has been very consistent since the start of 2014, with scores in the -9 to -10 range. That suggests the audience has pretty much made up its mind about MSNBC, and the network hasn't been able to do much to shake that overall negative impression. Since many of the big swings seem linked to major news events, it might also suggest that MSNBC's specific coverage of news doesn't change viewers perceptions about the network.
Source - Despite disaster coverage CNN perception declines, YouGov BrandIndex report
Friday, January 31, 2014
Public Trust in TV News: Not Good, and Highly Politicized
Polling firm Public Policy Polling recently released its fifth report looking at the public's trust of major TV news outlets. The results shouldn't really be that much of a surprise, although it will shock most media elites.
Its probably not a shock to find that the public places more trust in PBS as a news source than it does any of the major broadcast networks, and all but one cable news outlet. Or that that Fox News is that one cable outlet. The real shock is in the numbers themselves - and that more people trusted Comedy Central most than trusted MSNBC and NBC.
When asked which media outlet they most trusted for news (more than any other outlet) - 39% said Fox News; 14% said PBS; 11% said ABC; 10% said CNN; 9% said CBS; 6% said Comedy Central; 6% said MSNBC; and only 3% identified NBC as their most trusted source. Turning it around, Fox News was also identified as the "least trusted" by 33% of those surveyed, followed by MSNBC at 19%, Comedy Central at 14%, and CNN at 11%. The broadcast networks can take take some comfort that none were identified as least trusted by more than 5%.
The survey also asked about each TV news outlet individually. Most had similar numbers indicating that they trusted the outlet as said they didn't trust it. Only PBS was significantly "above water" in that 57% of the sample said they trusted PBS, but only 24% said they didn't. The two outlets that were significantly "underwater" were Comedy Central (29% trust, 38% don't trust) and MSNBC (34% trust, 44% don't trust). That's probably not a problem for Comedy Central, as they don't see themselves as an "objective" news outlet - but could be a problem for MSNBC unless it wants to be perceived more as an opinion outlet than a news outlet.
In a blow to the idea of "objectivity" by news outlets, the poll also found that trust for most news outlets is highly correlated with political affiliation. For instance, Fox is both the most trusted among those identifying themselves as Republican, and the least trusted by those identifying themselves as Democrats. In looking at the news outlets individually, there were 40-50 percentage point swings in trusted/not trusted between those who voted for Obama in the 2012 election and those who voted for Romney. This wasn't just Fox News and MSNBC, but was a coonsistent pattern for all of the news outlets examined. The pattern also held true against a more general measure of political ideology, and smaller differences across gender, ethnicity, and age.
Oh, and the survey found that Fox's Bill O'Rielly would beat Comedy Central's Stephen Colbert in a Presidential election match-up.
Sources - NBC, MSNBC sink to the bottom of who the public trusts , Washington Examiner
5th Annual TV News Trust Poll, Public Policy Polling report
Its probably not a shock to find that the public places more trust in PBS as a news source than it does any of the major broadcast networks, and all but one cable news outlet. Or that that Fox News is that one cable outlet. The real shock is in the numbers themselves - and that more people trusted Comedy Central most than trusted MSNBC and NBC.
When asked which media outlet they most trusted for news (more than any other outlet) - 39% said Fox News; 14% said PBS; 11% said ABC; 10% said CNN; 9% said CBS; 6% said Comedy Central; 6% said MSNBC; and only 3% identified NBC as their most trusted source. Turning it around, Fox News was also identified as the "least trusted" by 33% of those surveyed, followed by MSNBC at 19%, Comedy Central at 14%, and CNN at 11%. The broadcast networks can take take some comfort that none were identified as least trusted by more than 5%.
The survey also asked about each TV news outlet individually. Most had similar numbers indicating that they trusted the outlet as said they didn't trust it. Only PBS was significantly "above water" in that 57% of the sample said they trusted PBS, but only 24% said they didn't. The two outlets that were significantly "underwater" were Comedy Central (29% trust, 38% don't trust) and MSNBC (34% trust, 44% don't trust). That's probably not a problem for Comedy Central, as they don't see themselves as an "objective" news outlet - but could be a problem for MSNBC unless it wants to be perceived more as an opinion outlet than a news outlet.
In a blow to the idea of "objectivity" by news outlets, the poll also found that trust for most news outlets is highly correlated with political affiliation. For instance, Fox is both the most trusted among those identifying themselves as Republican, and the least trusted by those identifying themselves as Democrats. In looking at the news outlets individually, there were 40-50 percentage point swings in trusted/not trusted between those who voted for Obama in the 2012 election and those who voted for Romney. This wasn't just Fox News and MSNBC, but was a coonsistent pattern for all of the news outlets examined. The pattern also held true against a more general measure of political ideology, and smaller differences across gender, ethnicity, and age.
Oh, and the survey found that Fox's Bill O'Rielly would beat Comedy Central's Stephen Colbert in a Presidential election match-up.
Sources - NBC, MSNBC sink to the bottom of who the public trusts , Washington Examiner
5th Annual TV News Trust Poll, Public Policy Polling report
Tuesday, November 12, 2013
History of Market Research
Online research firm Vision Critical has produced an interesting moving graphic - "Evolution of Insight" - that tracks key events in the developing of marketing research since the 1890s.
It's worth a look.
Source - Evolution of Insight, Vision Critical moving graphic
Wednesday, March 6, 2013
Thursday, November 8, 2012
Polls - From Spin to Rewrite
News reporting of survey results have been increasingly problematic over the years. Very few report the methodological details the American Association of Public Opinion Researchers argues is necessary to understand the reliability and validity of survey results. Almost all "cherry-pick" what results to include in stories (and what not to include). They often rephrase the questions when reporting on them - sometimes for clarity or due to space limitations, sometimes to fit particular storylines. And when they run their own surveys, or pay for others to do the surveys, they often suggest topics or specific questions and wording (again to fit existing themes, memes, or storylines).
Despite all that, however, most people could trust that the survey results themselves were real.
A recent story in the Salt Lake Tribune suggests that even that last aspect of trust may be misplaced. The story describes how the editors at the Tribune thought that one set of survey results they had commissioned was "wrong," in the sense of being inconsistent with other survey findings. And so they told the research firm conducting the survey to "fix" them. And the research firm did - by changing how they weighted survey results. The reasons for the change, and the mechanism (changing weighting) were apparently not given when the paper printed a story on the "adjusted" results.
Now there are many reasons one poll result may be inconsistent with others, even if the surveys are competently done and are reported accurately. First, something could have actually changed people's opinions and the survey responses accurately reflect that change. Second, there could be differences in terms of target population or sample weighting - that the difference in survey results is linked to differences in who the results are generalized to (for example, adults, registered voters, likely voters; or sampling only urban voters and not voters from all areas candidates will represent). Again, here the difference is likely "real", but the results are about different things and shouldn't necessarily be directly comparable in the first place.
The inconsistent survey might result from having a "bad" sample, or outlier, in the sense that it is not representative. It's easy to forget that when generalizing a sample to a larger population, you start with the assumption that the sample is representative - and there's always a chance that assumption is wrong. That's what things like confidence intervals are for. The most common, the 95% confidence interval, still indicates that there's a 1 in 20 chance that your sample is really not representative. And in big election campaigns, with dozens of surveys and samples, it's not that unlikely that you'll get one that is way off.
If that was the case here, it's not unreasonable to conclude that that specific sample is problematic and unreliable. And the question becomes how to best deal with the survey and its results. In a later discussion of the issue, reporters at the Tribune asked some "experts" about handling such a case. An ethics expert at the Poynter Institute said the paper should have asked about the "inconsistent" results before publishing the initial report, and inform their readers about any concerns. A professor at Brigham Young University who focuses on political polling also said that a big shift in numbers should trigger some concern, and that the lack of transparency in reporting methods and issue was "troubling."
But the newspaper went beyond raising issues about validity and informing their readers of potential problems. They asked their pollster for a fix. The pollster indicated that the sample was, at least in terms of political affiliation, representative of the Utah electorate. But that particular poll focused on local Salt Lake City elections, and said that "intuitively" you would expect that urban area to have more Democrats and so, after the paper questioned the results, he changed the sample weighting formula to match his "intuition."
As an occasional public opinion researcher (and AAPOR member), I have a problem with "fixing" a bad sample. Changing sample weightings won't fix the sample if it is that rare "bad" one, it would just add more layers of potential error and distortion - particularly if you change it to a more "intuitive" sample breakdown, or because the news organization funding the survey thinks the results should be something else. I've been down that road - back when I worked for a consulting firm, I was asked to "fix" an analysis - that the client needed the value of a group of broadcast stations to be 30% higher than the models we used indicated. By shifting some of our growth estimates to the high end (rather than use median estimates) and tweaking some financial forecasting assumptions I came close (25% higher); but I also warned my bosses that while that model was defensible, it was different from our standard model and was likely to be questioned. They went ahead with the "fixed" number and analysis, and the IRS did question it (and ended up not accepting it as a valid estimate of "fair market value.")
If it's looks like one survey result is an outlier, the ethical action from a research perspective is to use it, but certainly label it as such and discuss why it might be so, and what that suggests about the accuracy and validity of the findings; or to drop the outlier as being unrepresentative. In journalistic terms - report the results, along with concerns about its accuracy and representativeness (basically what the Poynter and BYU experts recommended), or chose not to report the results if you have reason to suspect their accuracy or representativeness - and tell the news consumers why.
That's not what the Tribune did - they reported the problematic results initially as accurate, then asked the research firm to "fix the numbers", and then reported the adjusted numbers as accurate, saving any discussion of the issues or process for a follow-up piece - a piece that was, most likely due to questions about why the two sets of results were so glaringly different, or why the adjusted numbers were supposed to be any more accurate than the initial numbers.
That's bad enough, but they way that the Tribune editor and head of the research firm talked about it, it appeared that "fixing" poll results in this manner was not uncommon, or considered to be unethical or something they needed to inform readers about. It makes me wonder just how common this practice is, and how much trust we can place in sponsored survey results.
Sources - It's Now Public: Editors Rejigger Polls, Commentary Magazine
Revised poll: Crockett-McAdams race is a virtual tie, Salt Lake Tribune
Despite all that, however, most people could trust that the survey results themselves were real.
A recent story in the Salt Lake Tribune suggests that even that last aspect of trust may be misplaced. The story describes how the editors at the Tribune thought that one set of survey results they had commissioned was "wrong," in the sense of being inconsistent with other survey findings. And so they told the research firm conducting the survey to "fix" them. And the research firm did - by changing how they weighted survey results. The reasons for the change, and the mechanism (changing weighting) were apparently not given when the paper printed a story on the "adjusted" results.
Now there are many reasons one poll result may be inconsistent with others, even if the surveys are competently done and are reported accurately. First, something could have actually changed people's opinions and the survey responses accurately reflect that change. Second, there could be differences in terms of target population or sample weighting - that the difference in survey results is linked to differences in who the results are generalized to (for example, adults, registered voters, likely voters; or sampling only urban voters and not voters from all areas candidates will represent). Again, here the difference is likely "real", but the results are about different things and shouldn't necessarily be directly comparable in the first place.
The inconsistent survey might result from having a "bad" sample, or outlier, in the sense that it is not representative. It's easy to forget that when generalizing a sample to a larger population, you start with the assumption that the sample is representative - and there's always a chance that assumption is wrong. That's what things like confidence intervals are for. The most common, the 95% confidence interval, still indicates that there's a 1 in 20 chance that your sample is really not representative. And in big election campaigns, with dozens of surveys and samples, it's not that unlikely that you'll get one that is way off.
If that was the case here, it's not unreasonable to conclude that that specific sample is problematic and unreliable. And the question becomes how to best deal with the survey and its results. In a later discussion of the issue, reporters at the Tribune asked some "experts" about handling such a case. An ethics expert at the Poynter Institute said the paper should have asked about the "inconsistent" results before publishing the initial report, and inform their readers about any concerns. A professor at Brigham Young University who focuses on political polling also said that a big shift in numbers should trigger some concern, and that the lack of transparency in reporting methods and issue was "troubling."
But the newspaper went beyond raising issues about validity and informing their readers of potential problems. They asked their pollster for a fix. The pollster indicated that the sample was, at least in terms of political affiliation, representative of the Utah electorate. But that particular poll focused on local Salt Lake City elections, and said that "intuitively" you would expect that urban area to have more Democrats and so, after the paper questioned the results, he changed the sample weighting formula to match his "intuition."
As an occasional public opinion researcher (and AAPOR member), I have a problem with "fixing" a bad sample. Changing sample weightings won't fix the sample if it is that rare "bad" one, it would just add more layers of potential error and distortion - particularly if you change it to a more "intuitive" sample breakdown, or because the news organization funding the survey thinks the results should be something else. I've been down that road - back when I worked for a consulting firm, I was asked to "fix" an analysis - that the client needed the value of a group of broadcast stations to be 30% higher than the models we used indicated. By shifting some of our growth estimates to the high end (rather than use median estimates) and tweaking some financial forecasting assumptions I came close (25% higher); but I also warned my bosses that while that model was defensible, it was different from our standard model and was likely to be questioned. They went ahead with the "fixed" number and analysis, and the IRS did question it (and ended up not accepting it as a valid estimate of "fair market value.")
If it's looks like one survey result is an outlier, the ethical action from a research perspective is to use it, but certainly label it as such and discuss why it might be so, and what that suggests about the accuracy and validity of the findings; or to drop the outlier as being unrepresentative. In journalistic terms - report the results, along with concerns about its accuracy and representativeness (basically what the Poynter and BYU experts recommended), or chose not to report the results if you have reason to suspect their accuracy or representativeness - and tell the news consumers why.
That's not what the Tribune did - they reported the problematic results initially as accurate, then asked the research firm to "fix the numbers", and then reported the adjusted numbers as accurate, saving any discussion of the issues or process for a follow-up piece - a piece that was, most likely due to questions about why the two sets of results were so glaringly different, or why the adjusted numbers were supposed to be any more accurate than the initial numbers.
That's bad enough, but they way that the Tribune editor and head of the research firm talked about it, it appeared that "fixing" poll results in this manner was not uncommon, or considered to be unethical or something they needed to inform readers about. It makes me wonder just how common this practice is, and how much trust we can place in sponsored survey results.
Sources - It's Now Public: Editors Rejigger Polls, Commentary Magazine
Revised poll: Crockett-McAdams race is a virtual tie, Salt Lake Tribune
Thursday, November 1, 2012
Infographic: The Borderless Lifestyle
Verizon has released results from a study of "borderless customers" - those owning laptops or desktop PCs, mobile devices, and say that they "usually" have a device with Internet service at hand. Verizon projects that 40% of US adults now fit that description, and the study looks at their attitudes, preferences, and behaviors. (The report notes the sample was non-random and not necessarily representative of the general U.S. population). Selected results were presented in the form of an infographic..
Verizon believes the Borderless Lifestyle is a result of the convergence of wired and wireless broadband networks, and the unique user flexibility and personal empowerment that converged services enable. The Borderless Lifestyle is about eliminating the old technology boundaries that used to separate networks and service platforms, home and work, allowing customers to connect and accomplish what they want or need to do, whenever they wish, wherever they are, using the device that they prefer.
The study suggests that over half their sample felt that Internet service was their home's most important utility - more people than the 39% identified as having a "Borderless Lifestyle."
Still, as a whole, while the Borderless were equipped for anywhere-anytime media consumption and engagement, their digital activities remained tied to their physical homes, at least for now. They still prefer to watch TV at home on traditional TV sets, and many online program access points remain clustered in their homes. Where they differ from other folks is their eagerness for the potential of the "Borderless Lifestyle" to be realized.
- 90% look forward to having all home electronics interconnected, with access to the Internet, compared to 64% of the Non-Borderless.
- 82% would like all media content, and all of their digital files, available anytime-anywhere (46% of NB)
- 61% look forward to using smartphones or tablets to control their TV sets (23% of NB)
Friday, July 13, 2012
Tracking Social Media chatter over News Events
One of the fun things for researchers is the huge amount of public data available online, and particularly the ability to capture snippets of public opinion and reactions through social media. (Although the methodologist in me needs to remind everyone that social media content is not necessarily representative of general public opinion.) For example, here's some summaries of reaction to the Supreme Court's decision on the Affordable Care Act (Obamacare).
In addition to tracking social media, many search engines provide the capacity to track trends in the use of search terms. Google, for instance, provides access to current trends as well as the ability to track term usage over time (or at least back to 2004) for general usage, as well as in terms of news references. Yahoo, Bing, AOL, Twitter, and YouTube all offer similar services. Many of these also can provide user information in aggregate, and can be further broken down by geographic location.
Online metrics firm Crimson Hexagon recently provided a demonstration of
some of its Online traffic tracking services though an analysis of the
reaction to the Supreme Court's health care ruling. They posted some screenshots of their real-time analytic platform ForSight that provided both descriptive summaries and more qualitative analysis of themes expressed through social media.
As I said at the beginning, the online world's providing a lot of interesting ways to quickly capture and analyse public opinion, comments, and reactions. Get to work, folks.
Source - Healthcare Ruling Sparks 12,000 Tweets Per Minute, Mashable
What People Search For - Most Popular Keywords, Search Engine Watch
Obamacare: Online Reaction to SCOTUS Ruling (Update) Crimson Hexagon press release
In addition to tracking social media, many search engines provide the capacity to track trends in the use of search terms. Google, for instance, provides access to current trends as well as the ability to track term usage over time (or at least back to 2004) for general usage, as well as in terms of news references. Yahoo, Bing, AOL, Twitter, and YouTube all offer similar services. Many of these also can provide user information in aggregate, and can be further broken down by geographic location.Online metrics firm Crimson Hexagon recently provided a demonstration of
some of its Online traffic tracking services though an analysis of the
reaction to the Supreme Court's health care ruling. They posted some screenshots of their real-time analytic platform ForSight that provided both descriptive summaries and more qualitative analysis of themes expressed through social media. As I said at the beginning, the online world's providing a lot of interesting ways to quickly capture and analyse public opinion, comments, and reactions. Get to work, folks.
Source - Healthcare Ruling Sparks 12,000 Tweets Per Minute, Mashable
What People Search For - Most Popular Keywords, Search Engine Watch
Obamacare: Online Reaction to SCOTUS Ruling (Update) Crimson Hexagon press release
Subscribe to:
Posts (Atom)