Measurement Issues: Online, Social, & RTB/Exchanges

This seems to be a good week to write about metrics - measuring audiences and/or media use.  I posted yesterday about Nielsen's plans to use audio code readers and STB (Set Top Box) data from cable and satellite services to try to get a fuller sense of today's TV viewing behaviors, and a few days ago about Billboard revising how it measures a song's popularity in their Top 100 song charts.  In this post, I'll try to highlight some other news and discussion of metrics.

  Measuring media use has always been a bit problematic - but in the old world of media silos, media and advertisers were able to eventually agree on standard measures.  Audited circulation numbers for newspapers and other print media, ratings and shares for broadcasters - none were perfect, but with a bit of back and forth between media and advertisers, a general consensus was reached on specific formulas for calculating media use that were considered valid and reliable enough to be useful.  Reaching consensus was helped by the limited number of options available for consumers to access the content.
  The rise of digital networks and growth of media outlets disrupted things - increased competition fragmented audiences, making accuracy in measures more critical.  To illustrate, in the 1960s programs on the big three broadcast networks would see primetime programs getting ratings in the 20s - a typical night might see shows with 25, 22, and 19 ratings competing and those 2-5 point differences were well outside standard measurement error - while this fall's network premiere week saw the top network primetime programs earn ratings in the 2.5 to 3.1 range (and 1.9 ratings considered a flop) with the result that critical decisions on buying advertising and/or cancelling programs are based on differences of a single ratings point or even tenths of a ratings point.  While improved methods have reduced sampling and measurement error a bit, a lot of ratings based decisions today are based on differences that are statistically insignificant.
  The growth in media outlets and convergence of content markets created another problem.  With much content being accessible through multiple outlets, and with users also having the options of time-shifting or place-shifting, two critical issue for metrics emerges: which of these alternative uses should count, and can you develop measures that are comparable across the range of options?
  Consider a relatively simple case - a daily local newspaper that also puts stories online.  You have the ABC circulation numbers of how many papers were sold (and presumably read) that day.  And metrics abound on the Internet, so it's easy to count how many times the online paper's site was accessed; you could also count how many "unique" site visits have occurred.  But online new use is different than print - with print you can argue that subscribers and purchasers at least scan a large part of the paper.  But the online news reader may look only at a couple of specific pieces of information - weather forecast, a favorite columnists latest piece, etc.  Is accessing something on a newspaper website the equivalent of a sold copy?  And if you approach metrics from the advertiser's perspective, they want to know how many people saw their ad.  Should web use count if their ad isn't on the website? 
  Another issue is the web itself - which on one level can easily measure certain things - page downloads, unique visitors, click-throughs, posts, ad nauseum.  But none directly get at the real question - what the web user did with it.
  So you can understand that there's a lot of questions these days about who and what should be measured - and a lot of push from both media and advertisers/marketers to develop ways of measuring audience consumption, use, and impact of various types of content.

The Nielsen and Billboard posts talk about ways of expanding metrics to include new forms of consumption - Billboard adding online sales and plays on streaming services to the traditional radio airplays; Nielsen's audio code reader providing a mechanism to capture second screen viewing (among other things).  But there's still a lot to work on with respect to online media use metrics

  Gail Belsky has a piece on OMMA (a news blog covering online marketing and advertising) that considers a rising issue in the online video market - the lack of standard metrics.  Online video use is booming, as is online video advertising - but many advertisers are hesitent to enter the market without reliable metrics and standards that let them directly compare online video viewing with traditional TV viewing.

(Online video viewing) still lacks uniform ROI metrics to compare it to TV — a concern even for pioneer online advertisers. “At the end of the day, it’s really about measurement,” says Greg Milner, director of global interactive marketing at computer maker Lenovo. “Is your ad working or not?”

Surveys of advertising and marketing executives suggest most feel that developing consensual and reliable metrics are important for future development of the online advertising market.  For now, some potential advertisers are satisfied with online advertising's ability to reach highly targeted market segments, and the ability of online video to create a "viewing experience" and promote engagement. For the mobile advertising market, there is also concern about whether the small screen can deliver the same impact. But for many others, the lack of a uniform metric comparable to those in traditional media makes contribute to uncertainty as to the relative value and impact of online advertising - even if they're convinced that they need to be in the online market.  So for now, many find themselves testing the waters with small buys.

“It’s an environment where viewership is,” says Sacerdoti (CEO of video advertising network BrightRoll). “Advertisers should be present and testing that environment. There’s absolutely no doubt that if you’re a top 200 advertiser in tv, you should be testing mobile.”

  Meanwhile, over at GigaOM, Mark Ingram looks at metrics for social media.  More specifically, he notes that media outlets and the firms using advertising and marketing through media, are interested in how people got to that target site.  Social media seems to be driving a lot of traffic today, but it's not that easy to monitor or measure the impact of social media.  Alex Madrigal suggests that a lot of traffic and referrals prompted by social media posts come through means that most current web-analytics metrics toss into a catch-all category called "direct".  The IP header used to direct online traffic can include various bits of metadata, including a referral tag from the originating source of the message.

 There are circumstances, however, when there is no referrer data. You show up at our doorstep and we have no idea how you got here. The main situations in which this happens are email programs, instant messages, some mobile applications*, and whenever someone is moving from a secure site to a non-secure site.

 To illustrate his point, Madrigal used one metric from Chartbeat that broke the "direct" category into several subcategories, including one linked to social media use, and applied it to The Atlantic magazine's website (see piechart).

(Chartbeat) took visitors who showed up without referrer data and split them into two categories. The first was people who were going to a homepage (theatlantic.com) or a subject landing page (theatlantic.com/politics). The second were people going to any other page, that is to say, all of our articles. These people, they figured, were following some sort of link because no one actually types "http://www.theatlantic.com/technology/archive/2012/10/atlast-the-gargantuan-telescope-designed-to-find-life-on-other-planets/263409/." They started counting these people as what they call direct social.

Chartbeat's estimated that social traffic to media sites is significant - 17.5% of all referrals were social referrals (coming directly from a social media site, or were "dark social").  Only search engines generated more referrals (21.5%).  But as much as 70% of those social referrals were "dark", and missed by many web metrics services, in the sense that they were not identified as social referrals.  This dark social traffic comes from email, IM, chat apps, and a variety of other conduits, but are likely initiated by social media content.

A third post from John R. Osborn on the Online Video Insider blog also points to the importance of resolving measurement issues.  Osborn's focus is on the emergence of RTB/Ad Exchanges
 - RTB standing for Real Time Bidding - where advertising inventory is pooled from a wide array of media outlets, and auctioned off just prior to the advertising slot (in contrast with upfront ad buys that occur weeks or even months in advance).  The development of these exchanges offers significant opportunities for smaller media outlines (particularly those online) to tap into advertising and marketing revenue streams.  It also opens opportunities for advertisers or marketers to react quickly to opportunities and events, and to aggregate highly targeted audiences without having to seek out and negotiate with individual outlets.  The primary thrust of the post is that if these exchanges can capture some of existing advertising/marketing revenues, and develop added revenue from small firms and organizations who aren't in the traditional markets, the revenue potential could make ad-revenue business models viable for the smallest of online video outlets.
  The key to the long-term success of this advertising market segment (and thus its viability as a significant component of media outlet business plans), is developing a common basis for measuring ad exposure and establishing consensual price/performance measures that are comparable to other advertising formats and markets.  As Osborn notes,

Many players are focused on development of a common buying currency, (sometimes called “digital GRPs”) to help accelerate the use of RTB and video ad exchanges.

Furthermore, if that can be done, the ability to get near real-time metrics with online media and ads can provide RTB/Exchanges with a serious competitive advantage compared to upfront buys, by reducing uncertainty about actual audience size, demographic make-up, and current interests/focus.

Advertiser value will increase as constant, real-time feedback and adjustments get the message in front of the right audiences in the right context at the right time.

Development of these exchanges provides a number of opportunities for both online media outlets and content providers, and advertisers and others interested in reaching their targeted audiences with their own message.  Furthermore, the fact that these Exchanges promotes on-the-fly aggregation of both outlets and advertisers opens the way for many smaller-scale outlets and individuals on both the supply and demand side to get into, and benefit from, the rapidly growing online advertising market. 
  A recent study from Index Platform found that about half of advertisers and publishers are already testing RTB/Exchanges.  The study also suggested that their greatest concerns were essentially about metrics - buyers wanting transparency and reliable measures of audiences; publishers wanting to assure better that the ad value coming from RTB is comparable with other sources.  They also argued that developing a market-wide consensus on metrics, values, and practices would promote growth of this market segment.  While most thought RTB revenues would grow, there was an common underlying perception that developing commonly accepted metrics will be critical for this aspect of the market to fully develop.

Each of the posts raises and discusses some interesting issues about the inadequacy of current metrics for online content and outlets.  All are worth a closer read.

Sources -  Video Goes Godzilla,  OMMA blog
Dark social: Why measuring user engagement is even harder than you think, GigaOM
Dark Social: We Have the Whole History of the Web Wrong, the Atlantic
How RTB Video Exchanges Will Create a New T/V Business Model, OMMA blog
Real Time Bidding and Ad Exchanges: What works and what doesn't, report from Index Platform