Useless online metrics you might be tracking
By Matt Rosenberg
The good news about digital advertising is that you can measure everything. The bad news is that you’re often asked to try. Everything cannot tell you a story that makes sense, just as I cannot tell you the story of how I met my wife in a way that includes all the words in the Oxford English Dictionary. What matters is that you choose the right words, or in this case, the right metrics that will let you know if you’ve spent your money in a way that achieves your goals.
The problem is that we often select metrics very broadly, or for reasons that don’t have much to do with the campaign goals at hand. Sometimes we’re just trying to compare to benchmarks, spending goals, or even just reaching into the grab-bag of the familiar. We also have a tendency to try to measure a spectrum of different tactics with one rigid set of metrics, like deciding that length is the thing that matters most and then measuring water with a yardstick.
Get connected. Want to gain more insight into the future of web analytics, as well as current best practices? Attend ad:tech New York, Nov. 3-4. Learn more.
Choosing the right tools to determine the success of a campaign is crucial. Below are a few examples of how metrics and strategy can become unhinged — and how we might line them up again.
Click it where the sun don’t shine
I had a client who loved search for exactly the reason that search is lovable. CPC pricing is super-efficient and leads are all qualified, so we assume that the ROI must be high. But we dug into the client’s search results and found that the first page bounce rate was more than 90 percent. (So much for pricing efficiency.) That’s not search’s fault; search was doing its job. The search was driving to the site’s homepage, and there were a lot of different and very specific terms and short blurbs that were driving consumers there. Essentially, the paid search results were making a promise that the homepage couldn’t keep. So, don’t just look at clicks — look at what happens to those clicks once they land.
Of course, there’s another issue with clicks as a metric: click fraud. It just keeps getting worse because it’s so easy for crooks to automate fake clicks. Search isn’t the only channel facing this problem; CPC display is as well. Look for pricing models that are more resistant to game playing.
Success you can’t see
An automaker wants to reinvigorate its brand and move metal. The agency plans a great campaign using data tools like BlueKai to target “intenders” — that is, people planning to buy a car in the next six months. The agency knows that for a brand campaign, click-through is a poor metric, but there needs to be some measurement that shows movement down the purchase funnel. The agency agrees with the client that a visit to the automaker’s site to either configure a car or use the dealer locator will suffice, so they set up view-through tags to see how many people who have seen the ad wind up on the automaker’s site.
This is guaranteed to under-count the effectiveness of the campaign. The creative might have done a wonderful job of making the audience want the car, but there are a lot of ways to move down the funnel without ever going to the automaker’s owned and operated site. Going to Edmunds or Cars.com would furnish the same (or more) information to the buyer but not feel as pushy and limited as the manufacturer’s own site. Dealers can be contacted and quotes given through these sites. But people who take these incredibly positive actions will never hit the view-through meter, and the money spent exposing them will often go into the wasted bucket by default.
Instead, consider spending some extra money to do a bit of behavioral research with a panel provider to see whether people exposed to the campaign searched for the brand, visited endemic sites, or took other actions inside and outside the manufacturer’s site at higher rates than the non-exposed audience. Spending more money to find out if the money you’ve spent worked is not wasting money — it is being strategic.
Reach without reaching
CPM is a great way of getting pure reach at enormous scale. Advertisers accustomed to TV and print often set reach goals (usually expressed as GRP) for a good portion of the budget. It makes sense — it’s cheap and with a frequency goal there’s an expectation that you’ll at least imprint the audience with a simple message or value proposition. But there’s one problem with pure reach as a metric: Digital ads are a bit more ephemeral and hard to audit than those in print magazines or running on reputable broadcast networks.
Recommended for you
There are shenanigans we’ve seen that are every bit as repugnant as click fraud. Sites will put up impression burner pages laden with tens of ads — no way for a viewer to crack that clutter. Or sites will sometimes put ad tags on redirect pages. The audience doesn’t see, but the ad server doesn’t know. You’ve seen sites that require you to continuously click through page refresh after page refresh to consume an article. That’s setting up a bad user experience in order to burn impressions — impressions on pages with low natural time spent are not going to imprint a message on anyone.
One particularly nefarious charade we uncovered recently was a site that served itself into an ad banner. What’s that? Well, by playing this clever but dishonest game the site lived inside itself, so it was counting ad calls as impressions even though they were invisible. Imagine putting two mirrors facing each other and watching iterations of reality spin off infinitely. It’s like that. And worse, each iteration was reporting to comScore, so the site’s traffic looked phenomenal. When Joestoenails.com has Top 50 traffic, you’ve haven’t spotted a trend — you’ve spotted a game.
When looking for reach, it’s important that you validate that your impressions are actually hitting eyeballs and staying in front of them for at least a little while.
Dilution by abstraction
You’re an AMD, and you’ve found a great new model for delivering results to your client. However, the client has worked hard to develop a media mix model and holds you accountable for eCPM pricing standards. Here’s the problem: eCPM rolls up a lot of disparate tactics, each with its own unique metrics, and tells you how much you spent for them without assessing the relative value of those things.
You know what some buyers do to bring down their bottom-line eCPM? They buy the things that they know will be effective, and then they buy a lot of cheap, mass-impression bulk to bring down the eCPM. In order to appear to be saving money, they waste money. Should the clients be happy about that? Well, no, but they seem to feel good about telling their bosses how low they drove the eCPM. And you can fully understand why the buyer does it — it’s the only way they can deliver what they actually believe in while doing right by the wrong measure they’re held to.
Rigidly looking at cost as the success metric ensures that you are over-spending. Annoyingly ironic.
Correlation does not imply causation
There are a few reasons to buy pre-roll. It’s creatively easy, it’s familiarity to the broadcast TV structure appeals to less-adventurous clients, and you know you’re getting complete views. Pre-roll generally gets about an 80 percent complete view rate, which is great, right? Well, it would be if complete rate were actually a proxy for recall. But there is no evidence that it is.
Even more basically, recall might not really be a valuable goal in the first place. It’s measurable through surveys, but does recall get you any closer to intent to purchase? Creating recall seems to be important because buyers of brands are highly aware of the brands they buy. But is it that recall leads to purchase or merely that purchasers have high recall? There is, to my knowledge, no research that shows causation (rather than correlation) between recall and intent. That’s the Rosser-Reeves Fallacy.
A timely conclusion
In case it appears that I’m suggesting the above metrics are meaningless, I’m not. I’m suggesting that they are meaningful, but only in the right context and when implemented honorably. Context shifts rather a lot from campaign to campaign, and honor is too often fleeting in this industry.
Is there a metric broad enough to be simply applied across lots of different types of campaigns? Well, I’m a big fan of engagement (though I think engagement rate is a poor metric because unless you’re qualifying engagement rigorously, it’s too easy to game). Engagement can often be measured in time spent. Of course, not all time is spent the same way, and some creatives, who know that success will be measured on time, play games like hiding the close button, which gets good results on the spreadsheet without helping the brand.
Assuming that your materials adhere to honorable best practices, time is a great metric. We’ve seen studies that indicate that increases in time spent correlate with increases in brand lift; the more time, the greater the lift (up to, I’m sure, a point of diminishing returns). With the right qualifiers and more research into how different modes of spending time should be valued, time might be the metric that can best normalize across ad types and publishers while being more immune to gamesmanship.