by Dan Woychick
As more marketing happens on laptops, tablets, and smartphones, the demand for and trust in metrics continues to grow. If something can be measured, it will be, as return on investment (ROI) weighs on the minds of executives everywhere.
Analyze the following proposition. This product deprives you of sleep, makes unpleasant noises at inappropriate moments, is temperamental, requires constant attention, and costs a fortune. Babies. Who in their right mind would sign up for this? What’s the return on that investment?
We like to think we’re rational creatures, and that any situation can be measured, analyzed, and then systematically improved. And while many business metrics can be useful – even vital – they take the place of instinct, experience, and other available means of perception at our peril.
The truth is a lot harder to identify than it would appear at first glance. Individuals, each with their own beliefs and biases, can be relied on only to reveal one version of “reality.” One man’s trash is another’s treasure.
I have fond memories of watching Hogan’s Heroes as a boy and planning “escapes” from the basement with my brothers. I think the show is funny and still apply favorite lines to everyday situations. My wife thinks it’s one of the dumbest TV shows of all time.
In a million different ways we are all “reality challenged” and that’s a good thing – vive la différence! But we can also become blinded by our biases, form premature conclusions, and miss alternative points of view, as in this Awareness Test:
Cooking the books
People tend to seek out and believe numbers that support an existing assumption or preferred course of action. In other words, we see what we want to see. Marketers can shape or choose “facts” that feed this tendency.
There are three kinds of lies: lies, damned lies, and statistics. – Mark Twain
Television commercials are rife with examples. Four out of five dentists recommend sugarless gum for their patients who chew gum. The Ford F-150 offers best-in-class fuel economy. More people find love on Match.com than any other dating site.
The existing bias toward plausibly objective data is widespread and tempting for many organizations. Earlier this year, Claremont McKenna, an exclusive California college, admitted to inflating freshman SAT scores for six years to improve its place in the U.S. News & World Report’s widely-read college rankings.
Blind spots can be dangerous as well. During the lead-up to the invasion of Iraq, an independent weapons inspector found no stockpiles of WMD in the country. Since these findings didn’t support its strategic goals, the U.S. government simply used other measures to justify military action.
The collected wisdom of the general public is subjective and often flawed. This creates opportunities for data wonks to dazzle us with metrics that may or may not illuminate effective decisions.
Many social media consultants will happily rattle off statistics that have the imprimatur of legitimate insight: “We measure influence and engagement and have the pie charts to prove it. ROI? Have we got numbers for you!”
We know social media is important. That’s what everybody says, and everyone we know belongs to several social networks. Statistics may simply back up our existing beliefs. But, honestly, are you seeking out opportunities to engage in dialogue or conversation with a company, an institution or a brand? I’m not. Do these numbers reflect actual behavior that supports business objectives, or is it wishful thinking?
In preparation for the National Football League draft, teams put college players through a battery of tests. How many times can you bench press 225 pounds? How far can you jump from a standing start? How fast can you run 40 yards? While all those things can be quantified, in isolation – or even cumulatively – they do not reveal whether the athlete can actually play the game.
A measured response
Some things can and should be measured, but the quest for ROI is often more about minimizing risk than maximizing revenue. We must remain aware of our own biases and blind spots if we hope to transcend the data.
Gaining meaningful insights through research most often requires a balance of art and science – subjective and objective measures – because even though bean counters can tell you how many beans are in a jar, they can’t tell you how good they taste.