Measure Twice

by Dan Woychick

As more marketing happens on laptops, tablets, and smartphones, the demand for and trust in metrics continues to grow. If something can be measured, it will be, as return on investment (ROI) weighs on the minds of executives everywhere.

Analyze the following proposition. This product deprives you of sleep, makes unpleasant noises at inappropriate moments, is temperamental, requires constant attention, and costs a fortune. Babies. Who in their right mind would sign up for this? What’s the return on that investment?

We like to think we’re rational creatures, and that any situation can be measured, analyzed, and then systematically improved. And while many business metrics can be useful – even vital – they take the place of instinct, experience, and other available means of perception at our peril.

Blind spots
The truth is a lot harder to identify than it would appear at first glance. Individuals, each with their own beliefs and biases, can be relied on only to reveal one version of “reality.” One man’s trash is another’s treasure.

I have fond memories of watching Hogan’s Heroes as a boy and planning “escapes” from the basement with my brothers. I think the show is funny and still apply favorite lines to everyday situations. My wife thinks it’s one of the dumbest TV shows of all time.

In a million different ways we are all “reality challenged” and that’s a good thing – vive la différence! But we can also become blinded by our biases, form premature conclusions, and miss alternative points of view, as in this Awareness Test:

 

Cooking the books
People tend to seek out and believe numbers that support an existing assumption or preferred course of action. In other words, we see what we want to see. Marketers can shape or choose “facts” that feed this tendency.

There are three kinds of lies: lies, damned lies, and statistics. – Mark Twain

Television commercials are rife with examples. Four out of five dentists recommend sugarless gum for their patients who chew gum. The Ford F-150 offers best-in-class fuel economy. More people find love on Match.com than any other dating site.

The existing bias toward plausibly objective data is widespread and tempting for many organizations. Earlier this year, Claremont McKenna, an exclusive California college, admitted to inflating freshman SAT scores for six years to improve its place in the U.S. News & World Report’s widely-read college rankings.

Blind spots can be dangerous as well. During the lead-up to the invasion of Iraq, an independent weapons inspector found no stockpiles of WMD in the country. Since these findings didn’t support its strategic goals, the U.S. government simply used other measures to justify military action.

Buyer beware
The collected wisdom of the general public is subjective and often flawed. This creates opportunities for data wonks to dazzle us with metrics that may or may not illuminate effective decisions.

Many social media consultants will happily rattle off statistics that have the imprimatur of legitimate insight: “We measure influence and engagement and have the pie charts to prove it. ROI? Have we got numbers for you!

We know social media is important. That’s what everybody says, and everyone we know belongs to several social networks. Statistics may simply back up our existing beliefs. But, honestly, are you seeking out opportunities to engage in dialogue or conversation with a company, an institution or a brand? I’m not. Do these numbers reflect actual behavior that supports business objectives, or is it wishful thinking?

In preparation for the National Football League draft, teams put college players through a battery of tests. How many times can you bench press 225 pounds? How far can you jump from a standing start? How fast can you run 40 yards? While all those things can be quantified, in isolation – or even cumulatively – they do not reveal whether the athlete can actually play the game.

A measured response
Some things can and should be measured, but the quest for ROI is often more about minimizing risk than maximizing revenue. We must remain aware of our own biases and blind spots if we hope to transcend the data.

Gaining meaningful insights through research most often requires a balance of art and science – subjective and objective measures – because even though bean counters can tell you how many beans are in a jar, they can’t tell you how good they taste.

Related content:
Are Metrics Blinding Our Perception?
Do You Know Your Blind Spots?

One thought on “Measure Twice

  1. Beth Goodnight

    This is an excellent post, Dan! I, too, love Hogan’s Heros (sorry R!)

    We really do see… and believe… what we want to see/believe. This is one thing that makes politics and religion (among other things) so dangerous… IMHO. We are taught or shown something at an impressionable time in our life, and we trust the source of that information. So we believe it and never question it again.

    Ex: His father was Republican and what his father said/did made sense to him as a child, so he, too became a Republican. He never listens to what Democrats (or other political parties) say, because he ‘knows’ they are wrong based on what he learned and trusted from his father.

    We all need to open our eyes and ears to perspectives/ideas other than the ones we hold to see if we are still aligned with our previous views… or if there is something new we’ve not previously known. The world may have changed while we were stuck in our old ways/ideas.

    Gaining meaningful insights through research… you said it, brother!

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s