Decoding the data dilemma: How to strike a balance between metrics and instinct

Data-driven decision making can be both a powerful tool and a potential hazard. Sure, metrics are helpful, but what happens when they contradict real life results? Do you trust the data or your gut? The answer is: both. Success lies in finding that delicate balance between quantitative data and subjective judgment.

This week, Launch by NTT Data’s Chris LoSacco and Gina Trapani discuss the principle of being data-informed rather than data-driven, and when to challenge the evidence and trust your instinct instead. Check out the highlights below, then dive into the full episode to learn more about the nuanced relationship between metrics, user experience, and an organization’s overarching goals.

Bezos' approach to data

In a recent interview, Amazon founder Jeff Bezos advised not relying solely on data and emphasized the importance of trusting one's hunches even when data suggests otherwise. He gave an example using Amazon’s customer service, which was performing well on paper but was receiving a lot of customer complaints. To test the metric, Bezos himself called the helpline during a meeting and discovered just how wrong the company’s data was, proving that metrics do not always paint a complete picture.

Goodhart's Law

It is, however, possible to be too data-focused. Goodhart's Law states that when a measure becomes a target, it ceases to be a good measure. This concept underscores the potential pitfalls of hyperfocusing on specific metrics as targets without considering broader organizational goals. If you put blinders on in order to focus on one specific path forward, you might miss all the signs and better avenues along the way. 

User experience cannibalization

Which metrics actually move the needle and make a difference for your customers? Is the data you’re focused on actually providing business value, or is it a measure of vanity? If you’re prioritizing metrics like clicks and views, odds are you’re more concerned with your own perception rather than the user experience or the overall spirit of the product or service. A focus on sensationalist and clickbait content may lead to a race to the bottom and undermine the quality and integrity of your content, products, and services. Keep your customers front of mind and instead of fixating on how they’re engaging with your business, consider how your business can better engage them.  

Data-informed vs. data-driven

There’s no question that data is a valuable asset for business intelligence and decision making, but it is certainly possible to overdo it. Being solely data-driven may lead to overreactions or missing the broader context. Instead, aim to be data-informed. This involves using data as one of many signals in decision-making, but not the be-all and end-all. The objective metrics have their value and place in the process, but so do subjective insights.                                                

Balancing data and subjectivity

The best approach is one that balances both quantitative and qualitative signals to inform decision making. While data provides objective insights, there's an equal need for a guiding vision or strategy that aligns with the company's goals and values. At the same time, too many opinions can pull the business in conflicting directions and lead to the wrong priorities. Exercise critical thinking to avoid blindly following data or subjective opinions, and instead strike a balance between both. 

As always, don’t forget to subscribe to Catalyst wherever you get your podcasts. We release a new episode every Tuesday, and each one is jam-packed with expert advice and actionable insights for creating digital experiences that move millions.