Seven Ways to Think about Data and Why it’s Always Changing

What would you do if you were a content publisher and couldn’t measure your own traffic?

BuzzFeed became leaders in social, news and entertainment by focusing on how and why people share content online. We publish everything from breaking news to scripted shows on more than 30 platforms, including YouTube, Facebook and Snapchat, in addition to our owned and operated channels. Therefore, the usability of our tracking reports at times relies on the ad hoc data provided by these third-party sites.

As a result, we estimate that our current comScore metric of approximately 80 million UVs represents less than one-fifth of our actual global reach. Less than one-fifth! Most of us already know that UVs aren’t as important as they once were (we now care about more important metrics like engagement, social sharing and, of course, sales) – but in the fragmented media landscape we’re currently in, you need to learn how to get creative. Future media companies and brands that publish on multiple platforms, in particular, will need to look beyond simplistic approaches to performance and get comfortable with the more difficult and chaotic world of measuring the success of content.

We are tackling this challenge head on and have found some surprising (and sometimes entertaining) answers. Below are some of these insights.

1. Embrace the complex journey to a viral hit
To approximate overall reach, we look at a combination of metrics that are available across platforms and then determine the most resonant themes and messaging. For instance, engagement is one metric that helps us identify the best “container” (third-party platform) for every piece of content.

Our creative teams then consult the knowledge base we’ve accumulated over the years of experimenting with content on multiple platforms. By combining expertise in these variables with the insights a brand can bring to the table, you’re on your way to uncovering an idea that’s really special.

2. Be humble: Data is best used for learning, not vanity
Knowing some of our big topline numbers are useful for understanding trends (and for bragging), it doesn’t help us create better content or connect with our audiences. In the end, we care about data because we hope to learn something from it. We should look to other, smaller numbers for that.

3. Questions matter more than answers
It’s important to ask the right questions. At BuzzFeed, members of the data science team discuss with editors what s/he is trying to accomplish before they decide which metrics may best measure performance. And if such a conversation unearths a metric that we do not currently capture, we know to get started. As content becomes increasingly nuanced, asking the right questions will matter more and more.

4. Be skeptical
There is a pervasive belief that data equals truth. The cult of “big data” equates the volume of data with its trustworthiness.

The reality is that every data collection scheme is a set of rules coded by humans, any experiment can hide inherent biases and every model’s assumptions could be wrong. If the methodology is faulty, then it doesn’t matter how much data you have. Size doesn’t trump technique: both matter.

Data scientists are duty-bound to question the viability of data sets, reconsider methods of analysis and question the degree of “truth” that can be extracted. Only then can we get closer to understanding what is happening, which is always more complex than a single number can hold.

5. Data can tell you what happened but rarely why
Data can sometimes tell you what is likely to happen. We can create a predictive algorithm based on correlations seen in existing (i.e. past) data, but this does not mean we understand the “why” of what we’re trying to predict. Before we act on correlation, we need to remember that correlation and causation are not the same thing.

6. Be pragmatic: Look at the metric(s) that helps you achieve your objectives
We are trying to learn and achieve different things on each platform, so we fit our choice of metric(s) to the specific situation at hand. This can be complicated and messy, but it’s realistic.

Are you interested in scale or direct response? Do you care more about time spent or number of shares? Data should inform your choices, not determine your strategy. An excessive focus on optimization can result in very small numbers and zero room for serendipity.

7. Stay focused on what’s really important
People talk about big data, small data, lean data, smart data. We try not to get caught up in the crazy. Stay focused on the problems you’re trying to solve, and keep questioning, collecting, scrubbing, learning, analyzing, testing and making mistakes. Then do it again and again.

In conclusion…
Here are two final thoughts: (1) Metrics should reflect what a marketer cares about, and that will differ not only from company to company but even between individual campaigns, and (2) What and how you measure success should always be changing. Even now, as BuzzFeed adopts a “global cross-platform” strategy, we are dreaming up new ways to understand and learn from data.

Exhausting? Sometimes. Exhilarating? All the time. Stay smart and good luck!

Reaching your Audience across Platforms
The Power of Culture