David Rabjohns: Garbage in, garbage out: What you don’t know could be destroying your measurement accuracy — Live from the Brands-Only Summit

Coverage of this session by Bridgette Cude of SocialMedia.org. Connect with her by following her on Twitter.

11:00 — SocialMedia.org's Courtney Graham introduces MotiveQuest's David Rabjohns.

11:01 — David: When you're eating a tub of ice cream and someone insists on showing you the ingredients, you really don't want to know what's in it. That's me today. I'd like to show you some of the ingredients that go into your social media measurement: some tasty, some not.

11:02 — In one example, During the Beijing Olympics, Visa's metrics were telling them they were the most talked about brand at the Olympics, but unfortunately, it was all negative.

11:03 — However there were a few problems with their data:

  1. Most of their data was too old.
  2. Their data was too broad: They were looking at all conversations, not just the Olympics.
  3. They didn't have a great way of filtering out “visa” conversations about the brand versus things like travel, etc.

Turns out, they weren't the most talked about, but they did have the most positive sentiment.

11:04 — There are four things that affect the quality of your data:

  1. Data
  2. Organization
  3. Terms
  4. Analytics

11:05 — David: A lot of people think of social media as one big bucket of data. But it's actually very nuanced.

11:06 — Data Sources: Better inputs give you better insights. Twitter isn't a great driver for a prediction of sales. Community forums however can give you a rich understanding of what people are talking about and why they're talking about it. Split up the analtyics. Analyze Twitter differently from forums and forums differently Facebook, etc.

11:07 — For example, here are some strengths of each channel:

  • Facebook: Understanding how consumers interact with your brand
  • Twitter: Monitor news
  • Blogs: Deep dives into insights (but watch for spam)
  • Forums and message boards: Behavioral insights
  • YouTube: Complementary insights

11:08 — What are you doing with the data? Social media monitoring or human insights?

If you're trying to figure out what's driving advocacy, track opinions, or inform brand positioning, look into forums, message boards, and blogs. For social media monitoring, like tracking likes, share, and tweets, Twitter and Facebook are better indicators.

11:09 — Organization: One of the best tricks with social data is to organize it within someone's naturally occurring passion. Start with the source level data, not keyword data. Then, look at each category individually and determine what's going on in each bucket.

Imagine the magazine rack at Barnes and Noble: cars, motorcycles, fashion, babies, weddings. People cluster around what they're passionate about, not your brand.

11:10 — For example: A lot of people talk about pickles. But only 30 percent of conversations about pickles is around food and dieting.

People talk about pickle juice being a great source of hydration for athletes. In the personal finance category, people were offering advice on making your own. Some were adding jalepenos to their pickle jar for spiciness (an idea that Kraft reflected back with jalepeno flavored pickles.)

Kraft had the third highest ROI just from reflecting back to people the ways people were already using them in their community.

11:11 — Important: brands don't make up much of that conversation. If you want to understand what people care about, don't just do brand monitoring, do category monitoring.

11:12 — Terms: The trick is linguistic strings, but it's important to spend quality time with those linguistic strings. Use Bolean strings: Ands, not, but..

11:13 — For each of your brands, and each of your terms, you need elaborate on those explorations.

11:14 — Analytics: How you look at your data matters:

11:15 — Turns out, buzz had very little to do with sales, and sentiment was pathetic in terms of it. The only thing that was a good driver of sales was online recommendations (real time Net Promoter Scores)

That way, you can say: If advocacy is up in June, sales will be up in August. So what's driving advocacy? And what things are a waste of money?

11:16 — You have to take the data, crunch it through, and really think about what drives sales.

  • Advocacy is the best predictor of sales.
  • Detractors are not a good predictor of sales (because the people who detract your brand don't buy from you anyway).
  • Sentiment is a pathetic predictor of sales.

11:17 — Treat analytics like a scientist: Go in with a hypothesis, then take those metrics and test them. At the end of the day you can come back to your company with models that work for your brand or category.

11:18 — David: I know this is nothing new. But go back into your data, and see if there are other ways you can look at it and analyze it.


Q: Listening insight and the source: One of the biggest sources we have is Twitter and I don't feel like we have as much access to online communities. What do you think about the difference between data in owned and hosted communities?

A: I would analyze those two (hosted vs. owned channels) differently.

You're not going to like my answer to your first question. I've been trying to collect this data for four years, I've never found a vendor that's good at measuring forums. You basically have to replicate a human clicking through all of the different topics.

Q: Do you know if an advocate's recommendation is diminished if it's sponsored?

A: If people know it's sponsored, it's diminished. And people are very savvy about that. They can sniff those out very well. The sponsored conversations don't really get any engagement.

Q: For companies that don't have budgets for market research firms, what tools do you recommend to dip or toes in the water and voiuch for the budget?

A: My favorite is to start with Google. You're going to find people talking about your brand and qualitatively get a sense of what people are talking about. You can find out a tremendous amount if you take the time.