web analytics

The Battle Over The Accuracy Of Social Media-Based Consumer Insights Research

The Battle Over The Accuracy Of Social Media-Based Consumer Insights Research

In grad school, my marketing research professor used to say “n=1” as her is very geeky way of qualifying her opinion. Professor Escales’ quant humor not withstanding, I guess I will caveat this blog with “n=1” but you’ll find my argument reinforced which quite a few “n’s” so take from it what you will.

Late last night Gareth Price, a UK-based social media researcher, basically used Pew’s latest social media demographics study to debunk the use of social media as a consumer research tool. Though I get what Gareth is saying and agree that you can’t use social media to answer a question that hasn’t been asked, and that social media is still very much a convenience sample – I fundamentally disagree that its not a valid research tool, particularly for brands.

First off, you need to know that I conduct research via social media for brands for a living.  Not that this makes me any more or less of an expert but I have seen the use of this type of research before. I’ll also say that Gareth’s focus on Twitter is important because, in my estimation, anywhere from 40 to 75 percent of any topic conversation is based on Twitter, arguably more than any other platform.

The key thing I believe Gareth left out was the fact that social media users, and Twitter users in particular, tend to skew heavily towards users that are more influential offline. Said another way, social media may be a convenience sample but its one that is remarkably powerful in identifying the tip of a much larger consumer insights iceberg.

Sure, Twitter demographics alone speak to the fact it is not a representative sample of the populous:


In fact, Gareth is never more right in his implication that Twitter reaction to large public events should not  be considered scientific – the 2012 election proved that.  However, Twitter has proven to be surprisingly accurate as a tool to predict stock-market fluctuations and outbreaks of the flu.

More importantly for brands, social media users are a proven barometer of potential purchase intent.  This means that despite its flaws, Twitter can and should be a goldmine for helping to pinpoint not only valuable and powerful consumer insights but as a self-selected representation of those who might indeed be most likely to both purchase your product and to tell others about it.

Gareth, I love your insights but let’s admit that there’s much more that goes on, on Twitter that meets the eye.  Yes, we need more tools and more geeky marketing research applied to the medium to bring it up to the rigor used in most research tools.  However, there are just to many “n’s” out there that speak to the power and impact social media research can have on brands.

You Might Also Like


  • Gareth Price

    Thanks for following up on the article, Jared. However, I think you have mischaracterized my views somewhat.

    I certainly didn’t intend to ‘debunk’ social media research – not least because I work in the same field too! I agree it’s a perfectly valid research tool and, in some cases, the best approach to use.

    However, I think it’s important to recognise it is a skewed demographic, that isn’t reflective of the population at large (which invalidates it as a traditional quantitative approach) and that people aren’t talking about everything (so it can’t replace all forms of research).

    You stated that “the key thing I believe Gareth left out was the fact that social media users, and Twitter users in particular, tend to skew heavily towards users that are more influential offline”.

    However, I intended to convey that exact point when I said Twitter’s users are “early adopters [who] are more likely to be those interested in being part of and shaping the public sphere [which] demonstrates we generally capture the opinions of people who feel strongly about issues”.

    As you point out, in some instances, it can work as a predictive tool but the fact it doesn’t reflect the population at large means that, in things like an election, it can only act as a guide (not to mention the fact it could be ‘gamed’).

    I think, on the whole, we perhaps agree far more than we disagree and I can definitely confirm I didn’t intend to debunk this form of research. However, I think it is important that those of us who work in the area acknowledge its limitations and focus on where and how it’s best used.

    As I’ve written before, I think it’s best to think of social media research in more of a qualitative sense, with less focus on quantitative metrics (which frequently just act as a point of context): http://conversation.cipr.co.uk/posts/gareth.price/the-power-and-danger-of-the-story

    Thanks again for the response though.

    March 26, 2013 at 4:50 am Reply
    • Jared

      Hi Gareth,

      Thanks for pointing out the larger article! PR Daily, where I initially read the article, tends to cut off the details of the argument.

      It definitely confirms that we agree on pretty much every point and I agree that unless you qualify the sample you’re just begging for abuse of social media as a research tool. That being said, I think its worth applauding the people who do at least try to conduct some form of research because at least its better than going from no data at all.

      I actually think that Charles Wheelan (http://www.nakedeconomics.com/) does a good job detailing the issues we’re talking about. He explains that statistics can be pretty dangerous in the hands of someone that doesn’t understand how to qualify where the data comes from. What CW also does is make the case for better education around the subject which is exactly what you and I are doing although I didn’t see it at first.

      I agree – we need more of this discussion and more education. I think we would both agree we’d like to see more marketers get serious about serious research and apply a more rigorous and informed approach to to research,


      March 26, 2013 at 7:29 am Reply
  • Mauro Murgia (@Mauro_Murgia)

    Sorry to jump into this conversation with a comment that might be slightly off-topic, but I found very interesting how the principles that you are discussing could be applied to politics, not just consumer research.

    Probably you are not very familiar with the political situation in Italy at the moment, but there’s a lot of talking about “digital democracy” advocated by the new party Movimento 5 Stelle, which is pretty much the second biggest party following the last elections. They think democracy should be open to digital conversation and the decisions made by politicians influenced by it.

    I couldn’t agree more with Gareth when he says that “I think it’s important to recognise it is a skewed demographic, that isn’t reflective of the population at large”. Especially in Italy, with its ageing population and relatively small number of people engaging with social media, digital democracy would be fundamentally flawed, and frankly pretty scary.

    March 26, 2013 at 8:24 am Reply
  • Leave a Reply

    This site uses Akismet to reduce spam. Learn how your comment data is processed.

    The Power Of “Alpha Ideas” In Digital Marketing

    © 2015 – 2016 Degnan Media | All rights reserved.