Social media research is chock full of leading questions

by David Spark on June 29, 2009

Jeremiah Owyang, Forrester

Jeremiah Owyang, Forrester

“We did research and asked consumers who they trust. 90% [of the people responded] said ‘people like them.’ Under 10% [of the people responded] trusted corporate blogs,” said Jeremiah Owyang of Forrester, as reported by Louis Gray at the San Francisco American Marketing Association (SFAMA) event last week, “How Tech Bloggers are Leading the Marketing 2.0 Revolution.”

Owyang is an industry friend and respected social media analyst, but I find this result to be obvious, yet at the same time misleading to the entire story of what corporations can do in the social media sphere. The question of “Do you trust corporate blogs?” is a leading question. Of course we know how people are going to answer it.

I attended the SFAMA event last Thursday and because I got distracted, I didn’t get a chance to question Jeremiah further. If I had, I know I would get a more in depth story, since consulting about social media for large corporations is what Owyang does for a living. The problem is when someone of his stature uses his well respected research outlet to make public statements such as, “Only ten percent of people trust corporate blogs” it resonates as if there’s no hope for corporations in social media and they simply should give up.

What were the questions Forrester asked that yielded those results? If all you do is simply ask a question, “Do you trust your friends’ blogs or corporate blogs more?” of course you’re going to get people saying they trust their friends over corporations. Even if Forrester split that question into two questions, it still would be lopsided.

What I’ve learned from all the social media events I’ve attended and research I’ve read is that social media research is chock full of leading questions. I’m sure that Forrester asked more involved questions, but unfortunately “corporate blogs are not trusted” is the result that’s being quoted. What researchers need to do is spend more time questioning the reader to see if there is any preconceptions, confusion, or conflicting stories. That’s where the real stories in social media research lie.

For example, sure, go ahead and ask that leading question, “Do you trust corporate blogging?” But then dig deeper and see if there are inconsistencies. Start asking the respondents the following questions:

(Continue asking if they read or follow any other well known corporate brands in the social media sphere)

  • Can you remember anything you read? Have you ever commented, retweeted, or responded to anything they’ve said?

You’ll be surprised that the same people who fell in the camp of “not trusting corporate brands” are also huge fans of well known and trusted corporate brands such as Southwest Airlines and Zappos. If you read, respond, and have a personal connection with a blog or Twitter account, you know there’s a person behind it. In those cases, do those respondents who track Southwest and Zappos even consider them as corporate brands?

What I would have liked to hear Owyang say, “Yes, we got people saying they hated corporate brands, but when we dug deeper, we discovered that they loved corporate brands that had a strong social media presence.” The problem is there’s a preconceived notion when people are asked, “Do you trust corporate blogs?” Please note that I am making this up and it’s only my prediction, but I’m sure there’s plenty of truth to it and I invite Owyang and anybody else who researches social media to agree or refute my estimates.

IDC’s misleading research about online advertising

A year ago I attended another social media conference, the SWAT Summit in San Francisco, and a woman from the research outlet IDC conducted an analysis of consumer social media use and attitudes towards targeted online advertising on social networking services (SNS). In the study, IDC asked people the question, “Do you want services tracking your online behavior and information so as to serve you more targeted advertisements?” Once again, as you might imagine, the overwhelming response was no. At the same time, they also asked people if they use Web 2.0 services such as Facebook, Hotmail, Yahoo! Mail, and Gmail which all track user behavior to serve up more targeted advertising. Do people even know that? Would they stop using them if they knew that? Did they even connect the question they were being asked to the way those tools behave?

Here’s what I was amazed by. The IDC researcher didn’t cross reference those responses. There’s obviously a huge discrepancy between what people say they want and what they actually use when it comes to Web 2.0 services. At the time, the IDC presenter said that since she had the information, all she needed to do was cross reference the results and she’ll get her answer. She said she would send those results to me. Guess what? It’s been one year and there’s no information. I’ll send her a reminder.

BTW, as a side note. In every single case where I’ve asked the question of “Do you have this information?” I often get the response, “Not right now, but I’ll send it to you.” In all the years that I’ve asked that question, no one has ever followed up, ever. I think they think if I’m really interested I’ll follow up with them and hound them.

Research companies need to tell the stories of discrepancies

The cross referencing of IDC’s research, between what people say they want and what they actually use is the story they should have uncovered. And that’s exactly what research companies such as IDC and Forrester need to do. Go ahead and ask the obvious questions, but show where people are confused about the tools they’re using. Don’t let them get away with an answer such as “security is really important to me” without discovering how often they change their passwords.

Ultimately though IDC’s data didn’t go deep enough and I need to learn more about the Forrester study.

My question out to all of you reading this is, do you take social media research results at face value or do you question them? Do you question the research methodologies? Do you simply never question the studies because you implicitly trust the research outfit? I mean, geez, it’s Forrester, it’s IDC, they’ve got to know more about research than I do, right?

No, not always. Because they’re not going for the stories that the research can reveal. It’s your job to push them and to constantly ask the questions. There’s always a story to be told.

Previous post:

Next post: