According to a recent study by Knowledge Networks fewer than 5 percent of social-media users age 13-54 “regularly turn to [social media] sites for guidance on purchase decisions” in a range of common product/service categories.
The study created a flurry of dissent among social media marketers, who were quick to point out flaws in the research.
Certainly its sweeping conclusions seem to conflict with findings from other studies that show social media does influence decisions.For example, in a survey of women online (PDF), “45% of survey respondents decided to purchase an item after reading about it on a blog.”
Part of the problem, as pointed out by blogger Chris Baggot is that the research focused on use of social networks but the report uses the term social media.
One blogger in the UK opines that the study is a “perfect example of a lack of genuine understanding of the medium.”
So how does a respected research organization fall flat on its face? Let’s see why. Three significant featues of the study that make it statistically questionable.
- Conducted over a less than a week, from March 10 through 16, 2009.
- Used a closed group of “502 members of KnowledgePanel®”
- Defines “social media” as 27 pre-selected social networks (Facebook, MySpace, etc.).
Every college student who has been through Statistics 101 knows that the reliability of results depends on sample size. That is, using a pre-selected group within such a short time frame is bound to lead to questions of statistical validity.
The online results do not provide error estimates (plus or minus percentage points). Again, a small sample size will yield high error estimates, which hinders drawing reliable conclusions.
In the aforementioned study of women in social media, the researchers sampled a population of 2,281 women over a month (March 2009). Overall, 85 percent of women responded that a purcase decision had been influenced by a recommendation or customer experience posted on a blog. The Knowledge Network study omits blogs entirely, so it was wrong to suggest its conclusions apply to all of social media.
The age distribution of the sample population, characterized as 13 to 54 is also misleading. How many 13 year olds are turning anywhere for “guidance on purchase decisions”? A more meaningful approach would have to show the data divided up among age groups.
In their haste to jump on the social media bandwagon, Knowledge Networks made several basic marketing research errors. Shame on them for muddying the waters, when so many budgets and jobs depend on accurate conclusions about the value of social media.