With the publication this month of the Internet Advertising Bureau’s ‘Social Media Ad Metrics Definitions’, it seemed a good moment for me to write about the thorny subject, and have a look at some of the controversy around the measurement of ROI in social media. I’ll return to the IAB’s publication later, but in the meantime, let’s take a look at the background into which it appeared.
The ROI within social media has long been a bone of contention, and seems likely to become ever more so, with the equally lightning spread of both social media use and savage budget cuts. In a tightening economy, businesses need more to make sure that they’re getting a return on their marketing investment. Are they in the right places? Doing the right things? With the right people? And how can they tell?
Well, aye, there’s the rub. Quoting David Alston in MarketingProfs: “The discussion of ROI has focused mostly on the search for the holy grail of a metric, but adapting traditional metrics to fit social media would be akin to sticking a square peg in a round hole.” ‘Inability to measure ROI’ was named by marketers as one of the most significant barriers to the adoption of social media tactics by their organization from a poll quoted by MarketingSherpa: “Marketers are under constant pressure to measure everything they do. The result is often a default to tactics that are more easily and accurately measureable, regardless of their effectiveness. This is especially true in social media marketing which often requires qualitative measurement rather than quantitative metrics that are more familiar to online marketers.”
It’s becoming easier to say why companies should be entering the social media arena.
It’s where your audience are, and shapes how they think. Universal McCann’s Social Media Research Wave 3 research report (published in Spring 08), looked at 17,000 Internet users in 29 countries. According to this study, social media can have a dramatic impact on your brand’s reputation. “34% post opinions about products and brands on their blog and 36% think more positively about companies that have blogs.”
“In the past year, traffic to Blogger and Wordpress went up 44% and 50% respectively, while traffic to CNN, MSNBC, and the New York Times went up an average of just 17%.” (Augie Ray, quoting figures derived from compete.com)
Your customers are valuable resources. A recent study in 2008 by DEI Worldwide on the Impact of Social Media on Customer Behavior was able to provide the following interesting statistics:
- 70% of consumers have visited social media sites to get information
- 49% of these consumers made a purchase decision based on the info they found through social media sites
- 60% of people in the study said they are likely to use social media sites to pass along info to other online
- 45% of people who searched for info via social media sites engaged in WOM compared to 36% who found info on a company or news site
A customer is worth far more than their initial spend with your company: you need to factor in future purchases and the influence they may have through social media. Quoting Jacob Morgan in MarketingProfs: “If we are going to start looking at social media in terms of quantifiable numbers then we need to start understanding how much our customers are worth, and this means looking at more than just how much a customer spends. A customer’s value is not equal to how much they spend at your store. It's far more”. Here are a few questions he thinks organizations need to be able to answer.
- How much is a customer worth?
- How much does the average customer spend per transaction?
- Is a customer more likely to purchase a product after some sort of social media interaction?
- How much is it worth to convert an unhappy customer to a happy customer?
- On average how many people does a happy customer tell about a product or service?
- How much can an unhappy customer hurt you?
The other reason is that your competitors are doing it. The University of Massachusetts Dartmouth Center for Marketing Research recently released the results on the usage of social media in the Inc. 500. “Just over one quarter of the Inc. 500 reported social media was very important to their business/marketing strategy in 2007. That number has increased to 44% just one year later.” Alston again: “More importantly, social media isn't private. It's out there boldly for the entire Web to see (including the almighty Google), and a company's absence is only highlighted by such accessibility.”
But are the old metrics of online ROI still applicable to social media?
Measuring the impact of online advertising used to be relatively easy. It was all about analytics: Unique Visitors, Page Views, Cost per Clicks: lovely, measurable, safe, defined metrics. Feel the warmth of that bathtub? But those engaged in social media must now attempt a way of measuring not just the online advertising within social media, but the framework surrounding that advertising, which is like trying to play three dimensional chess with a mischief of breeding mice as chess pieces. Quickly evolving, spreading, amorphous, capricious...
This is where the IAB’s ‘Social Media Ad Metrics Definitions’ have gone some way to filling the gap. They provide a clear framework by which advertisers can gauge ad effectiveness across three main social media: Social Media Sites, Blogs, and Widgets/Social Media Applications. The document divides social media into three distinct categories and defines the metrics specific to each type:
- Social Media Sites
- Blogs
- Widgets & Social Media Applications
“Evaluating the most important measurement terms will help marketers, agencies and publishers quantify the value added by consumers as they distribute the content throughout their personal networks, one of the defining characteristics of the platform.” (IAB press release)
What this means in effect is that the IAB has clearly defined each platform, and then provided metrics by which the effectiveness of each might be measured when planning a campaign. In very other words, how to judge where to place your ad in order to obtain the best ROI. The metrics include:
- Social media sites: Unique Visitors, Cost per unique visitor, Page views, Visits, Return Visits, Interaction rate, Time spent, Video installs, Relevant actions taken.
- Blogs: Conversation size (no. of sites, links and reach of a conversation whose content includes conversation phrases relevant to the client), Site relevance (Conversation density, Author credibility, Content freshness and relevance)
- Widgets and Social Media Applications – Installs (no. Applications), Active Users, Audience Profile, Unique User Reach, Growth, Influence, Installs – (no. installed per User)
The IAB’s definitions have so far met a mixed response. Some had high praise for the idea, such as Catherine P Taylor, on the Social Media Insider: “In other words, compared to old-time metrics like reach, frequency and the click-through, these metrics are deep, not only measuring whether people are engaged, but how they are engaging. It's like being able to measure the temperature with a thermometer rather than opening the front door and declaring it either hot or cold [...] imagine that you're an advertiser who sorely needs to understand social media. Then imagine yourself suddenly finding that you can not only monitor discussion around a certain topic near and dear to your brand but that you can also mention the number of people talking about it and their level of passion. Suddenly, social media goes from a huge, indefinable blob of conversations into something that has contours around which you can engage, plan and buy. That's huge.”
However, others weren’t so impressed, picking up on the fact that the qualitative aspect of social media measurement was sidestepped entirely. Augie Ray was one such, writing on his blog: “Perhaps one of the most concerning aspects of this report is that a reader might get the idea that every action is equal. The report suggests that "Comments posted" are worth measuring, but it says nothing whatsoever about sentiment within those comments. In the entire report, the following words do not appear even a single time: "Sentiment," "Attitude," "Rating," "Positive," and "Net Promoter Score." Apparently the IAB thinks that all comments should be tabulated in aggregate, regardless of whether they are disparaging or complimentary.”
In a further blog post in The Customer Collective, he puts forward his own four ‘dimensions’ by which clients should analyse a blog’s suitability for their brand: “Evaluating which blogs are right for your brand requires consideration of factors that include reach, relevance, and credibility. Reach refers to the audience for a blog, including total number of readers, subscribers, and visits. Relevance concerns the extent to which a blog's content and audience is appropriate to the brand; this can be determined by evaluating the number and frequency of blog posts on a given topic, the blog's organic search results for desired terms, and the percentage of readers or visitors that match desired demographics. Credibility, the third dimension that determines if a given blog is right for a brand's outreach or sponsorship, is perhaps the most difficult to ascertain. Reach and Relevance can be determined by quantitative measures, but Credibility is more qualitative”
According to Ray, blog Credibility has dimensions that include: Independence, Affinity, and Transparency. “These four attributes--Independence, Affinity, Transparency, and Presentation--are important determinants of a blog's credibility. These are not the only factors that enhance or diminish credibility, but they are among the most important. Other factors that may be important, depending on the blog and brand, include Blog Attribution (the more frequently sites and blogs attribute their sources, the more credible they will be received [to be]) and Blog Reputation (presence of awards, Diggs, etc.).”
So. There you have it for social media metrics.
Except that you don’t, not really. Because try as I might, nothing I see above has really told me about anything other than new media versions of traditional media campaigns: where to place your ad, sponsored widget or sponsored conversation, and how to judge its effectiveness. How about real social media, finding out how your forum is contributing to your reduction in online support, how customer reviews are affecting sales path, whether your 30 second TV spot crowdsourced in an innovative UGC campaign really made your audience feel engaged? It’s really not all about online advertising anymore. It’s about what else is happening in the social networks, very likely somewhere your company isn’t. “Web Analytics provide traditional attributes of page views, unique site visitors, etc. This alone isn’t adequate anymore because customers are talking to each other (& much of it is not on your site)” says Connie Benson. It’s about engagement with your customers, your potential customers, and your critics, at every level of social interaction that modern communication has to offer. Therefore, whilst we shouldn’t forget about these useful new definitions of measurement for your paid-for advertisement, we need to look beyond that, into the buzz, the opinions, voices and experiences that people are sharing about your brand.
Jason Falls in Social Media Explorer: "Perhaps we shouldn’t measure social media ROI in the first place. The reason? The core reason social media programs are successful is because they’re about people, not money. Look at the social media buzz words – community, conversation, dialog, sharing – all of them are people-centric, consumer-centric. Social media isn’t about sales. It isn’t about market share. It isn’t about profit margins. It’s not company or brand-centric."
But you’ve got to measure something to know if what’s working and how ...
Well, agreed. The ubiquitous Jeremiah Owyang provided an excellent guide on how to measure your social media programme as far back as 2006. (Don’t let that put you off though: it’s still completely relevant.) He starts off with the basics: if you haven’t got a goal, then you can’t measure against it. Before you start, work out what your social media programme is trying to do. Increase sales? Listen to customer feedback? Drive awareness? (To which I would add that you need to put your benchmarks in place here in order to judge success: Increase sales by how much? What are you going to do with that feedback? What percentage of awareness amongst your target audience are you aiming at?) Then build in measurement of this before you launch – not as an afterthought; as part of the process.
He guards against trying to measure it all – it can lead to ‘Analysis Paralysis’, and don’t we know it! “Shel Israel suggests we don’t measure the value of a phone anymore – we know the value. At some point, business blogging will be like email –everywhere”. Importantly, he exhorts us to monitor alerts as they happen (versus only reading reports): “A negative meme, an exploding battery could shatter your brand: are you watching in real time?” and finally, he helpfully provides a list of attributes to measure:
- Activity (Web Analytics of blog or site)
- Tone (Sentiment)
- Velocity (Spread over time, URLs, Trackbacks)
- Attention (Duration on site)
- Participation (comments, trackbacks)
- Many qualitative attributes (comments, what did they say, what did they mean)
“ You may never measure the same way for each program, the goals of each program will change the method in which you measure. The attributes will stay the same, but you’ll just use them in different ways to create a new report [...] A bunch of facts and figures are not good enough to tell the story. What happened, where are we now, and where are we going? Did more folks come into the forum? How? And why? What are they doing now? [...] What really matters are the opinions, voices, and experiences that people are sharing. The written anecdote that persuades prospects to customers (or the other way around) is very important. Be sure to monitor and use these in your reports.”
Connie Benson is a big fan of Jeremiah’s, and based her post on Social Media Measurement on a webinar he gave in 2008. She cites another list of attributes:
“Attributes that need measurement:
- Attention = time on website (sometimes it’s not yours)
- Interaction = Click, Comment, or Embed
- Conversation Index: ratio between blog posts and comments+trackbacks
- Velocity * = Distance/Time (Rate of how fast your message is travelling in a given time) Jeremiah defines Velocity with examples
- Sentiment * = Tone, opinion
- Qualitative * = What did they say?
- Impacts * = What did they do? (influencers)
* Brand Monitoring Solutions - these are helpful with the last four above“Tracking the tone, perception and nuances within online conversations is getting easier. There are a number of solutions available that monitor discussion of your brand online. They offer options to track & create reports on frequency, influence & sentiment.”
Connie recommends using Techrigy SM2 for social media monitoring: “It offers an efficient way to measure brand presence, brand perception, opinion & tone. And it also identifies the conversations that the brand should be engaging with.” (Techrigy SM2 is also amongst a review of a massive 67 social media/web/reputation management tools and sites recently provided by Marc Meyer at Social Media Today : a useful reference).
She provides a great guide for the community manager’s role (versus the marketer’s), listing the new places to monitor/measure (social bookmarking sites, social networking sites, aggregators, micromedia, forums, competitor sites), a methodology for finding new communities in which to listen and participate, and finally, examples of things you can measure:
Using above methodology to find customers & prospective markets
- Listen, understand the conversation, then participate
- Measure the number of conversations
- Monitor the percentage increase of conversations over time
- Measure the reduced buying cycle & reduce support costs by encouraging self-support
- Increased sales due to increased customer satisfaction in product due to involving them in product development cycle
- Increased efficiency in developing products due to customer feedback at various stages
- Minimize brand damage by responding quickly to customer’s concerns online
Connie also recommends that community managers provide a report on a monthly basis, with:
- Ongoing definition of objectives
- Web analytics
- Interaction - Trends in members, topics, discovery of new communities
- Qualitative Quotes - helpful for feedback & marketing
- Recommendations - Based on interactions with the customers
- Benchmark based on previous report
So, all that was great – but where does that all leave the ROI question?Divided, I’d say, into three main camps.
The Deniers: What, really, is this obsession with ROI? This is the group who are most resistant to any attempt to measure social media; possibly because past attempts have been so clumsy and obviously missing the point.
The Definers: Never mind the community stuff. Just give me the click through rate. Please. IAB’s new metrics were made for this group. They are the ones who’ll be the best audience for the new definitions, and are likely to be hugely comforted by a large benchmark and a level playing field upon which to chalk it.
The Dedicated: It’s not a perfect science, and it’s a lot of work. But we’re getting there. This group are realising the importance of using every available system of measurement we have: web analytics, buzz monitoring, community management listening – the whole gamut – to try to bring in as near to a 360 degree picture of people’s reactions to and interactions with, a brand, as is currently possible.
I’ll leave you with a splendid quote from
Jeremiah Owyang: “Social media measurement is like driving a modern car. You may have a dashboard with all the lights, toggles, gauges, and metrics, but remember, the most important piece of data to have in front of you is the GPS screen. The GPS screen indicates where you want to go (your objective), where you are now, and how to get there.”
Read more...
