E-MAIL | PRINT | SAVE | LICENSE  

By Emily Wheeler

and Samara Omundson

Information source evaluation has always been part of the research and information professional's job. As information formats and delivery channels evolve, so must the approaches that we take to evaluate sources. The recent explosion of social media tools offers a unique and appealing opportunity in the evolution of source evaluation approaches.

In our particular organisation, we are consistently challenged to find a methodology for evaluating the importance and influence of media sources, regardless of format or delivery channel. For example, how does the New York Times website rank in comparison to The Huffington Post or Newsweek magazine? After some investigation, we realised that social media tools can provide pertinent data to answer just this type of question.

Subsequently, we developed a framework that leverages a variety of social media tools to evaluate the importance and influence of media sources. The resulting methodology is data-driven, easy to replicate, and applicable whether we are evaluating a list of blogs, podcasts, twitter accounts, or more traditional media publications.

 Creating the Evaluation Framework

Our first step in developing this methodology was to identify available metrics that could be indicators of the influence of various information sources and build a list of relevant social media and traditional research tools. In compiling our list of influence metrics, we tried to look beyond the traditional applications of specific social media tools to find underlying metrics.

For example, BlogPulse is a blog search engine that is geared toward general keyword searches, but its Trend Search feature maps the percent of the blogosphere referencing a topic by day. By searching for a website's URL using this Trend Search, we were able to identify a data point for the peak day when blogs linked to the URL in the past x months. This hidden data point was added to our list of metrics.

 Here is a list of metrics identified in our preliminary research:
 

For each Influence Attribute, we created a score based on the normalised data for each underlying metric. These scores could then be combined into a composite score, allowing us to rank lists of media sources by total influence or by the individual Influence Attribute, depending on client or project needs.   

 Applying the Evaluation Framework

To illustrate the application of the Influence Ranking framework, we collected sample data from blogs that cover the eReader or eBook industries. Although we focused exclusively on blogs to simplify this example, the framework could be applied to multiple content channels, including Twitter, podcasts, vlogs, print or online media.

 
1.       The first step was to collect data for each blog. The table below shows selected metrics under the Buzz Attribute.

2.       Next, we normalised the data for each metric on a five-point scale.
 
3.       We calculated a score for each Influence Attribute, again on a five-point scale (1 = low influence, 5 = high influence). Weights could be applied to Attribute scores at this stage, though none were used in this example. The table below shows the Attribute scores for each blog as well as a composite Total Influence score based on a sum of the Attribute scores.

When presenting Influence Ranking results to clients, we include a table similar to the one shown above as well as breakdowns by Influence Attribute and media type, as needed. Our eReader media list combined general technology blogs with niche industry blogs. The general technology blogs ranked highest in terms of Total Influence despite low Content and Audience scores. Separating the rankings by blog type would provide an opportunity to explain the differences between these blogs and recommend targeted actions.
                                 

Summary

 Our organisation has been successfully applying the Influence Ranking framework in research requests for over a year. In that time, it has significantly improved our ability to create robust media lists for our clients. The Influence Attribute scores have provided starting points for further discussion of the strengths and weaknesses of specific sources. And the consistent, replicable methodology has opened up opportunities for further research into media trends.

While we developed this framework in response to an industry-specific business need, we hope that the explanation of our process can help other information professionals develop similar systems to consistently evaluate information sources.

By Emily Wheeler

and Samara Omundson

Emily Wheeler is a Research Manager at Waggener Edstrom Worldwide, a global public relations firm. She has been with Waggener Edstrom for four years and specializes in secondary research and media analysis.

Samara Omundson is Research Director at Waggener Edstrom Worldwide, a global public relations firm. She has a decade of experience in media-related research and analysis. She is a past President of the Oregon Special Libraries Association.

FUMSI articles by Emily Wheeler »

Click here for copyright permissions!
Copyright 2010 Free Pint Limited.

Related articles:

• Social Networking: A Research Tool
• Second Life for Business: Ten Techniques
• Social media strategy: How I Became an Expert in Three Days
• Real-Time Monitoring: Tools and Trends to Watch

You may also be interested in:

• Subscribing to FUMSI »
  
  
• Browse the articles database for more Find articles »
  
  
• Search the articles database for more Find articles »
  
  
• Tools and reports for Finding Information »