Searching for 10,000 Missing Kittens

By Dmitria Burby | 0 Comments | Posted in in Analytics , Search | Permalink

Recently, the conversation of matching paid search clicks (from Google or Dart) to paid search site side reporting (Omniture or WebTrends) came back on the radar. I have had this same conversation many times in the past and have given many good reasons, but the truth of the matter is that the two sets of numbers will never match and we as a collective group should stop trying to get them to match. The two systems are both correct, it is not so much that there is a data ‘discrepancy’ - which implies error, as it is data ‘difference’ – meaning that there are different purposes for the two data sets. There are also implications with how the data is passed and measured (which we discuss below).
   
Data from paid search providers is concerned with reporting on individual actions (clicks), because it goes to how investments are made on individual keywords. Web analytics is generally concerned with data corresponding to individuals – which is why referral sources are often reported in terms of 'visits' or ‘visitors’ – and so if there is multiple search queries occurring from one ‘visitor’ web analytics will generally only report that single visitor as the referral.
   
Take for example that you have bought the keyword, "Kittens" (don't ask why I chose this word, I couldn't come up with something more appropriate). Google is reporting that you have 30,000 paid clicks on the keyword "Kittens," yet your site side reporting shows that you only have 20,000 paid click-throughs on "Kittens." Where are the other 10,000 clicks going?
   
I know that is hard to believe that both systems are correct when your clicks are 30% or even 40% higher than the click-throughs that are captured site side, but it is true. Think about it this way, the click is the intent to view content on your site and the click-through is the actual action of seeing content on your site. A lot can happen between the click of a content targeting link or keyword and browsing through site content.
     
Think about the fundamental differences in log file based tracking and javascript/tag based tracking. When the transition to javascript/tag tracking started we had several clients that wanted to compare the numbers from both sets of data. More often than not, we saw that the javascript/tag based tracking was between 20% and 50% lower than the log file based tracking. This shouldn't be surprising as tag based tracking was a much more accurate count of what visitors were doing on your site. The point of this statement is that there is a fundamental difference in the amount of content that servers serve up and the amount of content that is consumed by true consumers or visitors. Take the same approach with media reporting, there is a difference in the amount of content that is served up and reported as "clicked" versus the number of "click-throughs" that reach your site.
   
Some items that are noteworthy and difficult to change, but give some explanation of where those 10,000 clicks are:

• In addition to focusing on clicks vs. visitors, paid search assumes ‘match’ caveats for it’s keyword referral data. In other words "Kittens" may be the bidded term, but if matching parameters are tied to that term (broad, phrase) the data corresponding to that term would include dozens or hundreds of specific queries that included the word “Kittens”. Whereas site side analytics report on the actual keywords typed by the user, say "pink kittens," "stuffed kittens", etc.
• If paid search is using ‘content networks’ that click data will be reported as paid-search clicks, whereas web analytics tools will report those as site referrals like http://www.pinkkittendanceschool.com/blog/ (again, I apologize for the direction this example has taken)
• Some of the clicks on banners and paid search bounce from the site (or never reach the site) before the site side analytics tag fires. This happens more than you would think since the click is tracked on the search side before the redirect takes place.
• Every so often the tracking tags are dropped by the search engines.
• Filters on site side metrics can exclude clicks. Examples of this may be exclusion of internal traffic, spiders, etc.
• First Party Cookies and Third Party Cookies are handled differently by browsers.

     
With all of that being said, there are still ways to ensure that the numbers being reported are as close as possible.

• Ensure tags are placed on all of your paid search activities and all pages on your site.
• Ensure that the reporting attribution windows are the same in both tools.

   
Once you have taken the steps to ensure the data is as accurate as possible, do an audit to gain a baseline understanding of what the discrepancy is for your company. Understand, acknowledge and educate the consumers of the paid search data on why the data sets have a discrepancy and agree within your organization which source of truth you are going to use. Since most organizations are looking at behavioral data through tools like Omniture or WebTrends, it often makes sense to use these tools are the primary source of data to understand what people do once they land on the site.