Tuesday, November 27th, 2012

How Google Analytics *Really* Handles Referring Traffic Sources [Experiment] – Why Clicks and Visits Might Not Match Up

Google Analytics Referrals

Let me walk you through a common scenario in web marketing.  You have a website, and some people visit your site by clicking through links on other websites.  In your web analytics reporting, those visits are categorized as referring visits.  In Google Analytics specifically, those visits show up in your “Referrals” report under “Traffic Sources”.  And when visitors click on an outbound link on your site (a link to another website), your site shows up as a referring source in that website’s referrals report.

That’s pretty straight forward, but what I’m about to cover isn’t.  I find many marketers and webmasters don’t understand how Google Analytics handles that referring traffic during future visits to their websites.  For example, if someone clicks through to your site from sampledomain.com, leaves your site, and then returns the next day.

Do you know how that visit will be categorized in Google Analytics?  There’s a good chance you don’t, and I’m going to cover the topic in detail in this post.

Understanding Referring Visits is Important When Revenue and Cost Are Involved
I believe one of the reasons this topic isn’t understood very well is because it often doesn’t directly impact revenue or cost for many webmasters.  Sure, you definitely want to know how many people are coming from each referring site, but for many webmasters, the exact number doesn’t impact revenue, or payments to other webmasters.

But, for websites that need to track the monetary value of inbound visits and outbound clicks, accurately determining referring visits is extremely important.  For example, imagine you were charging certain partners for traffic you were sending from your site to theirs, or vice versa.  The fact of the matter is that checking referring sources could show different numbers than you think, and could be much different than the outbound clicks you see.  And depending on your own situation, the numbers could be way off…

The Core Disconnect – How Google Analytics Calculates Visits from Referring Sources (or any campaign, search visit, etc.)
Here’s the core disconnect.  When someone clicks through to your site via a referring source, the utm_z cookie is updated with traffic source information.  That cookie will not be overwritten unless another referring source or campaign takes it place.  Direct Traffic will not overwrite this value.  Let me say that again.  Direct Traffic will not overwrite the utm_z cookie value.  That means the utm_z value will remain the referring source of traffic when those visitors return to the site.

Google Analytics utm_z Cookie


What This Means To You

I know what you’re thinking. This guy is telling me about utm_z cookies?? What the heck does that mean to me?  OK, stick with me for a second.  Let’s say you had a partnership set up where another website pays you for traffic.  Maybe you’re both in the same niche and want to leverage each other’s traffic for more exposure.  You check your stats for the previous month and notice that you sent 500 visits to partner A.  Cool, so you contact them to check how the partnership is going and to make sure they are seeing the same number of visits.  They come back and say they’ve seen 700 visits from your site and thank you for the traffic.  The check will be cut soon.

Google Analytics Clicks and Visits Could Be Off

But that 200 visit discrepancy is bothering you.  Why is there a big difference between your partner’s reporting and the numbers you are seeing?  And let’s assume you have a solid setup for tracking clicks out of your website.  For example, maybe you are running outbound clicks to partners through a redirect that captures a number of important metrics.  The redirect then sends the visitor off to the correct URL on the partner site.  Basically, you know you are capturing all outbound clicks to the partner website.

This is where the native handling of referring sources in Google Analytics comes into play.  Sure, you are tracking clicks off your site, but your partner’s analytics package is capturing those clicks plus any return visits that are direct visits.  So, if someone clicks through to your partner’s site, then that’s one visit.  If they leave that site, and return directly (by typing the url directly in their browser or via a bookmark), then the visit will show up as a visit from the original referring source (your website).  That’s now two visits.  And if they do it again, that will be three visits.  That’s until another referring source or campaign overwrites the utm_z cookie.  In this example, there were 3 visits to your 1 outbound click!

An Example of How Google Analytics Handles Referrals

Based on this simple example, you can easily see how over a month’s time, some people would click through to your partner’s site and then revisit their site directly (and possibly a few times).  That would lead to more than one visit per user, and could sway the visit count from your website.

Still confused?  Let me clear this up via an experiment below.

Experiment – Calculating Referring Visits in Google Analytics
In the following simple example, I set up a webpage on a second domain that links to a landing page I set up on my website just for this experiment.  I didn’t want to skew the reporting by using an existing page on my site that gets a lot of visits.  Then I used several computers I have here with clean browsers to first visit the referring page that links to my new landing page, and then I clicked through.  The referring source should show up as the domain name of the referring site.  That would be visit #1.

Next, I would leave the new landing page on my site and revisit my website later by typing the exact URL into my browser (what most people would think is a Direct Traffic visit).  In theory, the referring site should show up as the traffic source, even though I’m entering the site as “Direct Traffic”.  Remember, the utm_z cookie will only be overwritten by another referring source or campaign.

Last, I would search for a keyword that my site ranks for, and then click through to the site.  And since this visit was from a search engine, the utm_z cookie would be updated with this new value, and my reporting would show Google as the referring source (along with the keyword I entered). Let’s find out the results of the experiment below.

The Results
1. First Visit

First, I visited the second domain and clicked through to my website.  Here is the first referring visit showing up in my analytics package:
Referring Sites Experiment - First Visit

2. Second Visit (Directly Visiting the Site)
Next, I left the site and returned via Direct Traffic.  Google Analytics shows the referring site as the source for this traffic, even though I entered via “Direct Traffic”. Also notice it accurately categorizes me as a “return visitor”:
Google Analytics Referral Experiment - Second Visit

3. Third Visit (Again Directly Visiting the Site, but the Next Day)
Just to underscore my point, I left and revisited the site the next day (again via Direct Traffic).  Google Analytics again shows the visit is from the initial referring source:
Google Analytics Referral Experiment - Third Visit

4. Fourth Visit, This Time From Search
Finally, I searched for my name on Google and visited my website.  Now Google Analytics shows the keyword that led to the site (from the traffic source “Google”).  Remember, the utm_z cookie will only be updated when another referring source is identified (versus Direct Traffic).
Google Analytics Referral Experiment - Search Visit

 

So there you have it.  Proof that your visit count by source may not be what you think it is.  Now, if you’re reading this post and are either generating revenue from referring visits, or you have to pay partners based on visits, then you might be frantically running to Google Analytics to rerun your reports.  Yes, this could impact things quite a bit.   I’ll leave it up to you how you handle the situation. :)

What You Can Do – The Importance of Clarity
If you do have a partnership where you are either generating revenue by driving traffic, or you are paying for traffic from other websites, then each party needs to clearly understand the arrangement.  Each website involved needs to be clear on the definitions of “traffic”, “clicks”, “visits”, etc.  For example, think about AdWords for a second.  You pay Google for clicks on ads, but don’t pay Google for direct visits back to your site (even though those visits will show up as campaign visits).  And by the way, most partners will not give you access to their reporting anyway… Therefore, you will only know the clicks out from your site.

If you are tracking outbound clicks, you can use event tracking in Google Analytics to track those clicks, including the pages or links where those clicks are originating.  If you don’t want to use event tracking, then you can run outbound clicks through a 302 redirect and capture the information you need to accurately track clicks.  If you are receiving traffic, then you can make sure the referring links contain querystring parameters so you can understand which partner the traffic is coming from (and that it’s not a standard referral from the site).  There are other ways to handle this, and those are just a few ideas.

Summary – Understanding Visits in Google Analytics
I hope you found this post explaining how Google Analytics handles referring visits helpful.  I know this topic can be confusing, and experiments always help clear up some of the confusion.  So now you know why visits might be higher or lower than you think, and how the utm_z cookie controls what shows up in your reporting.  I bet you’ll never look at referring sources the same again.

And let’s hope you’re not on the short end of the stick. :)

Happy Reporting.

GG

 

  • Damion Brown

    Fantastic post and analysis.

    Presumably there’s no way of calling a specific update to the utm_z cookie outside of the strict parameters set by GA?

    I can think of a lot of examples where a webmaster receiving traffic might only want to credit the first click; certain affiliate models, for example. Do other analytics packages allow for control over how repeat visits are handled?

    • http://www.hmtweb.com/marketing-blog/ Glenn Gabe

      Thanks Damion. I’m glad you enjoyed my post. That’s a great question regarding setting the utm_z cookie. I’m pretty sure you can’t do that in the standard implementation of Google Analytics, and I’m also pretty sure you can’t do that in Google Analytics Premium either.

      In premium, you can leverage attribution modeling, but that wouldn’t overwrite the utm_z values. It would just give you a stronger look at attribution across channels.

      Regarding other packages, I’m not sure if others allow you control how return visits are handled. In theory, you should be able to customize any of the major platforms, but I haven’t been part of an implementation that refined how this specific situation worked. I hope that helps.

  • calebdonegan

    Great article! One of those issues that I learned a while back, during the certification exam, but have long since forgotten. Do a lot of reports for clients, this is something I will need to consider moving forward.

    • http://www.hmtweb.com/marketing-blog/ Glenn Gabe

      Hey, thanks for your comment Cale. Glad you found my post post helpful. Regarding your reports, there’s so much confusion about how this works in GA, that walking clients through this process would be a great idea. I just did that last week with a client (which triggered the idea for this post).
      :)

  • http://www.ezmaal.com/ hyderali

    Fantastic Article!

    Never thought of it & never came to my mind that once referral visit will be same referral visit if visited by organic or direct.

    I was wondering, how could you solve this issue?

    Thanks a lot Glenn!

    • http://www.hmtweb.com/marketing-blog/ Glenn Gabe

      Thanks for your comment! It’s hard to rectify the problem, since that’s how the system works. You would have to rework the cookie handling.

  • Mirsad

    Thanks for your post, but I didn’t understand exactly how to setup it in Google Analytcs??

    • http://www.hmtweb.com/marketing-blog/ Glenn Gabe

      There’s nothing to set up. The post just explained how Google Analytics handles referring sources (and why you might see more visits than clicks). If webmasters (and marketers) don’t understand this, then the numbers can be misleading. I hope that helps.

  • Anna Zelenskaya

    Thanks for the straightforward explanation! I am facing just the opposite situation. We know that a banner on a partner website has been clicked (for instance) 500 times last month. But Google Analytic Referral traffic section is showing about 10% of this number! Any ideas? Thanks!

    • Charlotte Biggs

      Hi Anna, I seem to be having the same problem with google analytics only reporting a small percentage of the clicks. Did you ever find out anything useful on this issue? Thanks!

      • http://www.hmtweb.com/marketing-blog/ Glenn Gabe

        Check your banner tagging. Check the landing page tagging. Also, you need to make sure the banner is actually being clicked on 500 times. Some publishers aren’t going to give accurate numbers… Also, ask them how they are handling click fraud and bot clicks. Those might rack up high numbers, but not register in GA. Just a few thoughts. :)

  • http://www.hmtweb.com/marketing-blog/ Glenn Gabe

    Artur, that’s a great catch. Quickly checking that out reveals the same results that you saw. I need to check this out in greater detail when I have a free minute. I’ll try and write an update, based on my findings.

  • jim

    Besides an inaccurate representation of incoming traffic, we have another problem.

    In our analytics reports we excluded traffic that comes from our domain “exclude.com”

    The problem is, when someone came from exclude.com and returns later, no one of these direct visits will be in our stats.

    Is there a workaround for this problem?