Archive for the ‘web-analytics’ Category

Wednesday, February 4th, 2015

When The Hammer Falls – Analyzing Lyrics in the Google SERPs and Its Impact on Traffic [Case Study]

Summary: In the fall of 2014, both Bing and Google began surfacing song lyrics directly in the search engine results pages (SERPS). Since users could now find lyrics immediately in the SERPs, many wondered what would happen to lyrics websites that provided the same information, but required a click through to view the lyrics. This post provides findings from analyzing three large-scale lyrics web sites to determine the traffic impact of lyrics in the SERPs.

Song Lyrics Displayed In The Google Search Results

Article Contents and Quick Jumps:

In April of 2014, I picked up a major algorithm update that heavily impacted lyrics web sites. The drop in traffic to many key players in the niche was substantial, with some losing 60%+ of their Google organic traffic overnight. For those of you familiar with Panda or Penguin hits, you know what this looks like.

Lyrics Web Sites Hit By Google Algorithm Update in April of 2014

I ended up digging in heavily and analyzing the drop across the entire niche. I reviewed a number of lyrics sites across several countries that got hit and wrote a post covering my findings (linked to above). After writing that post, I had a number of lyrics sites reach out to me for more information. They wanted to know more about what I surfaced, what the problems could be, and if I could help rectify the situation. It was a fascinating algo hit to analyze and I absolutely wanted to take on the challenge of helping the sites recover. So I began helping several of the lyrics sites that were heavily impacted.

2014 – A Crazy Year for Lyrics Sites
I took on several of the lyrics sites as clients and began heavily analyzing and auditing the negative impact. That included performing a deep crawl analysis of each site, a heavy-duty technical SEO analysis, a thorough content analysis, while also using every tool in my arsenal to surface SEO-related problems.

I won’t sugarcoat my findings, there were many problems I surfaced, across content, technical SEO, and even links (in certain situations). It was hard to say if the specific update in April was Panda, a separate algo update that hammered lyrics sites, or something else. But I tackled the situation by covering as many bases as I could. Each remediation plan was extensive and covered many ways to tackle the problems I surfaced. As time went on, and many changes were implemented, the sites started to recover. Some recovered sooner than others, while other sites took many more months to surge back.

Lyrics Website Recovering During Panda Update

On that note, many of the large lyrics sites have ridden the Panda roller coaster for a long time. And that’s common for large-scale websites that haven’t focused on Panda-proofing their web sites. Over time, insidious thin content builds on the site like a giant layer of bamboo. And as the bamboo thickens, Panda smells dinner. And before you know it, boom, Panda hits the site (and for these sites, it hit them hard).

After recovering, each site would hold their collective breath while subsequent Panda updates rolled out. Based on the lyrics web sites I have assisted, only one has fallen again to Panda. The others have remained out of the gray area and are doing well traffic-wise. Unfortunately, one lyrics web site I was helping saw a temporary recovery after recovering relatively quickly (almost too quickly). Quick recoveries are rare when you’re dealing with Panda, so I did find that specific recovery odd. It typically takes months before you see a major surge after being pummeled by Panda. The site surged during the 9/5 update and then got hammered again during the cloaked 10/24 update. And Panda has not rolled out since 10/24/14, so we’re still waiting to see if the site comes back.

Lyrics Website Temporary Recovery from Panda

But enough about Panda for now. Actually, Google Panda could pale in comparison to what showed up in late fall 2014. We all knew it was possible, considering Google’s ambition to provide more and more data in the search engine results pages (SERPs). But it’s another story when you actually see it happen. I’m referring to the search engines adding lyrics directly in the SERPs. You know, when someone searches for song lyrics, and boom, the lyrics show up right in the desktop or mobile SERPs. No click through needed. I’ll cover how this unfolded next.

Lyrics Show Up in the SERPs
Bing was the first to add lyrics in the SERPs on October 7, 2014. That was the first bomb dropped on lyrics sites. It was a small bomb, considering it was only showing in Bing in the United States and Bing has approximately 19.7% market share (according to comScore Dec 2014 stats). Bing also drives Yahoo search (organic and paid), but lyrics are not showing in Yahoo yet.

Lyrics in Bing SERPs

But the writing was on the wall. Lyrics were coming to Google, and sooner than later. When lyrics hit Bing, I sent emails to all of my lyrics clients explaining the situation, providing screenshots, and sample searches. Not every song would yield lyrics in the SERPs, but this was still a major event for the lyrics industry.

Next up was the first move by Google. On October 24, 2014, if you searched for a specific song, Google began providing a YouTube video with some song and artist information at the top of the SERPs. And near the bottom of that unit was a line or two from the lyrics and then a link to Google Play for the full lyrics. Whoa, so Google was beginning their assault on lyrics by simply linking to Google Play to view the lyrics. Again, I immediately emailed my clients and explained the situation, knowing lyrics were coming to the main SERPs soon.

Lyrics in Google SERPs Linking To Google Play


December 19, 2014 – The Hammer Falls
And then this happened:

Lyrics in Google SERPs Finally Arrive on December 19, 2014

And here was my Google+ share, which ended up getting a lot of attention:

Google Plus Share of Lyrics in the Google SERPs


I shared this screenshot of Google including lyrics directly in the SERPs, and the G+ post got noticed, a lot. That share was mentioned on a number of prominent websites, including Search Engine Roundtable, TechCrunch, Billboard, and more.

To clarify what was happening search-wise, on December 19, 2014 Google began showing song lyrics for users in the United States, and only for certain songs. I’m assuming the limit on songs and geography was based on licensing, so this doesn’t impact every song available. I’ll cover more about the impact of those limitations soon when I dig into some stats, but it’s an important note.

For example, if you search for “bang bang lyrics” in the United States, you get this:

Bang Bang Lyrics in US Google SERPs

But if you search for “you shook me all night long lyrics”, you won’t see lyrics in the SERPs. Clearly Google doesn’t have the rights to present the lyrics to all AC/DC songs, but it does for “Bang Bang”.

You Shook Me All Night Long Without Lyrics in US Google SERPs

And by the way, that’s for the desktop search results. This is also happening in mobile search, in the United States, and for certain songs. Talk about dominating the mobile SERPs, check out the screenshot below. Where on desktop, you get the lyrics, but still see links to lyrics websites above the fold (typically), mobile is another story.

Check out the search for “bang bang lyrics” on my smartphone:

Bang Bang Lyrics in the Mobile U.S. Google SERPs

Can you see the massive difference? It’s just lyrics, and nothing else. And to add insult to injury, the percentage of users searching for lyrics is heavily skewed mobile. And that makes sense. Those users are on the go, hear a song, want to know the lyrics, and simply search on their phones. Or, they are in a situation where their phone –is their computer– so their searches will always be mobile.

Mobile Heavy Queries for Lyrics Globally


Death to Lyrics Websites?
Based on what I’ve explained so far, you know that Panda loves taking a bite out of lyrics web sites and you also know that both Google and Bing are providing lyrics directly in the SERPs (in the US and for certain songs). And you might guess that all of this means absolute death for lyrics websites. But wait, does it? I wouldn’t jump to conclusions just yet. There are definitely nuances to this situation that require further analysis and exploration.

For example, how much of a hit have the lyrics sites taken based on lyrics in the SERPs? How much traffic dropped for each song that yields lyrics in the SERPs? Was there an impact just in the United States or around the world too? And what about the difference between desktop and mobile? All of these were great questions, and I was eager to find answers.

So, I reached out to several of my lyrics clients and asked if I could analyze the changes and document the data in this post (anonymously of course). The post isn’t meant to focus on the sites in particular, but instead, focus on the impact that “lyrics in the SERPs” have made to their traffic. The lyrics websites I’ve been helping generate revenue via advertising, so a massive drop in traffic means a massive drop in revenue. It’s pretty much that simple at this point. That’s why Panda strikes fear in every lyrics web site owner and why lyrics in the SERPs can strip away visits, pageviews, and ad dollars. It’s a new one-two punch from Google.

Analyzing Three Large-Scale Lyrics Websites
Three of my clients were nice enough to let me move forward with the analysis. And I greatly appreciate having clients that are awesome, and are willing to let me analyze and share that data. The three sites I analyzed for this post are large-scale lyrics sites. Combined, they drive more than 30 million visits from Google organic per month and have approximately 6 million lyrics pages indexed. And as I explained earlier, a lot of that traffic is from users on mobile devices. Approximately 40-50% of all Google organic traffic is from mobile devices (across all three sites).

My goal with the analysis was to understand the impact of lyrics in the SERPs from a click-through and traffic standpoint. I dug into search queries driving traffic over time to all three sites while also checking impressions and clicks in the SERPs (via Google Webmaster Tools, both desktop and mobile). Then I also checked Google Analytics to determine the change in traffic levels to song pages since the lyrics hit the SERPs.

For example, if a query saw a similar number of impressions since the launch of lyrics in the SERPs, but clicks dropped off a cliff, then I could dig in to analyze the SERPs for that query (both desktop and mobile). I found some interesting examples for sure, which I’ll cover below.

An example of stable or increasing impressions, but clicks dropping off a cliff: 

Google Webmaster Tools Impressions and Clicks for Lyrics Queries


My analysis measured the impact right after lyrics hit the SERPs (from December 19, 2014 through the end of January 2015). The holidays were mixed in, which I tried to account for the best I could. Some of the lyrics sites saw steady traffic during the holidays, while one dipped and then returned as the New Year approached. The songs I analyzed and documented were not holiday-focused songs. I made sure to try and isolate songs that would not be impacted by the holidays. Also, Google Webmaster Tools data was sometimes wonky. I’m sure that’s no surprise to many of you working heavily in SEO, but it’s worth noting. I tried my best to exclude songs where the data looked strange.

Google Webmaster Tools & Advanced Segmentation in GA
When I began my analysis, I quickly found out that the straight reporting in both Google Webmaster Tools and Google Analytics wouldn’t suffice. Overall Google organic traffic wouldn’t help, since lyrics only rolled out in the SERPs in the United States. When checking traffic since the rollout, you really couldn’t see much overall change. But the devil is in the details as they say. So I used the functionality available to me in both GWT and GA to slice and dice the data. And that greatly helped me understand the impact of lyrics in the SERPs.

In Google Webmaster Tools, the search queries reporting enables you to filter the results. This was incredibly helpful, as I was able to isolate traffic from the United States and also view web versus mobile traffic. But there was another nifty filter I used that really helped. You see, many people visit lyrics websites for the meaning of the lyrics, and not just to see the lyrics. For example, “take me to church meaning” or “meaning of hallelujah lyrics”.

The reason I wanted to weed those queries out is because as of now, Google does not provide the lyrics in the SERPs for “meaning” focused queries. And that’s good for my clients by the way. So by adding the filters per site, I would able to isolate songs that could be impacted.

Filtering GWT Search Queries by Search Property, Location, and Negative Query:

Google Webmaster Tools Filters for Property, Location, and Query

After setting the filters, I was able to search for queries that yielded relatively stable impressions, but saw a drop in clicks and click through rate. And I always kept an eye on average position to make sure it didn’t drop heavily.

From a Google Analytics standpoint, I ran into a similar problem. Top-level statistics wouldn’t cut it. I needed Google organic traffic from the United States only. And then I wanted both Desktop and Mobile Google organic traffic from the United States only (separated). That’s where the power of advanced segments come in.

I built segments for Desktop Google organic traffic from the United States and Mobile Google organic traffic from the United States. By activating these segments, my reporting isolated that traffic and enabled me identify trends and changes based on those segments alone. By the way, I wrote a tutorial for how to use segments to analyze Panda hits. You should check that out if you aren’t familiar with segments in GA. You’ll love them, believe me.

Filtering Google Organic Traffic from the United States in GA Using Segments:

Google Analytics Segments for U.S. Desktop Google Organic Traffic


So, with the right tools and filters in place, I began to dig in. It was fascinating to analyze the queries leading to all three sites now that lyrics hit the SERPs. I cover what I found next. By the way, this posts focuses on Google and not Bing. I might write up another post focused on Bing’s lyrics in the SERPs, but I wanted to focus on Google to start.

The Impact of Lyrics in the SERPs – The Data
With multiple computers up and running, two phones, and two tablets, I began to dig in. I wanted to find queries and songs that typically drove traffic to the three sites that now yielded lyrics in the SERPs. And then I wanted to see what happened once those lyrics hit the SERPs, the impact on clicks, traffic, etc. I have documented a number of examples below. By the way, there are many more examples, but I wanted to just provide a sampling below. Here we go…


Spill The Wine Lyrics by War
Google Organic Desktop US Traffic Down 73%
Google Organic Mobile US Traffic Down 65%
GWT Clicks Down 56%


Sister Ray Lyrics by The Velvet Underground
Google Organic Desktop US Traffic Down 73%
Google Organic Mobile US Traffic Down 56%
GWT Clicks Down 84%


Rude Lyrics by Magic!
Google Organic Desktop US Traffic Down 41%
Google Organic Mobile US Traffic Down 32%
GWT Clicks Down 55%


Bang Bang Lyrics by Jesse J, Nicki Manaj and Ariana Grande
Google Organic Desktop US Traffic Down 32%
Google Organic Mobile US Traffic Down 47%
GWT Clicks Down 66%


Fireproof Lyrics by One Direction
Google Organic Desktop US Traffic Down 44%
Google Organic Mobile US Traffic Down 40%
GWT Clicks Down 29%


All of Me Lyrics by John Legend
Google Organic Desktop US Traffic Down 39%
Google Organic Mobile US Traffic Down 14%
GWT Clicks Down 61%


Country Road Lyrics by John Denver
Google Organic Desktop US Traffic Down 62%
Google Organic Mobile US Traffic Down 45%
GWT Clicks Down 36%


Come Sail Away Lyrics by Styx
Google Organic Desktop US Traffic Down 43%
Google Organic Mobile US Traffic Down 27%
GWT Clicks Down 55%


Midnight Special Lyrics by Huddie William Ledbetter
Google Organic Desktop US Traffic Down 53%
Google Organic Mobile US Traffic Down 85%
GWT Clicks Down 33%


Comfortably Numb Lyrics by Pink Floyd
Google Organic Desktop US Traffic Down 46%
Google Organic Mobile US Traffic Down 17%
GWT Clicks Down 43%


Yes, There’s A Serious Impact
As you can see from the statistics above, both desktop and mobile traffic to the song pages dropped significantly since lyrics hit the SERPs (for songs that yield lyrics in the SERPs). Again, these songs showed stable impressions during the timeframe, yet showed large drops in clicks from the SERPs, and subsequent traffic to the three lyrics sites I analyzed.

Some users were clearly getting what they wanted when searching for lyrics and finding that information in the SERPs. And in mobile search, the lyrics take up the entire results page. So it’s no surprise to see some mobile numbers absolutely plummet after lyrics hit the SERPs.

What Could Lyrics Sites Do?
Above, I provided a sampling of what I saw while analyzing the impact of lyrics in the U.S. Google SERPS. Clearly there’s a large impact. The good news for lyrics sites is that there are several core factors helping them right now.

  • This is only in the United States.
  • The lyrics only trigger when the query is structured in certain ways. For example, “magic rude lyrics” yields lyrics where “rude lyrics magic” does not. Also, if additional words are entered in the query, lyrics will not be shown (like “meaning” which I explained earlier.)
  • Not all songs are impacted (yet). I found many examples of songs that did not yield lyrics in the SERPs. Again, this is probably due to licensing issues.

If you look at the overall traffic numbers for the sites I analyzed (and the other sites I have access to), Google organic traffic overall has not been heavily impacted. Taking all global Google organic traffic into account, and across all songs, you clearly don’t see the huge drop like I showed you for the songs listed above. That said, this is still a grave situation for many lyrics sites. The content they have licensed and provided on their sites is now being surfaced directly in the SERPs. If this expands to more songs, more countries, and for additional queries, then it can have a massive impact on their businesses. Actually, it could very well end their businesses.

Moving forward, lyrics sites need to up their game from a functionality and value proposition standpoint. If Google can easily add lyrics to the SERPs, then lyrics sites need to keep driving forward with what Google can’t do (at least for now). They should develop new functionality, strengthen community engagement, provide member benefits, include more data and media for artists and songs, provide a killer mobile experience, etc.

Remember, there are many people searching for additional information related to songs. For example, people want to know the meaning of lyrics and seem to enjoy the community engagement about learning what each lyric means. And lyrics don’t trigger in the SERPs for those queries (yet).

And then you have the next generation of devices, social networks, messaging apps, gaming consoles, connected cars, etc. I would start thinking about how people are going to search for lyrics across new devices and in new environments. That’s a new frontier and it would be smart to begin building and testing lyrics applications that can work in those new environments. Mobile, wearables, voice search, cars, etc. provide a wealth of opportunity for business owners focused on music. It just takes the right ideas, time, resources, and of course, money.

But I’ll stop there. I think that topic can be an entire post and this one is getting too long already. :)


Summary – Moving Forward With (Expanding) Lyrics in the SERPs
In the short-term, it’s hard to say how this will expand. Google and Bing might drop the effort and keep things as-is, or they could keep expanding lyrics in the SERPs until every song and every country is covered.

Based on the current song and geography limits in Google and Bing, lyrics websites are still surviving, and especially for searches outside the United States. It will be interesting to watch this space over time, especially since I have several clients adapting to the new lyrics world as I write this post.

From an SEO standpoint, between Google Panda and content surfacing in the SERPs, lyrics web sites are fighting a battle on two fronts. If it’s not Panda attacking the site one night, it’s the Knowledge Graph pushing song lyrics front and center in the SERPs. And in this day and age, wars are won by technology, not brute strength. So lyrics sites need to up their engineering prowess, think two to three steps ahead of the industry, and then execute quickly and at a very high level.

That’s how they can survive and prosper in the coming years. Of course, that’s until we have a Google chip implanted in our brains that instantly provides the lyrics to every song ever written, from the around the world, since the beginning of time. Think about that for a second.



Wednesday, November 26th, 2014

Panda Analysis Using Google Analytics Segments – How To Isolate Desktop, Mobile, and Tablet Traffic From Google

Segments in Google Analytics to Isolate Traffic

In previous posts about Panda analysis, I’ve mentioned the importance of understanding the content that users are visiting from Google organic. Since Google is measuring user engagement, hunting down those top landing pages can often reveal serious content quality problems.

In addition, I’ve written about understanding the devices being used to access your site from the search results. For example, what’s the breakdown of users by desktop, mobile, and tablets from Google organic? If 50% of your visits are from smartphones, then you absolutely need to analyze your site through that lens. If not, you can miss important problems that users are experiencing while visiting your website. And if left unfixed, those problems can lead to a boatload of horrible engagement signals being sent to Google. And that can lead to serious Panda problems.

Panda Help Via Segments in Google Analytics
So, if you want to analyze your content by desktop, mobile, and tablet users through a Panda lens, what’s the best way to achieve that? Well, there’s an incredibly powerful feature in Google Analytics that I find many webmasters simply don’t use. It’s called segmentation and enables you slice and dice your traffic based on a number of dimensions or metrics.

Segments are non-destructive, meaning that you can apply them to your data and not affect the source of the data. Yes, that means you can’t screw up your reporting. :) In addition, you can apply new segments to previous traffic (they are backwards compatible). So you can build a new segment today and apply it to traffic from six months ago, or longer.

For our purposes today, I’m going to walk you through how to quickly build three new segments. The segments will isolate Google organic traffic from desktop users, mobile users, and tablet users. Then I’ll explain how to use the new segments while analyzing Panda hits.


How To Create Segments in Google Analytics
When you fire up Google Analytics, the “All Sessions” segment is automatically applied to your reporting. So yes, you’ve already been using segments without even knowing it. If you click the “All Sessions” segment, you’ll see a list of additional segments you can choose.

Google Analytics All Sessions Segment

You might be surprised to see a number of segments have been built for you already. They are located in the “System” category (accessed via the left side links). For example, “Direct Traffic”, “AdWords”, “Organic Traffic”, and more.

Google Analytics System Segments


We are going to build custom segments by copying three system segments and then adding more dimensions. We’ll start by creating a custom segment for mobile traffic from Google organic.

1. Access the system segments by clicking “All Sessions” and then clicking the link labeled “System” (located on the left side of the UI).


Google Analytics System Segments


2. Scroll down and find the “Mobile Traffic” segment. To the far right, click the “Actions” dropdown. Then choose “Copy” from the list.


Copying a System Segment in Google Analytics


3. The segment already has “Device Category”, “exactly matches”, and “mobile” as the condition. We are going to add one more condition to the list, which is Google organic traffic. Click the “And” button on the far right. Then choose “Acquisition” and the “Source/Medium” from the dimensions list. Then choose “exactly matches” and select “google/organic” from the list. Note, autocomplete will list the top sources of traffic once you place your cursor in the text box.


Creating a Segment by Adding Conditions


4. Name your segment “Mobile Google Organic” by using the text box labeled “Segment Name” at the top of the window. It’s easy to miss.


Name a Custom Segment in Google Analytics


5. Then click “Save” at the bottom of the create segment window.


Save a Custom Segment in Google Analytics


Congratulations! You just created a custom segment.


Create The Tablet Traffic Segment
Now repeat the process listed above to create a custom segment for tablet traffic from Google organic.  You will begin with the system segment for “Tablet Traffic” and then copy it. Then you will add a condition for Google organic as the source and medium.


Desktop Traffic (Not a default system segment.)
I held off on explaining the “Desktop Traffic” segment, since there’s an additional step in creating one. For whatever reason, there’s not a system segment for isolating desktop traffic. So, you need to create this segment differently. Don’t worry, it’s still easy to do.

We’ll start with the “Mobile Traffic” segment in the “System” list, copy it, and then refine the condition.

1. Click “All Sessions” and the find “Mobile Traffic” in the “System” list. Click “Actions” to the far right and then click “Copy”.


Copying a System Segment in Google Analytics


2. The current condition is set for “Device Category” exactly matching “mobile”. We’ll simply change mobile to “desktop”. Delete “mobile” and start typing “desktop”. Then just select the word “desktop” as it shows up.

Creating a Desktop Segment in Google Analytics


3. Since we want Desktop traffic from Google Organic, we need to add another condition. You can do this by clicking “And” to the far right, selecting “Acquisition”, and then “Source/Medium” from the dropdown. Then select “exactly matches” and enter “Google/Organic” in the text box. Remember, autocomplete will list the top sources of traffic as you start to type.


Creating a Google Organic Desktop Segment in Google Analytics


4. Name your segment “Desktop Google Organic” and then click “Save” at the bottom of the segment window to save your new custom segment.


Quickly Check Your Segments
OK, at this point you should have three new segments for Google organic traffic from desktop, mobile, and tablets. To ensure you have these segments available, click “All Sessions” at the top of your reporting, and click the “Custom” link on the left. Scroll down and make sure you have all three new segments. Remember, you named them “Desktop Google Organic”, “Mobile Google Organic”, and “Tablet Google Organic”.

If you have them, then you’re good to go. If you don’t, read through the instructions again and create all three segments.


Run Panda Reports by Segment
In the past, I’ve explained the importance of running a Panda report in Google Analytics for identifying problematic content. A Panda report isolates landing pages from Google organic that have dropped substantially after a Panda hit. Well, now that you have segments for desktop, mobile, and tablet traffic from Google organic, you can run Panda reports by segment.

For example, click “All Sessions” at the top of your reporting and select “Mobile Google Organic” from the “All” or “Custom” categories. Then visit your “Landing Pages” report under “Behavior” and “Site Content” in the left side menu in GA. Since you have a specific segment active in Google Analytics, the reporting you see will be directly tied to that segment (and filter out any other traffic).

Creating a Google Panda Report Using Custom Segments


Then follow the directions in my previous post to run and export the Panda report. You’ll end up with an Excel spreadsheet highlighting top landing pages from mobile devices that dropped significantly after the Panda hit. Then you can dig deeper to better understand the content quality (or engagement) problems impacting those pages.

Combine with Adjusted Bounce Rate (ABR)
User engagement matters for Panda. I’ve documented that point many times in my previous posts about Panda analysis, remediation, and recovery. The more poor engagement signals you send Google, the more bamboo you are building up. And it’s only a matter of time before Panda comes knocking.

So, when analyzing user engagement, many people jump to the almighty Bounce Rate metric to see what’s going on. But here’s the problem. Standard Bounce Rate is flawed. Someone could spend five minutes reading a webpage on your site, leave, and it’s considered a bounce. But that’s not how Google sees it. That would be considered a “long click” to Google and would be absolutely fine.

And this is where Adjusted Bounce Rate shines. If you aren’t familiar with ABR, then read my post about it (including how to implement it). Basically, Adjusted Bounce Rate takes time on page into account and can give you a much stronger view of actual bounce rate. Once you implement ABR, you can check bounce rates for each of the segments you created earlier (and by landing page). Then you can find high ABR pages by segment (desktop, mobile, and tablet traffic).

Combining Adjusted Bounce Rate with Custom Segments


Check Devices By Segment (Smartphones and Tablets)
In addition to running a Panda report, you can also check the top devices being used by people searching Google and visiting your website. Then you can analyze that data to see if there are specific problems per device. And if it’s a device that’s heavily used by people visiting your site from Google organic, then you could uncover serious problems that might lie undetected by typical audits.

GA’s mobile reporting is great, but the default reporting is not by traffic source. But using your new segments, you could identify top devices by mobile and tablet traffic from Google organic. And that’s exactly what you need to see when analyzing Panda hits.

Analyzing Devices with Custom Segments in Google Analytics

For example, imagine you saw very high bounce rates (or adjusted bounce rates) for ipad users visiting from Google organic. Or maybe your mobile segment reveals very low engagement from Galaxy S5 users. You could then test your site via those specific devices to uncover rendering problems, usability problems, etc.


Summary – Isolate SEO Problems Via Google Analytics Segments
After reading this post, I hope you are ready to jump into Google Analytics to create segments for desktop, mobile, and tablet traffic from Google organic. Once you do, you can analyze all of your reporting through the lens of each segment. And that can enable you to identify potential problems impacting your site from a Panda standpoint. I recommend setting up those segments today and digging into your reporting. You might just find some amazing nuggets of information. Good luck.



Wednesday, September 17th, 2014

How To Check If Google Analytics Is Firing On Android Devices Using Remote Debugging With Chrome [Tutorial]

How To Debug Google Analytics on Mobile Devices

We all know that having a strong analytics setup is important. Marketing without measurement is a risky proposition for sure. But in a multi-device world, it’s not as easy to make sure your setup is accurately tracking what you need – or tracking at all. And if your analytics code isn’t firing properly across smartphones, tablets, and desktop computers, your data will be messy, incomplete, and inaccurate. And there’s nothing that drives a marketer crazier than flawed data.

A few weeks ago, Annie Cushing tweeted a quick question to her followers asking how everyone was testing their Google Analytics setup via mobile devices. This is something many digital marketers grapple with, especially when you are trying to track down problems. For example, I do a lot of algorithm update work and often dig into the analytics setup for a site to ensure we are seeing the full drop in traffic, conversion, revenue, etc.

My knee-jerk response was to check real-time reporting in Google Analytics while accessing specific pages to ensure those visits were being tracked, in addition to events. That could work, but it’s not as granular or isolated as you would want. I also mentioned to Annie that using a chrome extension like User Agent Switcher could help. That wouldn’t document the firing of analytics code, but would let you see the source code when accessing a webpage via a specific type of smartphone or tablet. But again, you couldn’t see the actual firing of the code or the events being tracked. And that’s obviously an important aspect to debugging analytics problems.

A Solution – Remote Debugging on Android with Chrome
So I did what I typically do when I run into a tricky situation. I find a solution! And for Android devices, I found a solid one. Many of you might be familiar with Chrome Developer Tools (on your desktop computer). It holds some outstanding functionality for debugging websites and web applications. But although it’s extremely helpful for debugging desktop webpages, it didn’t really address the problem at hand (out of the box), since we want to debug mobile devices.

So I started to research the issue and that’s when I came across a nifty technique which would allow you to connect your Android device to your desktop computer and then debug the Chrome tabs running on your mobile device from your desktop computer. And since I could use Chrome Developer Tools to debug the tabs on my desktop computer, I could check to see if Google Analytics was indeed firing when accessing webpages via my Android device. Awesome.

So, I spent some time testing this out and it does work. Sure, I had to jump through some hoops to get it to run properly, but it finally did work. Below I’ll cover what you’ll need to test this out for yourself and how to overcome some of the problems I encountered. Let’s get started.


What You’ll Need
In order to debug GA code running on your mobile device, you’ll need the proper setup both on your desktop computer and on your Android device. In its simplest form, you’ll need:

  • Chrome installed on your desktop (version 32 or later).
  • Android 4.0 or later.
  • A USB Cable to connect your device to your computer.
  • Android SDK {this will not be required for some of you, but others might need to install it. More on that situation below}.

If you run into the problems I ran into, you’ll need the Android SDK installed. I already had it installed since I’ve been testing various Android functionality and code, so it wasn’t a big deal. But you might need to install it on your own. I wouldn’t run to do that just yet, though. If the straight setup works for you, then run with it. If not, then you might need to install the Android SDK.

If you are confident you have the necessary setup listed above, then you can move to the tutorial listed below. I’ll walk you through how to debug Chrome tabs running on your mobile device via Chrome on your desktop computer. And yes, we’ll be isolating Google Analytics code firing on our Android devices to ensure you are tracking what you need.

How To Debug Google Analytics on Your Android Device – Step-By-Step Instructions

  1. Enable USB Debugging on Your Android Device
    Access your settings on your Android device and click Developer Options. On my device, that was located in the more “More” grouping of my settings and under System Manager. If you don’t see Developer Options, then you need to enable it.You can do that by accessing Settings, tapping About Phone or About Device and tapping Build Number seven times. Yes, that sounds extremely cryptic, but that’s what you need to do. Once you do, Developer Options will show up in under System Manager in your phone’s settings.

    Enable USB Debugging on Android Device

    Then you can check the box to enable USB Debugging on your device. You will need to do this in order to debug Google Analytics in Chrome on your device.

  2. Enable USB Discovery in Chrome (on your desktop)
    Next, type chrome:inspect in a new tab in Chrome on your desktop. Ensure “Discover USB devices” is checked on this screen.

    Enable USB Discovery in Chrome Desktop
  3. Connect Your Phone To Your Computer via USB
  4. Allow USB Debugging
    When you connect your phone to your computer, you should see a dialog box on your phone that asks you if you want to allow USB debugging. Click OK. Note, if you don’t see this dialog box, debugging your mobile device from Chrome on your desktop will not work. I provide instructions for getting around this problem later in the tutorial. If you are experiencing this problem, hop down to that section now.

    Allow USB Debugging on Android Device
  5. Fire up Chrome On Your Mobile Device
    Start Chrome on your Android device and access a webpage (any webpage you want to debug).
  6. Inspect With Chrome on your Desktop
    Once you open a webpage in Chrome on your mobile device, access Chrome on your desktop and visit chrome:inspect. Once you do, you should see your device listed and the various tabs that are open in Chrome on your Android device.

    Inspect Chrome Tabs on Desktop Computer
  7. Click Inspect To Debug The Mobile Tab
    When you click “inspect”, you can use Chrome Developer Tools on your desktop to debug the mobile web view. You can use all of the functionality in Chrome Developer Tools to debug the webpage open on your mobile device.
  8. Click the Network Tab in Chrome Developer Tools
    By accessing the Network Tab, you can view all network activity based on the webpage you have loaded in Chrome on your mobile device. That includes any resources that are requested by the webpage. Then reload the webpage on your mobile device to ensure you are seeing all resources.
  9. First Check for GA.js
    When you load a webpage on your mobile device, many resources will be listed in the network tab. But you should look for ga.js to see if the Google Analytics snippet is being loaded.Tip: You can use the search box and enter “ga.js” to filter all resources by that string. It’s an easy way to isolate what you are looking for.

    Check for ga.js in Network Tab in Developer Tools
  10. Next Check for utm.gif
    After checking for ga.js, you should look for the tracking pixel that’s sent to GA named utm.gif. If that is listed in the network panel, then your mobile webpage is tracking properly (at least basic tracking). Again, you can use the search box to filter by utm.gif.

    Check for utm.gif in Network Tab in Developer Tools
  11. Bonus: Advanced Tracking
    If you are firing events from mobile webpages, then you can see them listed here as well. For example, you can see an event being fired when a user stays on the page for more than 30 seconds below. So for this situation, we know that pageviews are accurately being tracked and the time on page event is being tracked via mobile. Nice.

    Check event tracking in Chrome for Android


A Note About Troubleshooting
I mentioned earlier that if you don’t see the “Allow USB Debugging” dialog on your mobile device when you connect your phone to your computer, then this setup won’t work for you. It didn’t initially work for me. After doing some digging around, I found the legacy workflow for remote debugging on Android.

By following the steps listed below, I finally got the prompt to show up on my mobile device. Then I was able to debug open Chrome tabs on my Android device.


  1. Install the Android SDK (if you don’t already have it installed)
    You can learn more about the SDK here and download the necessary files.
  2. Kill the ADB Server
    Use a command prompt to access the “platform-tools” folder in the SDK directory and then issue the following command: adb kill-server. Note, you should use the cd command to change directory to the folder containing adb. That’s the platform-tools folder in your Android SDK directory.

    Kill ADB Server
  3. Revoke USB Debugging on Your Android Device
    Disconnect your phone from your computer. Then go back to Developer Options on your Android phone and tap Revoke USB debugging authorization.

    Revoke USB Debugging
  4. Start the ADB Server
    Now you must restart the adb server. Use a command prompt, access the platform-tools folder again, and enter the following command: adb start-server.

    Start ADB Server
  5. Reconnect Your Device To Your Computer
    Once you reconnect your device, you should see the “Allow USB Debugging” dialog box. Click “OK” and you should be good to go. This will enable you to debug Chrome tabs running on your mobile device via Chrome running on your desktop.
  6. Open Chrome on Your Android Device
    Go ahead and open a webpage that you want to debug in Chrome on your Android phone. Once it’s loaded in Chrome in Android, you can follow the instructions listed earlier for using the network panel to debug the GA setup.


Summary – Know When Google Analytics is Firing on Mobile Devices
So there you have it. There is a way to debug the actual firing of GA code on your Android devices and it works well. Sure, you may need to go the extra mile, use the legacy workflow, and install the Android SDK, but you should be able to get it working. And once you do, you’ll never have to guess if GA is really working on Android devices. You’ll know if it is by debugging your Chrome tabs on Android via Chrome running on your desktop. Good luck.




Monday, May 12th, 2014

How To Remarket 70+ Ways Using Segments and Conditions in Google Analytics

Remarketing in Google Analytics Using Conditions and Segments

I know what you’re thinking. Can you really remarket more than 70 different ways using segments in Google Analytics?  Yes, you can!  Actually, when you combine the methods I’ll cover today, there are many more types of Remarketing lists you can build!  So the total number is much greater than 70.

My post today is meant to introduce you to segments in Google Analytics (GA), explain how you can use them to remarket to people who already visited your site, and provide important Remarketing tips along the way.  I hope once you read this post, you’re ready to kick off some Remarketing campaigns to drive more sales, leads, phone calls, etc.

What Are Segments in Google Analytics?
Many digital marketers know about Remarketing already.  That’s where you can reach people that already visited your website via advertising as they browse the web.  For example, if John visited Roku’s website, browsed various products, and left, then Roku could use Remarketing to advertise to John as he browses the Google Display Network (GDN).  The Google Display Network is a massive network of sites that run Google advertising, and includes Google-owned properties like YouTube, Google Maps, Gmail, etc.  According to Google, the GDN reaches 90% of internet users worldwide.

Remarketing via The Google Display Network (GDN)

By the way, if you’ve ever visited a website and then saw ads from that website as you browsed the web, then you’ve been remarketed to.  As you can guess, this can be an incredibly powerful way to drive more sales, leads, etc.  It can also be extremely frustrating and/or shocking to users.  So be careful when crafting your Remarketing strategy!

When Remarketing first rolled out, you could only set up Remarketing lists in the AdWords interface.  That was ok, but didn’t provide a massive amount of flexibility.  That’s when Google enabled marketers to set up Remarketing lists via Google Analytics.  That opened up an incredible amount of opportunity to slice and dice visitors to create advanced-level Remarketing lists.  For example, you could create Remarketing lists based on users who visited a certain section of your website, or lists based on users completing a certain conversion goal, etc.  Needless to say, tying Google Analytics to Remarketing was an awesome addition.

Now, I started using Google Analytics Remarketing functionality immediately to help clients build advanced Remarketing lists, but I had a feeling that Google was going to make it even more powerful.  And they did.

Along Came Segments… Remarketing Options Galore
You might already be familiar with segments in Google Analytics, which was originally named “Advanced Segmentation”.  In July of 2013, Google released a new version in Google Analytics and simply called it “Segments”.  But don’t get fooled by the simpler name.  Segments enable marketers to slice and dice their users and traffic to view reporting at a granular level.  For example, I often set up a number of segments for clients, based on their specific goals. Doing so enables me to quickly view granular reporting by removing a lot of the noise residing in standard reports.

Using Segments to Create Remarketing Lists in Google Analytics

But starting in January of 2014, Google rolled out an update that enabled marketers to use those segments to create Remarketing lists.  Yes, now marketers had an incredible number of options available when creating Remarketing lists.  In addition, you could easily import segments you are already using! This means you could leverage the hard work you’ve already put in when creating segments in Google Analytics.

Although I thought I had a lot of flexibility in creating Remarketing lists leading up to that point, the ability to use segments opened the targeting flood gates.  I remember checking out the list of options when segments for Remarketing first launched, and I was blown away.

For example, using segments you could create Remarketing lists based on:

  • Demographics like age, gender, language, location, and more.
  • Technology options like operating system, browser, device category, mobile device model or branding, and more.
  • Behavior like the number of sessions per user, days since last session, transactions, and session duration.
  • “Date of First Session” where you could create lists based on the initial session date or a range (sessions that started between two dates).
  • Traffic Sources based on campaign, medium, source, or keyword.
  • Ecommerce options like transaction id, revenue, days to transaction, product purchased, or product category.
  • And you can combine any of these options to create even more advanced Remarketing lists.


Now, the options listed above are based on the major categories of segments in Google Analytics.  But you can also set Remarketing lists based on conditions.  Using conditions, you could leverage many of the dimensions or metrics available in Google Analytics to build advanced Remarketing lists.  Actually, there are so many options via “conditions” that I can’t even list them all here in this post.

For example, there are eight major categories of dimensions and metrics you could choose from, including Acquisition, Advertising, Behavior, Custom Variables, Ecommerce, Time, Users, and Other.  And each category has a number of dimensions or metrics you can select to help craft your Remarketing lists.

Using Conditions to Create Remarketing Lists in Google Analytics

Note, it can definitely be overwhelming to review the list of options when you first check this out.  Don’t worry, I provide some tips for getting started later in this post.  For now, just understand that you can use segments and conditions in Google Analytics to craft Remarketing lists based on a number of factors (or a combination of factors).  Basically, you have the power to remarket however you like.  And that’s awesome.

Examples of What You Can Do
Enough with the introduction.  Let’s get specific.  I’m sure you are wondering how segments in Google Analytics can be used in the real-world.  I’ll provide a few examples below of Remarketing lists you can build to get back in front of people who already visited your website.  Note, the lists you build should be based on your specific business and website.  I’m just covering a few options below so you can see the power of using segments to build Remarketing lists.

Example 1: Remarket to users who came from a specific referral path (page).
Imagine you knew that certain referring webpages drove a lot of high-quality traffic on a regular basis.  Based on the quality of traffic coming through those referring pages, you decide that you would love to remarket to those users as they browse the web (since you have a strong feel for the type of user they are based on the content at hand).

Using segments, you could create a Remarketing list based on the original referral path (i.e. the referring pages).  And once that list reaches 100 members, then you can start getting targeted ads in front of those users and driving them to your preferred landing page (whether that’s current content, campaign landing pages, etc.)

Using Referring Path to Create Remarketing Lists

And if you find several referring pages that target similar categories of content, then you could use Boolean operators to combine those pages from across different websites.  For example, {referring path A} AND {referring path B}.  For example, if three referring pages are all about Category A, then you could combine them to create a Remarketing list.  You can also use regular expressions to match certain criteria.  Yes, the sky’s the limit.

Using Boolean Operators to Create Advanced Remarketing Lists

Example 2: Reach a certain demographic that has visited your website.
Let’s say you just launched a new product targeting 18-25 year olds and wanted to remarket to users who already visited your website that fit into this category.  You know they showed some interest in your company and products already (since they already visited your site), so you want to reach them via display advertising as they browse the web.

Using segments, you could create a Remarketing list based on age using the Demographics category.  Simply click the checkbox next to the age category you want to target.

Creating Remarketing Lists Based on Demographics

Or to get even more targeted, you could combine age with gender to test various messaging or visuals in your ads.  Going even further, you could add location as another selection to target users based on age, gender, and geographic location (down to the city level if you wanted).

Combining Demographics to Create Advanced Remarketing Lists

Example 3: Target users of specific campaigns, ad groups, or keywords.
Let’s say you are already using AdWords to drive targeted users to your website.  Using segments in Google Analytics, you could build a Remarketing list based on specific campaigns, ad groups, or keywords.  For example, if you have an ad group targeting a specific category or product, then you could create a list containing the users that already searched Google and clicked through your ads related to that category.  It’s a great way to get back in front of a targeted audience.

Creating Remarketing Lists Based on Previous Campaigns

And by combining the targeting listed above with ecommerce conditions like the number of transactions or amount of revenue generated, you could create advanced Remarketing lists targeting very specific types of users.

Creating Remarketing Lists Based on Revenue

Example 4: Pages or Page Titles
If you have been building a lot of new content and want to reach those visitors as they browse the web, then you could create a Remarketing list based Pages or Page Titles.  For example, let’s say you have 25 blog posts about a certain category of content.  They rank very well, have built up a nice amount of referral traffic, etc.  You could build a Remarketing list by select a grouping of pages via urls or via page titles. Then you could reach those users as they browse the web and drive them to a targeted landing pages, knowing they were interested in a certain post (or group of posts) about a certain subject.

Creating Remarketing Lists Based on Page Titles

And you can combine those pages with conversion goals to add users to a list that completed some type of important action on the site.  For example, users that signed up for your email newsletter, users that triggered an event, downloaded a study, etc.

Creating Remarketing Lists Based on Page Titles and Conversion

Remarketing Tips

Based on the examples listed above, I hope you see the power in using segments and conditions to craft Remarketing lists.  But as I said earlier, it can quickly become overwhelming (especially for marketers new to Remarketing).  Below, I’ve listed several important tips to keep in mind while crafting your campaigns.

  1. Remarketing Lists Require 100 Members
    A list requires at least 100 members before you can start showing ads to users.  Keep this in mind when building lists to ensure you can reach that number.  If not, you will never get back in front of those users.
  2. Start Simple, Then Increase in Complexity
    Based on the 100 member requirement, start with simpler Remarketing lists and increase your targeting as you get more comfortable with Remarketing.  Don’t start with the most granular targeting possible, only to have a list of 3 people.
  3. Refine Your Tracking Snippet
    Google requires that you refine your Google Analytics tracking code in order take advantage of Remarketing.  Review the documentation to ensure you have the proper technical setup.
  4. Craft a Strategy First, and Your Lists Should Support Your Strategy
    Don’t create lists for the sake of creating lists. Always start by mapping out a strong Remarketing strategy before jumping into list creation. Your strategy should dictate your Remarketing lists, and not the other way around.  Spend time up front mapping out who you want to target, and why.  And once you have a solid plan mapped out, you can easily build your lists via Google Analytics segments and conditions.
  5. Use Display Advertising In Addition to Text Ads
    Remarketing enables you to use both image ads and text ads.  Definitely use both when crafting your campaigns.  There are a number of sizes and formats you can use.  I recommend hiring a designer to build your ads unless you have in-house staff that is capable of designing high-quality ads.  Use image ads where possible to grab the user’s attention and provide text ads as a backup when a site doesn’t support image ads.  You don’t have to choose one or the other.
  6. Measure Your Results! Don’t “Set It and Forget It”.
    Remarketing is advertising.  And advertising campaigns should have a goal.  Don’t simply set up Remarketing without knowing the intended action you want users to take.  Instead, make sure you set up conversion goals to track how those users convert.  Do not set up the campaign and let it run without analyzing the results.  Understand the ROI of the campaign.  That’s the only way you’ll know if it worked, if the campaign should keep running, and if you should base other campaigns on the original.


Summary – New and Powerful Ways to Remarket
After reading this post, I hope you see the power in using segments and conditions for creating Remarketing lists.  In my opinion, too many marketers keep going after new eyeballs and easily forget about the eyeballs that already showed an interest in their company, products, or services.  I believe that’s a mistake.  Instead, marketers can craft advanced Remarketing lists to get back in front of a targeted audience.  Doing so provides another chance at converting them.

Remember, a warm lead is always more powerful than a cold call.  Good luck.



Wednesday, April 16th, 2014

I’m Speaking at the Weber Shandwick Data Salon on April 24th – Learn About Google Algorithm Updates, Manual Penalties, and More

Weber Shandwick Data Salon on April 24, 2014

I’m excited to announce that I’ll be speaking at the Weber Shandwick Data Salon on Thursday, April 24th in New York City (from 6:00PM to 7:30PM).  Each month, Weber Shandwick invites leaders from various areas of digital marketing to speak, to spark conversation, and to share ideas.  I’m thrilled to be presenting next week to speak about the latest in SEO.

My presentation will cover some extremely important topics that I’m neck deep in on a regular basis, including Google algorithm updates, manual penalties, and the war for organic search traffic that’s going on each day.  I’ll be introducing various algorithm updates like Panda and Penguin, explain what manual actions are, and provide case studies along the way.  I’ll also introduce the approach that Google is using to fight webspam algorithmically, while also covering how manual penalties work, how to recover from them, and how to ensure websites stay out of the danger zone.

My goal is to get the audience thinking about content quality, webspam, unnatural links, and webmaster guidelines now before any risky tactics being employed can get them in trouble.  Unfortunately, I’ve spoken with hundreds of companies over the past few years that were blindsided by algo updates or manual actions simply because they never thought about the repercussions of their tactics, didn’t understand Google’s stance on webspam, or the various algorithm updates it was crafting.  Many of them learned too late the dangers of pushing the envelope SEO-wise.

So join me next Thursday, April 24th at 6PM for a deep dive on algorithm updates, manual penalties, and more from the dynamic world of SEO.  You can register today via the following link:

Register for Weber Shandwick’s Data Salon on April 24th:

Below I have provided the session overview.  I hope to see you there!

Weber Shandwick Data Salon #3
April 24, 2014 from 6:00PM to 7:30PM
Speaker: Glenn Gabe of G-Squared Interactive
Moderator: Kareem Harper of Weber Shandwick
909 Third Avenue, 5th Floor
*Refreshments will be available starting at 6:00pm

Front Lines of SEO

The Frontlines of SEO – Google Algorithm Updates, Penalties, and the Implications for Marketers
Explore Google’s war on webspam, learn about key changes and updates occurring in Search right now, and fully understand the implications for digital marketers.

There’s a battle going on every day in Search that many people aren’t aware of.  With millions of dollars in revenue on the line, some businesses are pushing the limits of what’s acceptable from an SEO perspective.  In other words, gaming Google’s algorithm to gain an advantage in the search results.

Google, with its dominant place in Search, is waging war against tactics that attempt to manipulate its algorithm.  From crafting specific algorithm updates that target webspam to applying manual actions to websites, Google has the ability to impact the bottom line of many businesses across the world.  And that includes companies ranging from large brands to small local businesses.  This session will introduce the various methods Google is using to address webspam in order to keep its search results as pure as possible. Specific examples will be presented, including case studies of companies that have dealt with algorithm updates like Panda and Penguin.  Manual penalties will be discussed as well.

Beyond battling webspam, the major search engines have been innovating at an extremely rapid pace.  The smartphone and tablet boom has impacted how consumers search for data (and how companies can be found).  And now the wearable revolution has begun, which will add yet another challenge for marketers looking to reach targeted audiences.   Glenn will introduce several of the key changes taking place and explain how marketers can adapt.  Glenn is also a Glass Explorer and will provide key insights into how Google Glass and other wearables could impact marketing and advertising.

Register today to learn more about Google’s war on webspam, to better understand the future of Search, and to prepare your business for what’s coming next.

You can register online by clicking the following link:



Wednesday, December 18th, 2013

Panda Report – How To Find Low Quality Content By Comparing Top Landing Pages From Google Organic

Top Landing Pages Report in Google Analytics

Note, this tutorial works in conjunction with my Search Engine Watch column, which explains how to analyze the top landing pages from Google Organic prior to, and then after, Panda arrives.  With the amount of confusion circling Panda, I wanted to cover a report webmasters can run today that can help guide them down the right path while on their hunt for low-quality content.

My Search Engine Watch column covers an overview of the situation, why you would want to run the top landing pages report (with comparison), and how to analyze the data.  And my tutorial below covers how to actually create the report.  The posts together comprise a two-headed monster that can help those hit by Panda get on the right track.   In addition, my Search Engine Watch column covers a bonus report from Google Webmaster Tools that can help business owners gather more information about content that was impacted by the mighty Panda.

Why This Report is Important for Panda Victims
The report I’m going to help you create today is important, since it contains the pages that Google was ranking well and driving traffic to prior to a Panda attack.  And that’s where Google was receiving a lot of intelligence about content quality and user engagement.  By analyzing these pages, you can often find glaring Panda-related problems.  For example, thin content, duplicate content, technical problems causing content issues, low-quality affiliate content, hacked content, etc.  It’s a great way to get on the right path, and quickly.

There are several ways to run the report in Google Analytics, and I’ll explain one of those methods below.  And remember, this should not be the only report you run… A rounded analysis can help you identify a range of problems from a content quality standpoint.  In other words, pages not receiving a lot of traffic could also be causing Panda-related problems.  But for now, let’s analyze the top landing pages from Google Organic prior to a Panda hit (which were sending Google the most data before Panda arrived).

And remember to visit my Search Engine Watch column after running this report to learn more about why this data is important, how to use it, red flags you can identify, and next steps for websites that were impacted.  Let’s get started.

How To Run a Top Landing Pages Report in Google Analytics (with date comparison): 

  • First, log into Google Analytics and click the “All Traffic” tab under “Acquisition”.  Then click “Google / Organic” to isolate that traffic source.
    Accessing Google Organic Traffic in Google Analytics
  • Next, set your timeframe to the date after Panda arrived and extend that for a decent amount of time (at least a few weeks if you have the data).  If time allows, I like to set the report to 4-6 weeks after Panda hit.  If this is right after an algorithm update, then use whatever data you have (but make sure it’s at least one week).  I’m using a date range after the Phantom update hit (which was May 8th).
    Setting a Timeframe in Google Analytics
  • Your next move is to change the primary dimension to “Landing Page” to view all landing pages from Google organic search traffic.  Click the “Other” link next to “Primary Dimension” and select “Acquisition”, and then “Landing Page”.  Now you will see all landing pages from Google organic during that time period.
    Primary Dimension to Landing Page in Google Analytics
  • Now let’s use some built-in magic from Google Analytics.  In the timeframe calendar, you can click a checkbox for “Compare to” and leave “Previous period” selected.  Once you click “Apply”, you are going to see all of the metrics for each landing page, but with a comparison of the two timeframes.  And you’ll even have a nice trending graph up top to visualize the Panda horror.
    Comparing Timeframes in Google Analytics
  • As you start going down the list of urls, pay particular attention to the “% Change” column.  Warning, profanity may ensue.  When you start seeing pages that lost 30%, 40%, 50% or more traffic when comparing timeframes, then it would be wise to check out those urls in greater detail.  Again, if Google was sending a lot of traffic to those urls, then it had plenty of user engagement data from those visits.  You might just find that those urls are seriously problematic from a content quality standpoint.
    Viewing The Percent Change in Traffic in Google Analytics


Bonus 1: Export to Excel for Deeper Analysis

  • It’s ok to stay within Google Analytics to analyze the data, but you would be better off exporting this data to Excel for deeper analysis.  If you scroll to the top of the Google Analytics interface, you will see the “Export” button.  Click that button and then choose “Excel (XLSX)”.  Once the export is complete, it should open in Excel.  Navigate to the “Dataset” worksheet to see your landing page data (which is typically the second worksheet).
    Exporting A Report In Google Analytics
  • At this point, you should clean up your spreadsheet by deleting columns that aren’t critical for this report.  Also, you definitely want to space out each column so you can see the data clearly (and the data headers).
    Clean Up Google Analytics Export in Excel
  • You’ll notice that each url has two rows, one for the current timeframe, and one for the previous timeframe.  This enables you to see all of the data for each url during both timeframes (the comparison).
    Two Rows For Each URL Based on Timeframe
  • That’s nice, but wouldn’t it be great to create a new column that showed the percentage decrease or increase for visits (like we saw in Google Analytics?)  Maybe even with highlighting to show steep decreases in traffic  Let’s do it.  Create a new column to the right of “Visits” and before “% New Visits”.  I would title this column “% Change” or something similar.
    Creating a New Column for Percent Change in Excel
  • Next, let’s create a formula that provides the percentage change based on the two rows of data for each url.  Find the “Visits” column and the first landing page url (which will have two rows).  Remember, there’s one row for each timeframe.  If your visits data is in column C, then the post-Panda data is in row 2, and the pre-Panda data is in row 3 (see screenshot below).  You can enter the following formula in the first cell for the new column “% Change”.=(C3-C2)/C3.Again, C3 is the traffic levels from the previous timeframe, C2 is the traffic levels from the current timeframe (after the Panda hit), and you are dividing by the previous traffic levels to come up with the percentage change.  For example, if a url dropped from 5,450 visits to 640 visits, then your percentage drop would be 88%.  And yes, you would definitely want to investigate that url further!
    Creating a Formula to Calculate Percent Change in Excel
  • Don’t worry about the floating decimal point.  We’ll tackle that soon.  Now we need to copy that formula to the rest of the column (but by twos).  Remember, we have two records for each url, so you’ll need to highlight both cells before double clicking the bottom right corner of the second cell to copy the formula to all rows.  Once you do, Excel automatically copies the two rows to the rest of the cells in that column.  Now you should have percentage drops (or increases) for all the urls you exported.  Note, you can also highlight the two cells, copy them, and then highlight the rest of that column, and click paste.  That will copy the formula to the right cells in the column as well.
    Copying a Formula to All Rows in Excel
  • Now, you will see a long, floating decimal point in our new column labeled “% Change”.  That’s an easy fix, since we want to see the actual percentage instead.  Highlight the column, right click the column, and choose “Format Cells”.  Then choose “Percentage” and click “OK”.  That’s it.  You now have a column containing all top landing pages from Google organic, with the percentage drop after the Panda hit.
    Formatting Cells in Excel


Bonus 2: Highlight Cells With A Steep Drop in Red

  • If you want the data to “pop” a little more, then you can use conditional formatting to highlight cells that exceed a certain percentage drop in traffic.  That can easily help you and your team quickly identify problematic landing pages.
  • To do that, highlight the new column we created (titled “% Change”), and click the “Conditional Formatting” button in your Home tab in Excel (located in the “Styles” group).  Then select, “Highlight Cells Rules”, and then select, “Greater Than”.  When the dialog box comes up, enter a minimum percentage that you want highlighted.  And don’t forget to add the % symbol!  Choose the color you want to highlight your data with and click “OK”.  Voila, your problematic urls are highlighted for you.  Nice.
    Applying Conditional Formatting in ExcelApplying Conditional Formatting by Percentage in Excel


Summary – Analyzing Panda Data
If you made it through this tutorial, then you should have a killer spreadsheet containing a boatload of important data.  Again, this report will contain the percentage increase or decrease for top landing pages from Google Organic (prior to, and then after, a Panda hit).  This is where Google gathered the most intelligence based on user engagement.  It’s a great place to start your analysis.

Now it’s time to head over to my Search Engine Watch column to take a deeper look at the report, what you should look for, and how to get on the right track with Panda recovery.  Between the tutorial and my Search Engine Watch column, I hope to clear up at least some of the confusion about “content quality” surrounding Panda updates.  Good luck.




Monday, December 17th, 2012

Trackbacks in Google Analytics – How To Analyze Inbound Links in GA’s Social Reports

Trackbacks in Google Analytics

In May of 2012, Google Analytics introduced trackbacks in its social reporting.  If you’re not familiar with trackbacks, they enable you to understand when another website links to your content.  So, using Google Analytics, and the new trackbacks reporting, you could start to track inbound links you are building from across the web.

Note, if you want to perform advanced-level analysis of your links, you should still use more robust tools like Open Site Explorer or Majestic SEO.  But, trackbacks reporting is a quick and easy way to identify backlinks, and right within Google Analytics.  It can definitely supplement your link analysis efforts.

If you’re in charge of content strategy for your company, or if you are publishing content on a regular basis, then checking trackbacks reporting in GA can quickly help you understand the fruits of your labor.  But since trackbacks reporting isn’t immediately visible, I’ve written this post to explain how you can find trackbacks, and then what you can do with the data once you access the reporting.

Social Reports and Trackbacks
First, if you’re not familiar with social reporting in Google Analytics, you should check out my post from March where I cover how to use the new social reports to analyze content.  Social reports are a great addition to GA, but I still find many marketers either don’t know about them, or don’t know how to use them.  And that’s a shame, since they provide some great insights about the traffic coming from social networks, and the conversations going on there (at least for data hub partners).

Below, I’m going to walk you step by step through the process of finding links to your content via trackbacks reporting.  Once we find them, I’ll explain what you can do with your newly-found link data.

How To Find Trackbacks (Step by Step)
1. Access your Google Analytics reporting, and click “Traffic Sources”, “Social”, and then “Network Referrals”.

Trackback Reporting in Google Analytics

2. Next, click a network referral in the list like Google Plus, Twitter, Facebook, etc. Note, “Network Referral” is new language used by Google Analytics for “Social Network” or “Source”.

Network Referrals in Google Analytics

3. Once you click through a source, you should click the “Activity Stream” tab located near the top of the screen (right above the trending graph).

Activity Stream in Google Analytics Social Reports

4. Once you click the activity stream tab, you’ll need to click the dropdown arrow next to the “Social Network” label at the very top of the screen.  Once you do, you’ll see a link in that list for “Trackbacks”.  Click that link.

Finding Trackbacks in Google Analytics

5. Once you click the “Trackbacks” link, you will see the links to your content that Google Analytics picked up.

Viewing Trackbacks in Google Analytics Social Reports

Congratulations, you found the hidden treasure of trackbacks in Google Analytics!  Not the easiest report to find, is it?  Now let’s find out what you can do with the data.

What You Can Do Once You Find Trackbacks
First, I’ll quickly cover the data provided in the trackbacks reporting.  Google Analytics provides the following information for each trackback it picks up:

  • The date the trackback was picked up.
  • The title and URL of the page linking to your content.
  • The ability to launch and view your content that’s receiving the link.
  • And a quick way to isolate that content in your social reports (to view all social activity for that specific page).

Next, I’ll cover four ways you can benefit from analyzing trackbacks data in Google Analytics, including a bonus at the end.  Let’s jump in.

1. Understand the source of the trackback (Who is linking to you.)
Linkbuilding is hard.  So when your content builds links naturally, you definitely want to understand the source of those links.  Trackbacks in Google Analytics provides an easy and quick way to identify links to your content.  But once you build some links, you shouldn’t stop and have a tropical drink with a fancy umbrella as you admire your results.  You should analyze your newly-found inbound links.

For example, you should determine if the links are strong, relevant, and how much will those links help with your SEO efforts.  You should also determine which authors decided to link to you, what’s their background, and where else do they write?m

One of the first things you’ll see in trackbacks reporting is the title and URL of the page linking to your content.  At this point, you can click the small arrow icon next to the URL to open the referring page in a new window.  You can also click the “More” button on the right side of the page, and then click “View Activity” to be taken to the page linking to your content.

Viewing Trackbacks in Google Analytics

At this point, you can check out the article or post linking to you, understand who wrote the content, what they focus on, link to their social accounts, find their contact information, etc.  Building relationships with quality authors in your niche is a great way to earn links down the line.  Therefore, analyzing the people who already link to your content is low-hanging fruit.  Trackbacks in GA make it easy to find them.

2. Understand Your Content That’s Building Links
When I’m working with content teams, I always get the question, “what should we write about?”  I’m a big believer that a content generation plan should be based on data, and not intuition.  And trackbacks provide another piece of data to analyze.  Let’s face it, the proof is in the pudding from a linkbuilding standpoint.  Either your content builds links or it doesn’t.  If it does, you need to find out why that content built the links it did.  And if it didn’t build links, you need to document that and make sure you don’t make the same mistake again.

As I mentioned earlier, there are some outstanding link analysis tools on the market, like Open Site Explorer and Majestic SEO, and I’m not saying that trackbacks in Google Analytics are the end-all.  But, you can definitely use the reporting to quickly understand which content is building links.

Once you find trackbacks and identify the content that built those links, you can start to analyze and understand what drove interest.  Was it breaking news, evergreen content, how-to’s, industry analysis, etc?  Which topics were hot from a linkbuilding standpoint, and were those the topics you expected to build links?  If you find a subject that worked well in the past, you can build a plan for expanding on that topic.  Also, are the pages linking to you providing ideas for new posts?  Do the comments on the page provide ideas, what did the author mention, etc?  Trackbacks provide a mechanism for supplementing your analysis.

3. Join the conversation, Engage Influencers
I explained above how you can find the people (and websites) linking to your content.  That’s great, but you shouldn’t stop there.  If there’s a conversation happening on that referring page, then you should join the conversation.  If someone went to the extent to mention and link to your content, the least you can do is thank them, and provide value to the conversation.

Adding value to the conversation and engaging a targeted audience can help you build more credibility and connect with targeted people in your niche.  And as I mentioned above, you can connect with the author of the post via email or via their social accounts.

4. Understand Linkbuilding Over Time
Using the trending graph in Google Analytics, you can visually understand linkbuilding over time.  The graph at the top of the screen will show you the number of trackbacks earned over the time period you have selected in GA.  I’m not saying that it’s better than using other, dedicated link analysis tools, but this is a quick way to find link data right within Google Analytics.

Trackbacks Trending in Google Analytics

In addition, if you click the “More” button for any specific trackback, and then click “Page Analytics”, you can isolate specific pieces of content receiving links.  Note, I’ve been seeing a test in Google Analytics where “Page Analytics” is replaced by “Filter on this Page”.  Personally, I like “Filter on this Page” since it’s more intuitive.  Regardless, after clicking the link you can trend linkbuilding over time for a specific piece of content.

Viewing Trackbacks for a Specific Page

In addition, you can always compare timeframe to see how links were built during one timeframe versus another.  You might find some interesting things, like a piece of content that built more inbound links months later versus when the content was first published.  Then you can dig into the links to find out why…

Bonus: Export The Data!
As with any report in Google Analytics, you can easily export trackbacks data.  If you are viewing any trackbacks report, you can click “Export” at the top of the screen, and then choose a format to quickly export the data for further analysis in Excel.  Then you can slice and dice the data, combine data from other reports, etc.  What you do with the data depends on your own Excel skills.  :)

Exporting Trackback Data in Google Analytics

Summary – Quick Link Analysis in Google Analytics
I hope after reading this post you’re ready to jump into Google Analytics to hunt down trackbacks.  Again, Google didn’t necessarily make it super-easy to find trackbacks, but they are there.  Once you do find them, you can analyze those links to glean important insights that can help your future content and linkbuilding efforts.  Although there are more robust link analysis solutions on the market, trackbacks reporting is a quick and easy way to identify and then analyze inbound links.  I recommend checking out the reporting today.  You never know what you’ll find.  :)



Tuesday, November 27th, 2012

How Google Analytics *Really* Handles Referring Traffic Sources [Experiment] – Why Clicks and Visits Might Not Match Up

Google Analytics Referrals

Let me walk you through a common scenario in web marketing.  You have a website, and some people visit your site by clicking through links on other websites.  In your web analytics reporting, those visits are categorized as referring visits.  In Google Analytics specifically, those visits show up in your “Referrals” report under “Traffic Sources”.  And when visitors click on an outbound link on your site (a link to another website), your site shows up as a referring source in that website’s referrals report.

That’s pretty straight forward, but what I’m about to cover isn’t.  I find many marketers and webmasters don’t understand how Google Analytics handles that referring traffic during future visits to their websites.  For example, if someone clicks through to your site from, leaves your site, and then returns the next day.

Do you know how that visit will be categorized in Google Analytics?  There’s a good chance you don’t, and I’m going to cover the topic in detail in this post.

Understanding Referring Visits is Important When Revenue and Cost Are Involved
I believe one of the reasons this topic isn’t understood very well is because it often doesn’t directly impact revenue or cost for many webmasters.  Sure, you definitely want to know how many people are coming from each referring site, but for many webmasters, the exact number doesn’t impact revenue, or payments to other webmasters.

But, for websites that need to track the monetary value of inbound visits and outbound clicks, accurately determining referring visits is extremely important.  For example, imagine you were charging certain partners for traffic you were sending from your site to theirs, or vice versa.  The fact of the matter is that checking referring sources could show different numbers than you think, and could be much different than the outbound clicks you see.  And depending on your own situation, the numbers could be way off…

The Core Disconnect – How Google Analytics Calculates Visits from Referring Sources (or any campaign, search visit, etc.)
Here’s the core disconnect.  When someone clicks through to your site via a referring source, the utm_z cookie is updated with traffic source information.  That cookie will not be overwritten unless another referring source or campaign takes it place.  Direct Traffic will not overwrite this value.  Let me say that again.  Direct Traffic will not overwrite the utm_z cookie value.  That means the utm_z value will remain the referring source of traffic when those visitors return to the site.

Google Analytics utm_z Cookie

What This Means To You

I know what you’re thinking. This guy is telling me about utm_z cookies?? What the heck does that mean to me?  OK, stick with me for a second.  Let’s say you had a partnership set up where another website pays you for traffic.  Maybe you’re both in the same niche and want to leverage each other’s traffic for more exposure.  You check your stats for the previous month and notice that you sent 500 visits to partner A.  Cool, so you contact them to check how the partnership is going and to make sure they are seeing the same number of visits.  They come back and say they’ve seen 700 visits from your site and thank you for the traffic.  The check will be cut soon.

Google Analytics Clicks and Visits Could Be Off

But that 200 visit discrepancy is bothering you.  Why is there a big difference between your partner’s reporting and the numbers you are seeing?  And let’s assume you have a solid setup for tracking clicks out of your website.  For example, maybe you are running outbound clicks to partners through a redirect that captures a number of important metrics.  The redirect then sends the visitor off to the correct URL on the partner site.  Basically, you know you are capturing all outbound clicks to the partner website.

This is where the native handling of referring sources in Google Analytics comes into play.  Sure, you are tracking clicks off your site, but your partner’s analytics package is capturing those clicks plus any return visits that are direct visits.  So, if someone clicks through to your partner’s site, then that’s one visit.  If they leave that site, and return directly (by typing the url directly in their browser or via a bookmark), then the visit will show up as a visit from the original referring source (your website).  That’s now two visits.  And if they do it again, that will be three visits.  That’s until another referring source or campaign overwrites the utm_z cookie.  In this example, there were 3 visits to your 1 outbound click!

An Example of How Google Analytics Handles Referrals

Based on this simple example, you can easily see how over a month’s time, some people would click through to your partner’s site and then revisit their site directly (and possibly a few times).  That would lead to more than one visit per user, and could sway the visit count from your website.

Still confused?  Let me clear this up via an experiment below.

Experiment – Calculating Referring Visits in Google Analytics
In the following simple example, I set up a webpage on a second domain that links to a landing page I set up on my website just for this experiment.  I didn’t want to skew the reporting by using an existing page on my site that gets a lot of visits.  Then I used several computers I have here with clean browsers to first visit the referring page that links to my new landing page, and then I clicked through.  The referring source should show up as the domain name of the referring site.  That would be visit #1.

Next, I would leave the new landing page on my site and revisit my website later by typing the exact URL into my browser (what most people would think is a Direct Traffic visit).  In theory, the referring site should show up as the traffic source, even though I’m entering the site as “Direct Traffic”.  Remember, the utm_z cookie will only be overwritten by another referring source or campaign.

Last, I would search for a keyword that my site ranks for, and then click through to the site.  And since this visit was from a search engine, the utm_z cookie would be updated with this new value, and my reporting would show Google as the referring source (along with the keyword I entered). Let’s find out the results of the experiment below.

The Results
1. First Visit

First, I visited the second domain and clicked through to my website.  Here is the first referring visit showing up in my analytics package:
Referring Sites Experiment - First Visit

2. Second Visit (Directly Visiting the Site)
Next, I left the site and returned via Direct Traffic.  Google Analytics shows the referring site as the source for this traffic, even though I entered via “Direct Traffic”. Also notice it accurately categorizes me as a “return visitor”:
Google Analytics Referral Experiment - Second Visit

3. Third Visit (Again Directly Visiting the Site, but the Next Day)
Just to underscore my point, I left and revisited the site the next day (again via Direct Traffic).  Google Analytics again shows the visit is from the initial referring source:
Google Analytics Referral Experiment - Third Visit

4. Fourth Visit, This Time From Search
Finally, I searched for my name on Google and visited my website.  Now Google Analytics shows the keyword that led to the site (from the traffic source “Google”).  Remember, the utm_z cookie will only be updated when another referring source is identified (versus Direct Traffic).
Google Analytics Referral Experiment - Search Visit


So there you have it.  Proof that your visit count by source may not be what you think it is.  Now, if you’re reading this post and are either generating revenue from referring visits, or you have to pay partners based on visits, then you might be frantically running to Google Analytics to rerun your reports.  Yes, this could impact things quite a bit.   I’ll leave it up to you how you handle the situation. :)

What You Can Do – The Importance of Clarity
If you do have a partnership where you are either generating revenue by driving traffic, or you are paying for traffic from other websites, then each party needs to clearly understand the arrangement.  Each website involved needs to be clear on the definitions of “traffic”, “clicks”, “visits”, etc.  For example, think about AdWords for a second.  You pay Google for clicks on ads, but don’t pay Google for direct visits back to your site (even though those visits will show up as campaign visits).  And by the way, most partners will not give you access to their reporting anyway… Therefore, you will only know the clicks out from your site.

If you are tracking outbound clicks, you can use event tracking in Google Analytics to track those clicks, including the pages or links where those clicks are originating.  If you don’t want to use event tracking, then you can run outbound clicks through a 302 redirect and capture the information you need to accurately track clicks.  If you are receiving traffic, then you can make sure the referring links contain querystring parameters so you can understand which partner the traffic is coming from (and that it’s not a standard referral from the site).  There are other ways to handle this, and those are just a few ideas.

Summary – Understanding Visits in Google Analytics
I hope you found this post explaining how Google Analytics handles referring visits helpful.  I know this topic can be confusing, and experiments always help clear up some of the confusion.  So now you know why visits might be higher or lower than you think, and how the utm_z cookie controls what shows up in your reporting.  I bet you’ll never look at referring sources the same again.

And let’s hope you’re not on the short end of the stick. :)

Happy Reporting.



Wednesday, November 14th, 2012

Hunting False Negatives – How To Avoid False Negatives When Checking Redirects After a Website Redesign or Migration [Screaming Frog Tutorial]

How To Check Redirects Using Screaming Frog

Every webmaster has to deal with a website redesign or migration at some point.  And redesigns and migrations often mean that your URL structure will be impacted.  From an SEO perspective, when URL’s need to change, it’s critically important that you have a solid 301 redirection plan in place.  If you don’t, you can pay dearly SEO-wise.

I wrote a post for my Search Engine Journal column last spring titled “How to Avoid SEO Disaster During a Website Redesign” and implementing a 301 redirection plan was one of the most important topics I covered.  I find many webmasters and marketers don’t understand how SEO power is built URL by URL.  As your URL’s build up inbound links and search equity, it’s important that those URL’s maintain those links and equity.  If you change those URL’s, you must notify the search engines where the old content moved to, and that’s where 301 redirects come into play.

So, when you change URL’s, you run the risk of losing all of the links pointing to the older URL’s, and the search power that the URL’s contained.  That’s unless you 301 redirect the old URL’s to the new ones.  A 301 redirect safely passes PageRank from an old URL to a new one (essentially maintaining its search equity).

Unfortunately, I’ve seen many companies either not set up a redirection plan at all, or botch the plan.  That’s when they end up with a catastrophic SEO problem.  Rankings drop quickly, traffic drops off a cliff, sales drop, and nobody is happy at the company (especially the CMO, CFO, and CEO).

Traffic Drop After Website Redesign

Meet the False Negative Redirect Problem, A Silent Killer During Redesigns or Migrations:
Needless to say, properly setting up your redirects is one of the most important things you can do when redesigning or migrating your website.  That said, even if you address redirects and launch the new site, how do you know that the redirects are in fact working?  Sure, you could manually check some of those URL’s, but that’s not scalable.  In addition, just because an older URL 301 redirects to a new URL doesn’t mean it redirects to the correct URL.  If you don’t follow through and check the destination URL (where the redirect is pointing), then you really don’t know if everything is set up properly.

This is what I like to call the False Negative Redirect Problem.  For SEO’s, a false negative occurs when your test incorrectly shows that the redirects are working properly (they don’t test positive for errors), when in fact, the destination URL’s might not be resolving properly.  Basically, your test shows that the redirects are ok, when they really aren’t.  Incorrectly thinking that 301 redirects are working properly by only checking the header response code for the old URL can trick webmasters into believing the redesign or migration has gone well SEO-wise, when in reality, the destination URL’s could be 404’ing or throwing application errors.  It’s a silent killer of SEO.

False Negatives can be a Silent SEO Killer

How To Avoid the Silent SEO Killer When Changing Implementing Redirects
The false negative problem I mentioned above is especially dangerous when changing domain names (where you will often implement one directive in .htaccess or ISAPI_Rewrite that takes any request for a URL at one domain and redirects it to the same URL at another domain).  Just because it 301’s doesn’t mean the correct URL resolves.  Think about it, that one directive will 301 every request… but you need to check the destination URL to truly know if the redirects are working the way you need them to.  Unfortunately, many SEO’s only check that the old URL’s 301, but they don’t check the destination URL.  Again, that could be a silent killer of SEO.

Screaming Frog Hops to the Rescue
I mentioned “scalable” solutions earlier.  Well, Screaming Frog provides a scalable solution for checking redirects during a migration or website redesign.  Note, Screaming Frog is a paid solution, but well worth the $157 annual fee.  Using Screaming Frog, you can import a list of old URL’s from your analytics package or CMS and have it crawl those URL’s and provide reporting.  Running a two-step process for checking redirects and destination URL’s can help you understand if your redirects are truly working.  For example, you might find redirects that lead to 404’s, application errors, etc.  Once you find those errors, you can quickly change them to retain search equity.

Below, I’m going to walk you through the process of exporting your top landing pages from Google Analytics and checking them via Screaming Frog to ensure both the redirects are working and that the destination URL’s are resolving correctly.  Let’s get started.

What You’ll Need and What We’ll Be Doing

  • First, we are going to export our top landing pages from Google Analytics.
  • Second, we’ll use the CONCATENATE function in Excel to build complete URL’s.
  • Next, we’ll add the URL’s to a text file that we can import into Screaming Frog.
  • Then we’ll fire up Screaming Frog and import the text file for crawling.
  • Screaming Frog will crawl and test those URL’s and provide reporting on what it finds.
  • Then we can export the destination URL’s we find so we can make sure they resolve correctly.  Remember, just because the old URL’s 301 redirect doesn’t mean the destination URL’s resolve properly.  We are hunting for false negatives.
  • Last, and most importantly, you can fix any problematic redirects to ensure you maintain search equity.

How To Use Screaming Frog to Hunt Down False Negatives:

  1. Export Top Landing Pages from Google Analytics
    Access your Google Analytics reporting and click the “Content” tab, “Site Content”, and then “Landing Pages”.  Click the dropdown for “Show rows” at the bottom of the report and select the number of rows you want to view.Export top landing pages from Google Analytics

    Tip: If you have greater than 500 pages, then you can edit the URL in Google Analytics to display greater than 500 URL’s.   After first selecting a row count from the dropdown, find the parameter named table.rowCount= in the URL.  Simply change the number after the equals sign to 1000, 5000, 10000, or whatever number you need to capture all of the rows.   When you export your report, all of the rows will be included.

  2. Export the Report from Google Analytics
    Click the Export button at the top of the report and choose “CSV”.  The file should be exported and then open in Excel once it downloads.
    Exporting a report from Google Analytics
  3. Use Excel’s CONCATENATE Function to Build a Complete URL
    When the URL’s are exported from Google Analytics, they will not include the protocol or domain name.  That’s the beginning of a URL with  Therefore, you need to add this to your URL’s before you use them in Screaming Frog.  Excel has a powerful function called CONCATENATE, which lets you combine text and cell contents to form a new text string.  We’ll use this function to combine the protocol and domain name with the URL that Google Analytics exported.

    Create a new column next to the “Landing Page” column in Excel.  Click the cell next to the first landing page URL and start entering the following: =CONCATENATE(“”, A8).  Note, change “” to your actual domain name.  Also, A8 is the cell that contains the first URL that was exported from Google Analytics (in my spreadsheet).  If your spreadsheet is different, make sure to change A8 to whichever cell contains the first URL in your sheet.  The resulting text should be the complete URL (combining protocol, domain name, and URL exported from Google Analytics).  Then you can simply copy and paste the contents of that cell (which contains the formula) to the rest of the cells in that column.  The formula will automatically adjust to use the right landing page URL for that row. Now you have a list of all complete URL’s that you can import into Screaming Frog.

    Using the CONCATENATE function in Excel to buld URL's

  4. Copy all URL’s to a Text File
    Since all we want are the URL’s for Screaming Frog, you can select the entire new column you just created (with the complete URL’s) and copy those URL’s.  Then open a text file and paste the URL’s in the file.  You can use Notepad, Textpad, or whatever text editor you work with.  Save the file.

    Copy the URL list to a text file

  5. Fire Up Screaming Frog
    After launching Screaming Frog, let’s change the mode to “list” so we can upload a list of URL’s.  Under the “Mode” menu at the top of the application, click “List”, which enables you to use a text file of URL’s to crawl.   Then click “Select File” and choose the text file we just created.  Then you can click “Start” and Screaming Frog will begin to crawl those URL’s.

    Using List Mode to Crawl URL's

  6. Review Header Response Codes From the Crawl
    At this point, you will see a list of the URL’s crawled, the status codes, and the status messages.  Remember, all of the URL’s should be 301 redirecting to new URL’s.  So, you should see a lot of 301’s and “moved permanently” messages.  If you see 404’s at this point, those URL’s didn’t redirect properly.  Yes, you just found some bad URL’s, and you should address those 404’s quickly.  But that’s not a false negative.  It’s good to catch low-hanging fruit, but we’re after more sinister problems.

    Viewing 301 redirects after a Screaming Frog crawl

  7. Find the Destination URL’s for Your Redirects
    Now, just because you see 301 redirects showing up in the main reporting doesn’t mean the destination URL’s resolve correctly.  If you click the “Response Codes” tab, you’ll see the redirect URI (where the 301 actually sends the crawler).  THOSE ARE THE URL’S YOU NEED TO CHECK.    Click the “Export” button at the top of the screen to export the “Response Code” report.  This will include all of the destination URL’s.
    Finding Destination URL's via the Response Code Tab
  8. Copy All Destination URL’s to a Text File
    In Excel, copy the destination URL’s and add them to a text file (similar to what we did earlier). Make sure you save the new file.  We are now going to crawl the destination URL’s just like we crawled the original ones.  But, this process will close the loop for us, and ensure the destination URL’s resolve correctly.  This is where we could find false negatives.

    Exporting all destination URL's to excel from Screaming Frog

  9. Import Your New Text File and Crawl the Destination URL’s
    Go back through the process of selecting “List Mode” in Screaming Frog and then import the new text file we just created (the file that contains the destination URL’s).  Click “Start” to crawl the URL’s, and then check the reporting.

    Using List Mode to Crawl URL's

  10. Analyze the Report and Find False Negatives
    You should see a lot of 200 codes (which is good), but you might find some 404’s, application errors, etc.  Those are your false negatives.  At this point, you can address the errors and ensure your old URL’s in fact redirect to the proper destination URL’s.  Disaster avoided.  :)

    Finding and Fixing False Negatives Using Screaming Frog

Screaming Frog and Actionable Data: Beat False Negatives
Going through the process I listed above will ensure you accurately check redirects and destination URL’s during a website redesign or migration.  The resulting reports can identify bad redirects, 404’s, application errors, etc.  And those errors could destroy your search power if the problems are widespread.  I highly recommend performing this analysis several times during the redesign or migration to make sure every problem is caught.

Make sure you don’t lose any URL’s, which can result in lost search equity.  And lost search equity translates to lower rankings, less targeted traffic, and lower sales.  Don’t let that happen.  Perform the analysis, quickly fix problems you encounter, and retain your search power.  Redesigns or migrations don’t have to result in disaster.  You just need to look out for the silent SEO killer. :)



Monday, September 10th, 2012

SEM Competitive Analysis – The Power of Understanding Your Competition in Paid Search

SEM Competitive Analysis

There are a lot of moving parts to developing and managing SEM campaigns.  First, you need to develop a strong paid search strategy, perform keyword research, map out a robust structure for your campaigns and ad groups, determine budgets, create effective ads, etc.  After the setup phase, you will be neck deep in ongoing campaign management, which involves refining your campaigns and ad groups based on performance. That includes refining keywords, ads, creating new ad groups when necessary, pausing ad groups or campaigns that don’t perform well, split testing ads, etc.  This includes managing both Search and Display Network campaigns.  As you can guess, SEM is definitely not for the faint of heart.

Based on all that’s involved with paid search, I think it’s easy for SEM’s to keep driving campaigns forward without taking a step back to analyze the competitive landscape.  For example, which companies are you competing against in SEM, which ads are they running, what types of landing pages are they using, how does their pricing stack up, etc.  That’s where a thorough competitive analysis can pay huge dividends.  There are so many important things you can learn from analyzing the competition that I’m surprised more companies aren’t doing it.

In this post, I’m going to explore five important insights you can learn from performing an SEM competitive analysis.  My hope is that once you read through this post, you’ll be eager to get started on your own analysis.  Let’s get started.

What’s an SEM Competitive Analysis?
Simply put, an SEM competitive analysis enables you to understand the companies also bidding on the same keywords and categories you are targeting in paid search.  Let’s face it, if you are bidding on a set of keywords, it’s important to understand which competitors are targeting the same keywords, where they are driving visitors, how aggressively they are bidding, the pricing they are providing for similar products, etc.  While performing the analysis, there are times you find incredible nuggets of information that can help enhance your own campaigns.  You can also understand why certain competitors might be outperforming your own efforts.

Competitive Analysis Tools
This post isn’t meant to provide a tutorial on how to use the various competitive tools in the industry.  There are many to choose from and you should test them out to determine which ones fit your needs.  Pricing-wise, some are paid solutions while others are offered for free.  For example, SEMRush and SpyFu are two paid solutions that enable you to view a wealth of competitive SEM data such as keywords, ads, cpc’s, volume of traffic, etc.

Competitive Analysis Tools

Google’s Ad Preview Tool is free and enables you to view an unpersonalized SERP, while also enabling you to specify geographic location, mobile vs. desktop, language, etc.  In addition, AdWords recently released Auction Insights, which gives you a view of the companies you are competing with on a keyword level (if there is enough data).  You can view a competitor’s impression share, their avg position, the overlap rate, the percentage of times they rank above your own ads, etc.  Again, there are many tools on the market, and my recommendation is to figure out the right combination for your needs.  Many of the paid solutions have free trials, so you can start using them immediately to gauge their effectiveness.

A screenshot from the Google Ad Preview Tool:

AdWords Ad Preview Tool

Analysis Scope
When determining the scope of your analysis, you can either start small and analyze a specific ad group, or you can analyze a larger campaign (or set of campaigns).  If you are just starting out, you might want to start smaller and just focus on an important ad group.  Once you determine the best process to use, along with the right tools, you can expand to other ad groups and larger campaigns.  I recommend choosing an ad group that’s important to your business, but one that might not be performing very well.  You never know, the competitive analysis could reveal why that is…

Let’s take a look at five things you can learn from an SEM competitive analysis that can greatly help your own SEM efforts:

1. Who Are Your *Real* Competitors (in Search)
Whenever I begin  helping a new client, I always ask them who their top competitors are.  It’s a trick question, since the standard set of competitors in the industry might not be the same competitors in SEM (or SEO).  Understanding which companies are present in the SERPs for target keywords is extremely important.  For consumers that don’t know which company to do business with, and start searching Google, the offline competition might not make a big difference.  That’s why you need to understand your true competitors in SEM.  That’s who prospective customers will be reviewing while researching online.

When I present my findings with regard to true competition, it’s not uncommon for my clients to fall out of their chairs.  Sure, they might find some familiar faces, but they might find some additional companies or websites that surprise them.  For example, say hello to, the biggest and baddest ecommerce retailer on the web.  If you are selling online, Amazon very well could be a core competitor in SEM.  If that’s the case, you better check out pricing on, how often they show up for your target keywords, which third party sellers are providing similar products, etc.  Let’s face it, low pricing and Amazon Prime membership is a killer combination that you’ll have to face and deal with at some point.  And you’re not alone.

You also might find comparison shopping sites, forums, answer-driven sites like Yahoo Answers, personal blogs, etc.  If you do, you might need to form a strategy for monitoring those sites to ensure you are represented (the right ways).  You might find manufacturer websites that provide links to online retailers that offer their products.  Are you listed there?  Should you be?  I think you get the picture.  Understand the real competition, dig deeper, and form a strategy for dealing with those “competitors”.

A list of competitors in AdWords for a target keyword:

Your True Competition in Search

2. Find the Keywords Your Competitors are Running
OK, so now you know which companies you are competing with in SEM.  Your next question might focus on which keywords they are running.  This is important for several reasons.  First, you want to make sure you aren’t missing important keywords or categories that customers are searching for.  Even if you performed keyword research, you might have missed something.  Analyzing keywords your competitors are running could help close the gaps.

Analyzing the keywords a competitor is bidding on:

Competitive Keyword Analysis in SEM

Second, you can start to gauge how much traffic each keyword or keyword category is driving to your competitors’ websites.  For example, if you see a larger percentage of traffic for certain categories, there might a good reason for that.  Maybe they are seeing outstanding performance from those keywords or categories, and they are allocating more budget to those keywords.

Note: there are many companies not managing SEM correctly, so be careful here…  If you see something stand out while analyzing the keywords that competitors are running, you can and should, test those yourself.  As long as you have a strong analytics strategy in place, you can easily identify high quality traffic, strong performance, etc.  I guess what I’m saying is that keyword intelligence is great to attain, but nothing compares to actual testing.

3. Competitor Landing Pages
Next on our list are the landing pages that competitors are using.  Let’s say you were running an ad group for an important category.  You are getting  a lot of traffic, but not many conversions.  You’re baffled why that is…  Well, analyzing the landing pages that competitors are using can tell you a lot.  Are they driving visitors to product detail pages,  campaign landing pages, lead generation pages focused on gaining contact information, mobile landing pages (for mobile traffic), etc?  All of this can help you better understand why your competitors might be outperforming you in SEM.

Understanding the landing page experience for prospective customers can help you form ideas for your own landing pages.  If you are driving visitors to a product detail page and competitors have set up dedicated campaign landing pages with a wealth of information, images, video, reviews, live chat, etc., you might want to refine your efforts.  Don’t pale in comparison to your competition.  It could be the very reason you are seeing less conversion (or no conversion).

A sample SEM landing page:

Landing Page Analysis

4. Ads, Ad Extensions, and PLA’s
Using competitive tools, you can review the text ads that competitors are running.  When prospective customers are facing a SERP filled with paid ads, it’s important to stand out (for the right reasons).  Are your competitors punching sales, deals, special offers, etc?  Are they providing actual pricing in their ads?  Are competitor text ads aligned with the landing pages they are driving visitors to?  All of this can help you understand why your own performance isn’t as strong as it should be.

Viewing competitor text ads:

Analyzing SEM Ads

And let’s not forget about ad extensions and product listing ads.  Are your competitors using sitelinks extensions, product extensions, call extensions, local extensions, social extensions, etc?  The extra information provided by ad extensions can be extremely valuable to prospective customers.  For example, you can drive visitors deeper to certain sections of your site, to specific products pages, show social connections, click to call phone numbers, etc.  And if you’re an ecommerce retailer, don’t overlook the power of seller rating extensions.   Those little stars can bring a level of credibility that can mean the difference between revenue or just a click.

An example of sitelinks extensions in AdWords:

Analyzing Ad Extensions in Paid Search

In addition to what I mentioned above, I have to cover the power of product listing ads.  Recently, Google transitioned Google Shopping to a pure paid model.  Product listing ads are an important part of that model, and are extremely powerful.  They are image-based ads for specific products, based on your merchant center feed.  They are CPC-based and can help drive strong performance for ecommerce retailers.  If your competitors are running PLA’s, and you aren’t, you better get in the game.  There are times text ads just don’t compare to the image-based PLA’s competing for attention in the SERPs.

An example of product listings ads in action:

Analyzing Product Listing Ads

5. Pricing
The final insight I’m going to cover is probably the most important – pricing.  Performing a competitive analysis will reveal the pricing your competition is providing for the same products you are selling.  The power of the internet is a double edged sword for many sellers.  You can now compete with the big boys, but you will also be compared with every other seller on the web.  And this can happen in mere seconds as people research products via Google, Bing, and Yahoo.

I find this step in a competitive analysis provides incredibly important insights for my clients.  They are sometimes floored by what they are seeing.  Actually, it’s not unusual for some clients to start yelling as I’m presenting my findings.  “How are they providing that pricing?”  “That can’t be right.”  “They are lowballing prospective customers!”  I’ve heard every possible comment under the sun.

Regardless, unless a consumer knows and trusts your company, you are going to have a hard time comparing to a competitor selling the same product at 20% lower than your own pricing.  Not every person will go with the lowest price (based on a number of credibility factors), but some will.  And when you are paying for every click, it’s important to keep those visitors on your site with the hope of converting them.

Analyzing competitor pricing:

Analyzing Competitor Pricing


My recommendation is to analyze each of the competitors for a category, and break down the pricing for each.  Try and determine if that’s the real pricing, how they are providing that pricing, understand their shipping costs, etc., and then form a strategy for dealing with the situation.  By the way, that could mean pausing your ad groups for that category.  If your ROI is pitiful, and your competitors are selling at pricing that makes no sense, then pause your ad groups.  You can find other more profitable categories to drive…

Summary – Competitive Data is There. Go Analyze It
Are you ready to get rolling with your own competitive analysis?  As I covered above, there’s a lot you can learn.  It’s important that you don’t get so caught up in your own campaigns that you forget to learn what your competition is running, how much they are spending, where they are driving visitors, and what type of landing pages they are using.  You never know, you might end up finding serious gaps in your own campaigns.  And that can lead to more revenue, profit, and a stronger ROI.  Good luck.