Candour

Blog

Episode 16: All about Google penalties

Mark Williams-Cook
Mark Williams-Cook , SEO Director
01.07.2019 • 23 min read • Podcast

Play this episode:

Or get it on:

What's in this episode?

What's in this episode?

Mark Williams-Cook will be taking a dive into the world of Google penalties!

We'll be talking about how Google handles and processes penalties, what the most common types of penalty are and why it might seem like Google is doing nothing about that pesky cheating competitor!

Show note links:

Report spam paid links or malware: https://support.google.com/webmasters/answer/93713?hl=en

Search Engine Roundtable interview with a former Google employee: https://www.seroundtable.com/why-do-some-google-manual-actions-seem-automated-21899.html

Google Webmaster hangout about recognising blackhat SEOs buying old domains and restoring old content: https://www.youtube.com/watch?v=u7JY_Tz2c4U&t=27m43s

Transcription

Welcome to episode 16 of the Search with Candour podcast recorded on Sunday the 30th of June 2019. My name is Mark Williams-Cook and today I'm going to be talking all about Google penalties. I'm going to cover a few different types of Google penalties, how Google finds and deals with them, and why it might feel like Google's doing nothing about that competitor of yours that's cheating.

So, this episode was inspired by one of your questions, a chap named Niall O’Gribin. He asked this question, he said:

“Here's one for you sir: How does one REALLY get Google to banhammer a site in SERPs which relies 99% on spammy tactics, site-wide paid links etc. .. when reporting them and Google's ‘AI algorithms’ do nothing to solve the issue?”

I thought this was a really good question and we've probably all been in that situation where we've seen a competitor is obviously doing things against Google guidelines but they're still ranking really well. I thought the whole subject of Google penalties was worth exploring and just taking a few steps back to talk about them more generally as it is quite a misunderstood area of SEO.

Types of google penalty I think is a really good place to start, and there's an important difference from the outset between a Google algorithm update and a penalty. So previously we've had the well-publicised, well written about Google algorithm updates that we've understood in terms of what they were targeting. We had the Panda update which was really going after sites with what Google was referring to as ‘thin or low-quality content’, we had the Penguin updates which were focused around link profiles and people with spammy link profiles. Of course, we had the recent, as it was just called, ‘June core update' where we saw that the Daily Mail lose around half its search traffic and it's really important to understand the difference between an algorithm and penalty.

So, if you experienced like the Daily Mail did - this drop in traffic after a major Google algorithm update this is not the same thing as a Google penalty. Philosophically it might be in that, you know, a Google algorithm update is saying “look we've just adjusted how we are measuring against our guidelines” which you need to produce good content. The site needs to have a good user experience, links to etc. so this is why Google put out the advice saying “if you do experience a large drop in ranking after an algorithm update, it's unlikely that there's going to be just one easy basic thing for you to fix.”

What's actually happening is that they've said “Okay, our algorithm now we feel better describes what we're trying to produce in the search results and as it happens we were misclassifying your site - we were giving you way more credit than actually you deserved.” So, all that's happening is other sites that we now believe to be better are taking their rightful place; whereas a penalty is something specific. You will normally get a message through the Google Search Console, the thing that used to be called ‘webmaster tools’ which will explain the type of penalty you've got, and you have a chance to interact with Google.

They're specifically saying, ‘for this reason we are penalising your website’ and there's lots of different types of penalties. They range from your site being suppressed in Google because it's been hacked so to protect users Google's maybe completely indexed part or all of your site, and that ranges all the way across things you've obviously intentionally done: whether it's buying links or just producing kind of spam content, and they're the things I want to cover in this episode.

The penalty process is where I want to start; which is how do penalties happen, and if you haven't seen it Google does actually have a form which is called ‘report spam, paid links or malware’ and I'll put a link to it in the show notes at search.withcandour.co.uk and this is a form where Google's actually asking you to submit things that you think is spam, paid links, or malware. It's quite interesting reading their descriptions as it gives you an idea of how they handle each of these different types of things they don't want in their search results. So, they say:

You'll notice there they specifically don't mention about devising scalable solutions there, and I'll get some more detail on that in a moment. Lastly they have ‘malware’ which says:

I did ask John Mueller if he could clarify this so I asked him:

We've got some other information from Googlers in a moment but my general understanding of this is that for spam, which we will classify later, as essentially how you're handling the content on your site - Google does appear to have a lot of algorithmic ways to deal with this and apply penalties. For paid links everyone I've spoken to, all the information I can get from Google, all the experience I've had from clients coming to us with sites that they need to have penalties removed, from sites I've run that have had penalties for, in sites I've run that should have had penalties and never got them, is that there definitely seems to be this manual human review before a penalty is applied. When I was discussing this with John, Barry Schwartz kindly posted me a link to an interview he had on search engine roundtable where he interviewed a former Google spam fighter Murat Yatagan, there that yet again and he spoke to him specifically about Google's automated and manual review methods for how they deal with spam and this is what he said:

in short, Barry surmises it seems like Google does use automated methods to locate a lot of spam but this is manually reviewed by many Googlers before the manual actions go out, and this manual work is done to analyse the data to ensure that there are very few to zero false positives out there. I think the really two important takeaways from what Murat [from the roundtable interview] is saying to firstly begin with the Googlers who are applying these manual actions. It seems that several people need to review it and after they're more experienced, a single Googler can apply a manual action to a site. He also specifically spoke about trying to push up this data to the Search engineer team for “this is how we're identifying spam to see if they can get a more scalable, algorithmic solution to getting closer to identifying this from a machine perspective.” So rather than having to rely on humans to do it, and that's really important because I think the key thing here is that it's always going to be economically cheaper, time cheaper, cost cheaper for spammers to create spam and bad link profiles than it is for someone to find and verify that it is spam.

So, it's for every 10 minutes you create spam it's going to take someone three times that to track it down, eliminate it and make sure there's as little collateral damage as possible.

Possible types of Google penalty is something I want to cover as well and before I get into this I want to talk about “duplicate content penalties” or more really that duplicate content penalties aren't really a thing, at least in the context I come across them a lot with clients and that's over many years I’ve had lots of people really worried that their site’s going to get a duplicate content penalty either because they've got the same content on multiple URLs on their site or they're worried their site's going to get a penalty because they've copied a single page from a press release site and that's also on their site or a competitor's copied a few pages of their pages. They're worried Google might think they're copying content so they're going to get a penalty and in 99.9/100 this isn't a thing.

What will happen is if Google identifies the same content it's just going to serve one of those pages in the search results. So, if you've got ten pages that are identical on different websites, Google's going to do its best to work out who was the original canonical source for that content and show that in the search results. All it will do is filter out the other ones, on the logic that if the content that they're ranking number one or two or whatever isn't any good. There's no point ranking it multiple times in the same search result page, so there's nothing to worry about in terms of that.

There's a slightly longer discussion if you're particularly being targeted as there are methods that blackhats can use to basically take your content and try and trick Google into thinking that they were the creators of it, but that's by far and away a rare instance in the context that most people are thinking about duplicate content penalties. It's not something you have to worry about. The types of penalties we do see, by far the most common is unnatural links to your site which is a penalty you'll get. That's the name of the penalty you'll get in Google Search Console and this is really because you've broken Google's Webmaster guidelines when it comes to acquiring links. There's been some really interesting discussions about this recently so I'll link to a Google Webmaster hangout and the specific time in that hangout where John Mueller is explaining how Google is recognising patterns of blackhat SEO including what I think is slightly more advanced PBN, which is a private blog network kind of activity, where you've got people buying expired domains and then looking on things like the Wayback Machine, archive.org to see what the old content on that domain was. They'll restore that old content and then start inserting links back to themselves on a large scale, and John was talking about how Google is working to recognise that and where they see it happening on this big scale to apply penalties to that.

Now everyone knows links do help in terms of SEO so it's quite a lucrative area and buying and selling links is still not uncommon. The difficulty if you see your competitor doing this is that this is one of those things Google has to identify, really, who's doing this because anyone could buy links to your website and certainly I've seen people try and do what they would call ‘negative SEO attacks’, which is where they buy a whole load of really low quality/hacked or spam links and point them at a competitor's site and then they'll report them. Obviously, Google needs to be very careful that they don't create a situation whereby it's economically more viable to put resources in to disrupting your competitor’s rankings rather than improving your own site and improving your own rankings.

So, ‘unnatural links to your site’ is one of the more common penalties just because links are sometimes the hardest thing to get, the hardest thing to do in terms of SEO but it's also one that can be difficult to try and get Google to action on your behalf if you see someone doing it.

‘Pure spam’ is another interesting penalty and unlike the other types of penalty you can get, this one is normally fastidiously enacted and quickly enacted because the webmaster can't plead ignorance or say it was someone else because this is content on their site. So, pure spam is when the webmasters are doing things like just generating content for instance. So they're just using algorithms to automatically generate the content, or they're just scraping content or maybe they're showing some okay content to the user but cloaking, which means showing different content to the search engines to try and get rankings so all of these things are really intentional.

So this is why, as Murat discussed earlier, Google can find ways which are more reliable to algorithmically detect what’s going on here because there's not this big range of uncertainty and if you do see competitors doing techniques like this, it's a lot easier to get Google to take action through filling out that form.

‘Thin content with little or no added value’ - this is a particularly interesting one because this isn't about spam. This is Google saying low quality or shallow pages will trigger this penalty, so they give examples like low-quality content such as guest posts - so if you're just farming out lots of content opportunities to a site to numb experts, just to try and target key phrases. Doorway pages which might be if you're running an SEO agency in your base in London but you create another hundred pages that say “SEO agency in Norwich”, “SEO agency in Leicester”, “SEO agency in Leeds” etc. you're not actually in those locations you're just trying to create a low-quality page to catch those searches.

The other thing which I think is really interesting, it hit affiliates quite hard when we had the Google Panda update - it includes things like OEM content (original equipment manufacturer content). So if you're reselling or you were an affiliate of someone, you're trying to sell their products it wasn't uncommon for them to provide you with a feed of their products which would say “here's our product title, here's a description, here's the technical specification, here's the reviews” and you could take that feed and populate your site with it. But the logic then was “well hang on, if this content already exists on the merchant site what value are you adding as an affiliate if you're essentially just re-publishing this content? The only reason you're re-publishing this is to try and get your commission, you're not actually adding any extra value over and above the original merchant.” So, there's some examples and which is why I think this particular penalty is interesting whereby perhaps the webmaster wasn't intentionally trying to dupe Google. They were just trying to do things in an efficient way, not particularly thinking so much about the end-user and what value they're adding.

I think those are the three most common types of penalties I've seen. There are some other less severe penalties: I've seen things like schema abuse, tricking triggering penalties which is where you've marked up a webpage - sometimes by accident and you've added schema that describes the information that isn't on the page or it's different to the page and rather than just kind of getting your site de-indexed or a massive chunk of it banned what's happened there is Google just disables and removes your ability to show rich results within the search pages until you fix that. But those are the three main types of penalty that you'll see.

How do you go about getting back after you've had a google penalty?

This is a really interesting question because it really depends, I hate to say that because that's the answer to so many SEO questions and I think something really interesting to think about is that remember Google's focusing on trying to give the user the best possible experience. This by definition means that a lot of really big brands have a get-out-of-jail-free card when it comes to Google penalties because if you’re in the UK and you're searching for cheap flights somewhere there are brands that you expect to be in those search results. You're going to expect to see someone like Ryanair, easyJet in those search results so if a company like that did some blackhat SEO or they bought loads of links and they got caught what tends to happen (and what we've seen happen historically) is that Google will apply them a penalty and it might be for days or a week or so. Obviously for these huge brands driving a huge volume this will be very painful for them and it will cost a lot of money, but they will go back in and they will go back in fairly quickly - they'll be at the front of the queue. That’s because those users expect those brands to be there and the search result is just not the same quality if they're not there.

This unfortunately isn't true for most companies and most SMEs, for most websites, you're all fighting for some space in the search result and if you do get banned people in general probably aren't going to be that bothered which means you tend to be near the back of the queue. If you do get one of these penalties as I said you'll get a notification/message through Google Search Console, which if you don't have you absolutely should have. It's your diagnostic free platform to get information back from Google about how your site is ranking and any kind of diagnostic issues, problems and certainly messages when you get manual actions. You'll get a notification saying you've had a manual action on your site, it will describe and give some examples of the type of thing you're doing wrong and you have to go through what's called a ‘reconsideration request’ which is basically you have to clean up the mess you've made to the best of your ability and send a very apologetic note back to Google explaining what happened, what you've done to clear up, why it's not going to happen again.

So for instance if you were caught buying links you would have to try and manually contact those sites to remove the links, or failing that if you can't do it you would need to use the Google disavow tool to remove them and try and include as much proof of this as possible and send it back to the re-inclusion team who will then review it within anywhere between one to several weeks. Obviously, that can be a very damaging amount of time not to rank for people. If you were ranking quite well previously and then obviously, hopefully you'll get re-included. They're keeping a record of this so if you're getting three penalties in a year or something there's a good chance, they're just not going to let you back in at all. So, swinging all the way back round to Niall’s original question, why is it basically that he can see competitors are doing stuff that he knows is blackhat - so they're buying links whatever and he can't seem to get Google to take action?

We've got to take in context what type of thing we're seeing people do here so some of you very kindly sent me some very clear examples because I asked around if you've had success with filling out the Google spam report form and seeing competitors being penalised for this. A few of you kindly sent me some quite clear examples where you'd reported competitors for buying links on scale and within a couple of weeks they had been penalised.

I'm not going to share those examples because nobody likes snitches and I haven't asked if I can share their names, so I won't. It's worth filling out if there's something you want to do. I do think it's worth as the kind of first thing just to fill out that form - to provide evidence that you can, that something's happening and there's a few things here I think it's just worth bearing in mind.

Firstly, there are certain types of things that Google would prefer to tackle algorithmically rather than on a one by one basis, so this might be one of the reasons you're finding that your reports aren't being actioned. It's that they're being received and Google saying “We've got 10/20/50/100/1000 sites that matches this kind of profile, so we can't just keep manually trying to find them all and review them all. We're going to try and strengthen our algorithmic method to identify these sites” so it may be that they're using this data just to build a more robust algorithm.

The second thing is from my experience is that things tend to get action more quickly where it's a clear case the blackhat SEO is “harming” the end-user. Take for instance the pure spam example. This result is terrible for users and it makes Google look bad, it's likely that action will be taken against blackhat SEO like this because if someone does a Google search and they just end up with gibberish automated content that's very unhelpful for them, Google will see that and it's a clear-cut thing that they can penalise. If we skip back to when we talked about reporting things like paid links that Niall said, it's quite common but it's very likely to be a much lower priority for Google for a couple of reasons. Firstly, even with paid links the content may be good, it may be the best content out there for users so Google's not going to be keen to remove it, it may be better for Google just to try and ignore links that they believe are manipulative so building a better, more robust algorithm to look at link wrath and rather than penalise sites just to ignore and not pass benefit for links they think are spammy. The other reason that's an advantageous strategy is that it's difficult for Google, even with a manual review to prove that those links are an intentional product of the webmaster. It's not unheard of as we've said for webmasters to try and damage other site’s rankings by buying them low-quality links in them and reporting them. So, Google needs to be really careful about how it processes these penalties otherwise they can make it too easy to do negative SEO and it becomes economically viable to focus resources on damaging competitor sites rather than improving your own. There's a whole mix of reasons if you report paid links to Google as to why there might not be an action quickly or at all.

Now Niall did specifically say he didn't want me to give the answer of just spamming John Mueller at Google when you find a competitor doing something like this, but the from the cases we've seen - the high profile cases, the ones that get talked about publicly are the ones that leave egg on Google's face - they do tend to get actioned. From talking to other SEO’s and about their experience when they've seen competitors do really obvious things, it does help if you do have someone at Google or you use one of their contacts that's publicly available just to say “Hey can you have a look at this? What do you think about this?” not necessarily even going to look at this this is terrible just asking their opinion and you know you can start a discussion around how Google might look at tackling that so there are a couple of different avenues you can take. It's certainly not guaranteed that you will ever be able to get Google to as it now puts it, “use the banhammer” on sites that you think aren't playing by the rules, unfortunately.

That's all I've got time for today it's already been over half an hour, so I'll be back on Monday the 8th of July and of course you can get all of the links to the stuff I've talked about today at the site on search.withcandour.co.uk so you'll find a full transcription of this podcast and links to everything I've spoken about. Until then hope you have a great week and I hope you found it interesting learning a bit more about Google penalties.

More from the blog