Almost half of GSC clicks go to hidden keywords and SERP annotations from unstructured data
Or get it on:
Show notes and links
In this week's episode, Jack Chambers-Ward is once again joined by his inimitable co-host, Mark Williams-Cook, to discuss all the latest SEO & PPC news. This week, Jack & Mark discuss:
- A study by Ahrefs revealed that almost half of GSC clicks go to hidden keywords
- New 'pros and cons' SERP annotations appear from unstructured data
- Reason number 5,445,131 why some SEOs destroy the reputation of SEO
Transcript
Jack: Welcome to episode 26 of season two of The Search with Candour podcast. My name is Jack Chambers-Ward. This week, I am once again joined by my co-host, Mr. Mark Williams-Cook. This week, we'll be talking about some SERP annotations from unstructured data, keywords being hidden for 46% of Google search console clicks and how someone managed to write a fake press release and gained a bunch of authoritative links from government websites in Italy. The whole case study, we'll get into it in a bit. Search with Candour is supported by SISTRIX, the SEO's toolbox. Go to SISTRIX.com/SWC, if you want to check out some of their fantastic free tools, such as their SERP snippet generator, hreflang validator, be able to check out your site's visibility index, and the Google update tracker.
Mark: Before we actually get into the show and we are talking about SISTRIX, there is one feature that we don't normally mention in our intro that I want to make listeners aware of, which is the compare SERPs feature. And I haven't seen any other SEO SaaS tool do this as well as SISTRIX. So I was using it this week on some proposals I was doing, and it is just such a brilliant visualisation. So if you've got SISTRIX or you've got access to SISTRIX, if you go into the keyword section, so obviously you're triggering this by just searching for a keyword you want to rank for. In the left hand side there is a sub menu you called trend and right at the bottom there is a link for compare SERPs. And what this will do is for that key phrase, you can select two date ranges. So for instance, today and one that might be six months, 12 months ago, and it will actually show you the top 20 ranking URLs from your first date on the left hand side of the screen.
And on the right hand, you can probably guess it will show you the second date. And the cool thing it does between the two is any that are present on the older SERP, it will literally draw an arrow, a colour coded arrow, so green if it's gone up, red if it's gone down, gray if it stayed the same, for how that SERP has changed. So as part of this pitch I was doing, they'd actually been through a migration and guess what? So many people we had talked about, they had lost some rankings. And as far as they were concerned, they had actually done quite a good job of the migration. But this was a really good way to look at the different key phrases where they had lost significant traffic, because they had lost rankings and actually seeing, well, who is ranking now? Is there any trend in the type of sites that moved up? Is it a new entrance? It was such a brilliant way to visualise it.
Jack: I think visualisation is the key there as well. Because you can go into impressions data or get a snapshot of something from any tool, millions of different SEO multi tools. But the fact that SISTRIX does this with a compare SERPs, you can present this to someone who doesn't really understand SEO. Like you said, if you're pitching to a brand new client or explaining it to someone who's higher up in your company and you're in house and they don't necessarily know the more technical side of SEO, you can say green is good, red is bad. And really show them arrows go up, arrows go down. This is how much it's changed.
You can get into discussions of volatility and how things are changing, whether competitors are shifting around, if there's any trends with competitors you see. You check multiple keywords and you see one competitor dropping off, or one competitor really climbing up. You can check out what they're doing and it gives you links there. It can send you off in so many different directions. I think it does a brilliant job of getting that really easy to understand clear visualisation for this data.
Mark: Yeah. We talked before about the SISTRIX tool that shows your percentage visibility loss in relation to the Google core updates that they know about. And this is a really nice tool as well, to use to set on a specific date before a core update, few weeks before, and then a few weeks after to see again, it's this type of site that's been moving up. So if you haven't tried it out yet, SISTRIX's keyword section, compare SERPs in the bottom left.
Jack: And funny, as you're mentioning SERPs Mark, because SISTRIX have just updated their US SERP database and are now covering over 50 million keywords all the way to position 100. Full archive for the US market. So if you're working with US clients, you're working on sites in the US and looking to rank there, your capabilities within your database have expanded dramatically recently, thanks to the work of people at SISTRIX.
Mark: That was actually very timely as I had a question on LinkedIn the other day, because I showed something on SISTRIX and someone said, "How EU centric is it versus US?"
Jack: There you go. That's your answer. The US SERP database has just been updated. So if you are looking to work on US sites and use SISTRIX, I highly recommend checking it out because the database has never been bigger, never been better.
Jack: So, kind of spinning off of SERPs, let's talk about some new SERP features, as we often do on this show, because Google is always doing something new and trying something weird. And we're actually seeing something, I can't remember if I've seen this before specifically, is the pros and cons included in the SERPs themselves underneath the article from unstructured data, which melts my brain a little bit.
Mark: I love you're getting to that stage now in SEO where you're like, have I seen this before? I can't even tell anymore.
Jack: Hat tip of course to Dr. Marie Haynes, who we shout out regularly on this show. If you're not following her on Twitter I highly recommend that. She is fantastic for catching all this stuff. And it's pulling from internal lists not structured data that you're trying to get featured as a snippet. It's pulling from the on page content of these articles and pulling it through to the SERPs. How, why, what. Apart from the whole conversation, which again I had with Cindy Krum recently as well, and I had conversation with Katherine Watier Ong when I had her on the show a few weeks ago, Google wants you to stay on the SERP. This is more of this stuff. They're giving the user more content on the SERP rather than letting them click through to the article that us poor SEOs and content writers have spent our hard earned money and blood, sweat, and tears crafting. No, you get that for free on this SERP. No clicks necessary. Great. Thanks. Thanks Google.
Mark: I think it's really interesting. I don't find it too much of a surprise for a couple of reasons, which is firstly, relative to search engines, the widespread use of schema labelled structured data is fairly new. So for the longest time, all search engines basically have been trying to process mostly unstructured data. Just the jumble of stuff that people put on the web and they've got various heuristics that work out, this is this and this is probably what we're talking about. So it doesn't surprise me from that point of view. And even actually in parallel with Google's other products, I believe we've talked about it. We definitely talked about it in the studio, maybe on the show, how Google Docs can do a summary of the contents of the document. Which is a similar thing, I think, to what's happening here, which is that it's summarising here the pros and cons, this review.
Jack: And I think, again, we're tying into all the reviews that are so important. We talked about the review update not too long ago, we've talked about EAT multiple times. This ties into this pros and cons authoritativeness. Are they an expert? The person talking about this product. There's beauty products as an example here, Dr. Marie's screenshots. So it's like, that's the thing you want pros and cons for? This works on dry skin, but this works on oily skin, and this is scented, and this is unscented and whatever. That's the product you're going to be wanting pros and cons and comparisons for and stuff like that. So I think it totally makes sense as we're getting into more authoritative, more expertise, more trustworthiness, all that stuff. I saw a couple of people talking around this topic as well on Twitter.
Again, anecdotal evidence here but I saw a couple of people discussing it, comparing very similar product review pages, product pages that have reviews on them. One of them was written in first person, the other three they were comparing or written in third person. And the first person one was consistently outranking the other three. As if there's that inherent trustworthiness of, I have tried this thing. I am the person who knows this product because I am writing about it in first person, rather than one of our experts blah, blah, blah. That doesn't necessarily mean anything, there's no way of proving that. When you have the author name below, they are writing in first person and they say I have tested this product or whatever, there's not only that level of like to the search engine, but to the user as well. I think it's that "put your money where your mouth is".
I've done that previously for sponsors on my other podcast where we always say we don't endorse things we don't want to use. Same is true here on Search with Candour. We use SISTRIX, we endorse SISTRIX, we work with SISTRIX. As you were saying Mark, you literally used it to compare SERPs features the other day for a proposal. And I think there's that level of trustworthiness you have to your readers, to your listeners, to your viewers, whatever your content is to then say, I am endorsing this thing or I'm reviewing this thing honestly, because I have used it, I have tested it. Whatever the product is.
Mark: I believe if my memory serves me correctly, that pros and cons are one of the things specifically mentioned in the guidance from Google around the product review update of something that's important to-
Jack: Correct. We discussed this because, again, it ties into that almost transparency and trustworthiness of including your cons in there as well. Even if you're writing about your own product, don't just say it's the best product ever. It's the best skin cream or whatever. You have to say that level of knowledge about the thing, the fact that you can analyse it and see there are pros and cons. And then also the transparency there saying, yes, we understand there are cons to this, or there are better alternatives or whatever, builds that trustworthiness with your audience as well.
Mark: So I have one theory about this, which is similar to how Google is letting organic free results creep in with shopping feeds to increase their inventory to most likely be on a level footing at least with Amazon, I've seen a lot of conversations really, and I think you've seen them as well, of especially Gen Z moving to TikTok to search for stuff. And one of the things that came up regularly for TikTok, apart from things like how-to guides, was product reviews. Is this any good 15 second video? So this SERP then just gives you what's it good for, what's it not good for in a few seconds. Which again, that's a theme that's come up on the podcast and in SEO in general, which is generally the path of least resistance will win in the long term whenever you're talking about user experience, usability of what's the energy cost for me to find this information and my time and effort.
Jack: Absolutely. I'm planning to have a guest on in the not too distant future to discuss TikTok as a search engine, how it differs from Google, how more people are searching and all that stuff. So stay tuned for that listen if you do want to learn more about TikTok and how it's being used as a search engine now, and now that it's taking over Google in many ways for many, as you said, gen Z, Mark.
Mark: Mark shakes head slowly.
Jack: Jack and Mark feel old.
Mark: Keywords hidden for 46% of Google search console clicks.
Jack: I love that headline.
Mark: That is the headline.
Jack: Credit to the writer. We'll get into this in a second, but credit to the writer of this article for that. That attention grabbing headline of you're losing 46% of your keywords. It feels a bit clickbaity, but it is legit and this is a really interesting study, I think.
Mark: So this is a study done by Patrick Stox, who's very well known and respected in the SEO community and is currently product advisor, technical SEO and brand ambassador, that's a lot of hats for Ahrefs. So it's an Ahrefs study and this study includes one month of data across 146,741 websites and nearly 9 billion total clicks.
Jack: Sorry, billion with a B?
Mark: Billion with a B. So more data than probably most people have in their-
Jack: Advantage of working at Ahrefs and having access to that data, right?
Mark: Exactly. We will of course link to the study and show notes, search.withcandour.co.uk. One thing I found particularly interesting about this study, and they've done a nice job visualising this, was they were looking at the bands of percentages of missing clicks by site. So when we say that 46% of these keywords are hidden, that's the median of what is pretty much, except for a few outliers, the bell curve of how much data is hidden. So they did find some sites have got pretty much all of their keywords and other sites have pretty much none. We're going up to the high 98% or something. So there's sites with 98% of their keyword data missing. And then there are sites on the other side of the coin with 98, 99% available.
Jack: I think this is particularly interesting because it's coming from Google search console. And I think so many of us in SEO, we use tools all the time and everybody says, I don't use so many tools and all this stuff. We've talked about it on the show before. Tools are a way of automating the things that will take you much more time. That's usually how you should be looking at most of the tools that we use. You could go through two different SERPs if you had taken a screenshot of something from six months ago and compare the top 20 results, but SISTRIX just does it for you in like 10 seconds.
So why would you do that? So many of these tools. AlsoAsked is another perfect example. You could go through and take notes and write down all of the different PAA data, or you could chuck into AlsoAsked and find the data straight away in seconds. I think this is a perfect example because often you look at Google search constantly, this is the data source. This is authoritative. Google Analytics as we know, we talked about many times on the show, questionable.
Mark: Not an exact science.
Jack: Not an exact science, exactly. But often I think we treat the search console as, this is the source of information from Google. This is not going through anyone, somebody screwed up the code on the website or anything. This is the raw data, this is the factual stuff. Nope, you're missing more than half your keywords maybe.
Mark: Yeah. It's interesting. Because we have that discussion with clients about first party, third party data. And the search console is in some ways a good source of truth about, well, how many clicks did we get from Google? That can be way off when you compare it to third party tools. But yes, then actually third party tools can be better when it comes to finding various opportunities and seeing that you were actually ranking for things before. Because a bit like Google Ads, Google is of course hiding some of this search data.
Jack: Yep.
Mark: What is their excuse for this?
Jack: The exact word is Google gives a few reasons for the discrepancy. And this is the quote from Google that Patrick included in his study. "To protect user privacy, the performance report doesn't show all data. For example, we might not track some queries that are made a very small number of times or those that contain personal or sensitive information." I am sceptical. I know, and again, we've touched on this a few times already, the move to Google Analytics is inevitably coming next year. It's very much driven by privacy and GDPR issues that Google is experiencing. We know Google really needs to sort their privacy stuff out, but come on.
Mark: It's a lot of clicks to lose.
Jack: 46%, and as you said Mark, that's the most common one. Some sites are losing 90 something percent of their clicks or at least the keyword data for their clicks. So it's, how do you justify? Unless it's an incredibly sensitive, informational, personal website thing, you could lose it there, I guess, and that goes through their privacy filters.
Mark: That's actually something I've got a note to ask Patrick about if there's any categorization.
Jack: Of the sites that they covered in the study?
Mark: Yeah. Because I'd be really interested if it's, well sites that cover, say, medical stuff that when people are Googling incredibly specific medical ailments, are they up in that where we've got the high everything's hidden. But then the popular D&D website, how many personally identifiable private queries are going to go through that? I'd say very low, but maybe the algorithms are misunderstanding them because it's talking about height and weight and stuff of a fictional character. And the algorithm's getting confused. But again, I'm very, very sceptical about that level of privacy being invaded.
Jack: An absolute credit to Patrick for this study, I think it's fascinating data to dive into. And as you said Mark, something that we mortals who work at SEO often don't get a chance to look at billions and billions of clicks and go through that and look at thousands of sites at once. So absolutely recommend you go and check out the study in more detail. Like we said, links in the show notes at search.withcandour.co.uk, as always.
Mark: As well, in that study there is a data studio template provided by Patrick. So you can hook up to your own Google Data Studio data and see what percentage of keywords you're missing as well. And he has actually asked if people are willing to just send those raw numbers in for him so he can anonymously aggregate them and just make sure everything's matching up to his data. But certainly he's made it very easy to do and it would be interesting looking across, especially if you deal with multiple websites again, if there's any pattern as to which ones are having a lot of data restriction.
Jack: So we were talking about trustworthiness and authoritativeness earlier on. Time to delve into the other end of that, I suppose, and talk about a manipulation of some authoritative things in a very untrustworthy kind way.
Mark: Yeah. So I found this really interesting. So this was a write up by Gianluca Fiorelli. I've pronounced that correctly, apologies if I have not. It's actually he's told the story, documented the work of Piersante, I would like to say. Again, my deepest apologies. I did Google how to pronounce your name properly, but I couldn't find anything that wasn't a robot doing a terrible job of it. So hopefully I've done better than that. The title of this article was, Reason Number 5,445,131 Why Some SEOs Destroy The Reputation of SEO basically. A basic title.
Jack: It is.
Mark: And I discovered this through the TLDR Newsletter. Again, highly recommend subscribing if you are not already.
Jack: Shout out to Saijo. Amazing. One of my favourite newsletters, has been for years. Fantastic work that he does.
Mark: And he summarised this article as a fake press release that generated high authority links by deceiving government websites, which is pretty much a very accurate description of what's happened here. So again, we'll link to the write up because it is an entertaining, quite long story, but I just wanted to give you the highlights from this. So this started when they found a blank link, and by blank link I mean it's got the link tag but there's no anchor text there, so this is just existing in the source code. On the French Senate website, and this blank link was linking to an Italian tariff comparison site. So energy tariffs and you can still find a link apparently in the pages HTML code. And obviously there are some reasons why that could happen, especially with "what you see is what you get" editors. Sometimes when you delete things, you get artefacts of code remaining where it hasn't properly polished up and cleaned up after itself.
Jack: Especially common in things like WordPress and stuff like that. Happens all the time.
Mark: But obviously the suspicious things about this were, it was the only blank link on the page. And it was obviously to a commercial website that had nothing to do with the French Senate. So it was, ah. And the story goes on about how this piqued their curiosity and they did, again using the tools we've been talking about in the episode, a bit of investigation work and they found links in the end on around about a thousand government owned websites. And around 10% of those were Italian and the number was growing quite steadily. They were finding that about 1 in 10 of these sites was removing the links as well. And when they looked into it, a lot of these were very small, tiny, little, very local government websites that sometimes have been inactive for years.
In other cases, they had text published and they had links to a company called Selectra, which is an SEO company. So they had these pages with links to these commercial sites and at the bottom they were signed by the mayor with an electronic signature. So the article goes into this big, how on earth are they doing this? Because from NS, from a very basic SEO point of view when we're talking about trust and things like this, you would generally think that links from government websites, I don't want to get into it too much, but there's been lots of discussions as well about particular top level domains, like.gov for instance in the UK, you can't purchase these. They're dished out. So there's some thought about, links from them must be worth more. Other people just surmise, well, because of the type of website they're just worth more because they get loads of good links themselves. But either way, I don't think there's an SEO in the land that would turn down-
Jack: A .gov link.
Mark: Links from government websites, whatever niche they're in. It's a good thing. So they were like, how on earth is someone at scale just getting these links constantly? And they got lucky and found one footprint basically on a website that started off. So the text that linked to the site. They said, "Good morning. This is to communicate ARERA, which is A-R-E-R-A, which is the Authority for Energy, Networks and Environment with resolution, blah, blah, blah, blah, has extended the extension of the terms for renewal." So basically it was a press release that looked like it came from the official authority for energy networks and environment. So you can imagine obviously these sites, that's exactly the thing that they would publish and why-
Jack: I assume they get these things all the time. And it ends there with, "We therefore ask you to publish the following information." Again, knowing people who have worked for the government, you get this stuff that happens all the time. You were told to just, this has come in, this is the new regulation. Just put it on the website, put it in the magazine, whatever it is, there you go. Just press send and don't ask questions basically.
Mark: Exactly. So once they found this footprint, it was easier and they uncovered hundreds of sites that were doing this. And some of them as well in the source code, you could see they'd copied the email styling and so it'd obviously just been copy and pasted directly from the email client into the website editor or how I imagine they're all on different platforms. So it looked like this is what was happening. And as you can imagine, the site that was on the other end of this, in terms of receiving links, just exploded in terms of visibility. Especially during the June 2021 and May 2022 core update, you can see very large jumps of 50, well actually 100% increase in visibility and then another around about 30, 40% increase in visibility around those core updates. So unsurprisingly it worked.
Jack: Yeah. And this is, as we've talked about a few times already, this is Google rewarding dodgy stuff, for lack of a better phrase, Google rewarding black hat practice because for whatever reason, they're not picking up on this stuff. And I know Gianluca touches on this in the article as well. If they could work this out as SEOs, how is Google not picking up on this stuff and discrediting those links or whatever, however you want to put it? It's really interesting that they're rewarded from core updates for doing really black hat dodgy stuff. They're like, okay, why do I bother doing all this white hat SEO if I could just sneak links into government websites and get away with it.
Mark: So I have to disagree with Gianluca on that, because to me that is the definition of how to break the rules well. And that's what you're trying to do. Every time you-
Jack: Pretending to be a governmental body, I think, that's illegal Mark.
Mark: Legality aside, but the purpose of doing so is effective... If you were, like, I don't care what Google's rules are. I just want to do effective SEO. The way to do that would be to make the footprint of that almost indistinguishable from this is legit, good.
Jack: Yeah. They've done a good job of...
Mark: I don't see how an algorithm could realistically tell the difference to not give a false positive.
Jack: Yeah. You would then get legit links that would then get discredited and you'd get those SEOs and those companies saying, hey, I got the link. What happened here?
Mark: Again, some people talk about page rank and trust differently. There's that very old Yahoo paper on trust, rank and seed sites, which apparently isn't how Google works anyway. But regardless, talking about if these sites are so valuable and you are getting false positives, that will have a big ripple effect throughout the link graph if they get false positives. So for once I'm not going to be over-critical of Google there. I think that's a very, very difficult thing to algorithmically detect and that's the thing. As Gianluca says, he doesn't normally write about, as he says, black hat SEO, because he's got no love lost for Google, I think, and everyone's just doing what they're doing. I think as well, I will read it out. So the head of SEO at Selectra Italia did respond to this being published. So I will just read out their response because it's fair to show the other side. And this is from a couple of LinkedIn comments. So Marino from Selectra says, "Hello, it's almost flattering..."
Jack: Almost.
Mark: Almost. "To be at the centre of attention of such an important SEO community. However, let's look at some facts. One, we were totally unaware about the empty link on the French Senate website until now, if you look at," and then he is given a web archive .org link to the French Senate site, "You'll see that it's always been present on the page and empty." It doesn't look like we contacted anyone to suggest a change. The only explanation we find here is that it could be a legitimate link left empty by mistake." And then this is two. "As any SEO market player, we reach out to different websites, public and private, offering them content."
Jack: Sure.
Mark: "In addition to gaining attention and links for us, this activity also allows many partner websites to provide updated and useful information to their visitors. Some of the information we were offering was related to market updates and official decisions made by the authorities, which in some cases has created some misunderstanding."
Jack: Yes. It's their fault for misunderstanding.
Mark: "Although no representative of Selectra has ever impersonated ARERA or other authorities and has never sent any emails from any other domain than Selectra.net, some communications sent were ambiguous as it was rightly pointed out. When alerting on this topic earlier this year, we investigated internally and set up the right process to control that all communications are matching our transparency standards. Finally three, we kindly ask you not to judge our entire work on SEO from that perspective, the fact that we have different websites in Italy does not make our business illegitimate. Each of them has truly useful content created by a wonderful team. And their job is the number one reason for ranking high on Google and attracting readers and other content editors placing links towards our websites. Hope this helps we're open for discussion. I'll be happy to get in touch with you to clarify any doubts any of you guys might have left." I just thought it was fair to read out their response. Again, we will link in the show notes so you can read the full write up. Gianluca put a link to the Twitter and LinkedIn responses if you want to read through them as well so you can make your own mind up about whether this was intentional or a misunderstanding.
Jack: Well, that's all we have time for this week. Thank you for joining me, Mark. It's been lovely to talk about some SEO and PPT news. I'll be back next week, perhaps with Mark, perhaps with a guest, we'll see what the schedule's doing. I've got a few guests lined up in the next coming weeks. Like I said, I'll be talking about some TikTok stuff for the first time, which is intimidating to me as a man in his thirties. I'll be talking about what SEOs should be learning about PBC to help them in their SEO processes. And a couple other things lined up further off down the line. So please do stay subscribed and stay tuned for those coming up. But in the meantime, have a lovely week and thank you for listening.