Episode 69: Blogspot.in, indexing low quality pages and the rich results tool
In this episode, you will hear Mark Williams-Cook talking about the Google...
Episode 70: Tab surprises, Google CTRs and All in One security flaws
Play this episode:
Or get it on:
What's in this episode?
In this episode, you will hear Mark Williams-Cook talking about the All in One SEO plugin: A major security flaw affecting 2M users of the Wordpress plugin, Google CTRs: A look at the Sistrix CTR study which shows why global CTRs and keyword volumes are no longer useful for forecasting and tabbed content: a surprising case study by Searchpilot showing Google's advice may not be optimal.
Show notes
https://www.sistrix.com/blog/why-almost-everything-you-knew-about-google-ctr-is-no-longer-valid/
Transcription
MC: Welcome to episode 70 of the Search with Candour podcast. Recorded on Friday the 17th of July 2020. My name is Mark Williams-Cook and today I'm going to be talking to you about vulnerabilities in the WordPress all-in-one SEO pack. We're going to be talking about the SISTRIX click-through rate study and what this potentially means for SEO and forecasting, and we'll look at an SEO split test by Search Pilot that has looked at results of bringing content out of tabs.
So we're kicking off with probably the most urgent, maybe not the most important, but certainly the most urgent of this news, which is that there is a medium severity in that it's maybe a little bit unlikely to happen but if it does happen it's very very bad. Vulnerability in the WordPress plugin all in one SEO pack, and this was discovered by WordFence - if you haven't heard of it, WordFence is a security plugin for WordPress, highly recommend it, they're really geared up. There's a lot of stuff the plugin does to harden your installation of WordPress and as you'll hear in their blog post, they are very proactive in that if threats are discovered, vulnerabilities are discovered in plug-ins, they can actually, in a lot of cases, offer some level of protection through their plug-in, in the interim between when these plugins are actually patched and those vulnerabilities are fixed by the plugin developers.
So I think it's worth mentioning and there's loads of stats that are always talked about around WordPress and security and is WordPress secure as a platform or not and we see all these terrifying stats about what percentage of hacks are done on WordPress. And I think it's important to say that while WordPress powers, I believe it's around about a third of all sites on the web, the core WordPress itself is pretty secure, of course if it's set up correctly. The issues generally that I see with WordPress sites being hacked and why you see so many of them, is a combination of the fact that a lot of websites rely on the functionality of third-party plugins for their website, and of course these third-party plugins are made by people not necessarily associated with wordpress. You know, anyone can make and publish a plugin for WordPress, and it's quite common that even with the more popular ones, these aren't written in a completely foolproof way and eventually security holes and vulnerabilities get found, which is true pretty much of all software. The issue normally comes when the website, administrators, the webmaster, whoever it is that's in control of the website is not keeping an eye on the updates.
So a lot of attacks that happen against WordPress sites happen on an automated basis because these publications happen and these security vulnerabilities are put into the public domain once they're patched. So for instance, this announcement by wordfence and we'll link to the blog post at search.withcandour.co.uk, so you can read it for yourself in full, but it goes into quite a lot of detail about how this exploit works. Now when this happens with other plugins what it means is that there are millions and millions and millions of installations of WordPress, with specific plugins. So people that wish to do so can write scripts themselves that try to automatically exploit these vulnerabilities. So a lot of the time when we see sites that are hacked, it's not because they have been individually targeted, it's because they've just been one of many that's been picked up with a script that's attempted to execute whatever it is, cross-site scripting, or exploit this vulnerability, and they've had success and then managed to do whatever that vulnerability allows them to do. So it's not that WordPress is inherently insecure, it's normally that it's a victim of its own popularity and its own success; in that, if a vulnerability is published there's a large pool of potential victims who may not be keeping their site up to date. So this one's particularly important as WordFence have written on their blog post, this affects potentially two million users, which is what they estimate as the installations for the all-in-one SEO pack. They said on July 10th, “Our threat intelligence team discovered a vulnerability in the all-in-one SEO pack, a WordPress plugin installed on over 2 million sites. This flaw allowed authenticated users with contributor level access or above the ability to inject malicious scripts that would be executed if a victim accessed the wp admin panel's all posts page. We reached out to the plugins team the same day of discovery on July 10th, and a patch was released a few days later on July 15th. This is considered a medium severity security issue that as with all XSS vulnerabilities - that's cross-site scripting - can result in complete site takeover and other severe consequences. So they strongly recommend immediately updating to the latest version of this plugin, which is at the time of writing, version 3.6.2 of the all-in-one SEO pack. And they mentioned there, as I said earlier, that if you are a WordPress/WordFence premium customer they automatically pushed out a new firewall rule on the same day that they found the exploit on the 10th, which would protect against this.
They go into some detail about how this exploit works and it's essentially down to the SEO meta data for posts, such as the title and descriptions, had no input sanitisation. This meant lower level users like contributors and authors had the ability to inject HTML and then Javascript into those fields, and therefore it would be executed when you view all pages. So ideally those page titles and descriptions aren't somewhere Google's going to be executing or browsers are going to be executing Javascript, so that's essentially what needed to happen and that has now been patched. So if you are running the all in one SEO pack on your WordPress site make sure you update it as soon as possible.
On this Tuesday SISTRIX posted a really nice study about Google CTR's - click-through rates. It was on the 14th of July, it's titled ‘why almost everything you knew about google ctr is no longer valid’ and what a lovely clickbait title. It's a really well researched bit of data and content. I can see this appearing in many decks over a long, long time. So this is normally what happens when SEOs get fresh data like this from a good source. So SISTRIX has said, ‘we've analysed over 80 million keywords and billions of search results in order to better understand click rates in Google search. We were surprised, as you are going to be in a moment. Throw your outdated knowledge about CTRs away, and let's start from scratch.’ and they've given a really nice summary at the beginning of the post of what they think are the important results.
So the few headline results here is, the average click rate for a first position in Google was 28.5 percent and they said, beyond position one the percentage falls quickly, the second position the click-through rate was 15.7 percent, and the third position was 11 percent and if you go all the way to the bottom of the first page, the tenth position had an average of only 2.5 percent of people clicking on this search result. So before your mind starts spinning as to how you're going to use these fresh out of the box, brand new figures in your forecasting, the next bullet point may dash those hopes. SISTRIX says, ‘Global CTRs across all types aren't very useful because depending on the search intent and therefore, the SERP layout, the CTR for position 1 will vary between 13.7 and 46.9 percent. So just to elongate all of the acronyms we use there, what they're saying is these average click-through rates aren't very useful because as the intent of the searcher changes, and by intent we mean whether it's a kind of a how-to informational search or it's a search for a very specific piece of information or an e-commerce like transactional type search, that will change the search engine result page, the serp layout. The features and verticals that appear in a universal search result page will drastically affect how people interact with those results. So the first position, in some cases was as low as 13-14 percent of clicks, and at the higher end was like 47 clicks. So that's like a four to five times difference which is going to make a massive impact.
So I just jotted down like an example for you, if you were going to forecast, if you had a keyword that you said, okay there's 10 000 searches a month for this keyword, let's try and forecast how much money we could make from that, if it's an e-commerce term and we said, okay we'll assume we're going to convert at between, our site will convert one to three percent of people into buying. On average those people will spend 50 pounds we'll say. So with our first set of numbers, if we had 10 000 people searching, and if we took the bottom number and we said 13.7 of people will click on our search result, we know obviously that's going to give us around 1370 visitors a month, and if we convert at 1% and we've got a basket value of 50 pounds - the maths, I'll do for you says, we know we'd only make about 685 pounds a month in sales which isn't great.
Taking the other end of the numbers that SISTRIX has given us and compounding them with our slightly higher basket rate conversion rate of three percent - so if we said we had our 10000 visitors again, but we had a type of search result where the 46.9 percent clicked on, that means we'll get around 4690 visitors a month, and if we convert at 3 percent it means we'll be making 7000 pounds a month instead of about 700 or under 700. So already, there's a 10x difference between these two potential forecasts and this is just playing with two variables. In reality when you start trying to build these forecasts and models, there's a lot of other numbers that you need to estimate and plug in and very quickly your worst case scenario looks so magnitudes different to your best case scenario, that really you haven't got a forecast anymore, you've just got a very broad range of things that could happen. And forecasting is a really interesting subject, I certainly know it's contentious, I've had a lot of discussions with people. Some people are very adamant that they can give accurate forecasts for seo in certain circumstances, and others not so.
My personal view is, I think you do need historic data, meaning as well you need to have tried to do some SEO because the other thing you've got to try and forecast obviously is, when you're doing these numbers, is the amount of effort you need to put in to change ranking and essentially how easily you're going to move through the competition. So if you've been doing SEO for six or twelve months, you'll have some idea of we're putting this much throwing this much resource at it and we've moved up this much, and we've gained this much traffic or we've changed results like this, and that's really helpful information, Whereas, if you're in the same position and you've put all this resource into it and nothing's budged, you might be able to get a clearer picture of what's possible.
So in terms of forecasting I mean to me, it means that you're modeling the future based on historic data within an acceptable range and there are lots of components to that. I really agree with SISTRIX here that you can't be using these global click-through rates anymore to try and make these sums work. There's some other really interesting snippets in here. So they've said searches for which site links are shown, so that's when you do a search and the result comes back and it's got an additional four links normally for the site have got a much better click-through rate than pure organic search engine result pages, so that's the 46.9, and the other without the site links is 34. So that's like a 12% difference there. The worst click-through rate is found on commercial searches where Google shopping 13.7, or Google Ads 18.8, feature is shown. So these are the commercial intent searches where Google's directly showing products and that really makes sense to me. We've seen this behaviour as well where especially with e-commerce sites, if people aren't using the shopping results they might jump straight into Google images and do a visual search that way. I think it's worth bearing in mind as well that we do know from previous announcements that Google is going to integrate that organic component into the Google shopping results. So this hopefully should show that the effect, or the effect of this cannibalisation of Google shopping results in the organic results will be lessened because we'll have organic results in there as well.
Featured snippets, 23.3% and knowledge panel features, 16.7%, also reduce the organic clicks and that's for me - so the knowledge panel and the featured snippets - we know feature snippets are when you have the position zero right at the top, where Google's taken a part of the text of a website and put it right at the top, and the knowledge panel is Google's knowledge graph, normally on the right hand side and that's way higher than I thought it would be to be honest that 16.7 is 17 percent of clicks. In general, the consensus is where there are more different elements and integrations and serp features, you'll find lower click-through rates which is fairly understandable. For SEOs this means the search intent of the keyword defines the serp layout and therefore how many organic clicks you can target. So this means that our search volumes alone are really not enough to be going on, we need to look at these different kinds of serp layouts and models based on these.
So we'll put a link to this study in the show notes, it goes through into quite a lot of detail and they give a lot of visualisations as well of the different click-through rates based on organic serps, search with site links, featured snippets, Google apps, knowledge panels, so it's a really nice one to go through and get a feel if you've got a client that's got these serp features appearing. The conclusion they give is, ‘search volume as the sole metric for evaluating potential clicks has had its day. As can be clearly seen in the analysis, the serp layout of the keyword must also be included in the evaluation. Only the combination of search volume and serp layout results in a realistic number of potential visitors. Google knows how to direct the flow of visitors, the unmistakable direction either to a pay click out of the platform, like ads and shopping, or by keeping the user on the platform and meeting the need for information directly through Google. Things like featured snippets, knowledge panels, Google apps. The relevance of the search intention continues to increase the user's search intention determines the serp layout and the serp layout determines how many potential clicks an organic result can get for that keyword.’ So really, really important to think about and yet no it should be no surprise there, I've talked about it before, it's something we teach in our SEO courses which is that Google, no surprise, is going to do the things that make it money, and that means trying to achieve their mission of indexing and showing relevant content, while at the same time either pushing you through their paid channels or keeping you inside their ecosystem.
Again, all of these things will change so some related news that I picked up to this via twitter again, I'll put a link to this tweet at search.withcandour.co.uk, a tweet by Jakub Motyka who showed an example of what appears to be another Google test which is Google enlarging the font size of the first PPC and the first organic ad. So if you have a look at that tweet, he's put some screenshots of examples that he's found where the first Google ad and the organic search result, the font size and the title is significantly larger. So again it's worth a bet that this is going to affect click-through rates as well. So none of this is ever going to be static and it might change since you do forecast, but if you are doing forecasts this SISTRIX study is a really good place to start, you shouldn't just be looking at search volume, you shouldn't be looking at global click-through rates, this is really helpful data for you.
Finally, I really wanted to talk about this, this is really interesting because it's a contradictory thing. it's a contradictory evidence to what I believed at least after hearing google tell me some things which is, an seo split test by Search Pilot around bringing content out of tabs, and what i mean by this is where you've had content that's by default when you load the page not visible because it's behind a tab that you need to click on. So for instance if you're looking at a product and you want to see like a technical specification maybe, or delivery information that might be behind a tab you need to click. So this is a blog post by Emily Potter on the 10th of July and again of course, we'll link to it in the show notes at search.withcandour.co.uk, and this test was for Iceland and what they did was they removed the tabs and accordions that were concealing product information like ingredients, nutrition facts, etc when the page was loaded, instead of this they made the text visible on the page and they had a control and this variant and they tested this over desktop and mobile.
Now, over the years we as SEOs have been told different things by Google. Originally I think it was around about 2013, we were told that any content that was hidden by default when the page loads may not be indexed because it's considered perhaps not that important. The logic being well, if it's not visible to the user when they load the page, how vital can that information be. Now several years have obviously passed, technology has changed, user behavior has changed, and as we know more people are accessing the web on mobile than desktop now, and this advice we were specifically told changed. So we were told by people at Google, hey we understand that real estate on a screen is now at premium because while more people are using sites on their mobile, it means there's way less pixels to or way less space to play with than when they're normally on a desktop large monitor. So the mobile versions of sites we have to try and maintain the best user experience and that sometimes will mean hiding content behind tabs.
So we were generally told, with a high degree of certainty, that there's no real difference in whether content is hidden behind a tab or not as long as Google can access that information it will be indexed and it won't be considered any more or less important than if it was immediately visible. This obviously made logical sense, and this test essentially contradicts that. Now, there are a couple of question marks around certain bits which I'll cover, but the results that Emily is reporting is that they had a 12% uplift in organic sessions when they put this change in place. So they're variant pages, which are the pages where they've taken the content out of the tabs, made it immediately visible, they're claiming they have 12 more visitors now because of this, which is evidence to counter what we have been told by Google.
Now, the only thing that I could think that might affect this would be I don't know how Iceland were originally hiding that content. so if, there are lots of different ways you can do this, if content was just kind of hidden by the CSS by the styling, it's still very easy for Google to access that information and I think that's what they mean when they say ‘hiding it’ and of course, there are various ways that you could use javascript to hide content and load it in, for instance when something was clicked on. And while Google will go through a rendering process where it tries to process Javascript, Google's not going to just process any Javascript that requires user interaction as well because they have to be careful around triggering things on websites, such as you wouldn't want Google trying to submit you know contact forms and things like that. So there is a possibility that how Iceland was previously hiding that content behind the tabs was, in a way, making it technically more difficult for Google to index that content. However, my gut would say based on the technical competence of this write-up from Search Pilot that that's unlikely, but I will follow up with Emily as well to see if she's answered that or if she can provide any more information on that, because that's really really interesting.
It is not the first time this has happened, we've had other situations and it's always around Javascript, in my experience, where we've been told one thing by Google and sometimes another thing turns out to be true. So the one that sticks in my mind was to do with canonical tags which was Google told us that if canonical tags were deployed via Javascript, or for instance via Google Tag Manager, same thing, that they wouldn't be honored and that was contrary to the experience that certainly my team had had because we had been deploying canonicals via Tag Manager and it certainly seemed to be working; the amount of index pages dropped, we got more search traffic and someone who was a little bit more thorough than me actually set up an experiment and proved that to be the case. And I don't think this is where Google is intentionally misleading us, I think it's important to make that clear. Generally, I think they try to be very helpful and there's no good reason for them to try to be misleading people about a detail like this, it just doesn't make sense and we need to appreciate that this process of crawling, rendering, indexing is very very complicated, there's always going to be edge cases and there are possibly other confounding factors that aren't immediately obvious in this study.
However, it is interesting and it would make me think twice now before just very very confidently giving the recommendation to hide that content. So I hope you found that as interesting as I did and I hope that raises some interesting discussions, especially for those running ecom sites with tabbed content internally.
And that's the end of this week's show, so we'll be back on Monday the 27th of July, with episode 71. As usual, if you are enjoying the podcast, please leave a review, please subscribe, link to me - love links - and I hope you all are getting on well and coming out of lockdown and I hope you'll tune in next week.
More from the blog
Episode 68 - SEO site audits and Sitebulb with Patrick Hathaway
In this episode, you will hear Mark Williams-Cook talking to Patrick...