Candour

Episode 93: Keyword research with AlsoAsked and Question hub, search sundry and more Q&A

Play this episode:

Or get it on:

What's in this episode?

In this episode, you will hear Mark Williams-Cook talking about:

AlsoAsked Beta: The new version of AlsoAsked that is going live this week and new features to expect.

Google Question Hub: How to find true gaps in longtail keyword research

Search sundry: Google Ads data exclusion updates, ranking shifts and Majestic link graphs

Listener Q&A: Your questions on 404s, links on page and conspiracies, answered!

Show notes

Google Question Hub

Use data exclusions to prevent conversion tracking issues from impacting your Smart Bidding

Majestic Blog

Transcript

MC: Welcome to episode 93 of the Search With Candour podcast, our first episode of 2021,It was recorded on Friday the 8th of January and, of course, my name is Mark Williams-Cook. Today, I'm going to be talking to you about keyword research with the new AlsoAsked beta and Google Question Hub, some changes to Google Ads with data exclusions, some sundry, a possible ranking update, and we've got some more listener Q&A; last time I put it out there I actually had about 30 or 40 questions come back, so we'll be working through these in the subsequent episodes where I can.

Before we get going, I just want to tell you, this podcast is very kindly and proudly sponsored by Sitebulb. Sitebulb, if you haven't come across it before, is a desktop-based SEO auditing tool for Windows and Mac. I'm surprised, actually, that I learned some other SEOs that I know quite well still hadn't used Sitebulb, which was strange to me, because I thought everyone used it. If you haven't tried it, yet they've actually got an offer with us where you can go to sitebulb.com /swc and you'll get a 60 day trial for free - you don't need to put in any payment details or anything so there's no catch, you can just give it a go.

Every week I normally talk about a different feature it's got and why we like using it, because we use it at the agency, and, this week, it's just one of the extra things it does really, which is, outside of the whole auditing process you can actually use Sitebulb to generate your xml sitemaps. Why would you want to do this? Well, some content management systems actually make problems when they generate sitemap files. That's because they will include non-canonical urls or even fragments of pages; some content management systems build pages by making lots of little sub pages and putting them all together to form a full page, and I've seen lots of those. Some of the quite popular CMSes actually include all of those fragments of pages as URLs in the sitemap, which obviously you don't want them indexed. Using Sitebulb, you can actually crawl your site as you wish; obeying robots.txt, not looking at pages which are non-canonical, and generate a sitemap from that, quickly and easily. Just another thing you've got in your swiss army knife of Sitebulb tools. That's at sitebulb.com/swc.

This week, I want to talk a little bit about keyword research. It's going to be incredibly self-serving, because I'm going to talk about AlsoAsked. For those of you don't know, AlsoAsked is a website that I founded and the Candour team built, November 7th 2019 I think we originally launched it and it's been in alpha for about a year. The alpha version, basically meaning that not all of the features we wanted were live, and some of the features may be broken. We left it live and we've got feedback, and I know people have really enjoyed using it. We're up to now between 100 and 150,000 users per month, using alsoasked.com and together again, on average, they're doing over a million searches a month.

We have come to this stage anyway, where we would have to actually look at charging for this tool, because it's now costing a fair bit of money to keep running in a reasonable fashion. We have to pay developers to work on the actual infrastructure and backend, because serving those million searches a month isn't quite as easy as you might think. If you haven't used AlsoAsked before, it's basically a tool that aggregates and organises the ‘people also ask’ results that you get in Google. So, for around 50 percent of searches that happen, everything from questions to one word brand searches, Google will show a little box that says ‘people also ask’, which shows questions that, funnily enough, people also asked that's related to your search term. If you click on those questions, what it will actually do is, it will branch out and open another set of questions that are related to the one that you clicked on.

Now, for content creators, for SEO people, copywriters, PPC people, that's really really useful information. However, from a content creation point of view it's very difficult to get that information directly from Google. Now there are ways you can scrape this information yourself and, on the AlsoAsked website we've always provided the open source solution in Python to do this through the command line interface, for free, so you can just use that on your own desktop, if you like. The idea of AlsoAsked was to, firstly, visually show that data to you, and by that I mean including the clicks. When you put a search term into AlsoAsked, it will show you the questions that are returned, the ‘people also ask’ questions, and it will also simulate clicks on all those questions and show you what happens a couple of levels down.

The end result is, pretty quickly, you get this tree view of questions. Why that's particularly important is, it also shows you the relationships between those questions. it can show you where you've been, maybe, blinkered in your understanding of the intent of a subject or a question. That's happened to me a few times, when you've got involved with a client and you're thinking about only their business and you put in a keyword, and then you see some completely unrelated questions and realize there's a whole different set of intent around that search term as well. The other thing it offers is you can actually click on any of those questions and it will redo the search and you can explore these trees for as long as you like.

From a data analysis analysis point of view, you can immediately export what you see on the screen to a .csv, and in that .csv file it will actually keep that same organizational hierarchical structure of those questions as well, so that's that level, that nuance of data, saved in the .csv file. You can also export straight to .png, currently, on the site. That is basically for, if people want to use the data for client presentations maybe, or internal presentations, it's a very quick way of showing the detail to people.

The reason I wanted to talk about AlsoAsked on the podcast is, hopefully, by the time you listen to this, this week we're going to be launching the beta version. Moving from alpha to beta, which is pretty much, we've got everything as we want it, we've got all the features as we like them, we're just not sure how they're going to stand up to a stress test, or not, as the case may be. We're immediately just going straight to an open beta, everything's still free at the moment, and I'll talk more about costs and etc in a bit. We've added some new features as well.

What's new in AlsoAsked is, firstly, you will generally now get more results in the question maps, so, if you compare the current version of AlsoAsked to the new one that launches, doing the same search, sometimes, you'll get as many as two or three times the amount of questions come through in the new version. That does mean that searches themselves will take longer, so at the moment a search on AlsoAsked takes normally around 10 to 20 seconds. On the new version of the site it can take 30 seconds to a minute for a search to complete because it's getting more data. The other thing that we've added is a login functionality and an account creation functionality. The reason for this is, we wanted to give users some cache management of their searches. When a search is performed we will cache the result that we get, because it's quite common that people do the same sets of searches. There are common sets of searches, so this massively speeds up the interface for the users, because they can instantly get those results back.

What the cache management offers you is a couple of things; firstly, we are going to introduce a limit for the amount of searches that particular users can do through free accounts, even if you're not signed in. You'll still be able to search without an account and there's several reasons for this, one of the main reasons is, you may have seen, if you follow me on Twitter AlsoAsked did come under attack a few times from an individual doing a Denial of Service attack. This was a very specific attack on our service that targeted some of our endpoints and was trying to do like hundreds of thousands of searches per second. When we put mitigation in place, this person was trying to again find ways around our mitigation, and it becomes very difficult to have a service that's free and open and easy for people to use, while still trying to have a level of security that mitigates determined people. This is why we can't have nice things, as the saying goes.

Just being able to have some easier more verified rate limiting, through account management, will really help us provide a better service for everyone, but also it comes back to this cache management, which is, you'll have a search history of things you've searched for and you can then recall your cached searches for free, if you like, in that it won't take away from the number of searches you're allowed to do, because that search has already been done. We've got the answer saved for you, so you don't need to use a credit to do that again. However one of the things that we learned over the last year with ‘people also ask’ is that the results actually changed quite quickly for some topics. Things, especially around like The Mandalorian series, was one I watched with interest as the first ‘people also ask’ questions appear for that, and how they morphed over time, certainly things around the elections, for instance, as well. With cache management, once you've done a search, you're going to have the option as well to clear that cache, so you can then get the latest, up-to-date ‘people also ask’ results for that.

The other main feature that we're introducing and, again, everyone's going to get this for free during the beta, is a bulk search capability. This means if you do have a set of keyword research that you need to do and you don't want to try and have multiple tabs open etc, even though we've got quicker ways to do that now, you can upload up to 100 keywords at once and what AlsoAsked will then do, is it'll put these in a queue, it'll chug away and, again, this is why we wanted account creation, because it will simply email you a compressed .zip file with all of the .csv data on all of the search terms there. I'd be really interested in what feedback you have on the new version of AlsoAsked, so it's available at alsoasked.com; I'm hoping, by the time this podcast comes out on Monday, that we will have this beta version live. If not, it will definitely be here at some point during this week, he says. I just love your feedback, do test it.

I said I’ll talk more about pricing. We are going to start charging, we will have two free versions; there will be a version for new people that have first seen it, you'll be able to do a few searches without doing anything. There'll also be a version where you can sign up for free and have a dozen or so searches for free every month, and export things to images. Then there'll be a freelancer type version, where you'll get a couple of hundred searches a month and you can export the data, and you'll have your cache management etc. Then we'll have this version where you've got a thousand or so searches a month and you'll also be able to do the bulk searching, which is the main difference there, so this is really for larger teams.

I'm hoping that people will find value in this, they'll find it useful and that we can keep the service running. Check it out, if you found anything that breaks just let us know, we'll have some feedback tools on there so really appreciate any thoughts you've got, and there will be new features we've got planned over 2021 to add. I know we've got a lot of users as well, outside of the UK and US, so non-English users, and we are going to look at having a few different multilingual, particularly Spanish versions, of the site live during the year as well.

Google Question Hub is the other thing I wanted to talk about, for keyword research. Again, I thought this fit in nicely because it's speaking more about people's questions, long tail keyword research, and finding these gaps. ‘People also ask’ is, as we discussed a really good way to see what people are searching for, but there is one snag there, which is that, if you have a ‘people also ask’ answer, it means that Google has an answer for that question. That's literally what you're seeing, you're seeing the question, but what Google's showing you is the answer. So, while it's great to understand that this is the content that you need, and this is what you need to be competitive, what it doesn't necessarily show you is where all the gaps are, which is where this Google Question Hub comes in.

This is actually a Google product and I think I spoke about this almost a year ago, because it was available in India for for quite a while now, and I've just seen quite a few more people talking about it recently, because it's now become available in the United States, India, Indonesia, and Nigeria, a really interesting cross-section of territories there. Google's hoping to expand to other countries over time.

What Google Question Hub actually is, is it's essentially Google's way of saying “look, these are the things that we've had searches for that we've not been able to provide any good answers for”, and it's giving publishers the opportunity to get in on that and actually start answering those questions, or just going back to their sites and start producing this content. Now, I've seen some of the questions that have come back in the US version and they are the ones I've seen, at least at this stage, are really really really specific questions, things you would expect to only be searched for very rarely, and that's fair enough, that's actually what you would expect because the web is very big, there's lots of searches and we are looking at the extreme long tail here. But the fact remains that these are still questions that Google cannot find answers to, so there is definitely going to be value in being the entity, the person, the website, the organisation that answers these questions. If you aren't in the United States, India, Indonesia, or Nigeria, they have got a wait list so you can get notified when it's launched in your country. It's definitely something that I would have on my radar for my content teams as well.

There's almost this stack of software for keyword research at each different level. For me, I like still using the the Google data you get for the very top of the funnel broad stuff, and then a lot of people use Answer The Public, Answer The Public often gets confused with AlsoAsked. Answer The Public uses primarily the Google Suggest API, so that's the suggested searches, and that's a really good topical overview.

Then I actually use AlsoAsked a level down from that so, once I've got the topic information from Answer The Public, I then put those specific topics into AlsoAsked to get questions about them, and then I would see perhaps even the next stage down, is where Google Question Hub would fit in there, so you build a nice natural taxonomy from going this top down, really broad searches, then into breaking off into topics, then AlsoAsked for questions within those topics, and then Google Question Hub for the unanswered questions within those topics. I put a link in the show notes, you can get the show notes at search.withcandour.co.uk, and there will be a link there that you can click to go sign up and join the waiting list or, if you're in the States, Indonesia, India, or Nigeria, just sign up for Google Question Hub.

There's a couple of bits of search sundry that I wanted to cover very quickly, before we got onto the user Q&A. I don't think they deserve their own sections but I found them interesting, and it's probably good that you know about them.

The first one is about Google Ads, and this was quite a quiet announcement, but Google Ads has launched data exclusions for smart campaigns. What does that mean? Smart campaigns, or smart bidding, and this is Google's description, is “a subset of automated bid strategies that use machine learning to optimize for conversions or conversion value in each and every auction, a feature known as auction time bidding.” If you run any Google Ads, you would have encountered these before, Google pushes them really hard, so this is all the stuff like Target Cost per Acquisition, Target ROAS, Maximize Conversions, Maximize Conversion Value, even Enhanced CPC. They're all smart bidding strategies, they're all the strategies that Google basically uses your Google Analytics conversion data for, to try and build models and basically improve your Google Ads for you, with mixed success. Again, as we've discussed before, this is really helpful.

What this data exclusions is offering is, there there are times and this has happened, I'd be surprised if an agency hasn't had a client where this has happened, where conversion tracking breaks, and by breaks that could mean either, it stops tracking or it's double tracking, or is tracking something it shouldn't, it's basically not working properly. The issue that you can have here especially, and this is the running joke of machine learning, is if you put bad data in, you'll get bad data out. This data exclusions allows you to go into Google Ads, into data exclusions, and choose a time frame for when you want to, say “please ignore this conversion data or lack of conversion data” in this time.

So, if you know that you had broken conversion tracking and it wasn't working, you can go and do this, and this will be excluded from the data set that is used to train the models that are going to be controlling your bidding. Very important because, even fairly small times of losing this conversion data can really upset these models, and they take a long time to train, so you've probably had it before where, if you've switched around your smart bidding, things tend to get very expensive for a while and, if you speak to the Google accounts manager, their stock answer will be “hey, yeah, the model just needs time to learn”, during which point you're paying out the nose, so you want to avoid these bad data situations. Again, I'll put a link if you want to read it in the show notes - search.withcandour.com - that will take you to the Google Support article that explains how to do this.

The other things I wanted to mention was, it looks like yesterday there may have been some kind of ranking update again. I don't think it's a core update because Google tends to announce them. We've seen a few tools again like the SEMRush sensor spike yesterday, which was the 7th of January. Some chatter on webmaster forums, opinions divided as to if this is still the last tremors of the December core update, but Google did say that was finished rolling out on the 16th. My view is, it is probably still something to do with that, so while it's finished rolling out, you still get these ripple effects that tend to go on for a week or two. I don't think there's going to be anything new there. The reason I'm telling you this is, if you've seen some changes at the beginning of January, so you've come back after holiday to hopefully improved rankings, hopefully not worse rankings, you're not on your own. This isn't something you've specifically done, or something that's broken, it seems lots of people are noticing this.

The last piece of sundry I wanted to mention was actually about Majestic which is; Majestic has launched a feature, so Majestic is the backlink analysis tool where it can actually show you graphs of how different websites, domains, link together. This is really cool information, from a practical point of view, it actually opens up a lot of neighborhoods of links that you might otherwise miss. The traditional way, if you like, or the way you'd go about finding new backlinks might be things like competitor research, and you might only go back one or two levels, but seeing this visually, this structure of how you and your competitor websites, and each neighborhoods are linked together, it can let you see what I would refer to as good neighborhoods of sites. A neighborhood, to me, is a set of topically relevant sites that are linked together, or all linked to a particular hub, and making sure that you have links from those hubs. If you just check out the Majestic blog, again, I'll put a link in the show notes for you, you'll see that feature there. Majestic is just one of the tools we use for backlink analysis, I highly recommend it.

To finish off I thought we would work a little bit more through the listener Q&A. I asked a few weeks ago, if you had any questions, submit them and we'll try to answer them on the podcast, and I got a lot back, which is great. Hossam Ismail submitted seven questions, and they were really great questions but, to get through everyone's questions, I've just picked two of them. Hossam's first question was “if we have an old website with a large number of 404 pages, what is the best practice here? Should we delete these URLs manually or should we just put them in robots.txt, with a no index condition? What should we do?”

The first thing that is important to realize here is, 404 pages aren't pages, they're a lack of page, they just a link to a non-existing page. They're going to exist in one of two situations, which is, firstly, if they are internally linked, i.e you have pages on your website which link internally to other broken pages. The easiest fix in that case, is just to update that internal link to the page, where it should be going. There's no reason why you should have broken links on your site.

The other possibility is, if you have external links which are linking to pages that no longer exist. Now, there's an array of ways to tackle this. The answer normally is not to have these pages 404, because that's a bad experience for the user, not even thinking about search engines. Generally you want to do one of two things. A lot of people will use a 301 redirect, so if someone is linking to a page of yours which is now broken. If you have a close matching or relevant page they should go to, then you would just 301 redirect them to that page. If, for instance, it's a product, so if it's an e-commerce site, and maybe the product is now not available, then I would recommend actually keeping that page there or recreating it, and explaining to the user that that product is no longer available, and you can link maybe to close alternatives.

There's a few advantages, as well, to that approach in that; you still then maintain the benefit of having that incoming link and passing that equity to other pages, and you've made it super clear to the user what's going on, so they're not confused if they click, expecting to get to a page, and then end up getting redirected. What you don't want to do is just 404 them, there's no reason, with 404s you want to be using robots.txt and certainly not no index, because 404 pages won't be getting indexed anyway.

The second question from Hossam was “when i'm looking for good backlinks, do I need to keep in mind the number of outbound links from that page? For example, if I have two options, a page with five outbound links and another with 20, does the number of outbound links affect the quality of the backlink, ‘from a Google perspective’?” I thought this was a really interesting question, not because the technical answer is interesting, I think it's an interesting question because the technical answer is useless. What I would say here, is if everything else is equal, which it will never be, and you had your two options, a page with five outbound links and a page with 20 outbound links then, yes, technically the page with the five outbound links, those links would be “worth more” from a Google perspective, because part of the modeling Google is doing with these links is working out reasonably based on text, link size, location, frequency, if users would click on those links, so if there's five links and you're one of them you have a slightly higher chance of being clicked on than if there's 20 links and you are one of them, therefore that's reflected in the link graph.

However, that makes absolutely no impact on your judgment as to whether you should get links from that site or not, and whether it's one or another, right? That page is either relevant for you to have a link from or it's not. It doesn't matter if it has five links or 500 links on it, if it's relevant you being there, then you should aim to be there. It's not bad if it's got lots of links on the page versus if it hasn't, and this is what I mean by “while there's a technically correct answer to this”, it's not applicable in any sensible realm of SEO. What you should be doing, I would always just make a judgment call on “is this a good page, does it make sense I'm linked to from here? If so I want a link where it's relevant and hopefully as most prominent as it can be”. I wouldn't worry too much about that, the only way you might think about this is if you're buying links, which I just wouldn't get involved with. It's just not something you should be worried about. I hope that answers your questions, Hossam, maybe we'll work through some of your other ones in the subsequent episodes.

I thought we'd kick off 2021, since it's one of those years already, even though I'm only on the eighth is two, what we'll call conspiracy questions. We're rife with conspiracies in the world at the moment, so I thought we would enjoy these. The first is from Martin McGarry and Martin asks “would you put it past Google to covertly sabotage the credibility of organic results through core updates in order to make the paid results more relevant and useful? Now that they're pushing for self-sufficient AI driven results, they want those to be the true answer to most peoples’ (commercial) search needs.” Martin's basically asking “would Google tank the relevance of organic commercial terms so more people click on ads?” My honest answer to that is “I don't know if I'd put it past Google doing something bad”, because they do apparently quite a few morally questionable things now, but I don't think they're doing it, because I don't think it makes commercial sense to do what you are suggesting.

The reason that, in my opinion, the reason that Google generates over $100 billion a year in ad revenue is because its organic results are so good. The reason Google can actually make the money from the ads is because they can put them in front of so many people. The majority of searches I would say don't have massive commercial intent and they all need to be served by this core algorithm as well, so it’s not one or the other, you've got research as well, from Ofcom, that shows around half of UK adults can't actually identify a paid ad in Google, so while this might play into this, the other half, when interviewed, about 75% of them say they skipped the ads. You would be disenfranchising a lot of people, especially those that skipped the ads when they saw them, if you started trashing these organic results. It just doesn't make commercial sense, to me, to drive more revenue from their search to do this.

I would say continue making the best search engine possible, there's loads of other ways that they can generate more revenue, we've seen Google, as we spoke about in our end of year episodes, step on people's toes in all kinds of industries, zero click results, these are all ways that I see Google eating in and making more revenue, but I don't think that sabotaging their own organic results is how they would go about this. I'd actually wonder, like you've suggested, if they're AI driven, what kind of granular control they would have for. Unless they actually set these fitness functions around destroying the relevance of those organic results, they wouldn't have that granular control to do it, because the whole point of the AI is it's going and tuning everything for them.

I hope that, or at least my opinion, has entertained you on that, Martin. The last question from Phil Fraser; one we've all heard before, i've answered before, but what better way to end the show and start 2021, with the question from Phil Fraser “will my SEO perform better if I spend more money on PPC with the big G, ‘you scratch our back with dollars, we'll scratch yours’. Again, I wanted to answer this, just to be super clear, because the first part of the question is, will my SEO perform better if I spend money on PPC?

The answer is possibly, because if you drive more people to your website, you have a higher probability that they will like your content, link to it, and do all the things that will actually give Google the signals it needs to improve your ranking, but you've gone on there to make the suggestion, is it a direct relationship that you spend money on PPC and it will improve your organic? No, no, no, no, no, we've covered that a lot before, Google's covered it before, again, I don't see that how that would make commercial sense for Google, apart from all the legal hot water they'd land themselves in, if someone's spending lots of money on Google Ads, and they're reliant on it, you actually surely want them not to perform well in organic, because then they wouldn't need to spend as much money that they are obviously spending on paid search. Someone's always going to fill that void anyway, so I just don't see what Google has to gain from linking those two things, but always happy to answer these questions, keep them coming and we will try and answer them on subsequent episodes.

That's all we've got time for, I hope everyone is having a great 2021 so far. We're only a week in but it's been pretty interesting. I will be back next Monday, which will be the 18th of January. I hope people have some feedback for me on AlsoAsked, apart from that I hope you'll have a brilliant week and do remember to subscribe to the podcast if you are enjoying it.

More from the blog