Episode 36: Large site SEO [part 2] with Andrew Smith

Play this episode:

Or get it on:

What's in this episode?

In this episode, you'll hear from Andrew Smith giving actionable insight into dealing with big site SEO from his experiences with companies such as Expedia, CheapFlights and eDreams on big site SEO.

Recorded on 7th November 2019 at SearchNorwich 9, Andrew goes into detail about reporting best practises, common pitfalls when dealing with big site SEO.

Full video:


Podcast transcription:

MC: Welcome to Episode 36 of the Search with Candour podcast! Recorded on Friday the 15th of November 2019. My name is Mark Williams-Cook and this week we're bringing you part two of big site SEO with Andrew Smith.

So last episode we had Andrew Smith, the ex-head of SEO for Expedia, Cheap Flights and eDreams, he came to very kindly do an interview with me, just talking about the ins and outs of big site SEO and we got a load of really good feedback about that interview. Lots of people really keen to hear more from Andrew and after actually, I spoke to Andrew, he did a talk at our Search Norwich SEO meetup event about big site SEO, so that's what I want to give everyone today which is the recorded talk of Andrew doing that. If you go and look at the show notes as well at you can actually see the video of Andrew doing his talk and you can download his slides there as well, so we did the interview, the talk is actually a lot more actionable if you like, so he's put together that content, he goes into the ins and outs of how to report on big sites when you're doing big site SEO and he looks at some of the common mistakes people make and where you need to focus your efforts on big sites, because there's some issues like we covered in the podcast that aren't really a thing for small sites but their problems actually scale with the size of the site and become a real pain in the ass very quickly, so this is Andrew Smith from Search Norwich on the 7th of November talking about big site SEO.

AS: Okay so yes big site SEO, I'll talk a bit about that but first a little bit about me, this is my moment. So SEO is pretty cool which has taken me to London, Seattle, Barcelona and now Norwich, which is like way up there, but I'm quite lucky in that respect! So I work for Expedia, I started up in the UK there and then I relocated to Seattle as SEO manager for and recently I moved to Barcelona to work for eDreams ODIGEO and SEO has been quite good to me. I've enjoyed it, you know I've done a lot of work at lots of cool companies.

Back in 2007 I was looking for someone to help on Expedia and Mark was writing about some really like black hat, well probably grey hat - no I don’t think so, I've got to bring this up - and I saw his blog and I thought, look if this guy knows about this stuff which is a kind of grey and borderline black hat, and the things that I need, he's probably going to find this really easy. So I hired him and so I've known him for like 12 years now and he'd done some really quite fantastic stuff for me at Expedia back in the day and you know, he's still doing some really cool stuff now.

Now I mostly do like SEO consulting, so I help out a few clients. My original vision was that I was supposed to be building a whole bunch of tools and lots of passive income, but then clients came to me and I didn't want to turn the work down, so that's what I've been doing for the last 18 months. Call it what you like - call it grey hat, black hat, white hat, inbound marketing and we really like to rebrand it but SEO is just SEO.

What I like about it is that the basis of it really stays the same. So what is important for SEO? Content, technical - you're technical information, architecture, external links, they stayed the same, it's always going to stay the same, I don't think there's anything that’s really going to change that. But it's kind of always evolving, there's always more to learn, there's always different things that Google throw at us. So for me, it's how it keeps it interesting, there's lots of different problems to solve and I really like that about SEO.

Okay a bit about big site SEO, so what is a big site? And this is something that Mark asked me during the podcast earlier, I guess I would define it as a site that maybe has a hundred thousand to a couple of million pages, and those are the type of sites that I've generally been dealing with. I'm going to go through a few tips, but I think the first one is to standardise your reporting and what I mean by that is, I see a lot of teams that are like this; oh sh#t, we've got to do our SEO report, got to go and give an update to the c-levels or the management team, whatever or even if it's an agency - I think you still get agencies that are like this, like crap - we're meeting the client tomorrow, what good stuff are we going to tell them, you know, what have we been doing the past month and this was also something my team were doing when I when I arrived at different places, the reporting is super disorganised, they don't really know exactly what they're talking about right from one week to the next and I think really it shouldn't be that way. So there's a few things to remember on this, especially when you've got many many many websites that you need to report on.

So I think the first one is just set the cadence. I would say if your team have daily reporting, like your team really are gonna care about the minutiae of what is going on an SEO. so at the template level, at the page level have daily reporting, if it's like a hundred keywords that you care about, a thousand keywords, 10,000, you know a million, try and have daily reporting for that so you can get an idea of what's really going on.

Focus on the KPIs that matter and those should be the KPIs that the business care about, not just the KPIs that the SEOs care about. So obviously traffic is going to be an important one, rankings is going to be an important one, like maybe you want to monitor the number of position one rankings you have in Search Console, but make sure if your business really cares about revenue, which all businesses do, if they really care about some specific KPI that's super important to that business, make sure you're actually tracking that but most importantly make sure your team understand what that metric is actually about because what it's probably going to mean is you have a budget that you need to actually hit, a target that you've got a hit, that is all about that KPI so it really matters.

Automate everything you can, set the format and stick to it. I think this is quite a simple one but, if your team is having to spend hours and hours and hours doing reporting on a fairly regular basis, that's bad - it's a wasted effort okay. It's kind of, effort of justifying your existence is how I always say and it's like, try to just automate what you can.

Adjust for the audience. So yeah, when it comes to talking to c-levels, don't give them all the details. They don't need it, they're not really that interested in it, what they care about is are you hitting the budget, so focus on that. If you've got other detail that you need to share, put it in the appendix, but don't go into it, it's just their eyes will glaze over, I've seen it so many times.

I think the other one that teams quite often forget about is the competition, so they don't do reporting that says, where are we versus the competition, you've all got access to tools I imagine that do this. Like HF does this pretty well, SEMRush does is pretty well, there's SISTRIX, there’s lots of tools right. They're gonna say - hey, our SEO is pretty good, in fact it's really good, look at where we're doing versus these competitors over here - so sometimes it’s an opportunity to make you look good, sometimes it might make you look a bit bad, but it's really important.

Google Data Studio is super super good for this. Why? So if you've got, you know, if you're dealing with like a 150 websites, 20 websites or however many you've got in your market. So for companies I've worked for, it's usually pretty much a website per country. It's really good for building and you can pull, and it’s dead simple right, it's super flexible, it's really simple. The first time I used Google Data studio I spent about three hours just clicking through stuff and then I was like wow I've made a report that is just going to self update every single day. So take the time to do it, you can pull data from Google Analytics, there's a whole bunch of other connectors in there, it's really flexible, but for me one of the best things about it was taking the budget which the team has come up with for the year, it has been agreed by the c-level execs, putting that into a Google sheet and then plunking that into Data Studio - so there's a line that I can see, that I know if I'm above that line I'm above budget, if I'm below it I'm below budget, it's really simple.

I think the other cool thing about it is it's not just reporting; you can create some really quite actionable reports, so if you want to see, I don't know, if you want to see the content that is underperforming, so you might worry, you can pull in a report from Search Console and you can see, look over the last seven days, here's the pages which have dropped the most in terms of traffic, so you can see what that flux looks like and you can say, look this is stuff that we need to go and fix over here. So to an extent it can become part of your monitoring and not just your reporting.

So next, like Stephen kind of talked a lot about this, but I would say monitor for cock-ups and clean house because this happens a lot and I think when you're dealing with big site SEO, sometimes these cock-ups can really just make things so much worse, like if you have a canonicalisation issue and it affects 100,000 pages, it's much worse than if it's not a small site and it's just a handful of pages that you're dealing with.

So these are some of the things that I'd watch out for, so there's good tools that monitor like robots.txt, I think maybe Content King does this? (yes) Okay cool. There's other ones, I used Roboto, there's a Little Warden, but monitor for this because sometimes people make changes to it and you don't realise, suddenly you've got chunk to your site that’s just been blocked, pages being mis-canonicalised, I’ve had this lots of times where my team are working on pages to get rolled out and that example with a staging environment, I've seen that happen as well at my own company that I've been working at. Developers roll it out and you're like, how the did that happen?, so you know it happens, so it's really important just to be part of that testing process and look out for it.

Stray like no index, no follow, so pages just getting no index, you didn't even realise, sometimes the CMS f#ck's up and things get no indexed. Hidden external links, so this is one that I had where - it's a bit like your example actually with the badgers out there - so I was working on our website and they provided content which got distributed elsewhere, so it was like their review content, they've got shown against other like the same products but on other web sites, what happened it was one big website, you probably all know it, it was TechRadar. Those guys were displaying reviews and they decided to hide the links to the source of the reviews, so it just made them hidden links, so they're still there within the content but they were hidden, that website lost 30, about 20/ 30 percent of its traffic just like that and nobody really knew. I was looking at it the next day, I was like what's going on here?, no manual penalty, nothing really, there was no algorithm update, tracking all worked absolutely fine and we were like, what's going on here? - turned out it was those links. So it took us about a week to track it down, it took them almost another week for them to flip the change and fix it and almost immediately that traffic came back, so we were pretty confident that's what it was.

Tracking issues, this is a big one. I was talking to Steven about this earlier but one interview question I ask people is, ‘if you come in tomorrow and traffic's gone down 40%, what would you do?’ and quite often that is the reason, it’s tracking issues. You see it and it's like, don't sh#t your pants, I think there must have been an algorithm update, look internally first okay, it's more likely that something got messed up internally, then you got affected by something external.

Some of these other things as well, but I think just be really aware of them, try and put a system in place that you're going to be not reactive to it but proactive to it, so the monitoring is going to tell you. So I mean these monitoring tools are super-important.

This one, I see clients like this all the time, like they say they want to do some super cool, new features but then you go into Google Search Console and they've got hundreds of thousands of errors and you're like how about we just fix this stuff first. Google is literally telling you to fix it. The cool thing about it as well is quite often it fixes scale really well, so you usually see the same errors in different markets right, so you've got ten websites and you're probably going to see actually if we fix this on one, it's going to fix all of them, so it's quite a good one to really focus on. But it's just so simple, Google's literally telling you these are things that are wrong so stay on top of them and quite often it can be pretty simple straightforward things.

This one, so Google's just released, actually changed this; so there's lots more reporting for it but again, it's another thing that scales well, so quite often if it's a speed issue on one site, it's gonna be a speed issue on another site ,so focus on the speed issues. But I think one thing that I would really recommend for this and it kind of annoys me a bit, so the speed stuff quite often seems to sit with the SEOs as like, oh this was your responsibility - it’s really not SEO is responsibility at all, it’s like SEO channel gets a benefit from it, but what I found worked quite well here is to try and put the onus as an overall company target to say, ‘look this is our target for load time, let's everyone buy into it and let's try and achieve that’, because the only way you're really going to fix it is probably doing lots and lots of small things over time, rather than saying, ‘oh look, we're gonna tackle site speed and you know, we're going to do a big project’, and then six months later, after you've done that you see the speed starts going down because it wasn't an ongoing target. So I'd say it's a really good idea to try and set that as a target and get buy-in at the senior level.

I think lastly, it's just so much more important in a mobile world this, like desktop first, it wasn't so much of a big deal, this is now super super important and I see stuff with clients right now where I think the biggest difference is actually their mobile page speed, like their performance on my boss was quite poor. Maybe you yourself if you go in Search Console, you might not see that you've got any fast pages, so I see that in a lot of the clients that I've got, they haven't really got any pages which Google will actually deem to be fast. So I think there's a heck of a long way to go on this.

Next one is leverage or information architecture; so this is really quite common, so this site, 87% of their pages have got less than 20 internal links, so you know there's like over 200,000 pages here where Google is a bit like, ‘well how important is this page, you can't even be bothered to link to it more than 20 times’ and I think this is something that on big websites gets really really underutilised. So if you think of it this way; I've got a website, I'm going to link to stuff more frequently that's more important, it's like a really simple way of telling Google - look, this is more important than that over there - now is it easy to do that? No it's not that easy to do that. On a thousand page website, yes you can probably get everything four clicks away relatively easily, on a million page website it's much harder to do that.

But there is another way of doing it. So one way and one approach that's work for me is to understand the keywords and say, ‘okay look if I've got, let's say I've got 20,000 keywords that really matter here, I know what pages those keywords are relevant for, I know I've optimised at a template level, so I know my flights to New York page and my flights to Las Vegas page and I know what each of those are for because I've generated those pages from geography data, so I can take that search volume and match up my internal links for my demand’, so effectively what i’m going to do is, I'm going to distribute internal links through my site, based on the search volume - so the higher the search following, the more internal links it's going to get.

So these graphs, it's kind of gives you a bit of an idea, but if you imagine you've got like a pyramid- your sites like pyramid - you're going to try and link up from the bottom and push that equity all the way up. I would say, on a big website that's not doing this properly, you have a big change. So this graph at the top left - we use like a page rank ruler for this - basically we adjusted the way the internal links were going, so the purple part - that's how that internal page rank was being distributed before, we changed the internal links and effectively what we've done is we've boosted internal page rank to a huge number of pages; this was like the top two or five thousand pages, the ones that really mattered and you can do this, you can basically sculpt your information architecture just with internal links. A few samples that are getting this right - so Etsy is a pretty good one in retail, they do some cool linking to their category pages. HomeToGo - so this is quite a different one because HomeToGo have a pretty small website but what they do on that website is really really good, so if you want a great example of SEO - HomeToGo. Other sectors, like jobs, this works pretty well in jobs too, they're really like going at it when it comes to like all different facets of describing jobs in different ways and CV Library have got quite a good example of it.

Last thing, so my last tip is run a/b tests. So how do you run a/b tests in SEO? Well it's quite tricky, but if you've got a decent number of pages - so let's say for example, you've got 10,000 pages which are about a particular subject, so maybe its products for example on a retail site, you've got 10,000 pages and the template for those products is fairly similar, it's pretty consistent, yes alright the content is going to be a bit different. Just think of something that you're going to do on those and apply it to some of those pages, if you've got 10,000 pages, do a random sample of five thousand and then apply it to those. Measure that 5,000 against the other 5,000, there's your a/b test for SEO. All you're trying to do is increase traffic to those 5,000 pages, if it works do it, roll it out on all of them, keep doing that. I would say the simplest thing to try and do it with is, title tags, it's really simple change for you to make and you'll be able to measure it quite easily. This is just really for the slide share, so there's some links here of useful things and bits and pieces from what I've just been talking about. Thank you.

MC: I hope you enjoyed that talk by Andrew Smith. If you are in or around Norwich or you can get here, we run the Search Norwich events every two months, you're more than welcome to join us, it's free to attend, you can find out more at If you would like to see the transcription of this episode, get Andrew’s slides from his talk or watch the video of him doing talk, you can see that at and you'll see write-ups transcriptions, all the links, resources for all of the podcasts that we've ever done. With that, I'll leave you and I'll be back with you on Monday 25th of November, have a great week.

More from the blog

Get in touch

Please call us on
+44 (0)1603 957068

or email
[email protected]

Alternatively, if you fill and send this form we will get back to you shortly:

Artboard Created with Sketch.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.


Thank you for your enquiry, we will take at look at your request and get back to you shortly.