Candour

Episode 97: Site monitoring with Dom Hodgson from Little Warden

Play this episode:

Or get it on:

What's in this episode?

In this episode, you will hear Mark Williams-Cook discussing site monitoring with Little Warden founder Dom Hodgson. Together, they'll cover the importance and use cases of site monitoring, the history of Little Warden and Dom's other wild projects for charity, how companies like Google and Microsoft have dropped the ball with monitoring, why prevention rather than cure gives the best return and use cases for web developers, digital marketers, and agencies.

Show notes

https://www.businessinsider.com/this-guy-bought-googlecom-from-google-for-one-minute-2015-9?r=UK

https://www.rundom.run/

https://www.facebook.com/hodgsonlights/

https://www.penguinawards.com/

Transcription

Welcome to Episode 97 of the Search With Candour, podcast recorded on Friday the 5th of February, 2021. My name is Mark Williams-Cook and, in this episode, we are speaking to Dom Hodgson from Software-as-a-Service site monitoring tool, Little Warden, and we're going to be chatting all about the benefits, the pros, the cons of monitoring sites going down, what happens if your domain expires and someone nicks it. We're going to be getting into that and looking at why this is something we should be doing.

Before we kick off, I want to tell you that we are sponsored by the lovely people at Sitebulb to do this podcast. Sitebulb is a desktop based SEO auditing tool for Windows and Mac. Every week, if you listen to his podcast regularly, which I hope you do you'll know I normally go through one feature that I particularly like about Sitebulb. This week, however, I just want to talk about it in terms of where it fits into our stack, because we use Sitebulb. We've used it for a long time at the agency and it's good to talk about it in context to a tool like Little Warden that we're talking about today.

Little Warden obviously is this Software-as-a-Service tool, as you'll find out it's like, if you're doing a health check, Little Warden is like the temperature check on your forehead that you're doing every day to make sure that you're okay. Sitebulb is a tool that you run, you choose when to run it, it's not like it's always running in the background, but you can actually schedule crawls with it. But from our stack, what we do is, we can use tools like Little Warden to basically find out if there is some issue going on.

As soon as we detect something, this is where you can roll in a tool like Sitebulb, hit go, and it does these huge, really broad audits that cover everything from internal linking to page speed, even touching on things like security. While it's not a security pen testing tool or anything like that, one thing we'll cover in this episode is just that, the importance of the overlap with these tools, because just having one type of check for something can give you blind spots, whether it's because certain people only have control of them or just because he gets missed in that tool.

I really enjoy using Sitebulb as part of our toolkit. The great thing is they have a free trial for listeners of Search with Candour that’s extended to 60 days, so you don't need any credit card, any payment details. If you go to sitebulb.com/swc, that’s sitebulb.com/swc, you'll get this extended trial. Go give it a try now!

In this episode, we are lucky enough to be joined by Dom Hodgson, founder of Little Warden. Little Warden is a site monitoring tool, it's designed for people who are responsible but might not have full control of lots of sites and domains. It does things like expiration checks, status checks, change monitoring, indexability, content checking, sitemap validation, monitoring, malware detection, and all sorts. We're going to talk to Dom about that. Dom, welcome!

DH: How you doing, mate? Good to hear from you.

MC: Very good. Thank you. So where I wanted to start with you, and I thought it would be a lovely place for the listeners, would be, for the people listening, I'm talking to Dom right now, and he's sitting in a shipping container with millions of tools behind him, which I think is a great start because you really like making things, right?

DH: Yes. I'm not very good at it, but that hasn't stopped me yet. I like creating things. I like projects. I consider myself a hacker at heart, not in the sense of the 1980s film Hackers, it's where you hack stuff together. You have an idea and you produce something.

Many, many years ago, before kids, and when I could drink red bull, we did things called hack days in the UK and around the world. That is, there were events where you have 24 hours, you came to an event, you have 24 hours to produce an idea and then pitch it. Then your fellow competitors or judges would judge. I was part of a team, we were pretty good at that. We won a couple of PayPal events where one of the prizes was to go and compete in San Francisco at the PayPal conference and speak there, which I've done. We that won that one as well, and I consider myself PayPal world champion, but they've asked me not to say that but I keep mentioning it as much as I can.

MC: You've said it now.

DH: I have, yeah! I really enjoyed that because you had that burst of creativity, that rush. The first few hours of working on new project is just so full of promise and hope and excitement. We just like to keep doing that, much to my wife's dismay.

MC: I think it's worth mentioning. You're quite well-known within the SEO community for a lot of the charity work you do. So I've personally watched you sweating on a treadmill in various costumes, and I've watched you on Instagram doing a much better job soldiering than I do, um, with lots of crazy lights. So do you want to just tell us a little bit about that? What the charity stuff you've been doing is, and maybe what plans you've got for this year.

DH: Yeah. I support a local children’s hospice called Martin House, and I supported them for many, many years. For the last five years, I'm on a proper campaign of ‘let's do something that goes beyond’. I'm one of those people that when I ask for money from somebody, it can't be doing something that somebody else has already done. I can't be, I'm not judging anybody, but everybody I know does a 5k, everybody I know does a 10K.

For me, if I was going to ask for money from somebody, it can't just be a 10K. Iit needs to be something that causes pain, either emotional or physical pain. It's like people that ask for money to go to do a skydive. It always gets me a little bit because, you know what, you want to do a skydive and it’s something to tick off your bucket list so, really, I'm helping you do that. You're enjoying it! You shouldn't do! If you want my money, a sky dive where your parachute only lasts a hundred metres. Have a bit of risk involved, come on!

Every year I try and do a little bit more. That's another thing, I can never ask for money to do the same thing twice, I always have to go more. The first year we started off doing two half marathons in costumes. The next year, I think we did five or six, then the year after we.. anyway, last year I did 26 half marathons and each one was in a completely different costume. Because we couldn't really leave the house, each one was on the treadmill. We live-streamed, we chatted to people, we played games on the live stream, while doing the half marathon. Now, I will point out to your listeners, that a lot of it wasn't running, there was a lot of walking involved because I don't say I run half marathons, I say I do half marathons.

I don't know if you realise it, but when the London marathon was canceled, there was this 2.6 challenge where everybody did 2.6 and I was thinking “do you know what? I'll just do 26 half marathons, it's fine.” And I thought, “oh, that'll get a lot of press attention” but, by the time I'd finished it, nobody remembered the 2.6 challenge because social media. Nobody remembers that Bernie meme’s, it’s gone and dead now, it's just so quick.

MC: I think that’s a good lesson to learn, that the internet's attention span is very short, But you stuck it through, right? I remember seeing your final one.

DH: Yeah, we finished the final one on my birthday and we managed to find a birthday cake costume. One of the toughest parts about it is, you can see this cause we're on Zoom right now, but I'm not in peak physical condition. A lot of people do comment on the fact that “how can you do 26 half marathons where you’re the size of a small bus?” I will tell you that it's just determination. This just goes to show, if you don't concentrate on food when you exercise that it doesn't matter, you won't lose weight. There’s a bit of a health tip for people. Food is very important. But yeah, we finished it on my birthday. We're taking a year off of running this year. I do have something special planned, I just need to look through the logistics. We also do the Penguin Awards, which you've known about before. I believe you've won one haven’t you?

MC: Yes. We won the Best Agency Dog.

DH: I love awards but they're quite expensive. They're also a little hoity-toity. You pay £170 to enter, you get shortlisted, you go to a swanky London venue where you pay £100 for a ticket, you get your sit down, get your suit, and get your hotel. That always felt a little bit much for a lot of the people in the industry. I was like, right, let's just do something funny, so we created the Penguin Awards, which are £10 to enter. 100% goes to charity and we have categories such as Best Dog or Best Office Pet, because we were more inclusive this year. Best Industry Beard, Worst Industry Beard, Messiest Desk, Worst Client Pitch, Best Spam Email. We just have categories like that.

We have sponsors, again, all the sponsorship money goes to charity. We send out legitimate trophies, we pay for it. It costs me so much in posting; a lot of the winners last year were American and, oh my God, sending trophies to America. I should’ve bought them in America. Next year I'm just going to wait until everybody's one and then order everybody a generic trophy with their label on it. We get them all engraved and what's funny is people's Zoom calls now, and I'll see in their background, their Penguin Award next to their Search Award and their Daddy Award, and all that makes me proud because it's just a bit silly, but we raised a lot of money with that, just with something fun with the Penguin Awards.

MC: Wow. So I think it's fair to say that Dom is quite the force for good. In the show notes, at search.withcandour.co.uk, you will be able to have a look at all the links for the various projects Don's been doing and how maybe you can get involved. Since you're here why don't we talk about Little Warden for a bit.

DH: That’d be nice.

MC: I think it'd be a good place to start on Little Wardern about, I introduced it as obviously a site monitoring tool, just give us your views, very briefly, on where do you see that setting, fitting in with SEO agencies, freelancers, or even web developers? What are the most common scenarios that you see Little Warden being used?

DH: Yeah. We designed it with agencies in mind, because I used to work for an agency. I saw some of the issues that were there. An agency is designed to focus on billable hours, to focus on billable things. What is the next piece of work that the client wants? And that's all great, but sometimes things get missed, because it's nobody's responsibility to check for the domain name, the agency doesn't look after the domain name and that's fine, that's not your remit, but I often say to the agency, “okay, how much SEO work are you going to get billed for if their domain name expires?” It's just embarrassing, isn't it? It's like when you go home for Christmas and your mum thinks because you work on the internet, you know everything about printers.If somebody is doing your marketing, in a lot of cases, there is an assumption that they're looking at the domain names and they're looking at things.

Also things that can help you with no effort. I'll give you an example, say, ao.com. Every year, ao.com leave their old domain name, appliancesonline.co.uk, they leave it to the last day to get that renewal. We monitor about 10,000 brands and we just keep a look out and occasionally we'll just tweak them and go “Hey! Your domain…” ao.com panic me every year, because all it takes is that renewal script, one year to fail and 90% of their links are going to appliancesonline.co.uk.

If you're an SEO agency and you lose the old domain name, you lose the old redirect, that is a lot of equity that you're losing. So what we did was we created something that was literally fire and forget. You paste the URLs, you spend a couple of minutes configuring it, and then you leave it alone. We should never have to alert you, you should forget about Little Warden until something messes up. That's how we, that's how we've designed it. That's basically what we perform is that pre-launch checklist, because everybody has a pre-launch checklist and they go through it, what they don't tend to do is go through it six months later. What we do is we can do it every hour or we can do it every day, and that's what we look for. I'm trying not to make this into too much of a pitch, but that’s my introduction.

MC: It was a good introduction. I think you touched on a few important things there and things that I've learned over the years. So I've been involved in agencies for quite a while now, about 15-16 years, and one thing that I think that I talk about, especially to newer agency owners, is remembering that, at its core, it is still a customer service business, regardless of what you're providing. The interesting thing you said to me about the domain name and my view on this is if the domain name expires, nobody really cares that it's not really your responsibility to look after that, it's still your fault. If you're doing the SEO and those small things, helping clients stay on track with the things that, technically they're not your responsibility, really help you out in the long run and keep clients staying with you.

The other thing I think that a lot of SMEs don't realize, we'll talk about this more later is that, if you've got a domain name, that's got lots of links or is popular and it ranks well, there's a queue of people waiting in the wings to grab that domain, if it does expire. it's not the case of, “oh yeah, well, if the domain does expire well, we’ll re-register straight and we'll be offline for a day.” That's not going to happen is it? If these domains drop, someone is going to be in there very quickly buying them. What's the situation then as a small business, you know, if someone's grabbed your domain?

DH: We've seen this happen time and time again. If somebody grabs your domain and you've got a trademark and you've got a brand authority, it happens to Microsoft. In the past five years, we've seen it, Microsoft, Airbnb… I've got a list that I didn't prepare, but I'll tell you last year, Mark 2, the agency, that made headlines, it made BBC news that they forgot to renew the domain name. They're worth other a billion dollars and they forgot to renew the domain name. It's just embarrassing, if you're a site and you forget to renew your domain name, it's embarrassing.

So a few years ago, we were talking about hack days, I used to run a hack day in Leeds called Leeds Hack. It was really popular, it was one of the popular ones. I woke up one day and I was updating the WordPress blog and I logged on and I just saw a lot of Chinese characters and I was like, “hmm, that's new”. Then I just went to /wp-admin and I was like, “Oh, 404, this is not good.” So I looked at the thing and I was like, “Oh I don't own it anymore”. So I bought the dotnet and basically there was no way I was ever getting that domain name back. Subsequently, they have tried to sell it back to me for five grand because that's how it works, they'll get the domain name, they'll see if it's got any traffic, they'll try and utilize it for affiliate stuff or link selling. If they can't do that, they'll try and look at who owns other domains and try and sell it to them.

MC: How much would they have paid for it if they're selling it back to you for five grand?

DH: They would've paid the registration fee.

MC: And how much would that be, for the benefit of our listeners?

DH: Oh, $10.

MC: They're grabbing it off you for 10 bucks and trying to sell it back for five grand.

DH: It's a whole industry called drop catching and it is and absolutely competitive industry. Take UK domain, for instance, when a UK domain name expires, it goes into something called pending and that can be up to 60 days. After that, at any point it can drop and there's no sort of like time. So they aren't people, there’s a service called drop catch, which will check Nominet every second, every millisecond to say “is this domain name available?” and you pay them a fee, if they catch it. We're talking, this industry is millions of pounds because domain names are expensive and people are buying servers closer to Nominet to try and get that lower ping times from getting that connection. It's a bit like the stock market.

MC: Yeah, I knew they did that in the stock market. I didn't know that was a thing they did for drop catching as well.

DH: Yeah. They look and try and utilise as small latency as possible. That's the sort of thing that, if they're in this industry and they they're doing that on the .coms, the .co.uks, I’m not kidding you, if your domain name is popular and you don't have a brand like a trademark and you can't appeal it then what are you going to say? The domain name isn’t yours, you’re renting it. A lot of people consider a domain name property, but yeah, you're not buying a house. You're renting it for a period of years.

And so when I made that mistake, I've got a little black book of ideas, I wrote it down, and then I closed it. Then, a few years later, we created Warden and it's amazing how many people I'll talk to about Warden. They'll go, “oh, I had the idea to do that.” I was like, “oh why?” “Oh yeah, because I lost this domain name” because it happens all the time.

MC: Let's talk about that quickly. How long did it take then from being in that book to being a working product? And was this just you that built it?

DH: This is a funny story. So as I say, I've got a little black book full of ideas. I like to start ideas. I've got a very patient wife, but at one point it was about 2016-2017, she was like, “Right, we need to stop. I need a break. I need a holiday,, and you need to not start a company while I'm on holiday.” I was like, all right, that's fine. So we've booked, we're big Disney fans, we booked a Transatlantic cruise. We went, we did a bit of Disneyworld, we got on a Disney ship from Florida to Copenhagen across the Atlantic. Everything like that, 15 nights. It was absolutely amazing.

However, what she didn't account for is the fact that, oh, and also no internet, well, we did have internet, but 100mb was a $100 or something. I'm from the North... I'm not paying that money. We accounted for that. What she didn't account for the fact that Scarlet, at that point, was one year old and they liked to go to bed early. So you're on a cruise ship, your daughter's in bed, in the same room in the cot next to your laying in bed with your wife. What do you do? Well, you get your laptop out, don't you?

Well, I'm sick of playing games now. I'm not really good at games, so I don't enjoy them. I like working stuff. And I was like, let's have a look at them idea blockers and have a play. Obviously you're in the middle of the Atlantic ocean, you've got no internet, what do you do? You build a tool that scrapes websites for errors. That's the perfect logical assumption.

I started playing with it and I started playing with some code, and I will always say that my wife is the brains behind our relationship. She's got a PhD in Maths. I couldn't even finish university, I dropped out. She is amazing. She's the best coder. What she can't stand is me messing up. She can't start my code a lot of the time, I've got the essence, but what happened was, I had a few nights of “I'll get this working, I'll get that working. And she'd be like, “you did that wrong”. After about four days, she was like, “right, give it here. I'm going to make it, you work on marketing and then we'll make it.” That's how Little Warden was born.

From that cruise ship, it probably took us another 3-4 months until we had our first release and the first release just did status checks, just on domain name didn’t do it on sub-domains and just did it on whois. We've added so many more checks since.

MC: That's something I wanted to talk to you about because I find it quite interesting. In your opinion, do you think websites have more issues nowadays than they used to? It used to be you just connect via FTP and you bang some files up. Nowadays we're like building stuff with Docker and then we've got continuous integration, and it's not a matter of just uploading a few HTML files. I feel like there's a lot more that can go wrong now, between someone wanting to do something on a website and it actually being there and staying there is that you're seeing as well?

DH: There are so many other variables that can go wrong. Especially with all the technology, like you're talking about Docker, we've got JavaScript rendering, loads of people have staging sites, development sites, things like that. Even little things like, the amount of times we see REL canonical pointing to the staging website on the homepage or on everything. We see that, I would say, at least once a week, it's so often because people are utilising the technology. That's what we do, we catch mistakes, we point it out to them and they go, “Oh, well we'll go and fix that.” But yeah, they are getting more complicated.

Websites are also getting bigger and more departments are focused on it. One of the things that we use a lot, and it's really funny, is the SEO team. We do something called block level changes. So you can select say a div or a paragraph object, and say, “right, tell me if that changes, tell me if this word disappears” and we get SEO teams monitoring texts so that the content team or the PR team doesn't change it without them knowing, because they don't trust each other internally and that always amuses me quite a lot.

MC: That's the thing. Firstly, I've seen a REL canonical pointing to a staging site within the last two months with a client and, working with bigger clients, I fully understand that is an issue when you've got multiple teams. When you've got a PR team or you've got a content team internally, an external SEO team, from my point of view, it's not always a trust thing, it's just that not everyone understands everyone else's jobs and is painful when things get changed. Essentially, you look a bit silly because as I said, nobody really cares why it happened, but it happened on your watch. I think that's a really important point here, which is about this, two things actually, which is prevention rather than cure. I think that’s a good mantra for life in anything. It does, in my opinion, apply, especially for SEO, because if things go wrong with your site, there's going to be that lag time between when you fix it and if Google has picked up on that.

Even things like, so you mentioned earlier about how you can do the malware checks for sites with Google's list, that's super important because a lot of site owners, especially SMEs, aren't watching Google Search Console. So even when Google has worked out that something's happened to their site, they probably won't notice because a lot of the sites are getting hacked, they're cloaked, meaning that it's only search engines that are seeing all the dodgy links that get injected. So they only notice when they eventually end up Googling their own site and then the damage is done, it's not a quick fix.

DH: In that specific instance, a lot of people find out when their customers will tweet them and say, “Hey, um, I went to your website in Google and it's just this big red page, which says malware warning and a big DANGER.” So yeah. Google have this ability, they have a big database that we download a few times every day and basically say “right, put a URL in here and is this URL in it?"

The downside is they don't let you put it on a domain, you have to put a specific URL link. But that’s fine because that's how Little Warden works anyway. But yeah, you can say, is it in there or is it not? We'll give it a bit more information about why. There is a delay between finding that and then curing it because you have to say to Google, “please I've fixed it, could you please come and…” Oliver Twist style go to and Google “please Sir, can I get re-indexed? Google is very busy, it might take a few days, but the sooner that you can work that out the better. Prevention is always better than the cure, so what we always say is, when something like this happens, take a look at why it happened, put processes in place and make sure that it doesn't.

MC: That's something actually we talked with Aleyda about when we talked about doing even technical audits which is, once you've identified things, once you've put them in place it's really important that you monitor, they stay that way as well. We don't want those root causes of why things have gone off piece, don't happen again.

Features wise in Little Warden, one of the things I love is and it fits in with workflow, but it just made me chuckle, which is you've got this: the tool works, obviously you put in these URLs, they get patrolled and you essentially have this dashboard of when there's an issue that you need to look at and how serious it is. When it raised issues, I love, we just have this forward to someone else, which is just like, “Oh, there's a problem.” and you just immediately delegate it.

Talk to us a little bit about that dashboard and what people can expect, because I laughed at it, because I do love a good bit of delegation, but that's important because you can hook this up right to the people that are responsible. We can say look, if the domain name or the SSL certificate is expiring, we need to let the IT person know about that.

DH: Yeah, that, that was basically the nature behind that. Let's face it, a lot of people aren't given SSH access to the SELT. So a lot of the things that we monitor with, but you can't fix, you don't have the ability to fix. The first thing that you can do is, and this was requested a lot by some of the clients was, first of all, they wanted to mute something. “Oh yeah. That's not our problem. So can we for a week” and I didn't really want to do that because actually you’re just hiding the problem and it's just easy to go mute and then not send it to anybody. “They'll sort that”, “that's their department” and that takes it into what's the point of using the software in the first place?

So, by saying forward issue, what we do is we send... here's what the current state says, and here's a log of the last few times that we've checked it, here's where it should be. You can put a note in, because that's the reason, because we realized that the person that's doing the monitoring and a lot of the cases, isn't the person that's going to be fixing it. Also before we had teams implemented, which came live last week, this was a way of basically assigning jobs to specific people. We saw that a lot and we do see that forward issue message a lot.

When you log into Warden, we have this thing called outstanding issues and every task that we have, and we try and group any checks that are the same, so if you've got the same redirect on the same domain, we'll check it and we'll say, “right, once you do this, it affects all of them.” Some of our clients monitor a thousand, ten thousand pages on one domain, which isn't our use case, we'll go into that later, but it works.

So what we do is, we automate the group all of their checks together and say, right, okay, in this instance, when you want this and you can, you can disable it. You can override it. Let's say that you've put in your domain name. Let's say AlsoAsked for instance, and let's say we discussed, we discovered that actually you've bought a new website and the domain name has 301’d. Our categorisation is green for all good, yellow for warning, and red for danger.

The first thing we'll say is, we expected best practice for a URL to respond with the status code 200. That will be what's green, but actually 301s because you've rebranded. What you can do is you can say, actually, this is fine, whis is what I want it to do. So that instantly becomes the new green. Then, if that ever changes, then it becomes danger and warning. We've tried to make that process as easy as possible. We've got something we're working on right now called Smart Warden, which is a lot more advanced to be able to try and detect if its a staging environment and try and guess some of those things for you.

But there are just so many, the Internet's a terrible place. There are just so many, race conditions and edge cases and “well actually this is our staging site, but this part of the staging site is open to the public” and all that. It's just such a nightmare that’s been in testing for about six, seven months now, we're still not happy with it.

MC: Yeah, I can well see that. I've worked with one company and they've had completely different environments for their staging, for the same site, we've had some areas that are password protected, like you say, some that we're doing some funky stuff with IP whitelists, three completely different environments for basically the same site. So, while we're talking about benefits, I think it's good just to talk about money because almost every client we speak to, money always comes up when we're talking about SEO or PPC or websites, and we start talking about return and things like this.

I guess the things that are worth talking about are, like you said, you encountered a few people that were like, “Oh yeah, I'm going to build that”. I think it's quite easy for especially agencies just to think “oh, well nah we get the emails, we'll just keep an eye, we'll get someone to keep an eye on those things” or the classic that I've learned the hard way many times is, “oh, well we can build that.!” So just talk us through those scenarios and realistically what happened.

DH: The first thing that we've had is we get this a lot is, “Oh, we don't need you. We've got loads of processes” and stuff like that. A few years ago I had an agency say that to me, their MD was like, “do you know what? I like it, but we don't need it. We're good here.” Then, at 5:00 PM on a Friday, he called me up and said, “right, what's the best deal you can do. We've just had a client lose the domain name. I want to make sure this never happens again.” I just said, I said, “right, if you signed up for a year, I'll give you 20% off. Is that done?” Now that's the deal on the website so he didn't get anything better. They've been a customer ever since. I did think about having an arsehole tax, but I didn't.

That's the sort of thing, but we do get “we'll build it ourselves.” And this, to me, it doesn't make financial sense because if you have a developer that's any good, you're going to be paying them I would say about £60,000, in the SEO industry you’re going to be paying them about £40,000, so it's going to take them a month or two to get at least feature parity with some of the things that you want us to do, such as, let's say you're just doing domain names, let's say, you're just doing a SSL.

Now don't get me wrong, you can write a PHP script to get them done in a day. Right. But you'll have no adding, you'll have no logging, you have no interface, you’ll have no emails. And then, let's say you've budgeted two months to do this, including the crawling. Now, I can tell you that we got a small version of Little Warden up, but it didn't scale. Every time that added things we had to have server and we had some capacity. Little Warden use these nodes and we've got nodes all around the world to replicate other countries.

People often think about the initial cost of development, but if you've got a developer that's £40,000 and they spent two months working on it, well, that would pay for a Little Warden for five years. But then you've got to host the thing. Then you've got to update it because let me tell you, like I said earlier, the internet’s a terrible place. HTML does not make sense. It might look good in a browser, but it'd be terrible to parse. So there will be time the developer might need at least two, three hours a week in maintenance and that time adds up. So why would you not just give that to somebody else? Why not just make that my problem for £30 a month? We really tried hard to fight and make. Little Warden as cheap as possible, because I wanted something that, every year, around April, there’s a tool budget review, and I wanted something that had to be, “look, we need that. I don't care. It has saved us.”

And that is the thing, or it might save us and we, we replaced some there's so much time taken, even if you had one of the SEO executives that's just joined from university. All right. Yeah. “Just go and check all the websites. Just just typing the whois for every domain name of every client.” That's going to take hours when you just buy our software, that's what I say. It just frustrates me, the short-term thinking.

MC: Yeah. I can attest to that from our development of AlsoAsked, the first version we got up and running in weeks, a couple of months. Then, to get the scalable version that properly works pretty much took us the best part of a year. And then, like you say, pot peppered in between that and still ongoing is just maintenance stuff that crops up that has to be done. That's obviously our first foray into SaaS stuff and that's been wildly more expensive and difficult than I thought it would be. In terms of while we're still talking about catching these areas, it's got little wounds, we've got integrations, haven't we with stuff like Slack?

DH: Yep. We've got Slack, we've got Zapier. We have a push API and a pull API. We've got Google Sheets. People want Data Studio and we get this requested a lot. I'm like, “tell me what you want. I need a bit more information because how would you use it in Data Studio?” Nobody ever comes back to me. I will integrate Google Data Studio tomorrow but there has to be a point to it. Tell me what sort of graphs you'd make and how you do it, because I don't use it and I don't, I can't visualise it. But we're really receptive to ideas like that. We want people to be able to use it in all ideas. We've got a Slack bot, so you can actually add checks and remove checks and URLs from the Slack bot. You can talk to robot Dom in your own Slack room, just give him hell and just be mean to him.

MC: I think that's actually really important because emails get lost. I get a lot of emails a day. I get a lot of alerts, so anything like that, having it in Slack for us, and I know a lot of other agencies use Slack, I think it's really helpful. On this, I just can't get off this thing of just ignoring the situation.

Looking at your site, there was a brilliant story on there that I found from Business Insider, this is from 2015 and I'll put a link to this in the show notes, which is this ex-Googler, ended up buying google.com, right. This was a chap called Ved, Sanmay Ved and he was essentially just playing around on Google domains, typed in google.com and it came up as available and he bought it for $12. It didn't last very long, obviously. There was obviously some alarm tripped in Google there, but he had access to the Search Console for Google and could have done all kinds of damage. Like you said, the same happened with Microsoft, with Foursquare, and these are big companies that have IT teams in place that have lots of processes in place. So if it can happen....

DH: I'll give you, I'll give you an example from yesterday and that is Cisco. Cisco has an email monitoring system called SpamCop, where all the emails go through them, they give it a spam rating and then it says, pass it on or not. They let that domain name expire! All of these thousands and tens of thousands of emails that will go through SpamCop. It happens and we see it happen time and time again, it works in the bigger companies because every department it's another department's thing to do.

MC: Wow. So do you have any stories? I don't know if you can be anonymous, of course, about people catching big mistakes with Little Warden, or what are the most common things you see?

DH: The most common things we see are indexability. Sites accidentally pushed to live, either noindexed or they're staging websites being pushed indexed and ranking their main site in Google. We catch it before that happens but we haven't seen that happen. That's such a good sales technique for us because if people tweet about it and all that we get tagged in. We've had plenty. We had one really big client and I got on the phone with them and they were like, “yeah, we're thinking about cancelling”. They were like, “well, this is just not accurate.” I was like, “well, what's going on?” They said our website goes down on a night that all the pages don't work and I'm like, okay, that's fine.

I looked at the log and it was like, right. Okay. So we're saying that between two and four every night, the whole website 503s. That's a bit weird, so I waited up until 2:00 AM and I checked it was like, yeah, your whole website goes down, and this is a massive international brand. It turns out the developers during the deploy time, were just 503ing the whole site for two hours and all the international ones as well, while they deployed to make sure everything was good. Their first complaint was like, “Oh Little Warden must be broken” rather than asking the developers. So they took the Little Warden logs and said “yeah, you need to fix this.”

And that's the thing, we get used a lot with either internal struggles, sometimes lawsuits between agencies and clients. We're Switzerland, we're a neutral third party, we don't make any more money if we find lots of errors or no errors. We're not paid by the era, we just want to make sure that your website is working the way that you think it should. We've seen this happen time and time again. When you were doing the introduction, and listing off those features, it's boring, let's face it, it's boring.

Our little logo tagline was “monitoring the tedious” because you can't get excited about 301 redirect checks. You can't get excited about WW to non WW checks, but they need to be done. WW to non WW is an interesting one, and that was originally because of duplicate content. That's less of an issue now, what it's doing is it's splitting link records we've seen. People are going to the wrong one and then linking to the different ones so that Google doesn't know which one to actually rank. Google doesn't combine the two. Even if you did it in Webmaster tools and said, “look these are the same”, then what does Bing do? What do the other search engines do? Just sticking that one line of 301 just fixes all those issues and make sure that you've got one canonical version.

MC: Apart from it being tedious, I don't think is particularly tedious, but maybe that's just me, I think one of the issues that a lot of companies have is that there's things that you don't know and there's things you don't know that you don't know. What you've demonstrated, I think really well there, with that 503 issue is that there was a company that didn't know they didn't know they had a problem. It's very hard for someone to make a decision to buy something about problems they don't know they have. It's much like security, I've worked with a couple of cybersecurity firms and they always have big issues trying to get into firms to talk to them about security, because nobody cares about security. It's really boring until you get ransomware on your computer and then the most important thing. I think the same applies for Little Warden and people need to think ahead about, look the whole point of this is protecting you against things you're not even aware are going on and those things can be expensive.

DH: It is. One of the other things that we'll talk about is money. We've seen agencies use Little Warden for sales. Two ways we've seen it make money. There's probably more but we've seen people put prospects in there. So they've looked at anyone they're trying to pitch, they'll run that through a Little Warden and, and go, “hi we're pitching you. Did you know that your site does this, this and this. Did you know that we spotted this, we'll fix this.” The other thing that we've seen, and we've seen this a lot, is because you get a lot of URLs with Warden, and Warden’s designed to, ideally, look at domains and subdomains. We're not looking at individual pages, although we can do, but my mantra is, do you know what? If you've got a million pages on your website, then what you want is something like Sitebulb to look at it, do the checks, do the changes and then run that like once a month. You don't want notifications every hour, every day, that a million pages of change. You're just going to get overwhelmed by notifications and you're not going to pay attention to it.

Every day, I get annoyed and we actually do customer reviews where we look through all the notifications that we've sent them and go, how can we do this better? What are we annoying them with? I forgot my point in that. Because URLs are quite cheap with Warden, if a client leaves, we say keep them in. Keep them in and monitor them. What we've found is that people six months down the line have gone, “oh, that client we lost, the website just noindexed or they've just lost the subdomain of the most important brand with all the links. Well what's going on there?” and they just sent them an email saying, “hi, I know we don't work together, but we just want to know, are you still doing this? The next time that contract comes up for renewal, that's quite an easy conversation to have, because you become a proactive agency rather than a reactive and that’s something that we should all be encouraging.

MC: I thought you were going to say that they get the alert, buy the domain and then try and sell it back to them for $5,000.

DH: Well, we both know that in this industry, if you didn't have ethics, you'll make a lot more money. We've always tried to have ethics. We've always tried to be reasonable. That's why, if you look at our Twitter accounts, sometimes we'll just tweet brands and go, “Hey, just to let you know your domain name is expiring.” I'll add companies on LinkedIn and just say, “Hey, just to let you know, your domain is expiring” and there's no sales message there. Normally they're embarrassed, so I don't even get a thank you, but we just try and be a force for good on the internet and create a little bit of joy.

MC: I think that's a really good place to end it there, Dom. Thank you so much for your time. It's worth saying that you've just started a free trial of Little Warden, so anyone can go sign up, you don't need a credit card or anything, littlewarden.com. You can give it a go, put some URLs and get them patrolled and see if you like it. I even saw that you can book a one-to-one with Dom.

DH: You can listen to the sultry sounds of my voice for 15 to 20 minutes while we go through the products, if you want. But no, thank you very much for the time, mate. I appreciate it. I'm sorry it's taken me so long to respond to you much lightly. As a product owner, the free trials were actually meant to do it two years ago, but it's one of those things where you're like, oh, we only get one chance. So we’ll just launch this feature, we'll just launch this feature and then we'll just do this. Patrick from Sitebulb, who I'm sure sponsors this show, but he’s not paying me, has been bugging me for over two years to launch it. Every time I see him, when you could meet people, that's the first question and the last question, “when’s trials?” But thank you very much, I really appreciate it. I look forward to not listening to me because I can't stop my voice.

MC: Brilliant. Thank you, Dom. You can find Little Warden at littlewarden.com or just Google it, of course, it'll come up. That's everything we've got time for this episode. We'll be back in one week time, which is Monday the 15th of February and hope you have a lovely week.

More from the blog