Filter Server Logs to Googlebot

Googlebot is one of the most impersonated bots around. Scraping tools will often use this particular user-agent to bypass restrictions, as not many webmasters want to impede Googlebot's crawl behaviour.When you're doing server log analysis for SEO purposes, you might mistake a scraping tool's 'crawl everything' behaviour for Googlebot behaving erratically and 'wasting crawl budget'. It's important that you don't solely use the reported user-agent string for any analysis you might be conducting, but instead combine this with other information to confirm what you are seeing is actually Googlebot (and not you auditing the site with Screaming Frog).In this post I'd like to offer snippets of code you can play around with to achieve this. You probably shouldn't just copy and paste code from [...]

Could Rotating GIFs Improve Performance?

GIFs are funny.Take a look at the following beautiful gradients in GIF format:The vertical GIF gradient weighs in at 12.6KB, whilst the horizontal GIF gradient weighs in at 63.9KB. (I have wpsmush running on here, but those are the sizes pre-upload. The larger image lost 2.08KB, whilst the smaller lost 37B).Why is this the case? "GIFs follow a compression algorithm that removes horizontal redundancy. So by introducing extra vertical details or noise, we’ve increased the file size of the GIF." - Lara Callender Hogan These images are dramatically different sizes because the GIF format offers significantly better compression in one direction.So what if we were committed to displaying horizontal GIF based gradients, but wanted the performance of vertical GIF based gradients?We're [...]

Learning to Type Faster

You probably spend a lot of time typing each day.Although we get a lot of passive practice with typing, we rarely sit down to work on our technique or drill for speed. As a result, we don't really improve. You'll probably understand this if you play an instrument.If most of your time at work is spent attached to a keyboard, then increasing your typing speed is probably worthwhile. There isn't a neat 1:1 between typing speed and meaningful productivity. Improving your typing speed will not solve all of your problems, as you hopefully spend more time thinking about what you're going to type than you do typing. However, working on your typing speed might just speed up your Google searches, slacking, and inbox clearing enough to get more done each day. Training to Type FasterGet the [...]

Automate Linkedin Stalking Your Own Employees So You Can Have Awkward Conversations About Their Profile Updates

Amazing insight: If your employees are updating their Linkedin profiles, it's one of the better indicators that they've started looking for other work. Now, your employees might be savvy enough to change this setting in their profile first:If this is the case, you won't just see their passive-aggressive profile updating in your news feed whilst you're checking out other roles. You could take some time to stalk their profiles daily in incognito mode, but this could be time consuming. You might be in charge of a lot of employees! You could automate this.Linkedin obviously aren't too fond of being crawled or scraped. Check out their robots.txt, or their scraping policy. If you were going to go ahead with this, here's how it might work:Cron job to download each profile page at regular intervals [...]

Faster Google Penalty Removal

Your website is only as good as Google's picture of it. So, if you're working on a website under penalty and are actively trying to get it out of penalty (or just trying to preempt future updates), you should do everything you can to make sure Google is up to date with the link profile so that their image reflects reality.I've used the following method for just over two years, though I haven't seen it getting any serious coverage (though I'm sure it's quite widely used). In short - get Googlebot to crawl the links that you've removed or disavowed. Penalties and Disavow The problem is that terrible links aren't crawled as often as you'd think. Google have been hinting at this in most Webmaster communications since the disavow tool was introduced: "For a disavowed link to be 'counted' [...]