Announcing SEO Sleuth

Tuesday 8th August was an adrenaline pumping day for me. Sitting on the ferry I was going through newly downloaded files on my laptop's desktop and came across these large gzips a mate had sent me that morning. I was thinking it was going to be that Google N-gram data. But browsing through it, it was clear these were searches, millions of them! I let out an involuntary audible expletive and then day I wondered round the city thinking up things to do with the data.

The most obvious site, it seemed, would be one that allows ordinary people to read this goldmine of personal information, instead of just people who knew how to read large files. (I was surprised by a number of news stories on the subject where the journalist disclaimed that they hadn't seen the data themselves). But I thought that was a bit unethical. As it turns out, there's no shortage of such sites out there now with dontdelete.com and aolsearchdatabase.com among the more popular.

I also started to see more research-oriented sites crop up that looked at it completely in aggregate - what proportion of people click which ranked result for all terms, how long is the tail and such.

With my background in search, what was really interesting for me was the possibility of generating search engine reports for any site. Except for the rare occasion when a log reports are made publically available, we really don't know what people search for to find our competitors' sites.

And the results are certainly interesting. For example, The Open Directory Project's incoming keywords are predominently adult. The top term is actually "adult" itself.

Next thing I did was a transpose: for given keyword, it will tell you which sites get the traffic and where they rank. Obviously simply performing a search and seeing what ranks best is the traditional way of doing this, but very often lower ranking sites get more traffic. Knowing both where the results rank and how much traffic they got, and being able to look at the original search (just click on the keyword title) you can then use the tool to work out which sites had the best titles and descriptions (or perhaps brand power).

Anyway check it out. The site as had some success with mentions on Threadwatch and the New Scientist blog.