Google’s new infrastructure

Most people who visit here probably have no idea what SEO even is.  Well, at this point, almost no one visits here anyway.  I’ll make another post sometime about SEO and my history doing SEO work.  I’m currently semi-retired.  Nonetheless, I continue to follow the news and trends with some interest.

Google Caffeine is the new architecture Google started talking about in August 2009.  In and of itself, it wouldn’t cause any significant changes in the results you and I would see.  However, as the new system is faster it should allow some interesting options.  While some have said that Google Caffeine is not now live, I have to believe that the architecture is being used to create some of the results we are seeing during the Mayday update.

The Google Mayday update is the dramatic change in the search engine results many people (mostly webmasters) noticed in May 2010.  This seems to be one of the biggest algorithm updates Google has had.  There are always minor updates to the Google formula that determines which sites will be at the top for any given search.  In the early years, it was fairly clear what the most influential factors were.  I was very successful at SEO (search engine optimizing) during that time.  Over time, especially after Google decided to become a publicly traded company, the influential factors became less clear.  More complexities were added to the “secret sauce”.  It was the addition of some of this complexities that led me to semi-retire from SEO work.  The obvious lucrative sources were much more competitive.  As I’m a full time psychiatrist, I really didn’t feel I could adequately compete with some of the smart people doing SEO work full time.

This most recent Mayday update, however, has some of those bright minds really perplexed.  Ironically, my sites have not been too dramatically impacted by this update.  I did have some fluctuations on some of my sites starting in April, but between all of my sites, they seemed to balance out.  I really don’t have a big enough personal sample, nor did I take the time to explore too deeply, what was being impacted on my sites.  However, just look at search results for some of the common searches I was doing did show some interesting changes I hadn’t noticed before.

Before I get into what I noticed in the search results itself, I’ll comment on some of the things I noticed the minds of Google saying.  After reading this blog entry about improving Arabic searches I keyed in on the phrase “historical search data”.  To me this raised my suspicion of a few things.  1 – Google may be looking at the frequency with which certain terms are being searched; but 2 and more likely – Google may be looking at click through rates in detail to see which sites best match for what people are looking.

Tracking click through rates is nothing new for search engines, but it is something that generally wasn’t viewed as a big ranking factor for Google.  Some of the oldest search engines used click through rates as a major factor in how they ranked their results.  The problem is that clicking on your own sites would raise them in the results, making manipulation of the index pretty easy.  Google must have some pretty sophisticated ways to combat robot-clicking and self-clicking of search results to combat this problem.  Google has been using click-through data for a long time with the Adword program, and rankings for paid members do improve with higher click-through rates; however, people aren’t going to click on their own links in the adwords program because they have to pay for each click.  This problem doesn’t exist in the organic results.  So far, I haven’t heard of anyone doing a test to see if a broad attack of your own sites with robot generated clicks would improve organic rankings.  It’s only a matter of time before someone tests that out.  A new way for black hat SEO’s to spam the index, assuming they can outsmart Google, and of course assuming Google is putting click through rates in the algorithm as a significant factor.

In May 2010, Google started to allow webmasters to see a significant amount of data on web rankings and click through rates.  I believe the way they present the data supports the idea they are looking at click through rates as an important measure of quality in their results.  Clicking a result, though, just means that the title and snippet seem to be what the person is looking for.  Titles are often misleading, but the snippets usually give a pretty good idea of whether the site will have what you are looking for.

Along with the release of this information, many people started to notice that the search engine results fluctuate much more than in the past.  A specific search may show one site as the 2nd or 3rd result at one time, and at another time that site may be on the 3rd page.  There doesn’t seem to be significant rhyme or reason, but I’m guessing this is part of a randomization of the results, combined with tracking of click throughs to see which sites best match the search.  There are probably other ways Google can measure quality of their results – such as someone not clicking any link and doing another similar search, or someone going to a page and then returning back to Google.  I’m sure the smart folks at Google have come up with all sorts of creative ways to track users search habits (the Google toolbar and Google Chrome definitely come to mind – as Google does have a privacy warning and I wouldn’t be surprised if those tools track users browsing habits in some detail).

Other things I started to notice in the search results were the types of results given for different queries.  For example, doing a search for someone’s name used to give a list of a bunch of websites with that user’s name.  Recently, I started to see many more 1st page results which were a variation of the person’s name.  Often a letter replaced with another letter that is nearby on the keyboard – perhaps trying to see if certain searches were conducted as a common typo maybe.  I’m sure there are various categories of these types of search nuances that Google is testing, and probably testing in different ways for different types of searches.

I also think that different categories of websites are being impacted in different ways.  I have definitely noticed differences with blogs vs. discussion forums.  Several of my other websites are discussion forums.  I was noticing that new posts on those forums weren’t showing up for long periods of time.  I started this blog around that time and noticed that the posts from this blog were showing up in the results immediately – and this blog had no backlinks.  I think the current backlinks are only trackbacks from other blogs I’ve linked to – and I doubt those count for much in Google’s algorithm.  Not only did posts from this blog show up immediately, but they were actually ranking fairly decently for the main terms.  I found this kind of surprising.

I haven’t written any posts here that I’d view as competitive terms.  However, my section on Keter Bellevue sheds ranked well for a few days and is starting to drop a bit in the results.  Nonetheless, I got some traffic to those pages with really no offsite SEO.  The pages aren’t money terms and not particularly competitive, but nonetheless, I was surprised as in the past I wouldn’t expect those pages to have ranked as well as they did as quickly as they did.

I’m being paged to get to bed by Mrs. F.  Gotta go and perhaps I’ll post some more on this topic.  I will be very surprised if this page actually ranks as I’d consider these terms a bit more competitive and much more commonly written about in recent days.