The Chatterbox

Search Engine Overlap?

There has always been something of sense out there that the search engines are falling all over themselves trying to deliver exactly the same 10 (or 15 or whatever) sites on their first page SERPs as the other engines do for identical searches. For example, since Google is supposedly the King of search, that Yahoo, MSN, Ask Jeeves and all the rest should be trying to duplicate what Google produces.

This accepted "truth" could not be farther from the truth. Each search engine has its own way of doing things and its own ways of measuring Relevancy, which is what they're all really shooting for.

To illustrate the point I'll mention a July 2005 survey conducted by Dogpile, the meta search engine. But first a little background and a bit of a disclaimer.

Dogpile is a Meta Search engine. What this means in English is that they get results from all of the big four (Google, Yahoo, MSN and Ask), including any paid placement ads, then try to amalgamate that data to "Show you the best of the best" as it were. Since they conducted the study some might ascribe a certain ulterior motive to the results.

I don't buy into this conspiracy theory. Sure Dogpile has motivations for studying and releasing such data, but it doesn't automatically follow that they would intentionally or unintentionally try to skew results of the survey. This is the second such study in the series --the last didn't include MSN since they didn't have their own search engine then. Frankly, the results provided in the Dogpile research match what I've seen with my own two eyes when I've looked at the question across some search phrases that are of interest to me. It also matches data compiled by others in recent years.

Okay, let's get on to the facts and figures. For your reference the full release of the study is here (30 pages) and the summary (1 page) is here. In the earlier report they released only the shorter Whitepaper edition. Warning: All are in PDF format.

I would encourage you to grab a copy of each for further study and to have a historical reference. Try to ignore the Dogpile hype, as that is to be expected IMO.

I rather like the fact that they released a fuller version of the second survey since it gives us a good overview of the methodology being employed, the analysis and even has a good discussion of the assumptions made and how conclusions were drawn. This alone gives the final data considerably more staying power in my view.

What the study proposes to do is try to set a benchmark of sorts in determining what percentage of the same sites rank on the first page (only) across each of the four most popular search engines for a random selection of search phrases. The current study researches 12,500 such search phrases. The reason they use only the first page results, regardless of how many results that may be depending upon each search engine's default, is that it's a pretty well proven point that the vast majority of searchers choose which site to visit from those first page listings. Note that the actual position on the page is not given any weight, only whether the site appears on Page 1 of the SERPs.

The results will likely surprise you, especially if you were under the impression that all of the search engines are chasing each others' tails..

When viewing the totality of all 12,500 searches, the same site appeared on the first page of all four engines a remarkably low 1.1% of the time. Remember, we're not talking about a #1 position correlating to #1 on another engine or anything of that nature. We're talking the First Page here. So it could be a case where a site was listed at #10 in one search engine and not at all in the others, or #1 in one engine and not at all in any others.

Instances where a single site appeared on the first page of three of the search engines for the same term came in at 2.6%, which is still less than the 3% noted in last year's report that only looked at Google, Yahoo and Ask.

11.4% of the time the same site would show up in two of the four engines for the same term.

And an astonishingly (to many) 84.9% of the time a web site's first page ranking was unique to a single search engine.

Let me say two of those again another way, just to stress the point.

If your site ranks on the first page in any of the big four, the statistical chances of it ranking well on a second, third or fourth engine are extremely remote, all things being equal. Almost 85% of all sites that rank on the first page of any engine will not rank that well on any other engine.

While the chances of even a well optimized site getting a first page ranking on all four search engines is a scant 1 out of 100.

What does it all mean?

Well for those of us who are in the SEM industry it means that very, very few sites are successful at getting a top (first page) ranking across all of the various search engines for the same phrases.

For searchers/users it means that they may want to use more than one search engine if they want to receive a better selection to choose from, because each engine is going to give you totally different results the vast majority of the time. Or use some sort of Meta Search Engine like Dogpile, which is of course the point they're trying to hammer home.

Now, what does it really mean for SEO's and SEM's?

It's quite simple really. The results of the study point out, in my most humble opinion, that far too many people have been focused on chasing the algorithm of XYZ search engine, thinking that will work for all of them. When nothing could be further from the truth!

Do you want to know how to rank well on all of the engines? I'm sure you do!

It's really quite simple, and is in fact the same thing some of us have been preaching for years now. Create good content that is geared towards your potential audience; you know, the folks that you want to visit your site because you know many of them will buy from you. Build your site to be the best it can be. The best it can be for users, not the search engines. And especially not for any single search engine.

Make your site a Truly Great Experience by paying particular attention to making it as user friendly as humanly possible. Then throw a well thought out link building campaign into the mix, one that will drive traffic to your site without the search engines, but will also give you an advantage with the engines link popularity measurements and you have it licked!

I know that sounds exceedingly simplistic, and that some of you will think it just has to be harder than this; that there has to be some secret I'm not telling you. You would be wrong.

I'm here to tell you that the above is exactly what I do and every single one of my sites is listed on the first page of at least 3 engines for the phrases that are most prized in their niche. Many of the sites are in that miniscule 1.1% that get listed First Page on all four of the major search engines. And even those where only three engines have me on the first page, the fourth engine always has me listed high up on the second page.

Go figure. Who would have thought it was really that easy and that simple? There's really no need to chase any search engine's algorithm, just build the best site that you can.

Oh, and to the above question about who would have thought it would be that easy, the answer is: Those of us who were around back in the Web's Stone Age when there were a lot more search engines to optimize for, but who realized that you didn't really need to do anything special to rank well across all of them. ;-)