Search Engine Relationships

Search Engine Relationships

They’re all related. It’s true, you can’t get away from it. Somehow each directory or search engine will use results from other search engines or directories. I will try to explain the relationships as they stand today. However, tomorrow, they could change.

Does it matter that you know all of these affiliations? Not really. My best advice is, if it is confusing, don’t worry about it. You can still optimize and submit your pages to the search engines, you can submit your home page or content-specific pages to every directory and not be penalized – we will explain more about submitting when we discuss each specific directory and engine (because even though they’re related, they all have different rules.)

Here’s what I see today, September 1, 2002. (I’m only giving you the big ones, there are many other small ones and I’ve probably missed some of the relationships.):

ODP or Open Directory Project (directory – free)
feeds the queries of Netscape (Netscape basically “owns” ODP), AOL, Google, Lycos, Hotbot, AlltheWeb, Teoma, and countless others.

Inktomi (search engine – submit through positiontech, must pay.)
provides a database for LookSmart, AOL, Hotbot, MSN, Overture, IWon, and a lot more!

Google (search engine – still free)
provides a database for itself, Netscape, AOL, and Yahoo!

AltaVista (search engine – free submit but also a pay for spidering)
provides a database for itself (owned by Compaq) and AltaVista family of sites and Excite. AltaVista has seen a resurgence of popularity with it’s paid for spidering.

Fast (search engine – free submit but also a pay for spidering)
provides a database for Lycos, Fast or “AllTheWeb” is a new one so look for this engine to partner with others in the future.

LookSmart (directory – must pay for inclusion and not highly recommended.)
feeds the queries at Iwon, AltaVista, MSN, CNN, Juno, TimeWarner.

Teoma (engine)
feeds the queries Ask Jeeves.

Overture (pay per click engine)
partners with Yahoo, Teoma, AskJeeves, Excite, AltaVista, MSN, Iwon, Hotbot, Netscape, AlltheWeb, and Lycos.

Now you can see that some of these overlap. However, just because Yahoo and AOL both take information from the Google database, that does not mean that they rank that information in the same way.

Why do they all interrelate? Because every search site wants to have the freshest data on the net. Some of them compete for the “biggest” index of sites on the net. If they take from different sources, it helps them to maintain a fresh database and a large one.

Another important thing to remember: it is a safe assumption that a surfer will find a particular search engine and stick with it. They learn how to use it; they become familiar with it. The interrelations between search engines and directories does not matter to the average surfer. The average surfer wants fresh content.

It is also possible that a surfer would like to go to an engine and get different content every time. AltaVista rotates algorithms. This means that today you may type in a keyword and bring up a specific set of websites. Then, tomorrow, you can type in the same keyword and bring up a totally different set of results.

This brings me again to the point that you cannot worry too much about what the search engines are doing. It is more to your advantage to create content-rich pages, that focus on one or two particular keywords and submit these pages to each search engine.

More Resources

Search Engine Partners: Another list of Partnerships between engines and directories.
Who Are the Major Search Engines? A short explanation of each major search engine and its relationships to the directories.