Inverting the Research Pyramid

Turning the search engine model upside down is a good start in more targeted research.

Rodolfo Carpintier [DAD-Digital Assets Deployment] | POSTED: 03.03.04 @20:56

A search engine is based on power, raw power. Let's face it: there are millions of new pages created on the World Wide Web every week. No one can keep track of it all. Even major search engines like Google are only a small help.

The philosophy of a search engine is to gather information, as much information as possible, and then—based on advanced mathematical algorithms and data-mining capabilities—try to provide best bets to searchers. A graphical representation would be a triangle like an upside-down pyramid: information gathered at the entry points (top two angles) is slowly distilled down to get individualized results (the bottom angle).

The question is, is this the only way? What if I already know, more or less, where the information that I am looking for is stored? Many professionals know their industry well—through the magazines, the blogs, and even the forums that are relevant for their specific business or scientific interests.

Does such a professional need to use always a search engine? Is there not a different way to organize his or her information requirements? Could information be gathered in a much more selective way? Try inverting the search triangle mentioned previously, this time like a pyramid with the single angle at the top.

Industry professionals should then be able to get very selective and industry-specific searching environments, having the ability to experiment with them and readjust them to their requirements regularly. In a manner we would be moving away from the centralized searching offered by the search engine and toward a system that is decentralized and powerful in a very specific manner, without needing tremendous computing power every time.

A sort of "dashboard" would allow users to adjust their requirements from an industry-specific model to specific search and research requirements. Instead of handling millions of pages in a search, this system would handle a very targeted hundreds of thousands.

That there is a requirement for it is part of the reason why IBM has produced WebFountain. While it is obvious that IBM has a captive audience in all of its existing customers of on-demand computing, it has, again, followed the path of the search engines and missed an opportunity to develop what I have just described. WebFountain provides specific search environments, but they are based on the raw power of the system and are provided as an expensive add-on to an outsourcing and computing-on-demand contract.

There is a clear need for the specific search environments I describe, and I believe Intelliseek in the United States and DiceLaRed in Spain are the companies closest to suiting this need.

I encourage other members of the AlwaysOn Network and Zaibatsu to let me know what you think.

Rodolfo Carpintier is an analyst for Digital Assets Deployment.

+ HOME

Tell us what you think of this post using our On or Off rating system. Only your most recent vote will count.

WAY OFF								ON THE $
	[1]		[2]		[3]		[4]		[5]

Join the Discussion

NOTIFY?

I agree with your analysis and suggestion. The current search engines serve the purpose of general purpose search for information but fall short for any vertical/niche/focussed search. In the B2B world it is critical to be able to search for actionable information (which is available) in marketing, sales and other areas - but current search engines search the whole web and the actionable information one is looking for is lost in the thousands of results as those rich-content pages may not be SE optimized for those keywords! Very ineffective. my-personalized-search-engine may be the answer.

Vaibhav Domkundwar
www.better-marketing.com