DoubleTrust Experimental Search

This blog concerns a small web programming project I wrote over a weekend (my first !) to combine and present search results from leading authorities in compact and useful ways. Its more of a learning experience, but hopefully its useful too !! I've been using it for my personal use with some benefits.

Tuesday, May 31, 2005

Rainbow Experimental Search launched

I really wanted to learn web programming since the last one year, so finally spent some time on it this weekend. As learning by doing is the best way, I decided to create a search hack Rainbow Experimental Search, an idea which had been going around my head for a month.

Initially I was just curious to write a small script to understand how useful the common results from Google and Yahoo can be. As it started to take shape, I think it now incorporates the following main ideas:

Relevance opinion from two authorities is better than one

The main idea was to see how the intersection of Google and Yahoo would look and present them in a useful way to be of practical utility I wanted to know that are the results from these two search engines very similiar or different. If they are different, then am I missing out on some good links not covered by one engine ? Different people had different opinions. Many thought they would be quite similiar. But it turns out, its somewhere in the middle: some results intersect , while many don't. Fraction of each depends on the search term. Popular terms like "movies" show lot of intersection. Searching for "virtual machine adaptation" doesn't.

Showing the intersection shows the most relevant results according to BOTH engines. Then the ORPHANS show the difference. Interesting results can be gleaned from the ORPHANS as well as the INTERSECTION.

Presentation Format

I was also curious how can we show results in a very compact form while still being very useful. In google or Yahoo it can take a while to scroll through all the results, esp if I want to see 20-40 results for a query.

Also wanted to create more choice in choosing results other than the ranking. The current results color code domain type, and show size bars to easily select a page with most content: useful if you are searching for lot of material on some topic and want to compare based on size.

Current format is an experiment with a table format with DHTML balloons to give a concise representation. The idea is to clearly show different page properties against one another, so that the user can make a better informed decision and quickly too. Using Google or Yahoo, it is very difficult for me to identify for example the page with the largest content in the first 10 to first 40 results.

The current format shows:
  1. Popup balloons showing description of each pages. This allows very concise summary of all the results and the user can look at descriptions of interesting pages.
  2. A graph for page sizes: allows very quick comparison amongst different page sizes.
  3. Color coded domains: allows easy seperation of different types of domains like .edu , .com etc.

It is hacked together in Perl CGI and Javascript, both of which I am very new at.
It's got some bugs Im sure. A known bug is in parsing results when searching for some weird queries.

I don't have time to work on this during the week, perhaps could spare time on the next weekend, a little busy with the quarter finishing and preparing for an internship.

Any feedback would be welcome !

11 Comments:

  • At 11:54 PM, Anonymous Geet said…

    This is a neat! I am now curious of a few things:

    1) Where did the Rainbow term come in?

    2) Does your "combined relevance" algorithm have the ability to combine multiple engines like a meta-search but in the way you're doing it? For example, sometimes scholar.google.com returns some results that are relevant that Google's main engine does not. In the context of Rainbow, this leads me to think that being able to choose engines for any given meta-search could be useful (maybe a bit to cluttered)

    3) http://en.wikipedia.org/wiki/Metasearch_engine

    4) What do you think about the number of "clicks" being something that increases "relevance"? Basically, rather than relevance by mostly cited, factoring in a relevance by popularity. This does shift from the meta-search paradigm to a search-paradigm, because data about individaul sites must be stored.

    4) You might find it appropriate that my first search term entered in was: Reciprocal Altruism. It seemed to get the relevant mix of yahoo and google searches.

     
  • At 1:50 AM, Anonymous Ashish said…

    hehe, Reciprocal Altruism is a great term for this !

     
  • At 5:30 PM, Anonymous Ashish said…

    Rainbow is just a name I liked, I am looking for any other suggestions for names, let me know. I thought of Prismo (domain available).

    2. A good idea, though the current focus is just on the two.

    3. Yes, I am aware of metasearch, this is just a very simple combination of results focused on Yahoo and google. Other employ much more sophisiticated algos from multiple search engines.

    This one also allows the user to see clearly what comes from where, what doesn't match and shows it seperately in a table with ranks.

    4. Clicks/feedback definitely can be a good indicator of popularity, I don't know if any search site incorporates that. Though one of my research projects incorporates this notion of popularity (Magnolia p2p search)

     
  • At 5:05 PM, Anonymous A. Seo said…

    Very practicle tool - the lack of customization and "emphasis" options of SERPs was one major flaw with Meta Search Engines -


    Now, just add MSN Search - they are now a major player with intelligent Algos

     
  • At 12:25 PM, Blogger www.inktraders.com said…

    How could ads work on this site like www.inktraders.com ?

     
  • At 1:19 AM, Blogger cc Infopage said…

    Hello,

    I am searching for fresh information
    for my cc Infopage, 30,000 daily updated Information Pages about all kind of subjects.

    It might interest you to know that your blog has been visited and has been read. I hope you enjoy your "Blogging".

    I wish you all the luck I can, keep the good work going!

    Kind regards,
    Jos
    Today's News From & About Google

     
  • At 1:53 AM, Blogger cash at home said…

    Hi i am totally blown away with the blogs people have created its so much fun to read alot of good info and you have also one of the best blogs !! I was just blog surfing and found you! If you are interested, go see my

    href="http://www.domaincity.ws/BlogSubmitterPro.htm">san diego seo
    related site. It isnt

    anything special but you may still find something of interest.

     
  • At 10:14 AM, Anonymous Anonymous said…

    I am very impressed by your blog, It is full of great information. I found it by searching for " seo optimization " on Google.

    I have started a blog directory and would like to personally invite you to post your blog. Its free and will help you in your search rankings.

    if you have any questions email me at admin@big3post.com and I will get back to you within a 24 hrs.

     
  • At 5:38 AM, Anonymous Anonymous said…

    learn nice interesting site -

    [url=http://trailfire.com/lortab] lortab online pharmacy [/url]

    http://trailfire.com/lortab
    [url=http://trailfire.com/lortab] buy lortab online [/url]

     
  • At 5:03 AM, Anonymous Anonymous said…

    look nice favorite site -

    [url=http://trailfire.com/vicodin1] order vicodin [/url]

    http://trailfire.com/vicodin1
    [url=http://trailfire.com/vicodin1] vicodin drug test [/url]

     
  • At 8:27 AM, Anonymous Anonymous said…

    see my interesting site -

    [url=http://trailfire.com/tramadol] tramadol cheap overnight inexpensive [/url]

    http://trailfire.com/tramadol
    [url=http://trailfire.com/tramadol] tramadol drug [/url]

     

Post a Comment

<< Home