DoubleTrust Experimental Search

Recent history of DoubleTrust

2005-07-21T21:02:00.000-07:00

I've been very busy with other stuff , and with a summer internship.

Some updates to DoubleTrust have been:

1. Addition of descriptions to the site
2. Change of layout in the search page
3. Addition of some witty lines amidst the orphans. I just like this quote collection I have so I decided to display it ! :)
4. Addition of a user forum for user discussion. I think getting user feedback is utterly imporant for the site to improve. So please leave even the slightest thought which comes to your mind.

Yahoo problems

2005-07-21T20:58:00.000-07:00

After the mention in Businessweek, it looks like Yahoo is blocking search requests from the server after a certain traffic amount. I have no clue how to fix that.

One stupid hack is to alternate between different servers ! That's what I am doing right now. I alternate servers !! when and if I learn there is a problem !!

DoubleTrust getting lot of attention

2005-07-21T20:56:00.000-07:00

Businessweek recently featured DoubleTrust in their annual issue and after that its been getting lots of traffic + emails .

I never expected a simple project like this (initially) to get that popular. It was a small weekend project for my own personal learning and I never thought of releasing over the Internet too. Its amazing how new stuff catches on.

Apparently it also was featured in ABC News three weeks ago. I can't get hold of the TV clip unfortunately, even after mailing the ABC guys.

Compare Toolbar button: The simplest but the most useful feature of DoubleTrust

2005-06-15T15:30:00.000-07:00

Last to last weekend I added this very simple feature: the Compare link.
I think it is one of the most important and useful feature of DoubleTrust. Just drag this link to your browser toobar (works with ANY browser).

Then whenever you are doing a query on your favorite search engine like Google, Yahoo (supports Ask Jeeves and MSN as well), you can just click the Compare button , which will take you to the DoubleTrust search comparison page for the same query. You don't need to enter any further input.

This is very useful if you want to get a quick set of compact results (around 80 !) from the two major search engines for a particular query. Gives you more choice and quick view of both sites.
The important point is that you can still use your favorite search engine as usual and use DoubleTrust as a search accessory, if you really want to stick with your current engine.

Here is the link: http://www.cs.northwestern.edu/~agupta/doubletrust/about.html#compare

More statistics in DoubleTrust

2005-06-09T16:04:00.000-07:00

Domain coverage analysis for Google and Yahoo for 100 results. It can give some idea about commercial and non-commercial web sites coverage of different search engines.

This option is available in the advanced preferences. Just click on more options and select "More statistics" check box.
Screenshots from DoubleTrust

Summary of ideas behind DoubleTrust

2005-06-06T13:53:00.000-07:00

Here is a summary of the main ideas behind DoubleTrust:

1. Main motivation: It is based on the simple premise that trusting two authorities is better than one. By presenting the intersection and non-intersection of Google and Yahoo very clearly, it shows the best results according to their MUTUAL agreement, and also the remaining sites which are unique to each engine (the ORPHANS), giving more options to the user.

2. Trust-o-meter and TrustRank: Meta-search has this problem of doing its own thing with results and taking control away from the user. TrustRank gives user control of the ordering of the common results by defining their own trust level for each engine. A weighted score is then computed using the user's trust options.

3. Compact Interface: Highly compact but convenient representation of around 80 results on page using tables and DHTML, gives the user a quick view of the results and allows quick selection and searching, saving time.

4. More freedom to choose: By presenting important properties of each search page like size, domain etc in a clear comparative representation, DoubleTrust allows the user freedom in choosing the results using other than just ranking. For example, looking for an essay on American History , one may prefer pages with more content say 50K than the smaller pages. A clear comparison using graphics and colors gives the user an immediate way to compare these other properties, which are usually buried in conventional search results.

Sample screenshots from DoubleTrust Website

2005-06-05T22:50:00.001-07:00

About Page from DoubleTrust
Screenshots from DoubleTrust

2005-06-05T22:50:00.000-07:00

The results page from DoubleTrust
Screenshots from DoubleTrust

Feature and debug update

2005-06-05T15:00:00.000-07:00

I worked on DoubleTrust some more yesterday.

Some bugs have been removed and some weird queries work now. But there might be other ways to break it. But most queries work fine

Features Added

I also managed to add some new features. The top features are:
1. Firefox Plugin available now !! Majority of DoubleTrust visitors are Firefox users.

2. Trust-o-Meter and TrustRank !
Meta-search has this problem of doing its own thing with results and taking control away from the user. TrustRank and Trust-o-Meter give the control back to the user on the search engine preferences. Its quite simple: Your trust level decides the ranking of the common results amongst the two engines. If you trust Google more , Google results get pushed up more than Yahoo's.
So you get to keep your beliefs, while you combine results at the same time !!

3. A search box on the results page. Someone had requested this and I think its a very useful feature too. Easy to repeat and refine searches.

4. Can Launch Google or Yahoo directly from the results page.

Known bugs

2005-06-03T10:47:00.000-07:00

A known bug right now is if the result generates a stock quote in Google search (like "DF" for dean foods) , it messes up.
There are some other weird queries where it fails too.

I need to take care of that, will do on the weekend.

Any other bug reports will be appreciated !!

Changed the name and the URL to DoubleTrust

2005-06-02T01:27:00.000-07:00

I got a domain in the name of DoubleTrust.net and changed the title to DoubleTrust ! I think it more accurately reflects the philosophy behind this site.

The logo thing took me some time to figure out !

Rainbow Experimental Search launched

2005-05-31T15:57:00.000-07:00

I really wanted to learn web programming since the last one year, so finally spent some time on it this weekend. As learning by doing is the best way, I decided to create a search hack Rainbow Experimental Search, an idea which had been going around my head for a month.

Initially I was just curious to write a small script to understand how useful the common results from Google and Yahoo can be. As it started to take shape, I think it now incorporates the following main ideas:

Relevance opinion from two authorities is better than one

The main idea was to see how the intersection of Google and Yahoo would look and present them in a useful way to be of practical utility I wanted to know that are the results from these two search engines very similiar or different. If they are different, then am I missing out on some good links not covered by one engine ? Different people had different opinions. Many thought they would be quite similiar. But it turns out, its somewhere in the middle: some results intersect , while many don't. Fraction of each depends on the search term. Popular terms like "movies" show lot of intersection. Searching for "virtual machine adaptation" doesn't.

Showing the intersection shows the most relevant results according to BOTH engines. Then the ORPHANS show the difference. Interesting results can be gleaned from the ORPHANS as well as the INTERSECTION.

Presentation Format

I was also curious how can we show results in a very compact form while still being very useful. In google or Yahoo it can take a while to scroll through all the results, esp if I want to see 20-40 results for a query.

Also wanted to create more choice in choosing results other than the ranking. The current results color code domain type, and show size bars to easily select a page with most content: useful if you are searching for lot of material on some topic and want to compare based on size.

Current format is an experiment with a table format with DHTML balloons to give a concise representation. The idea is to clearly show different page properties against one another, so that the user can make a better informed decision and quickly too. Using Google or Yahoo, it is very difficult for me to identify for example the page with the largest content in the first 10 to first 40 results.

The current format shows:

Popup balloons showing description of each pages. This allows very concise summary of all the results and the user can look at descriptions of interesting pages.
A graph for page sizes: allows very quick comparison amongst different page sizes.
Color coded domains: allows easy seperation of different types of domains like .edu , .com etc.

It is hacked together in Perl CGI and Javascript, both of which I am very new at.
It's got some bugs Im sure. A known bug is in parsing results when searching for some weird queries.

I don't have time to work on this during the week, perhaps could spare time on the next weekend, a little busy with the quarter finishing and preparing for an internship.

Any feedback would be welcome !