Surf Canyon--enhanced Web search

Stringer Tim brought this to my attention...Surf Canyon plugin for the common browsers that can potentially search deeper into the Web. I have not tried it. If you have, offer your opinion and its strengths and weaknesses.

"Digging Deeper in Web Search"

A personalization search tool reveals links buried deep within page results.

by

Kate Greene

January 29th, 2009

Technology Review [MIT]

One of the hottest frontiers in Web search is finding ways to improve results based on the searchers' preferences. Already, when you log in to Google, the search engine tries to personalize results by mining your search history: for an eighth grader who has performed lots of searches on marine life, a search for "dolphins" might provide more results for the animal than for the football team.

Now Surf Canyon, a startup based in Oakland, CA, is adding its own spin on personalization. Its software, which can be downloaded and installed into Firefox and Internet Explorer Web browsers, enhances individual searches on major search engines by evaluating which links you click on, and then instantly giving you revised search returns--including three sites that relate in some way to the site you clicked on. "We have invented real-time personalization," says Mark Cramer, CEO of the company.

For example, a Google search for "thermoelectric cooler" using Firefox with Surf Canyon installed provides 10 standard results. In my case, the eighth result, from freescale.com, a chip maker, seemed promising. I clicked on it, scanned the page, and then hit the "back" button. When I subsequently looked at the results page, three new suggestions appeared directly under the freescale.com result. Surf Canyon had elevated these links from the earlier 100 pages of results because its algorithm determined that these recommendations related to the information on freescale.com, including technical explanations of how thermoelectric coolers work.

Crucially, these new results are cleverly slipped into the search results so that the original results page doesn't look drastically different when a user navigates back. It would be off-putting to users, Cramer says, if they had seen a link in the original results that they wanted to click on but, when they went back to the results, found it missing. Therefore, recommended results only appear automatically below the link that was clicked on. "We don't want to jar the users," Cramer says. "[Surf Canyon is] specifically engineered to be as unobtrusive as possible."

Behind the scenes, an algorithm makes the personalization possible. Among other things, the algorithm analyzes which results are clicked, which are ignored, and how much time a user spends looking at the page. Importantly, says Cramer, the algorithm semantically deconstructs a page to determine what it means and how similar it is to others in the results. The results are cumulative: after a couple of clicks, the algorithm can determine if you're most interested in a Canon camera, an SLR camera, or, specifically, a Canon SLR, Cramer says.

Marti Hearst, a professor at the School of Information at the University of California, Berkeley, says that Surf Canyon succeeds in presenting the reordered links in a clear, useful, and unobtrusive way. It doesn't require people to do any extra work, as does Google's WikiSearch, a feature that lets users personalize their results by voting them up or down.

However, in her test cases, Hearst found that the algorithm's re-ranked results weren't completely useful. "Where personalization works is where queries are ambiguous," she says, but queries have become increasingly longer over the years, and they tend to provide clues that help the engine disambiguate the results on its own. Additionally, in Hearst's tests of Surf Canyon, she found that it only untangled the different meanings of the acronym ACL (which could mean both anterior cruciate ligament and Association for Computational Linguistics) to a certain point: it kept including mixed results even when she felt that her clicking choices had made it clear that she was interested in the linguistics group.

Cramer and his team say that they have gotten more positive results. In a study they performed, some participants saw a second page of search results that were reordered according to Surf Canyon's algorithm, while others saw a second page with standard results. The researchers found that the participants who had access to reordered results clicked on them 30 to 40 percent more frequently.

Surf Canyon