The Dào of Search

You used to be expected to type for yourself. But today people have come to expect a reasonable amount of help at even this task. Our phones now help us form correctly-spelled words, our browsers fill in long addresses after we’ve typed only a few characters, and search engines recommend searching for “Humphrey Bogart” after we’ve typed just “boga.”

But not all as-you-type search suggests are created equal. Careful observation seems to reveal three different approaches: completion, suggestion, and instant results. These approaches range in cognitive burden on the one hand, and utility on the other. We’ll look at several examples of each and consider when they should be used.

When a search engine is brought to bear on content with restricted access, it becomes evident that security and preserving the integrity of permissions can be an important and often thorny issue. Consider the case when enterprise search is applied to a document management system. Each and every document might well end up in the same index, meaning that when searches are conducted it is imperative that confidentiality is maintained and users only see results to which they have access.

Facet information displayed with TwigKit as 3D pie chart

Most modern enterprise search platforms provide some inherent capability to illustrate the shape and nature of the data within. Take for example faceted search. Facets will quickly break down the dimensions in all the data we’re storing or even just the stuff that meets our search criteria. In either case we can get some form of statistical feedback e.g. on which top-level categories exist, their names and how many documents each represents. This will not only give the user insight into what information is available, but also guides them in their search, allowing them to slice and dice the data to get to the information they’re after. The question is, how do we best represent this information and make it useful (and meaningful) to us?

Just before Christmas we put the finishing touches on a prototype internal search application for British broadcaster ITV. We’ll be working with ITV in the coming months to roll out the application across their entire organisation, but we wanted to give you a sneak peak in the meantime.

Last.fm's bar list

One of the greatest achievements of modern search engines is the ability to see the relationships between many different facets of the data. This is enabled by what’s known as “deep faceting” – knowing exactly how many results there are for each facet of the data.

The authors of DataFlow put it this way: “By giving shape to data, we… provide access and insight to the hidden patterns of meaning.”

In my earlier post, Precise to a Fault, I argued that result counts are most useful when they are used to indicate proportionality between facets. In this post, I’d like to look at some of the methods of presenting search-based information that help yield insight into how the data is related.

The purpose of search is to help people find what they’re looking for as quickly as possible. Search engines attempt to facilitate this by taking the user’s query and responding with results, placing what it thinks are the most relevant results first.

Unfortunately, pagination doesn’t always do the best job of guiding the user through the search results in the most beneficial manner. I’ve noticed three common problems.

Can a number be so precise that it actually hinders users rather than assists them?

Take search results for example. It is common practice to indicate the total number of results found for a given query. Google tells me that there are 221,000 web pages about “rubber duckies,” while a recipe site indicates that there are 113 salads and 7 desserts with “avocado.”

I can think of two good reasons for why showing the number of search results has become a standard. First of all, it provides a sanity check. If “Barack Obamaaa” only returns 3 results, it is an indicator that I’ve made a mistake in my search query. Second, it helps me compare one collection with another. The fact that there are twice as many two bedroom flats as one bedroom flats in Greenwich informs me about the area.