MastHeadRamana Rao's Home PageRamana Rao's BlogRamana Rao's TrailsRamana Rao's Sensemaking

Archive for the ‘Search’ Category

REAP and its kissin’ cousin memes
Thursday, November 16th, 2006

When it comes to posts that go at “Beyond Search is ___”, one has a good chance in the blogosphere, of, well, being mostly ignored … By now, you can more or less be overloaded just by search results for Simon “Poverty of Attention”. Hence my threshold, following my first post on REAP remains high, but this doesn’t by itself prevent some flights of fancy. Think of it as a meme tasting party of some memes that have reached their time

First up is Sensemaking, originating in the foothills of Palo Alto, and as noted, my all time fav for naming the human process itself. It scales to the biggest questions that make us humans, and anthropologists and philosophers alike have been there at increasing levels of abstraction. (more…)

Beyond Search is REAP
Wednesday, November 1st, 2006

… I’m sure I’m repeating myself and plenty of others by now, but it’s time to resurface “the vision” of Information Flow. It’s easy with long days and a real and full life with varied encounters with strangers, friends, and family to fall back on the crutch of calling what I do, “Beyond Search stuff.” But here in my open conversations with you it’s time to remint the concept, especially in the fresh dirt as the sprout of a new company labors to push its head out.

Beyond Search is not the same thing as Search getting better. Search can get better. Boy, can it?! However, Beyond Search is something else. Not just the litany of algorithms and technologies i.e. linguistics, statistics, extraction, categorization, clustering, personalization, visualization, social this that and the other. There is a holistically different experience to be had! And it will be at least as valuable as Search to some people, and that some is a huge many. It’s everybody that has a long term stake in subject matters and that utilizes such knowledge to get things done in the world. It’s poets, priests and politicians; it’s lawyers, doctors, and professors; journalist, analysts, and researchers; and it’s business professionals, financiers, and even thoroughbred horse rearers.

At PARC as we saw a future shift from document creation to information access coming, we, speaking Academicese called it Intelligent Information Access. And at Inxight, in Marketingese, we called it Discovery. And in the idiolect (and that doesn’t mean a language spoken by an idiot) of Ramanese, it’s the difference between “Users Chasing Documents,” and Information Flow enabled by “Statistics over Statements.” How we utilize information available over the Internet in professional activities is still fundamentally challenged. I think everybody can tell their own version of the story.

In throwaway lines, I’ve said to many, “you could called our new venture, a Vertical Search play, but it ain’t vertical, and it ain’t search.” That line is a disservice in its aint-i-ness, because things worth doing should be expressed in the positive (hmm, maybe). Now, the drum roll … we are building a REAP system (now you got it, huh?), but REAP is much bigger than us, much, and so I’m going to start talking about the bigger.

Beyond Search is REAP. REAP acro-expands into Retrieve, Extract, Arrange, Present. And REAP systems help people support that full process using those technologies mentioned above. Certainly, any of these verbs can be picked on, and many alternatives like Collect, Analyze, Organize, Keep, Share, hey even Remix, so fly to mind. But in aggregate, the point without niggling, is that we’re still not there with the deeper tools and the whole experience we desire for truly reaping the value from all that’s available to us on the Internet. Consider the typical information work flow of a professional:

  • Retrieve — collect information from a variety of sources
  • Extract — extract data, facts, examples
  • Arrange — arrange documents and facts for use now or later
  • Present — compose information into artifacts of value

Search is just about the retrieve and the -eaping is pretty much left to the person. Does knowledge worker come to mind? My all time favorite word for the human process (hat tip of fifteen years back to friends Dan Russell and Mark Stefik) is sensemaking because it reaches to even the deepest levels and the broadest extent of what we are after. Some of my friends, especially these two—actually pretty much all of my friends and yours too—are incredible sensemaking machines.

We all use the aggregate of tools and means at our disposal to make sense of subjects and the world and to guide our actions. However, I’m sensing a widespread frustration with a great deal of wasteful manual labor as we die, gently reaping. And though I am a huge believer that doing things by hand often gives you a depth of understanding, there’s a lot I don’t do because of the effort required. The difference between productivity and new capability isn’t always in kind, but often in degree. If you can take many more steps, you arrive at places you’d never get. And I’ll continue to invest my time on the bet that huge social opportunity lies ahead in making us all smarter by mainstreaming REAP systems.

Speak up, come along …

Thoughts on Search Panel at AlwaysOn Stanford Summit
Friday, August 11th, 2006

At the AlwaysOn 2006 Stanford Summit in July, Bambi Francisco of Marketwatch moderated an excellent Panel focused on what the Search Guys were learning from the data they have on our search behaviors. The panelists were Usama Fayyad, Chief Data Officer, Yahoo; Jim Lanzone, CEO,; Peter Norvig, Search and Quality Director, Google; and Michael Yavonditte, CEO, Quigo

Blog posting, by Dan Farber, Renee Blodgett, and Francisco herself, cover the panel well, here mainly I’m adding my reactions. Noticeably missing was Microsoft, but thrown into the mix, is Quigo as an Ad player. The panel was mostly devoid of company posturing, though plenty of data-supported commentary, mostly true to respective corporate positions.

Relevance and Advertising. Let’s start here, since really, no matter how much beyond search talk gets dished out, standard consumer search, i.e. QIRO (query in, results out), with its marvelous money printer, advertising, is not fading soon. The pressure will remain on market size, market share, and monetization of share. This means an ongoing obsession with extending the range of queries that “relevance” on top 10 result pages serves well. And on increasing the relevance of the ads to get greater return on marketshare. All panelists, mostly indirectly, reflected an alignment to this business reality.

With Quigo buying millions of keywords, Yavonditte emphasized Search works quite well now. Lanzone pointed out that Ask shed the fancier “natural language” approach with 10 words that wasn’t really doing better than two words queries on Google, and that they are now there with a search engine that produces world class relevance with two word queries. Fayyad pointed out, even amidst “change the game” and “search is not enough” comments, that they had to be careful not to mess up search, and I think he was alluding to more than whether it works for finding things or not. Sure, search works well. For what it works well for. Though all the panelists showed nuance, not utterly blinded by economic interests, their organizations are subject to such realities.

It’s not too early to see that the incumbents will face their own version of the innovator’s dilemma. Microsoft has the best incentive for disrupting the current model, because they have other ways to make money. Yavonditte pointed out that Microsoft was doing an interesting thing in sharing some of their data with their advertisers. Google must find another rocket ship that goes straight to the moon or other places without ordinary gravity, while evolving quickly to prolong and parlay the golden days of their current weightless franchise. Yahoo, with its stash of login accounts, range of Internet services, and opportunity around social media, is in-between.

Richer Interaction. These current interests aside, all panelists talked of better interactions for the user, albeit cautiously not just for financial reasons. Lanzone made the point that lots of things that sound like good ideas don’t necessarily help the user. People don’t like messing with a bunch of dials and buttons up front. They like to get going and then refine things as they go. The simple white search box allows them to do that.

In general, nobody seems to be using user profile data to improve relevance for a given user. Both Norvig and Lanzone explicitly said a user’s search history isn’t used for this purpose and isn’t likely to be useful. However, Ask and the others are all down the path of providing more than “ten blue links.” Enrichening the dialogue loop, not so much on the query in, but exactly on what comes out. Norvig used the common metaphor of the dialogue with a librarian.

Vertical Search Lanzone commented that nobody wants to remember 26 sites to go to search for different things, while Norvig pointed out that custom interfaces do make sense, as in Finance, and in Travel. These apparantly for and against points are really both for points. For effective user experiences. A single point of entry is about minimizing user load on getting started. While the right interface provides the scaffolding for the user in getting something done, a good kind of rigidity.

Supporting effective information seeking depends on both the content and user side of the dialogue. Content in certain domains is quite, say, slanted, to the point that it can even look like a separate language, e.g. say medicalese. Fayyad talked about the value of knowing the context you are in when searching, using the example of searching for Jaguar on an automobile site. Norvig certainly would emphasize that with enough data you can start to pull structure out that just wasn’t feasible to do so before. True enough, but as Fayyad points out in a separate interview, an ounce of knowledge is worth a ton of data.

Same goes for competent user experience design. It’s not clear that Google or the others have all the advantage because of content or query streams. In specialized searches, solid information architecture and functionality design around structured search and faceted navigation can be created by newcomers. The biggest advantage the incumbents have is being the starting points of Internet experience. The first box doesn’t mean one box is sufficient, but it does mean control over other boxes.

Portals and Social Media. Yahoo has broader and deeper Internet login services than any of the others. Knowing what people do in many aspects of their Internet life has got to be worth more than having ever more data on just queries. In the old days, using data to improve products would be great, because people paid directly for products. In the new days, you have to figure out how to connect data to money through advertising. Fayyad pointed out ownership of a greater share of time allows for more effective repetition of advertising. But, surely, we can get beyond not just search, but beyond advertising as the only model too.

Though people are willing to pay for experiences or experience goods offline, direct pay models on the Internet are still far behind the advertising model. Music hasn’t quite yet blazed a success yet, but I think it will. Meanwhile with video, as Norvig, Lanzone, Fayyad all pointed out, it’s very early days. Mostly just emailing around of links to outrageous videos of the day. Here too an obsession with advertising with Yavonditte pointing out that these early days pose challenges to advertising. Sure and such challenges will get sorted out, since some portion of video media will make sense under this model.

In sorting out how to support our broader digital experiences, Yahoo has the relative advantage, though my understanding is that they are viewed as uncool amongst teens and young adults. Shocking perhaps for a company that is not much over a decade old, until you read this as not a changing of old minds, but a generational shift to new minds. Though this talk of young and cool may seem to stick us to advertising models, perhaps not, consider games.

Given that time and attention are the bounded resource, it seems likely that people will be ever more willing to pay directly for digital products or Internet-based experiences. We all pay for computing and communication infrastructure now. We can see the beginnings of consumer pays in Internet games, music, subscriptions, hosted service extensions and so on. Ultimately, direct pay is necessary to create the broader range of products and services that work well across the niches and slices of life.

Friday, July 25th, 2003

Steven Johnson’s Slate article on some weaknesses in Google created lots of noise in the discussion area. He explains his intentions on his blog.

The three so-called google holes are pretty well understood among people that really understand Internet Search, but not so by most Internet users. Most people do tend to carry many misconceptions including: 1) there is a right answer to a search 2) google is better than other search engines by a long shot at coming up with the right answer and 3) searching a universe of everything makes sense. Though the three google holes that SBJ is pointing at don’t exactly correspond to these three points, along with the blog comments SBJ’s holes cover these points and more.

Over time people are getting more sophisticated about the limitations of Internet Search engines as they get more road miles behind them. And thus will start to entertain solutions that involve a bit more knowledge on their part. I’ve been saying for a long time that users won’t do more than one word queries, but it’s coming time to pull the other way. Somewhere between one word queries and a bard’s 14 line depiction of injury at the hands of a lover is the possibility of some elaboration by the user.

Similarly, we can start to expect more from the systems. Why should it deliver back the myth of a uniform list of results from a universe of equivalent things? Especially when it “knows” that it has lots of different kinds of things (not just several kinds of apple, but also many different formats and genres and kinds and ages and orientations of sources and documents). And that even if the most likely thing users want is one thing, that in fact most users will want one of many other things. Why not present this structure and diversity back to the user in a way that the user can better understand the options and then help themselves.

Google is fanatically loved, but as much because the first round of Internet Search companies forsook search. And of the next round, Google executed better than anybody else. Particularly in staying focused on search and in endearing themselves to their users. Hat’s off on that. That doesn’t mean that they have solved the problem of finding what you are looking for in all cases. There are yet miles to go …