Dr. Jay's Blog

Every time we go to New York we buy more equipment

Last week I was out attending Search Engine Strategies in New York. I got the chance to meet Richard Edelman, who had some fantastic ideas for us.  The show frankly wasn’t that great from my limited perspective as a technologist – although it was interesting to hear firsthand what Jeeves was doing after being let go and it was awesome to chat with Chris Pirillo, I didn’t see anything all that new and exciting there from a technology standpoint.

We were about to head to the IBDNetwork Web 2.0 event in San Francisco, and Al and I realized we didn’t bring any monitors for demos. If you saw us at the event, you know that we had the best screens, and, not arguably the best software. Luckily, New York City has one or two computer stores.  Here’s a photo of me carrying the screens from our hotel to the cab we took to the airport.  Why do I always get stuck lugging the equipment?

Last time we were in this situation, it was the night before C3 Expo.  We had all the signs and the materials ready and went to go duplicate them at Kinko’s, when we realized that it would cost us more to duplicate copies of our fliers than just to buy a color printer and pay for toner, etc.  So of course we bought the color printer and did the duplication ourselves.  The result was some great coverage on the local CBS morning news.

The demos at the Web 2.0 event went really well, and we met some very cool people, including Reg Cheramy, who was podcasting.

Anyway, to make a long story short, now I have a nice new monitor in my office.  Now I just have to take it out of the box!  And write an entry about Web 2.0.

-j

On the Road

I’m on the road a lot these days.  I just got back from Seattle, where I had lunch with Eric Horvitz.  Eric has been doing work in intelligent user interfaces for decades and I am always impressed with what he’s been up to.  He has this app that he’s deployed at Microsoft that uses predictive algorithms to infer a future traffic map based on things like the weather, the DOT reports, and events like football games.  It actually has a notion of “surprises” so that when something irregular happens you can know right away.  This was really interesting to me, because if Watson could know when there was something really important that you just couldn’t miss, then we could take special action.

Anyway, I thought his system sounded cool during our meeting, and really interesting from a research perspective.  But there’s a big gulf between research and use, and I hear about lots of things all the time that sound cool in principle (augmented reality, ambient displays, visualization, etc.) but don’t seem to have any impact.  So I was skeptical.  Later that day when we were leaving to go back into the city for dinner, someone pulled out their smartphone to use Eric’s system, and it prevented us from taking the 520, which had a wreck, and would have added an extra 30 minutes to our drive.  If only Eric’s system worked for other areas of the country, maybe I could justify a new smartphone!

There’s nothing better for a software maker than to have your software used.  Every time someone writes in with a story about how they’re using Watson is a great day for me.  When they say, hey I found this auction on eBay and Watson saved me money.  Or, it found me a blog about my product and that resulted in a new feature.  Or, even when people say, I tried it and hated it, but would love it if you could make Watson search for people on MySpace, understand PDFs, etc.  Chances are if you say something like that, and we’re able, we’ll find a way to put it in.

Next week I’m of to Danny Sullivan’s latest Search Engine Strategies conference in New York, and then to this IBDNetwork Under the Radar event in the Bay Area.  It’s a bicoastal week.  I’m really looking forward to seeing all the new Web 2.0 stuff at the IBDNetwork event.  If you’re going to be at either event, drop me a line and we’ll meet up.

-j

You don’t have to give up your privacy for personalization, remote search

People are up in arms about the recent release of Google Desktop 3.0 beta.

There are different sides to the issue.  The EFF has taken a stance, and so have bloggers from the industry rags (like eWeek and InformationWeek). The discussion goes back and forth about whether certain features of Google Desktop 3 compromise your privacy.

Either way this is old news.  Google Desktop 2.0 profiled users over time, gathering data about what they click on so that it could present news clips tailored to end-user interests.

There are a lot of problems with profiling.  But it all boils down to the fact that somewhere, some software has a record of your interests over time.  Someone could steal that record, or subpoena it, or even worse, your interests might change, but you’re stuck as a Britney Spears fan for the rest of your life.

The same issue is present with the new “remote search” feature of Google Desktop 3.  Google says they need an intermediary server in order to make this feature possible.  They also say that you need to have Google Desktop installed on the PC from which you’re doing the searching.  Interesting.  So your data has to go through Google, and you have to install their software on another machine.  The Google desktop website says that all of this is necessary.

The reality is you don’t have to give up your privacy or share your data with third parties in order to get either of these functionalities.

When we launched Watson, we demonstrated you could have personalized content, without sending any data to our servers or by building a user profile that could be inspected.  This is a shameless plug, but we’re proud of it.

Gnutella and other, more legitimate file-swapping utilities like FolderShare show that you can have a distributed architecture for remotely accessing files on your PC.  Google could have developed remote searching with all the benefits of redundancy, etc., with no central server to hack, subpoena, or monitor your activity.  In fact, why didn’t they just allow you to enter a username and password to access the web-based search interface to Google Desktop?  This would have been a simple hack.

There must be something else going on.  I’m not a conspiracy theorist, and don’t think Google is out to harm people or violate their privacy intentionally.  But it is a little suspicious that such smart engineers are willing to risk so many users when there are proven designs that don’t require a single bit be stored on a central server.

-j

Tags: Watson, Privacy, Google Desktop, EFF, Profiles, Search

You don't know what you don't know

What do most people do to find information.  They use a search box.  The search box seems simple, elegant, and easy to use.  When you realize you need something, just go to the right search box, type what you want, and you have all the “information at your fingertips.”  It’s like magic.  So what’s all this about search being broken?

The first issue with search is that you don’t know what you don’t know.

Search technologies assume that you know when to use them.  In order to get the most out of them, you need to be able to recognize when there’s information out there that could help you.  Either you know that the information exists and you just need help locating it, or you think it might exist and make an explicit decision to stop what you’re doing to go and check.  That leaves a lot on the table.  There are too many times when, for one reason or another, you don’t decide to go look, and so you simply don’t know what you’re missing.

How many times have you completed a presentation only to find out that someone else in your company had already done one that you could have used?  Or maybe you’ve bought a product only to find out later that it has serious flaws, and when complaining to your friend, they say, “Oh yeah, I bought one too, I hated it.  In fact, it was so bad I blogged about it.  Didn’t you read my blog?”

While writing this entry, Watson found me blogs about Yahoo! search and search in Microsoft Office 12.  I wouldn’t have ever thought to go look for them.

If you didn’t know to ask in the first place how can all of the information now available help you?  Search engines can have all of the information in the world indexed and organized, available for you to question using powerful algorithms, but if you don’t think to stop and go find it, the information in them has no value. Making information available just doesn’t cut it anymore.  The information has to be made useful to you.

Like Don Norman suggested in one of his last essays, the simplicity of search user interfaces is deceiving.  I agree with this statement, but for different reasons.  The simple “elegance” of search user interfaces is a chimera.  Maybe it’s elegant in principle.  But in practice, if you don’t know what’s in there, how do you know what to ask for and when?

This Scientific American article really drives this point home.  The article goes into detail on current and future search technologies but stops with search as something that users have to go and do.  The future is a search interface for analyzing and visualizing richer data.  While this will obviously be useful for some tasks, it still won’t help you know what to ask for. 

You see, to me, search is something you do only because you have to.  It’s a last resort.  Search isn’t a task in and of itself.  It’s always instrumental to achieving some other goal.  You search because you’re trying to contact a company and you need their phone number.  You search because you’re looking for a product to buy.  You don’t go ‘searching’ for its own sake. 

If that’s the case then search should be tied more closely with what you’re doing.  Watson is our first step in making that happen.

Maybe the issue is the term “search.”  If what we mean by search is what we know today then, fine.  But if we take it to mean something broader—helping people get to the information they need---then today’s search user experience is due for an upgrade.

Apparently Andrei Broder concurs.  Who knew?  Watson did.  And it told me so while I was writing this.

But then again, none of this is anything all that new.  The difference is, now it’s real.

Welcome to My Blog

So, who am I and why am I starting this blog?  I'm Jay Budzik, the co-inventor of Watson and CTO of Intellext.  I've spent the past ten years researching and developing intelligent information systems, trying to fix the traditional user experience of search.  You see, we don't think the problem with search is search technology.  The problem with search is that it's not integrated into the user experience.  What is needed is not better search but a total rethinking of what it means to access information.  We need to turn the user experience of search on its head.  Watson is the product of that effort.

When I finished my Ph. D. at Northwestern, I had already proven some of this in principle.  But when we introduced Watson a year ago, we began to show the world we could make it a reality.

As far as search goes, now is a particularly dynamic and challenging time.  The search user experience is seriously broken to begin with.  But on top of that, there's too much information.  As people start making more and more content available online, from traditional publishing initiatives to the individual voices that contribute to the blogosphere.  This proliferation is making search seem even more broken because the stakes are so much higher.

I started this blog to have a place where I can share my thoughts on where we've been and where I think this is headed, share what we're doing about it and hear from people about what they think about all this.

Any feedback woud be great, whether it's topics you'd like this blog to cover or comments on posts.  Please feel free to speak up.

Thanks for checking in.