MIT Technology Review

Friday, July 29, 2011

Would an iPhone 'Assistant' Really Help?

Apple may be building the technology into iOS. But can it succeed where others have failed?

Are we on the cusp of an era of ubiquitous "virtual personal assistants"? If Steve Jobs has his way, we just might be.

Back in the spring of 2010, Apple acquired Siri, a company that produced an app that described itself in just those terms. Now, clues dug up recently by 9to5Mac, a site dedicated to scrutinizing all things Apple, suggest that Apple may be ready to introduce Siri-like features in the next version of iOS, its operating system for the iPhone, iPod touch, and iPad.

If Apple is indeed about to launch a personal assistant, it could help set the iPhone apart from other smart phones in the market. Android's voice-command system is considered one of its chief advantages over the iPhone, but a Siri-derived personal assistant would add more voice functionality, eliminating Android's advantage. But it will be a gamble, as other efforts to foist a personal assistant upon computer users have backfired badly. Remember Clippy, the animated paper clip that would pop up every time you tried to write a letter in Microsoft Word?

In a screenshot that 9to5Mac turned up, apparently from the menu on an iPhone "test unit," one button reads "Assistant"; another reads "Speaker," suggesting that the assistant can talk back, if you want it to; and a tab reading "MyInfo" suggests that the assistant will be able to use data on your phone such as address book contacts and location to help find the information you want. 9to5Mac further claims to have plumbed the depths of an iOS software development kit and found lines of code that correspond to the features in the screenshot.

Siri's original app, which licensed voice recognition technology from Nuance, a company based in Burlington, Massachusetts, enabled users to perform searches and make appointments or reservations using voice commands. It worked remarkably well for these simple tasks. (You can see a video of it in action here.)

Work on Siri began about eight years ago, when DARPA funded a massive AI initiative called CALO (Cognitive Assistant that Learns and Organizes). The idea, says Norman Winarsky, vice president of ventures at SRI, based in Menlo Park, California, the prime contractor for CALO, was to develop a virtual personal assistant as good as the character of Radar O'Reilly on the TV show M*A*S*H. "Radar always knew what the captain wanted before the captain knew what the captain wanted," says Winarsky.

As the CALO program wound down, SRI recognized a massive market opportunity in the research it had been doing. Over a period of a few years, SRI built the company Siri and launched an app.

Apple scooped up the company less than three months after the Siri app launched. Since then, we've all been held in suspense. Once Apple acquires a company, says Winarsky, "they go into radio silence, and believe me, they don't share with SRI or anybody" as to just what their plans are.

But, even if Apple is ready to offer a virtual personal assistant to every iPhone 5 buyer, does that mean every iPhone 5 buyer is ready for a virtual personal assistant? Not if it doesn't at least outperform Clippy.

Jason Hong, an associate professor at Carnegie Mellon University and a member of its Human-Computer Interaction Institute, says Microsoft Word's Clippy failed for two reasons: he was intrusive, interrupting you when you had already begun a task, and he simply wasn't very smart, often failing to understand your intentions, when you bothered to indulge him, that is. Hong found an explanation at a talk given by Eric Horvitz, the Microsoft researcher who worked on some of the AI behind Clippy. "They had to lobotomize all the machine learning they used, to make it primitive enough to run fast and in real time" on your desktop, says Hong.

Now, though, smart phones are blisteringly fast, and complex processing can be outsourced to the cloud, which means we can fully leverage the fruits of AI research even from relatively simple hardware. What's more, adds Hong, Siri is crucially "driven by what the user is explicitly asking"—it doesn't pop up officiously, like that insufferable paper clip.

Winarsky is betting that virtual personal assistants will be ubiquitous, and widely accepted, sooner than many expect. Looking beyond the restaurant reservations that Siri handled so well, Winarsky foresees an era when virtual personal assistants offer advice and recommendations on a range of topics. Eventually, he says, the technology will be folded into the desktop and the Web, and it will make people rich. "Within 10 years," he says, "we will see the value associated with virtual personal assistants throughout our marketplace to be in the many tens of billions—and [it] optimally might reach the 100-billion-dollar level."

The biggest stumbling block ahead might just be how willing people are to be heard constantly issuing commands in carefully enunciated English into their iPhones. "I used to play a game of guessing whether people I saw talking to themselves were drunk, crazy, or on the phone," says Hong. "And sometimes it was pretty hard to tell."

In that sense, the most important feature visible on the leaked 9to5Mac screenshot may well be the button labeled "OFF."