Philip Haine's articles on Product Vision, Innovation and Design

Using gestures and voice for access to key tasks on a mobile device

How might the iPhone afford direct access to key apps and tasks without defiling its exterior with another dastardly button?

My rant from a few months ago about the inefficiency of the iPhone calendar application continues to strike a chord.

One of the things I criticized was how many steps it takes just to navigate to the calendar in order to check, tweak, or add an appointment.  On the iPhone it ranges from 3 to 7 steps, with some of those being heavyweight steps that pull eyes and your attention of other things.  On the ancient PalmPilot and its newer descendants, it is one button press.  Extremely frequent task was rightfully given top-tier treatment, with a physical button on the device.

But Apple isn’t really into buttons. (Nor are they into acknowledging that the iPhone is really more of a PDA than it is a phone.)

Can we have our cake and eat it too?  Can we have direct access to key tasks while also accommodating Apple’s pathological aversion to real buttons?  Buttons that you can actually find without looking at the device, which are always available, regardless of the mode you are in, and which have the gratifying haptic feedback of… clicking?

Here’s one way: from the iPhone’s “slide to unlock” screen (or even from standby mode) let the user jump directly to an app by drawing a gesture.  C for calendar, M for mail, F for facebook.  It would be configurable.

Gestures could go deeper than just launching apps and get you to most used tasks.  Draw an A to create a new appointment.  Draw a T to go to today.  Each apps could publish its candidates for direct-access tasks, and the user could assign them to gestures.

Here’s an even better way to give immediate access to key tasks without buttons: make voice recognition the main way to get to most frequent tasks.  Press a physical “listen to me” button and say, “Go to today” or “new appointment for next Thursday at 5:30 pm” or “Call Leslie” or “new contact” or “Address book find Edwin” or “Facebook” or “Yelp nearby sushi” or “Montreal weather” or “Apple stock price”.  These were scenarios I painted several years ago.  Now they are starting to take shape at Google and with iPhone add-ons like Say Who (which actually works well) and Say Where (which doesn’t work as well yet).

A good implementation of voice command would suddenly make all that iPhone goodness a heck of a lot more efficient.  It could be a key part of an iPhone-neutralizing device.

Posted by Philip Haine on Friday, November 14th, 2008 at 1:50 pm.
See similar articles in: Commentary, Designs to Steal.

2 Responses to “Using gestures and voice for access to key tasks on a mobile device”

  1. Anecdotes of a Dog Part 4 | Anecdotes of a Dog wrote on November 15th, 2008 at 1:37 pm :

    [...] Steal This Idea » Using gestures and voice for access to key tasks … [...]

  2. Dave Cortright wrote on September 29th, 2009 at 9:20 pm :

    the iPhone has Voice Control now in version 3. It’s not 100% accurate, but it does OK.

Leave a Reply