Akio Yoshioka, Hiroyuki Toki, Noboru Takahashi, Shunji Ito
ACCESS CO., LTD. is a global provider of Internet technologies to the mobile and beyond-PC markets. It is the world's leading software-licensing company for embedded systems, with over 2.3 billion units and 800 devices. Its NetFront browser, widely recognized as one of the most advanced Internet browsers in the world, is a cornerstone of the company's vision of enabling Internet access anywhere and on any device. NetFront technologies are delivering full Internet browsing and related services to next-generation mobile devices and consumer electronics ranging from digital TV to automobile telematics. The FAU experience (Flagship ACCESS User Interface) is a new controller+interface paradigm developed by ACCESS's in-house design team to offer a more natural interaction between a mobile phone and a home-electronics environment.
We believe that in the near future mobile phones will play an important role in Home Gateway usage. The FAU project explores a new user-interface concept that uses a mobile phone as a remote control for consumer electronics. It replaces the current static menu paradigm with a more fluid user experience that we think enables the user to select applications easily and appropriately.
Point, Pull, Push. Pointing and Pull-Push information are FAU's key concepts. ACCESS's FAU combines technology and user interface to achieve a specific user experience that marries the digital and the physical world. It combines the concepts of remote control, mirroring and redirecting displays, and physical triggering to bring a real-world-oriented interface to the mobile phone. The interface concept uses point, pull, and push techniques: Point at a device you wish to control; pull information from the controlled device; and push information to it.
While there is a range of complexity available in mobile phones today, most of them have a surprising number of functions. Some of them are extremely deep and complex (especially in Japan, where media editing and management are expected from camera phones). To deal with this complexity, most interfaces employ a deep, multilayered set of options. Even finding the applications themselves can be difficult depending on how a carrier or handset manufacturer chooses to present the first set of choices to the user. Finding and selecting specific applications and features often feels very cumbersome.
Let's think about how the user makes a selection in the real world. In most cases, this involves pointing to the object, identifying it as "this" or "that." Leveraging this direct style of interaction, we decided that the FAU style should be similar: Make it very easy to select the remote-control application on the consumer-electronic device, then also make it very easy to select the consumer-electronic device one wishes to control.
Prototype. We showed a prototype of the FAU technology at the 3GSM tradeshow in Barcelona, Spain, in February of this year (see Figure 1). For this demo, we attached a trigger mechanism to the back of a WiFi-equipped mobile phone and connected it to a monitor display. To move an image from the cell phone's display to the monitor display, the user moves the trigger key forward, "pushing" the information to the monitor. When the user moves the trigger key backward, it pulls information from the associated monitor.
This correspondence between pulling a trigger to pull information, and pushing a trigger to push information, creates a close association between the user's physical control movements and the results on both screens.
We used the trigger push/pull concept to use the mobile phone as a remote control for the monitor. Program information on the mobile-phone display offered the user some choices; selecting a television show started a mini-display of the selection on the mobile phone. Pushing the trigger "pushed" the show onto the connected monitor. The response to this demo was positive and intuitive; users don't have any trouble understanding the relationship between the controlling device and the controlled device. The hardware was easy to use, with the trigger mechanism moving smoothly. Pointing at the controlled device was easy and the results were fast and easy to interpret.
Use Cases for FAU. Now that we have a working prototype of the technology, we are exploring uses for it that revolve around the home, digital storage and display devices, and other digitally controllable elements of home life.
Setting up your FAU mobile. Point your new FAU IrDA/WiFi-equipped mobile phone at a device to get a control signal. The phone's display shows incoming arrows, indicating that the IrDA connection is made (see Figure 2).
Now pull the trigger key, and you receive a remote-control application from the network. What can you do with it?
Do what I say! Switch channels by pointing your FAU-enabled mobile phone at your television. Pull the trigger to take the TV channel with you as you walk away from the TV.
The family-photo slideshow on your mobile camera is fun to watch; when you want to show it to your family crowded around your shoulder, push the trigger to redirect the slideshow to your widescreen TV.
Point your FAU mobile at your air conditioner, your light fixture, or your robotic vacuum cleaner to pull a signal to download its control application from the network. Then push your instructions to the device and enjoy life from your armchair, while walking around your home, while tending to your garden.
Limitations. We expect that FAU will be easy to use, but there are some limitations. Consumers are accustomed to using IrDA for remote controls, which requires a direct line of sight as well as directional pointing. But it doesn't require any kind of setupit just works. FAU requires a registration process for the FAU mobile to locate and identify devices it can control, and WiFi to receive downloads of controller instructions. Today, high-end remote controls are expensive and cost even more to be professionally programmed, since they are difficult for the average consumer to understand and modify at home. FAU can't avoid the setup routines, but we believe that the process can be vastly simplified from the complex experiences users have today with sophisticated remote controls. In addition, FAU is not a completely generalizable controller; it participates in a home-gateway ecosystem with devices that are designed and built to be networked.
Future Possibilities. FAU is conceived for use in home environments with mobile phones that have home-gateway functions. We are continuing to develop the user-interface environment from both a GUI and a hardware-design standpoint, working on setup, feedback, and operational user experience. We believe it is possible to have a portable player or remote-control application that uses a protocol like DLNA.
Seamless and intuitive setup and control opens up many new possibilities, not just for the home environment, but for public and community spaces. Entertainment, information sharing, collaborative activities, and education and instruction are all application areas for the FAU point-pull-push technology, between not only mobile phones and consumer electronics, but embedded computing devices and anywhere a remote control or secondary display is useful.
ABOUT THE AUTHORS
Akio Yoshioka is a designer working on experimental and prototype systems.
Hiroyuki Toki is a designer with ACCESS Co., Ltd., based in Tokyo, Japan. He is currently working with ACCESS's wholly owned subsidiary, PalmSource, Inc., developing application designs for the ACCESS Linux Platform.
Noboru Takahashi is the design manager for the in-house ACCESS design team.
Shunji Ito is a designer for the in-house ACCESS design team.
©2006 ACM 1072-5220/06/0700 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2006 ACM, Inc.