CHAPTER 4

CONSUMER DEVICES AND ENTERTAINMENT INTERFACES

Tim Skelly

DEEP BACKGROUND AND THE BIG PICTURE

Computers have become ubiquitous features in the landscape of everyday objects. Mostly, they are hidden from the people who use them, embedded in other machines, as in the case of automobiles. These computers provide service without any extraordinary or even visible interface with the user. However, as power and flexibility become cheaper, the options for control offered by even the most trivial consumer device can quickly overwhelm users and the designers of these products. This, and the possibilities for interconnection between devices, creates a situation where the ordinary consumer is placed in the position of becoming, at least part-time, a programmer/technician. This has become a regular, unpleasant surprise for most people buying electronic goods. Consumers rightfully expect the tools and amusement activities they purchase to ease their lives, not make them more complex. The success of the consumer electronics industry is based upon shortening the distance between users and their desires. My goal was to determine what strategies, novel and conventional, technical and psychological, are being developed in Japan to shorten that immeasurable distance.

Before we left for Japan, the panel was asked to prepare a list of questions for our Japanese hosts so that they could prepare themselves for our visit and tailor their presentations to our interests. My questions, in accord with my opening comments, were these:

  1. What factors do you feel you must take into account for international consumers?
  2. With consumer products becoming increasingly sophisticated and flexible, some consumers feel many devices are too difficult to operate beyond a few simple functions. Are you developing any strategies to deal with this problem of overload?
  3. Is your work leading you towards user interfaces that are standardized across different products, or do you find that the design solution for each product needs to be unique?
  4. Do you believe that there are technologies (manufacturing, hardware, or software) not yet available that will significantly affect how your products are designed in the future?
  5. What role does product testing have in your work? Focus groups? Human factors?
  6. In the context of HCI, how do you balance the number of features and capabilities of your products against their eventual cost?
  7. What HCI features do you consider to be the most important to the consumer?
  8. Do the customs and habits of your users play a role in your design work? Do you feel it is important to preserve cultural knowledge of how things are supposed to work and behave?

Good questions, I thought, but I received few good answers.

Why?

Because Japanese industries are altogether too much like U.S. industries. That is, technology comes first, industrial design second, and HCI gets squeezed in there somewhere, usually in focus groups beforehand and user testing after the product is nearly ready to go to market. This is an old rant of the HCI community, that HCI is not given an important enough role in product development. I agree with this, but my opinion is tempered by the verity of the consumer market -- keep the cost down. Effective HCI is cost-effective in the long run, in that it results in a better, and thus more popular, product. However, in the short run, the costs associated with making products comprehensible and easy to use are not cheap. For instance, today nearly half of the code in a modern personal computer operating system is devoted either to help functions or to making the computer handle technical chores that were once left to the user. It wasn't until these operating systems began to be judged by their ease of use that developers were willing to pay the price required for a good HCI to be competitive.

In a volatile market such as personal electronics, products are rushed to market as cheaply and as quickly as possible. There are two reasonable and pragmatic reasons for this. The first reason is simple: the first person into a new market has an advantage. The second reason for speed and extreme economy in production is that most consumer technologies are novel. Who knows what people want in a personal digital assistant or on their interactive television? Why waste money adding and refining features when it isn't even known if consumers will pay for these devices at all?

Ironically, if a new product fails to succeed in its intended market, possibly because of poor human factors or interface problems, the product is simply scrapped, and no postmortem is performed to determine the cause of failure. As a result, no compelling reason is ever discovered to argue for increased research and development in the area of HCI.

The process of product development, in the United States and Japan, usually breaks down as follows:

  1. A new technology appears.
  2. Someone (usually with heavy marketing input) conceives of a product that can use, or is improved by, the new technology.
  3. Prototypes are mocked up by industrial designers, either foreign or domestic, and tested with consumers for "appeal." (They are not often tested for usability or ease of use, because at this stage there are probably no working prototypes available.) Industrial design in Japan, while very well done, does not differ significantly from that done worldwide. You need only walk the aisles of the semiannual Consumer Electronics Show to see proof of this.
  4. After some testing for major problems, they build it, ship it, and cross their fingers.

Now, if a consumer product is successful, manufacturers and developers may begin to listen to feedback from their customers and fix (or attempt to fix) problems that are reported. But sometimes the fix is worse than the original problem, or features are added simply because they are possible. Good HCI design is still regarded as an unnecessary luxury.

So, as I likely would have found in the United States, I received very little direct response in Japan to my questions -- probably because no one had ever bothered to consciously ask or answer them before. The following are the answers I either perceived, received, or inferred from prior experience. These are only my personal opinions, but I've seen, heard about, and experienced much consumer product manufacturing and development.

  1. What factors do you feel you must take into account for international consumers?

    UNITED STATES: Make sure it's in the right language and that the name of the product isn't insulting or stupid in that language.

    JAPAN: Get people in the target country to design their version of the product.

  2. With consumer products becoming increasingly sophisticated and flexible, some consumers feel many devices are too difficult to operate beyond a few simple functions. Are you developing any strategies to deal with this problem of overload?

    UNITED STATES: No, we haven't given it much thought.

    JAPAN: Same.

  3. Is your work leading you towards user interfaces that are standardized across different products, or do you find that the design solution for each product needs to be unique?

    UNITED STATES: Mixed response. It depends if a product is original or a variation on an established product. For instance, we want our product to operate similarly to other established products of its kind, but better.

    JAPAN: Same.

  4. Do you believe that there are technologies (manufacturing, hardware, or software) not yet available that will significantly affect how your products are designed in the future?

    UNITED STATES: No, we don't give it much thought beyond our next product.

    JAPAN: Same.

  5. What role does product testing have in your work? Focus groups? Human factors?

    UNITED STATES: Again, a mixed result, depending on whether or not there already exists a proven demand for the product.

    JAPAN: Virtually unheard of [but with good reason -- see below].

  6. In the context of HCI, how do you balance the number of features and capabilities of your products against their eventual cost?

    UNITED STATES: No contest -- make it as cheaply as possible.

    JAPAN: Same.

  7. What HCI features do you consider to be the most important to the consumer?

    UNITED STATES: Whatever we think is important, unless they tell us, and then it varies, product to product.

    JAPAN: Same.

  8. Do the customs and habits of your users play a role in your design work? Do you feel it is important to preserve cultural knowledge of how things are supposed to work and behave?

    UNITED STATES: Not very, if at all.

    JAPAN: Yes, it is very important!!!

Over the last several years, the United States has been lagging seriously behind Japan in one specific area of the entertainment market. Despite years of R&D, product design, and testing, the United States appears to have gained little ground in its efforts to catch up to its Japanese competitors. Rumor has it that this could have an effect upon trade relations with Japan that is more serious than the automobile or Kodak furors, yet it is the United States' own fault that we have yet to close THE KARAOKE GAP!

Ha, ha... Just joking, just joking. Why was karaoke developed and popularized in Japan and not in the United States? Ask someone Japanese, I wouldn't know. Why is it popular in the United States? Karaoke is a cultural FLUKE! A lucky accident. But then, there are plenty of examples like that in the United States. Maybe not as bizarre, but plenty.

Why is that? Two simple statements:

The United States has a heterogeneous culture. Japan has a homogeneous culture.

This isn't to say that all individuals in Japan think and act alike -- they don't. Artists are very unlike salarymen, salarymen are very unlike rural Japanese, and so on. But all three groups understand and are embedded in Japanese cultural tradition and environment. The people(s) of the United States do not all share a common heritage, nor is a single cultural norm enforced (though some might disagree with the latter). This explains the few differences in the answers to the questions I provided above:

Difference 1: What factors do you feel you must take into account for international consumers?

UNITED STATES: Make sure it's in the right language and that the name of the product isn't insulting or stupid in that language.

JAPAN: Get people in the target country to design their version of the product.

I was a part of the video game industry when it was still a solely American phenomenon. That monopoly didn't last long, of course, and the Japanese quickly began developing and exporting excellent products, many of which became highly successful in the United States. Little things like "Space Invaders" and "Pac-Man" were licensed and built by U.S. companies but designed in Japan for the Japanese market. Despite their many successes, Japanese video game companies, from the beginning to this day, have created American design groups in the United States to design products for the American market. This has always puzzled me. Why go to the trouble? Having to manage a group of (young and immature) Americans from halfway around the globe seems to be much trouble for not much return -- and, in fact, there never has been much return from these endeavors.

Over the years I have taken every possible occasion to ask many Japanese in the game industry to explain this to me. Their answer has always been the same, "We want our games for the American market to have the American..." feel, flavor, personality, you name it. "But why?" I would ask. "Your games do just fine over here as is." The people I questioned would not (or could not) give me an answer.

American video game developers have always found it difficult to create games that are successful on Japanese soil. This failure certainly has not been due to lack of effort on the part of American developers to target that market, nor has it been due to restrictions, tariffs, or any other trade barrier. Sega, for instance, established just after World War II, was until very recently an American-owned company with all principal operations in Japan. When I worked for a U.S.-based, Sega-owned game company, we still found it difficult to create games that would appeal to the Japanese. And that was the problem. It wasn't that we couldn't sell our product into the Japanese market; it was that we, Americans, were not Japanese. The dense, complex, and ancient Japanese culture was (and is) something that the Japanese believe you have to be born into to truly be a member of.

This isn't to say that the Japanese have not found much of American culture to their liking. For example, see the promotional image (Fig. 4.1) for the Sony Playstation entertainment titled 1950 American Dreams -- which features cartoon iconography from not one but several American decades. No, for Americans to purposely and consciously design a product that will appeal to the Japanese is as futile as designing artwork for bears -- how could U.S. designers possibly know what would entertain Japanese consumers? Again, refer to 1950 American Dreams. Does it amuse you, and if so, are you amused for the reason intended by its designers? If you do not read Japanese, can you even guess the purpose of this product?

It is a hard idea for Americans to grasp, given that we have no singular culture and we still somehow manage to produce goods and amusements that have universal appeal. But it is the only possible explanation for why Japanese game companies find it essential to have Americans design games for the American market.

I understand that some people may have reservations about this hypothesis, and I could be misinterpreting events as I have viewed them. However, I have one (of many) concrete examples to back up my theory. When Sega of Japan developed the game Sonic the Hedgehog, the parent company in Japan did not want it distributed and sold in the United States because they felt it was "too Japanese." Luckily, the head of Sega of America disagreed, and Sonic and his spin-offs have earned, mostly in the United States, nearly two billion dollars.


Fig. 4.1. 1950 American Dreams (Sony).

In short, since the Japanese feel that no outsider can design consumer products specifically for them, they also believe that they need the assistance of developers within their targeted country to develop the product for that country. Not a bad idea, but a bit overdone, in my opinion.

One last example taken directly from the JTEC panel's experiences in Japan: On a visit to IBM in Japan, members of our group observed (but were not officially shown) a very small IBM laptop computer -- one much smaller than models available in the United States. When asked if the laptop was about to be introduced in the United States, our group was told that the machine was intended for the Japanese market only, because the keyboard was too small for Americans. Our hosts seemed unmoved by the argument (made by a female member of the JTEC team) that many American women have small fingers.

Difference 2: What role does product testing have in your work? Focus groups? Human factors?

UNITED STATES: Again, a mixed result depending on whether or not there already exists a proven demand for the product.

JAPAN: Virtually unheard of, but with good reason.

The Japanese appear to feel that if you are truly Japanese (born into the culture), then you are like other Japanese at a fundamental level. Therefore, if the product being developed feels right and is satisfactory to the Japanese who are developing it, then most Japanese will feel the same way. This appears to be true, certainly to a much larger degree than what one would experience in other countries.

Difference 3: Do the customs and habits of your users play a role in your design work? Do you feel it is important to preserve cultural knowledge of how things are supposed to work and behave?

UNITED STATES: Not very, if at all.

JAPAN: Yes, it is very important!!!

I believe I have already said more than enough on this topic. But let me give you an example of how living in Japan can affect consumer product design: Earlier I mentioned environment as a cultural factor in Japan. One reason for this is that there is an extreme (by American standards) lack of personal space in Japan. It affects the lives of the Japanese in many significant ways -- for instance, in the cost of goods and housing, the necessity for social order, and the desire for cleanliness (if for no other reason than the even more extreme close personal proximity of other travelers on some Japanese commuter trains.)

Late one evening as the panel passed through a Tokyo district known as a hangout for teenagers, we were interested to note the large number of neatly dressed young women and young men and the mild atmosphere of "see and be seen." We asked Dr. Gene Gregory (our guide and personal trainer) if it was unusual for young women to hang out at this late hour. He informed us that due to the small size of family quarters in Tokyo, the children, once a certain age, are encouraged to go out in the evenings so that the parents can enjoy some relative peace and extra space. The streets in Tokyo are quite safe in the evenings, so this social solution (not regarded as a good practice for American families) to a spatial problem is quite reasonable.

The tight quarters of Tokyo has also led to another cultural phenomena. For families with younger children who must stay home in the evenings, the Nintendo game console has proved to be a great boon. The Nintendo and later, Sega home game units have given small children an activity they can enjoy in a limited space without making an undo amount of noise. I had this theory confirmed for me one evening when the panel was being entertained by employees of Japanese Rail. Hoping to make some cultural connection, I mentioned to one young man that I had been the art director for Sonic the Hedgehog II. His reaction was quite surprising to me -- he claimed never to have heard of Sonic (despite the fact that his image is plastered across Sega entertainment centers across Japan) and informed me that home video games were "for the small child."

As I have alluded to earlier, analysis becomes guesswork when an American probes Japanese culture. However, our panel did observe one trend in HCI design that several Japanese corporations shared, and with far greater emphasis than is found in the United States. At Microsoft, we call it "the social interface," as illustrated by Peedy the Parrot (Fig. 4.2), the star of Microsoft Research's Persona project, and the recently released Microsoft BOB. The Japanese embrace a more all-encompassing design philosophy known as kansai. Broadly, this means taking a more human-oriented approach to HCI design. But specifically, whatever name one chooses to use, the fundamental idea behind the work we observed is to make computers behave as humans would, obeying social rules and communicating with emotional feedback.


Fig. 4.2. "Peedy the Parrot" (Microsoft).

For example, Dr. Shuzo Morita, Deputy General Manager of Fujitsu Laboratories Ltd., presented a video of his work with artificial creatures (Fig. 4.3). An "immersion" effect was achieved for the audience with a very large screen rather than head-mounted displays, allowing multiple users to participate. As Dr. Morita is fundamentally interested in exploring the emotional side of human-computer interaction, his creatures did not speak (though they did sing) and behaved rather like playful pets. Another experiment with these creatures, which incorporated broadcast video and phone-in callers, produced an entertaining example of "interactive television."


Fig. 4.3. Artificial creatures (Fujitsu).

Probably the most captivating demonstration, though, was an entertainment product currently under development. Called "Teo, another Earth," it featured an extremely well-executed refinement of Dr. Morita's artificial creatures. "Phink" (renamed "Phin" for English-speaking audiences), the featured character in this product, is a dolphin-shaped bird that sings, eats, and generally behaves like a shy wild creature that gradually befriends the user (Fig. 4.4). It was very charming, and while designed for entertainment purposes, it nonetheless points the way to more emotionally engaging (and supportive) computer interfaces. Fujitsu has taken the project quite seriously, using the talents of several in-house programmers and a full animation team of about twenty-five.

Another demonstration of the use of emotional "gesture" took place at Sony. Senior Researcher at Sony Computer Science Laboratory Inc., Dr. Akikazu Takeuchi, is doing research focused on facial expressions, voice tones, and gestures, and how they express emotions and feelings. The SCSL staff believe that this research will help to introduce emotional factors into human-computer interactions. As Dr. Takeuchi put it, "In the future, a virtual human, which can interact emotionally with a real human through a display and a video camera, may be a reality."

Dr. Takeuchi showed video tapes of two examples of his system in trials. The first presented the face of his son (mapped onto a 3-D model) answering questions about new Sony products (Fig. 4.5).


Fig. 4.4. Fujitsu's "Phin" (formerly "Phink").


Fig. 4.5. Virtual human expressions (Sony).

This version of the system uses speech recognition and synthesis, syntactic and semantic analysis, plan recognition, response generation, and most interestingly, correspondence between conversational situations and facial displays. In the demonstration, the user asked questions of the boy's image on the computer screen. The system then interpreted the question, planned an appropriate response and animated the boy's image with both lip movement and emotional facial responses.

Dr. Takeuchi's second system involved two users playing a card game. The cards were displayed upon the screen, along with a woman's face. Through various facial gestures the computer attempted to give hints and encouragement to one of the players. A video camera was used as input to detect the position of the players.

Dr. Shigeru Akamatsu of ATR Human Information Processing Research Laboratories heads a group of researchers studying different aspects of the human face, primarily to advance telecommunications. One extremely interesting off-shoot of the work we saw was a system that could combine facial images to produce the "ideal" image for representing a certain emotion or communicative attitude -- for example, "gentle" or "fierce" (Fig. 4.6). This is a system quite wider in range than the Facial Action Coding System developed by Ekman and Friesen. That study attempted to identify facial positions that were recognized as representing the same emotions in all cultures. The resulting collection was a mere seven faces, hardly the full catalogue of human facial communication. The ATR system, based on a database of thousands of images of Japanese faces, appears to be capable of synthesizing a much broader range of human emotion.


Fig. 4.6. "Gentle" and "Fierce" (ATR).

Again, this is a system strongly rooted in Japanese culture. But at least in this instance, the same technique could be used with some certainty to identify and synthesize comparable facial "words" for other cultures as well.


Published: March 1996; WTEC Hyper-Librarian