Most mobile devices including cellular phones and portable video players cannot generate a "pull" or "push" sensation. The reason is that they lack the external fulcrum to generate force vector.

Our concept is using nonlinearity of human perception to induce force vector; humans feel rapid acceleration more strongly than slow acceleration. Our system uses neither grounding styles (such as PHANToM, SPIDAR) nor traditional nongrounding styles (gyro effect, air jet). Instead, we designed the force-feedback device that exploits the nonlinearity of human haptic perception. The revolution is in the perception method of force sensation; humans strongly feel rapid acceleration of the slider (right gray ball in the animation below) more than slow one since its acceleration is asymmetric. (humans do not notice low-amplitude force of long duration.) We utilized the haptic "wash-out" to generate a pull sensation.

The proposed force perception method subjects a mass to periodic translational motion; the mass is accelerated more rapidly in one direction than the other. If scaled correctly, humans translate the asymmetric acceleration into a one-directional force because of their perception characteristics. While this force display system can be categorized as a non-grounded device (it has no fulcrum), it can generate constant force by repeating one cycle of motion.

Buru Navi (Laval Virtual 2007 GrandPrix)

We evaluated the characteristics of force perception when the crank-slider mechanism generating asymmetric acceleration was held in the air. Our results indicate that the swinging force generated by the motion of linkages prevents us from well perceiving the force sensation; the motion of linkages makes it difficult for users to distinguish between a simple vibration and the directed force sensation. Moreover, the results revealed that resolution with which the force direction can be discriminated with the antiphase tandem pair is significantly smaller than that with the inphase one. i.e. limiting the direction of asymmetric acceleration to one axis is effective when people hold the force display in the air.

The force display would replace vibrators (or use together) in all mobile devices. Applications include the cellular phones draw your hand where you want to go.

The world is distorted when we perceive it.

The basic mechanism to create an asymmetric oscillation and its acceleration profile.

First prototype

Flyer of Buru-Navi for SIGGRAPH 2006

SIGGRAPH 2006
(two-dimensional version)

A waiter (user) in a cafe wants to deliver a drink ordered by a customer (target). The waiter does not know where the customer is sitting. However, his "smart tray" creates an attraction force centered on the customer and guides the waiter to him/her. Since the guidance is based on force sensation, the guidance information is useful regardless of the waiter's age or language ability. Moreover, since the guidance directions are transmitted via touch, the other senses remain available to the waiter, making it easier to move through even the most crowded areas. Finally, the instructions remain entirely private; no one else can discover that the waiter is receiving instructions.

In the demos, the users moved to the target according to the direction of the perceived force sensation. The direction of the force display was controlled so that they face the target based on the posture detection system.

The user chooses one customer by stopping in front of the target, projected movie or bear-shaped robot. If this choice is correct, the customer says "thank you"; otherwise, the customer says, "I did not order this".

Force feedback"tray" for novice waiters consists of Buru-Navi, rotation mechanism,and position and posture identification system. The system consists of a tray (approximately 750 g in weight) held by the user (waiter), a small bag containing a battery and a control device (approximately 300 g), and a position and direction identification system based on infrared LEDs and a wide-angle camera. Our haptic interface (force display), infrared LEDs, and stepper motor are embedded in the tray.

Demonstration in France

Demonstration in Singapore

Some emergency situations, such as fires or earthquakes, require that evacuation to a safe area, often through an emergency exit. This is especially difficult for people with visual disability. Here, we propose a new device, a haptic direction indicator, which will help blind pedestrians intuitively and safely escape from dangerous area by means of haptic navigation.

We performed an experiment for predefined route guidance using the force display to evaluate the time required for completing a walking task,correctly perceived directions, user preferences, and the level of easeof understanding. We built a human-size maze in the gymnasium of the Kyoto Prefectural School for theVisuallyImpaired, Japan. Since the streets and avenues in Kyoto are mainly laid out in a grid, the maze was designed with checkerboard-shaped routes.

Pilot study with people with visual impaierment

Empirical study

Weight perception has been of great interest for over three centuries. Most research has been concerned with the weight of static objects, and some illusions have been discovered. Here, we show a new illusion related to the perception of the heaviness of oscillating objects. We performed experiments that involved comparing the weight of two objects of identical physical appearance but with different gross weights and oscillation patterns (vibrating vertically at frequencies of 5 or 9 cycles/s with symmetric and asymmetric acceleration patterns).

The results show that the perceived weight of an object vibrating with asymmetric acceleration increases compared to that with symmetric acceleration when the acceleration peaks in the gravity direction. In contrast, almost no heaviness perception change was observed in the antigravity direction. We speculate that the reason for the divergence between these results is caused by the differential impact of these two hypothesized perceptual mechanisms as follows: the salience of pulse stimuli appears to have a strong influence in the gravity direction, whereas filling-in could explain our observations in the antigravity direction. The study of this haptic illusion can provide valuable insights into not only human perceptual mechanisms but also the design of ungrounded haptic interfaces.

Psychometric functions for the perceived weight change generated by asymmetric versus symmetric acceleration.

Perception of time over sub-second scale is fundamental for many aspects of human activities. However, its neural mechanisms for the time perception are not clarified well: modality specific and distributed mechanism vs. universal and centralized one in the brain.

We report tactual timing perception using the adaptation paradigm. The experimental results implicate that the duration perception is encoded by spatially localized temporal mechanisms.

Universal clock? Distributed?

Experimental procedure

GVS is an electrical stimulation technique which can generate virtual acceleration toward anode side. (The original method was discovered over 100 years ago.) Our team made applications for GVS interface as one of key technologies for parastic humanoid project.

The two males' vestibular system, which controls their senses of balance, has been controlled by weak electrical currents delivered just behind her ears.

When a weak DC current is delivered to the mastoid behind your ear, your body responds by shifting your balance toward the anode. If it is strong enough, it not only throws you off balance but alters the course of your movement.

SIGGRAPH 2005 (human remote control)

SIGGRAPH 2005 (racing video game). Feel the "G" (centrifugal force) in the corners.

SIGGRAPH 2005, (synchronization with music rhythm: Shaking the World)

Application possibilities of augmented reality technologies in the field of mobility support for the Deaf-Blind is discussed.

We propose the navigation system called Virtual Leading Blocks for the Deaf-Blind, which consists of a wearable interface for Finger-Braille, one of the commonly used communication methods among Deaf-Blind people in Japan, and a ubiquitous environment for barrier-free application, which consists of floor embedded active radio-frequency identification (RFID) tags. The wearable Finger-Braille interface using two Linux-based wristwatch computers has been developed as a hybrid interface of verbal and nonverbal communication in order to inform users of their direction and position through the tactile sensation. We propose the metaphor of "watermelon splitting" for navigation by this system and verify the feasibility of the proposed system through experiments.

Glove-style interfaces seem to be the most suitable design for the Finger-Braille interface because they are easy to wear. However, they cover the palm or the fingertip which has the highest tactile sensitivity. Therefore glove-style interfaces prevent Deaf-Blind people from obtaining environmental information by tactile sensation. In our previous work (Figures below), two types of ring-shaped devices were developed using two kinds of actuators: (a) vibration-motor type and (b) solenoid type, as shown in Figure. The vibration-motor type consists of six small-size, lightweight DC motors (CM05J; TPC) the frequency of which is approximately 116 Hz, since it has been reported that 120 Hz is the optimal frequency to stimulate the back of the finger. The solenoid type consists of six tubular solenoids (S-50S03; Shindengen Electric Manufacturing Co.) weighting 15 g each. Each actuator is fixed onto the setting of the ring. Each type is connected to a computer via a USB 1.1 port.

A new wireless prototype using Bluetooth technology was implemented. The actuator is the same vibration-motor as previous one from the previous experiment. Our system consists of a wearable computer and two Finger-Braille interfaces, and we linked them via wireless communication using two Linux-based wristwatch computers, called WatchPad1.5 codeveloped by IBM Japan and Citizen Watch.

The user can read letters through the tactile channel by wearing the two wristwatch computers and two modules which consist of batteries and circuits for actuators on the wrists. In addition, WatchPad1.5 is equipped with a serial connector for communication. The signals from the extended serial port are transmitted to parallel signals by a one-chip microcomputer (PIC16F84A; Microchip Technology Inc.) on the interface module. The total weight of the equipment, including the battery, is approximately 170 g for a single hand.

Finger-Braille Interface

Finger-Braille was invented for the Deaf-Blind to communicate in real time in Japan, and it became one of the commonly used communication methods among Deaf- Blind people in Japan. In Finger-Braille method, the index finger, middle finger, and ring finger of both hands function like the keys of a Braille typewriter. A sender dots Braille code on the back of fingers of a receiver as if typing on a Braille typewriter.

Some examples of the translation are shown in the table below. Regardless of the language, Braille systems consist of 6 dots; the pattern for "na" under the Japanese Braille system is the same as "K" under the English one. Therefore it is easy to translate Braille into Finger-Braille under all languages. The Finger-Braille method enables Deaf-Blind people to obtain textual information as if they are listening in real time.

Wearable Finger-Braille Interface with WatchPad (a Linux-based watch)

We developed a wearable interface for textual input on the basis of Braille input method. The device, named OBOE, is operated by both hands, which is good for portability and can be used while standing. The users get their input operations confirmed clearly by feeling the click since the buttons of the proposed device are the same mechanical switches as used in keyboards for desktop computers. The results of an experiment of learning effect revealed that the users who had no experience of Braille input could type Japanese phrases at 35.4 Braille codes per minute, and who had experience at 112.4 Braille codes per minute. Thus novices can master the proposed device and experts can input text very fast by using OBOE. Based on the results of questionnaire by the subjects, we discussed the comparison with a Braille typewriter, the optimum layout of keys for OBOE, and the posture of holding.

We carried out an experiment to evaluate the device by measuring input speed and error rate. In the experiment, there were five trials to examine the effect of learning. All six subjects were male, right handed and did not have experiences with Braille typewriters. The experiment consisted of a tutorial (non-semantic phrase session) and a semantic phrase session. The tutorial session was a 10-minute trial, followed by the 10-minute semantic phrase session. In each session, the subjects were told to input as quickly and accurately as possible and not to sacrifice speed for accuracy. The subjects could confirm their input through auditory feedback by a TTS (text-to-speech) engine, IBM Protalker97, as well as through visual feedback from a window on screen. The results are shown in Figures.

We carried out another experiment with those people who have experiences with Japanese-to-Braille transcribing. The transcriber subject showed a higher input speed, 112.4 Braille codes per minute, and a lower error rate, 4.1 %, than the average of the novice subjects, with a little learning in the semantic mode. The transcriber subject commented in the tutorial mode that characters that rarely appear in usual Japanese sentences prevented the speedup. (e.g., "dya" is almost replaced by "jya" in Japanese sentence but the user had to input that kind of characters in the tutorial.)

OBOE = Oboe-like Braille interface for Outdoor Environment

Result of experiment

OBOE with wearable Finger-Braille interface