Buru-Navi Buru-Navi Buru-Navi
2004 -
NTT CSL
server tray Smart tray Smart server tray
2007 - 2008
NTT CSL
Heaviness illusion Heaviness illusion Heaviness illusion
2007 - 2008
NTT CSL
Haptic compass Haptic compass Haptic compass
2007 - 2009
NTT CSL
Tactile percepion Tactile diration percepion Tactile diration percepion
2006 -
NTT CSL, JST, UCL
Wearable Finger-Braille Interface Finger-Braille Wearable Finger-Braille Interface
2002 - 2004
RCAST, the Univ. Tokyo
OBOE OBOE Oboe-like Braille input interface
2004 - 2005
RCAST, the Univ. Tokyo
Menu
Research Interests
update: May 2010
Haptic Illusions / Haptic Science
in NTT CS Labs. ('04-)

"Lead-Me" Interface: Buru-Navi

Buru-Navi

Most mobile devices including cellular phones and portable video players cannot generate a "pull" or "push" sensation. The reason is that they lack the external fulcrum to generate force vector.

How to generate force sensation in the mobile devices

Our concept is using nonlinearity of human perception to induce force vector; humans feel rapid acceleration more strongly than slow acceleration. Our system uses neither grounding styles (such as PHANToM, SPIDAR) nor traditional nongrounding styles (gyro effect, air jet). Instead, we designed the force-feedback device that exploits the nonlinearity of human haptic perception. The revolution is in the perception method of force sensation; humans strongly feel rapid acceleration of the slider (right gray ball in the animation below) more than slow one since its acceleration is asymmetric. (humans do not notice low-amplitude force of long duration.) We utilized the haptic "wash-out" to generate a pull sensation.
The proposed force perception method subjects a mass to periodic translational motion; the mass is accelerated more rapidly in one direction than the other. If scaled correctly, humans translate the asymmetric acceleration into a one-directional force because of their perception characteristics. While this force display system can be categorized as a non-grounded device (it has no fulcrum), it can generate constant force by repeating one cycle of motion.

You need to install Adobe Flash Player to watch this content.

Adobe Flash Player

**You can change the speed of the crank by the upper two buttons.

Buru Navi
(Laval Virtual 2007 GrandPrix)

Tandem pairs to counteract the side-to-side force generated by the motion of linkages

We evaluated the characteristics of force perception when the crank-slider mechanism generating asymmetric acceleration was held in the air. Our results indicate that the swinging force generated by the motion of linkages prevents us from well perceiving the force sensation; the motion of linkages makes it difficult for users to distinguish between a simple vibration and the directed force sensation. Moreover, the results revealed that resolution with which the force direction can be discriminated with the antiphase tandem pair is significantly smaller than that with the inphase one. i.e. limiting the direction of asymmetric acceleration to one axis is effective when people hold the force display in the air.

Force sensation more than rumble

The force display would replace vibrators (or use together) in all mobile devices. Applications include the cellular phones draw your hand where you want to go.

[reference]

  • T. Amemiya, T. Maeda, "Directional Force Sensation by Asymmetric Oscillation from a Double-layer Slider-crank Mechanism", Trans. of the ASME Journal of Computing and Information Science in Engineering, Vol. 8, No. 3, 2008.
  • T. Amemiya, H. Ando, T. Maeda, ""Lead-Me Interface" for a Pulling Sensation from Hand-held Devices", ACM Trans. on Applied Perception, Vol. 5, No. 4, 2008.
  • T. Amemiya, H. Ando, T. Maeda, "Directed Force Perception When Holding a Nongrounding Force Display in the Air", In Proc. of EuroHaptics 2006, pp. 317-324, 2006.
  • T. Amemiya, H. Ando, T. Maeda, "Perceptual Attraction Force: The Sixth Force", ACM SIGGRAPH 2006 Emerging Technologies, 2006.
  • T. Amemiya, H. Ando, T. Maeda, "Virtual Force Display: Direction Guidance using Asymmetric Acceleration via Periodic Translational Motion", In Proc. of World Haptics Conference 2005, pp. 619-622, March 200

Buru-Navi

nonlinear
The world is distorted when we perceive it.
mechanism
The basic mechanism to create an asymmetric oscillation and its acceleration profile.
Phantom-DRAWN
First prototype
Flyer of Buru-Navi
Flyer of Buru-Navi for SIGGRAPH 2006
2D version
SIGGRAPH 2006
(two-dimensional version)
Force feedback "Tray" for novice waiters
in NTT CS Labs. ('07-08)
A waiter (user) in a cafe wants to deliver a drink ordered by a customer (target). The waiter does not know where the customer is sitting. However, his "smart tray" creates an attraction force centered on the customer and guides the waiter to him/her. Since the guidance is based on force sensation, the guidance information is useful regardless of the waiter's age or language ability. Moreover, since the guidance directions are transmitted via touch, the other senses remain available to the waiter, making it easier to move through even the most crowded areas. Finally, the instructions remain entirely private; no one else can discover that the waiter is receiving instructions.
In the demos, the users moved to the target according to the direction of the perceived force sensation. The direction of the force display was controlled so that they face the target based on the posture detection system.
The user chooses one customer by stopping in front of the target, projected movie or bear-shaped robot. If this choice is correct, the customer says "thank you"; otherwise, the customer says, "I did not order this".
Force feedback"tray" for novice waiters consists of Buru-Navi, rotation mechanism,and position and posture identification system. The system consists of a tray (approximately 750 g in weight) held by the user (waiter), a small bag containing a battery and a control device (approximately 300 g), and a position and direction identification system based on infrared LEDs and a wide-angle camera. Our haptic interface (force display), infrared LEDs, and stepper motor are embedded in the tray.

[reference]

  • T. Amemiya, T. Maeda, H. Ando, "Location-free Haptic Interaction for Large-Area Social Applications", Personal and Ubiquitous Computing, 2008.
  • T. Amemiya, T. Maeda, H. Ando, "Bear's Beer", DAT, Singapore Science Center, Singapore, December 2007.
  • T. Amemiya, T. Maeda, H. Ando, "Come Over Here, or Catch You", Laval Virtual ReVolution, Laval, France, April 2007.
back to top

2D Buru-Navi

demo in  Laval
Demonstration in France
demo in Singapore
Demonstration in Singapore
Tray
Tray
Mobility support for people with disabilities
in NTT CS Labs. and Kyoto City ('07-09)

Direction Indicator through Force Sensation

Some emergency situations, such as fires or earthquakes, require that evacuation to a safe area, often through an emergency exit. This is especially difficult for people with visual disability. Here, we propose a new device, a haptic direction indicator, which will help blind pedestrians intuitively and safely escape from dangerous area by means of haptic navigation.
We performed an experiment for predefined route guidance using the force display to evaluate the time required for completing a walking task,correctly perceived directions, user preferences, and the level of easeof understanding. We built a human-size maze in the gymnasium of the Kyoto Prefectural School for theVisuallyImpaired, Japan. Since the streets and avenues in Kyoto are mainly laid out in a grid, the maze was designed with checkerboard-shaped routes.

[reference]

  • T. Amemiya, H. Sugiyama, "Haptic Handheld Wayfinder with Pseudo-Attraction Force for Pedestrians with Visual Impairments", In Proc. of 11th ACM Conference on Computers and Accessibility (ASSETS 2009), pp. 107-114, Pittsburgh, PA, October 2009.
  • T. Amemiya, H. Sugiyama, "Navigation in Eight Cardinal Directions with Pseudo-Attraction Force for the Visually Impaired", In Proc. of IEEE International Conference on Systems, Man and Cybernetics (SMC 2009), Texas, October 2009.
  • T. Amemiya, "Haptic Direction Indicator for Visually Impaired People Based on Pseudo-Attraction Force," eMinds: International Journal on Human-Computer Interaction, Vol. 1, No. 5, pp.23-34, March 2009.
  • T. Amemiya, H. Sugiyama, "Design of a Haptic Direction Indicator for Visually Impaired People in Emergency Situations", In Proc. of 11th International Conference on Computers Helping People with Special Needs (ICCHP 2008), pp.1141-1144, Linz, Austria, July 2008.
back to top

Haptic compass

concept picture

a pilot study with visually impaired people

Pilot study with people with visual impaierment

Picture of the haptic handheld wayfinder

Overview of navigation with the haptic handheld wayfinder

Empirical study

Heaviness illusion
in NTT CS Labs. ('07-09)
Weight perception has been of great interest for over three centuries. Most research has been concerned with the weight of static objects, and some illusions have been discovered. Here, we show a new illusion related to the perception of the heaviness of oscillating objects. We performed experiments that involved comparing the weight of two objects of identical physical appearance but with different gross weights and oscillation patterns (vibrating vertically at frequencies of 5 or 9 cycles/s with symmetric and asymmetric acceleration patterns).
The results show that the perceived weight of an object vibrating with asymmetric acceleration increases compared to that with symmetric acceleration when the acceleration peaks in the gravity direction. In contrast, almost no heaviness perception change was observed in the antigravity direction. We speculate that the reason for the divergence between these results is caused by the differential impact of these two hypothesized perceptual mechanisms as follows: the salience of pulse stimuli appears to have a strong influence in the gravity direction, whereas filling-in could explain our observations in the antigravity direction. The study of this haptic illusion can provide valuable insights into not only human perceptual mechanisms but also the design of ungrounded haptic interfaces.

[reference]

  • T. Amemiya, T. Maeda, "Asymmetric Oscillation Distorts the Perceived Heaviness of Handheld Objects", IEEE Transactions on Haptics, Vol. 1, No. 1, pp. 9-18, Jan-Jun, 2008. .

Well-known haptic illusions such as ...
(the link above is animation examples) phantom sensation / apparent movement / cutaneous saltation / masking
back to top

Heaviness Illusion

demo in  Laval
Psychometric functions for the perceived weight change generated by asymmetric versus symmetric acceleration.
Tactile duration perception
in NTT CS Labs. JST, and UCL('06-)

Time Perception in haptic modality

Perception of time over sub-second scale is fundamental for many aspects of human activities. However, its neural mechanisms for the time perception are not clarified well: modality specific and distributed mechanism vs. universal and centralized one in the brain.
We report tactual timing perception using the adaptation paradigm. The experimental results implicate that the duration perception is encoded by spatially localized temporal mechanisms.

[reference]

  • J. Watanabe, T. Amemiya, S. Nishida, A. Johnston, "Independent local adaptation of tactile frequency and duration", European Conference on Visual Perception (ECVP 2008), Utrecht, Netherlands, August 2008.
  • J. Watanabe, T. Amemiya, S. Nishida, A. Johnston, "Distortion of tactile duration perception", In Proc. of NEUROSCIENCE 2007, Society for Neuroscience, San Diego, CA, Nov. 2007.
  • T. Amemiya, H. Ando, S. Nishida, A. Johnston, and J. Watanabe, "Tactile Duration Perception using an Adaptation Paradigm", In Proc. of VRSJ Annual Conference, pp.33-34, 2006.

Tactile duration perception

universal clock or multiple represensation?
Universal clock? Distributed?
Procedure
Experimental procedure
Human interface applying galvanic vestibular stimulation (GVS)
GVS is an electrical stimulation technique which can generate virtual acceleration toward anode side. (The original method was discovered over 100 years ago.) Our team made applications for GVS interface as one of key technologies for parastic humanoid project.
The two males' vestibular system, which controls their senses of balance, has been controlled by weak electrical currents delivered just behind her ears.
When a weak DC current is delivered to the mastoid behind your ear, your body responds by shifting your balance toward the anode. If it is strong enough, it not only throws you off balance but alters the course of your movement.

[reference]

  • T. Maeda, H. Ando, Tomohiro Amemiya, M. Inami, N. Nagaya, M. Sugimoto, "Shaking The World: Galvanic Vestibular Stimulation As A Novel Sensation Interface", ACM SIGGRAPH 2005 Emerging Technologies, 2005.
back to top

GVS for VR application

human remote control
SIGGRAPH 2005 (human remote control)
racing video game
SIGGRAPH 2005 (racing video game). Feel the "G" (centrifugal force) in the corners.
synchronization with music rhythm
SIGGRAPH 2005, (synchronization with music rhythm: Shaking the World)
List of demonstration:
Wearable Finger-Braille Interface for the Deaf-Blind People
in The Univ. of Tokyo ('02-04)

Wearable tactual device for communication among the Deaf-Blind

Wearable Finger-Braille Interface

Application possibilities of augmented reality technologies in the field of mobility support for the Deaf-Blind is discussed.
We propose the navigation system called Virtual Leading Blocks for the Deaf-Blind, which consists of a wearable interface for Finger-Braille, one of the commonly used communication methods among Deaf-Blind people in Japan, and a ubiquitous environment for barrier-free application, which consists of floor embedded active radio-frequency identification (RFID) tags. The wearable Finger-Braille interface using two Linux-based wristwatch computers has been developed as a hybrid interface of verbal and nonverbal communication in order to inform users of their direction and position through the tactile sensation. We propose the metaphor of "watermelon splitting" for navigation by this system and verify the feasibility of the proposed system through experiments.
Glove-style interfaces seem to be the most suitable design for the Finger-Braille interface because they are easy to wear. However, they cover the palm or the fingertip which has the highest tactile sensitivity. Therefore glove-style interfaces prevent Deaf-Blind people from obtaining environmental information by tactile sensation. In our previous work (Figures below), two types of ring-shaped devices were developed using two kinds of actuators: (a) vibration-motor type and (b) solenoid type, as shown in Figure. The vibration-motor type consists of six small-size, lightweight DC motors (CM05J; TPC) the frequency of which is approximately 116 Hz, since it has been reported that 120 Hz is the optimal frequency to stimulate the back of the finger. The solenoid type consists of six tubular solenoids (S-50S03; Shindengen Electric Manufacturing Co.) weighting 15 g each. Each actuator is fixed onto the setting of the ring. Each type is connected to a computer via a USB 1.1 port.

Vibration motor type and solenoid type
Previous Version of Finger-Braille Interface

A new wireless prototype using Bluetooth technology was implemented. The actuator is the same vibration-motor as previous one from the previous experiment. Our system consists of a wearable computer and two Finger-Braille interfaces, and we linked them via wireless communication using two Linux-based wristwatch computers, called WatchPad1.5 codeveloped by IBM Japan and Citizen Watch.
The user can read letters through the tactile channel by wearing the two wristwatch computers and two modules which consist of batteries and circuits for actuators on the wrists. In addition, WatchPad1.5 is equipped with a serial connector for communication. The signals from the extended serial port are transmitted to parallel signals by a one-chip microcomputer (PIC16F84A; Microchip Technology Inc.) on the interface module. The total weight of the equipment, including the battery, is approximately 170 g for a single hand.

[reference]

  • T. Amemiya, K. Hirota, M. Hirose, "Wearable Tactile Interface for Way-Finding Deaf-Blind People using Verbal and Nonverbal Mode", Trans. of VRSJ, Vol. 9, No. 3, pp. 207-216, 2004. (in Japanese)
  • T. Amemiya, J. Yamashita, K. Hirota, M. Hirose, "Virtual Leading Blocks for the Deaf-Blind: A Real-Time Way-Finder by Verbal-Nonverbal Hybrid Interface and High-Density RFID Tag Space", In Proc. of IEEE Virtual Reality Conference 2004 (VR 2004), pp. 165-172, 2004.
back to top

Finger-Braille

Prototype
Finger-Braille Interface
How to type in Yubi-tenji
Finger-Braille was invented for the Deaf-Blind to communicate in real time in Japan, and it became one of the commonly used communication methods among Deaf- Blind people in Japan. In Finger-Braille method, the index finger, middle finger, and ring finger of both hands function like the keys of a Braille typewriter. A sender dots Braille code on the back of fingers of a receiver as if typing on a Braille typewriter.
Using Finger-Braille
Some examples of the translation are shown in the table below. Regardless of the language, Braille systems consist of 6 dots; the pattern for "na" under the Japanese Braille system is the same as "K" under the English one. Therefore it is easy to translate Braille into Finger-Braille under all languages. The Finger-Braille method enables Deaf-Blind people to obtain textual information as if they are listening in real time.
example of the translation a letter into a braille code and a Finger-Braille code
Wireless version with Linux watch
Wearable Finger-Braille Interface with WatchPad (a Linux-based watch)
Oboe-like Braille Input Interface for Wearable Computing
in The Univ. of Tokyo. ('04-05)

Mobile Chord Keyboard using Mechanical Switches for outdoor use

OBOE

We developed a wearable interface for textual input on the basis of Braille input method. The device, named OBOE, is operated by both hands, which is good for portability and can be used while standing. The users get their input operations confirmed clearly by feeling the click since the buttons of the proposed device are the same mechanical switches as used in keyboards for desktop computers. The results of an experiment of learning effect revealed that the users who had no experience of Braille input could type Japanese phrases at 35.4 Braille codes per minute, and who had experience at 112.4 Braille codes per minute. Thus novices can master the proposed device and experts can input text very fast by using OBOE. Based on the results of questionnaire by the subjects, we discussed the comparison with a Braille typewriter, the optimum layout of keys for OBOE, and the posture of holding.
We carried out an experiment to evaluate the device by measuring input speed and error rate. In the experiment, there were five trials to examine the effect of learning. All six subjects were male, right handed and did not have experiences with Braille typewriters. The experiment consisted of a tutorial (non-semantic phrase session) and a semantic phrase session. The tutorial session was a 10-minute trial, followed by the 10-minute semantic phrase session. In each session, the subjects were told to input as quickly and accurately as possible and not to sacrifice speed for accuracy. The subjects could confirm their input through auditory feedback by a TTS (text-to-speech) engine, IBM Protalker97, as well as through visual feedback from a window on screen. The results are shown in Figures.
We carried out another experiment with those people who have experiences with Japanese-to-Braille transcribing. The transcriber subject showed a higher input speed, 112.4 Braille codes per minute, and a lower error rate, 4.1 %, than the average of the novice subjects, with a little learning in the semantic mode. The transcriber subject commented in the tutorial mode that characters that rarely appear in usual Japanese sentences prevented the speedup. (e.g., "dya" is almost replaced by "jya" in Japanese sentence but the user had to input that kind of characters in the tutorial.)

[reference]

  • T. Amemiya, K. Hirota, M. Hirose, "Development and Evaluation of Oboe-like Braille Input Interface for Wearable Computing", Trans. of IPSJ, Vol. 46, No. 7, pp. 1701-1710, 2005. (in Japanese)
  • T. Amemiya, K. Hirota, M. Hirose, "OBOE: Oboe-like Braille interface for Outdoor Environment", In Proc. of 9th International Conference on Computers Helping People with Special Needs (ICCHP 2004), pp. 498-505, 2004.
back to top

OBOE: input device

OBOE
OBOE = Oboe-like Braille interface for Outdoor Environment
learning effect
Result of experiment
OBOE meets Finger-Braille interface
OBOE with wearable Finger-Braille interface