New haptic illusion (using characteristic of human perception)
and force-feedback haptic device
Most mobile devices including cellular phones and portable video players cannot generate a "pull" or "push"
sensation. The reason is that they lack the external fulcrum to
generate force vector.
How to generate force sensation in the mobile devices
Our concept is using nonlinearity
of human perception to induce force vector; humans feel rapid
acceleration more strongly than slow acceleration. Our system uses
neither grounding styles (such as PHANToM, SPIDAR) nor traditional
nongrounding styles (gyro effect, air jet). Instead, we designed the
force-feedback device that exploits the nonlinearity of human haptic
perception. The revolution is in the perception method of force sensation;
humans strongly feel rapid acceleration of the slider (right gray ball in the animation below) more than slow one since its acceleration is
asymmetric. (humans do not notice low-amplitude force of long duration.) We utilized the haptic "wash-out" to generate a pull sensation.
The proposed force perception method subjects a
mass to periodic translational motion; the mass is accelerated
more rapidly in one direction than the other. If scaled correctly,
humans translate the asymmetric acceleration into a one-directional
force because of their perception characteristics. While this
force display system can be categorized as a non-grounded device
(it has no fulcrum), it can generate constant force by repeating one cycle of motion.
**You can change the speed of the crank by the upper two buttons.
Buru Navi (Laval Virtual 2007 GrandPrix du Jury Winner)
Tandem pairs to counteract the side-to-side force generated by the motion of linkages
We evaluated the characteristics of force perception
when the crank-slider mechanism generating asymmetric acceleration
was held in the air. Our results indicate that the swinging force
generated by the motion of linkages prevents us from well perceiving
the force sensation; the motion of linkages makes it difficult for
users to distinguish between a simple vibration and the directed
force sensation. Moreover, the results revealed that resolution
with which the force direction can be discriminated with the antiphase
tandem pair is significantly smaller than that with the inphase
one. i.e. limiting the direction of asymmetric acceleration to one
axis is effective when people hold the force display in the air.
Force sensation more than rumble
The force display would replace vibrators (or use together) in
all mobile devices. Applications include the cellular phones draw
your hand where you want to go.
T. Amemiya, T. Maeda, "Directional Force Sensation by Asymmetric Oscillation from a Double-layer Slider-crank Mechanism",
Trans. of the ASME Journal of Computing and Information Science in Engineering, Vol. 8, No. 3, 2008.
T. Amemiya, H. Ando, T. Maeda,
""Lead-Me Interface" for a Pulling Sensation from Hand-held Devices", ACM Trans. on Applied Perception, Vol. 5, No. 4, 2008.
T. Amemiya, H. Ando, T. Maeda, "Directed Force Perception When Holding a Nongrounding Force Display in the Air", In Proc. of EuroHaptics 2006,
pp. 317-324, 2006.
T. Amemiya, H. Ando, T. Maeda,
"Perceptual Attraction Force: The Sixth Force", ACM SIGGRAPH 2006 Emerging Technologies, 2006.
T. Amemiya, H. Ando, T. Maeda, "Virtual
Force Display: Direction Guidance using Asymmetric Acceleration via Periodic Translational Motion", In Proc. of World Haptics Conference
2005, pp. 619-622, March 200
The world is distorted when we perceive it.
The basic mechanism to create an asymmetric oscillation and its acceleration profile.
Flyer of Buru-Navi for SIGGRAPH 2006
Force feedback "Tray" for novice waiters
in NTT CS Labs. ('07-08)
A waiter (user) in a cafe wants to deliver a drink ordered by a customer (target).
The waiter does not know where the customer is sitting. However, his "smart
tray" creates an attraction force centered on the customer and guides the waiter
to him/her. Since the guidance is based on force sensation, the guidance
information is useful regardless of the waiter's age or language ability. Moreover,
since the guidance directions are transmitted via touch, the other senses remain
available to the waiter, making it easier to move through even the most crowded
areas. Finally, the instructions remain entirely private; no one else can discover
that the waiter is receiving instructions.
In the demos, the users moved to the target according to the direction
of the perceived force sensation. The direction of the force display
was controlled so that they face the target based on the posture detection
The user chooses one customer by stopping in front of the target, projected
movie or bear-shaped robot. If this choice is correct, the customer
says "thank you"; otherwise, the customer says, "I did
not order this".
Force feedback"tray" for novice waiters consists of Buru-Navi,
rotation mechanism,and position and posture identification system.
The system consists of a tray (approximately 750 g in weight) held by the user
(waiter), a small bag containing a battery and a control device (approximately
300 g), and a position and direction identification system based on infrared LEDs
and a wide-angle camera. Our haptic interface (force display), infrared
LEDs, and stepper motor are embedded in the tray.
T. Amemiya, T. Maeda, H. Ando, "Location-free
Haptic Interaction for Large-Area Social Applications", Personal
and Ubiquitous Computing, 2008.
T. Amemiya, T. Maeda, H. Ando, "Bear's
Beer", DAT, Singapore Science Center, Singapore, December 2007.
T. Amemiya, T. Maeda, H. Ando, "Come
Over Here, or Catch You", Laval Virtual ReVolution, Laval, France, April 2007.
Demonstration in France
Demonstration in Singapore
Mobility support for people with disabilities
in NTT CS Labs. and Kyoto City ('07-09)
Some emergency situations, such as fires or earthquakes, require that
evacuation to a safe area, often through an emergency exit. This is especially
difficult for people with visual disability. Here, we propose a new device, a
haptic direction indicator, which will help blind pedestrians intuitively and safely escape from dangerous area by means of haptic navigation.
We performed an experiment for predefined route guidance using the force display to evaluate the time required
for completing a walking task,correctly perceived directions, user preferences, and the level of easeof understanding.
We built a human-size maze in the gymnasium of the Kyoto Prefectural School for theVisuallyImpaired, Japan.
Since the streets and avenues in Kyoto are mainly laid out in a grid, the maze was designed with checkerboard-shaped routes.
T. Amemiya, H. Sugiyama, "Haptic Handheld Wayfinder with Pseudo-Attraction Force for Pedestrians with Visual Impairments", In Proc. of 11th ACM Conference on Computers and Accessibility (ASSETS 2009), pp. 107-114, Pittsburgh, PA, October 2009.
T. Amemiya, H. Sugiyama, "Navigation in Eight Cardinal Directions with Pseudo-Attraction Force for the Visually Impaired", In Proc. of IEEE International Conference on Systems, Man and Cybernetics (SMC 2009), Texas, October 2009.
T. Amemiya, "Haptic Direction Indicator for Visually Impaired People Based on Pseudo-Attraction Force," eMinds: International Journal on Human-Computer Interaction, Vol. 1, No. 5, pp.23-34, March 2009.
T. Amemiya, H. Sugiyama, "Design of a Haptic Direction Indicator for Visually Impaired People in Emergency Situations", In Proc. of 11th International Conference on Computers Helping People with Special Needs (ICCHP 2008), pp.1141-1144, Linz, Austria, July 2008.
Pilot study with people with visual impaierment
Buru-Navi3: tiny but mighty being pulled sensation (New)
in NTT CS Labs.('14-)
We have succeeded in developing a thumb-sized force display for experiencing a kinesthetic illusory sensation of being continuously pulled. Previous version having a crank-slider mechanism succeeded in producing a similar sensation, but had limitations in its size and weight. We overcame these limitations by using a thumb-sized actuator that oscillates asymmetrically. User quantitative evaluation indicates that specific asymmetrical vibration is effective to create kinesthetic illusory sensation of being pulled. Small and light-weight force display will be useful for a handy somatosensory-based navigation system.
We have fabricated two prototypes: a one-DoF force display and a two-DoF force display. The size of the
one-DoF force display is greatly decreased by 95% (to 18x18x37 mm3) compared with the earlier one. The weight of the one-DoF force display is greatly decreased by 92% (to 19 g). Using two orthogonally placed actuators allows the holders to feel a force sensation in the four or eight principal directions on the azimuth plane.
Are you satisfied with a nibbling sensation created by conventional fishing games? If not, try Buru-Navi3.
T. Amemiya, H. Gomi, "Buru-Navi3: Behavioral Navigations Using Illusory Pulled Sensation Created by Thumb-sized Vibrator", In Proc. of ACM SIGGRAPH 2014 Emerging Technologies, Vancouver, Canada, August 2014 (accepted).
T. Amemiya, H. Gomi, "Distinct pseudo-attraction force sensation by a
thumb-sized vibrator that oscillates asymmetrically", In Proc. of Eurohaptics 2014, Versailles, France, June 2014. [Best Demonstration Award]
Weight perception has been of great interest for over three centuries. Most research has been concerned with the weight of static objects, and some illusions have been discovered. Here, we show a new illusion related to the perception of the heaviness of oscillating objects. We performed experiments that involved comparing the weight of two objects of identical physical appearance but with different gross weights and oscillation patterns (vibrating vertically at frequencies of 5 or 9 cycles/s with symmetric and asymmetric acceleration patterns).
The results show that the perceived weight of an object vibrating with asymmetric acceleration increases compared to that with symmetric acceleration when the acceleration peaks in the gravity direction. In contrast, almost no heaviness perception change was observed in the antigravity direction. We speculate that the reason for the divergence between these results is caused by the differential impact of these two hypothesized perceptual mechanisms as follows: the salience of pulse stimuli appears to have a strong influence in the gravity direction, whereas filling-in could explain our observations in the antigravity direction. The study of this haptic illusion can provide valuable insights into not only human perceptual mechanisms but also the design of ungrounded haptic interfaces.
T. Amemiya, T. Maeda, "Asymmetric Oscillation Distorts the Perceived Heaviness of Handheld Objects", IEEE Transactions on Haptics, Vol. 1, No. 1, pp. 9-18, Jan-Jun, 2008. .
Psychometric functions for the perceived weight change generated by asymmetric versus symmetric acceleration.
Tactile duration perception
in NTT CS Labs. JST, and UCL('06-)
Time Perception in haptic modality
Perception of time over sub-second scale is fundamental
for many aspects of human activities. However, its neural mechanisms
for the time perception are not clarified well: modality specific and distributed mechanism vs. universal and centralized one in the brain.
We report tactual timing perception using the
adaptation paradigm. The experimental results implicate that the
duration perception is encoded by spatially localized temporal mechanisms.
J. Watanabe, T. Amemiya, S. Nishida, A. Johnston, "Tactile Duration Compression by Vibrotactile Adaptation", NeuroReport,
Vol.21, No. 13, pp. 856-860, August 2010.
J. Watanabe, T. Amemiya, S. Nishida, A. Johnston, "Distortion of tactile duration perception", In Proc. of NEUROSCIENCE 2007, Society for Neuroscience, San Diego, CA, Nov. 2007.
T. Amemiya, H. Ando, S. Nishida, A. Johnston, and J. Watanabe, "Tactile Duration Perception using an Adaptation Paradigm", In Proc.
of VRSJ Annual Conference, pp.33-34, 2006.
Tactile duration perception
Universal clock? Distributed?
Human interface applying galvanic vestibular stimulation (GVS)
GVS is an electrical stimulation technique which can generate virtual
acceleration toward anode side. (The original method was discovered over 100 years ago.)
Our team made applications for GVS interface as one of key technologies
for parastic humanoid project.
The two males' vestibular system, which controls their senses of balance, has been controlled by weak electrical currents delivered
just behind her ears.
When a weak DC current is delivered to the mastoid behind your ear,
your body responds by shifting your balance toward the anode. If it is strong enough, it not only throws you off balance but alters
the course of your movement.
T. Maeda, H. Ando, Tomohiro Amemiya, M. Inami, N. Nagaya, M. Sugimoto, "Shaking The World:
Galvanic Vestibular Stimulation As A Novel Sensation Interface", ACM SIGGRAPH 2005 Emerging Technologies, 2005.
GVS for VR application
SIGGRAPH 2005 (human remote control)
SIGGRAPH 2005 (racing video game). Feel the "G" (centrifugal force) in the corners.
SIGGRAPH 2005, (synchronization with music rhythm: Shaking the World)
Wearable Finger-Braille Interface for the Deaf-Blind People
in The Univ. of Tokyo ('02-04)
Wearable tactual device for communication among
Application possibilities of augmented reality technologies
in the field of mobility support for the Deaf-Blind is discussed.
We propose the navigation system called Virtual Leading
Blocks for the Deaf-Blind, which consists of a wearable interface for
Finger-Braille, one of the commonly used communication methods among
Deaf-Blind people in Japan, and a ubiquitous environment for barrier-free
application, which consists of floor embedded active radio-frequency
identification (RFID) tags. The wearable Finger-Braille interface using
two Linux-based wristwatch computers has been developed as a hybrid
interface of verbal and nonverbal communication in order to inform users
of their direction and position through the tactile sensation. We propose
the metaphor of "watermelon splitting" for navigation by this
system and verify the feasibility of the proposed system through experiments.
Glove-style interfaces seem to be the most suitable design
for the Finger-Braille interface because they are easy to wear. However,
they cover the palm or the fingertip which has the highest tactile sensitivity.
Therefore glove-style interfaces prevent Deaf-Blind people from obtaining
environmental information by tactile sensation. In our previous work
(Figures below), two types of ring-shaped devices were developed using
two kinds of actuators: (a) vibration-motor type and (b) solenoid type,
as shown in Figure. The vibration-motor type consists of six small-size,
lightweight DC motors (CM05J; TPC) the frequency of which is approximately
116 Hz, since it has been reported that 120 Hz is the optimal frequency
to stimulate the back of the finger. The solenoid type consists of six
tubular solenoids (S-50S03; Shindengen Electric Manufacturing Co.) weighting
15 g each. Each actuator is fixed onto the setting of the ring. Each
type is connected to a computer via a USB 1.1 port.
Previous Version of Finger-Braille Interface
A new wireless prototype using Bluetooth technology was
implemented. The actuator is the same vibration-motor as previous one
from the previous experiment. Our system consists of a wearable computer
and two Finger-Braille interfaces, and we linked them via wireless communication
using two Linux-based wristwatch computers, called WatchPad1.5 codeveloped
by IBM Japan and Citizen Watch.
The user can read letters through the tactile channel
by wearing the two wristwatch computers and two modules which consist
of batteries and circuits for actuators on the wrists. In addition,
WatchPad1.5 is equipped with a serial connector for communication. The
signals from the extended serial port are transmitted to parallel signals
by a one-chip microcomputer (PIC16F84A; Microchip Technology Inc.) on
the interface module. The total weight of the equipment, including the
battery, is approximately 170 g for a single hand.
T. Amemiya, K. Hirota,
M. Hirose, "Wearable Tactile Interface for Way-Finding Deaf-Blind People using Verbal and Nonverbal Mode", Trans. of VRSJ, Vol. 9, No. 3, pp. 207-216, 2004. (in Japanese)
T. Amemiya, J. Yamashita, K. Hirota,
M. Hirose, "Virtual Leading Blocks for the Deaf-Blind: A Real-Time Way-Finder by Verbal-Nonverbal Hybrid Interface and High-Density
RFID Tag Space", In Proc. of IEEE Virtual Reality Conference 2004 (VR 2004), pp. 165-172, 2004.
Finger-Braille was invented for the Deaf-Blind to
communicate in real time in Japan, and it became one of the commonly
used communication methods among Deaf- Blind people in Japan.
In Finger-Braille method, the index finger, middle finger, and ring finger of both hands function like the keys of a Braille typewriter.
A sender dots Braille code on the back of fingers of a receiver as if typing on a Braille typewriter.
Some examples of the
translation are shown in the table below. Regardless of the language,
Braille systems consist of 6 dots; the pattern for "na"
under the Japanese Braille system is the same as "K"
under the English one. Therefore it is easy to translate Braille
into Finger-Braille under all languages. The Finger-Braille method
enables Deaf-Blind people to obtain textual information as if
they are listening in real time.
Wearable Finger-Braille Interface with WatchPad (a Linux-based watch)
Oboe-like Braille Input Interface for Wearable Computing
in The Univ. of Tokyo. ('04-05)
Mobile Chord Keyboard using Mechanical Switches for outdoor use
We developed a wearable interface for textual input
on the basis of Braille input method. The device, named OBOE, is operated
by both hands, which is good for portability and can be used while
standing. The users get their input operations confirmed clearly by
feeling the click since the buttons of the proposed device are the
same mechanical switches as used in keyboards for desktop computers.
The results of an experiment of learning effect revealed that the
users who had no experience of Braille input could type Japanese phrases
at 35.4 Braille codes per minute, and who had experience at 112.4
Braille codes per minute. Thus novices can master the proposed device
and experts can input text very fast by using OBOE. Based on the results
of questionnaire by the subjects, we discussed the comparison with
a Braille typewriter, the optimum layout of keys for OBOE, and the
posture of holding.
We carried out an experiment to evaluate the device by measuring
input speed and error rate. In the experiment, there were five trials
to examine the effect of learning. All six subjects were male, right
handed and did not have experiences with Braille typewriters. The
experiment consisted of a tutorial (non-semantic phrase session) and
a semantic phrase session. The tutorial session was a 10-minute trial,
followed by the 10-minute semantic phrase session. In each session,
the subjects were told to input as quickly and accurately as possible
and not to sacrifice speed for accuracy. The subjects could confirm
their input through auditory feedback by a TTS (text-to-speech) engine,
IBM Protalker97, as well as through visual feedback from a window
on screen. The results are shown in Figures.
We carried out another experiment with those people
who have experiences with Japanese-to-Braille transcribing.
The transcriber subject showed a higher input speed, 112.4 Braille
codes per minute, and a lower error rate, 4.1 %, than the average
of the novice subjects, with a little learning in the semantic
mode. The transcriber subject commented in the tutorial mode
that characters that rarely appear in usual Japanese sentences
prevented the speedup. (e.g., "dya" is almost replaced
by "jya" in Japanese sentence but the user had to
input that kind of characters in the tutorial.)
T. Amemiya, K. Hirota, M. Hirose, "Development and Evaluation of Oboe-like Braille Input Interface for Wearable Computing", Trans.
of IPSJ, Vol. 46, No. 7, pp. 1701-1710, 2005. (in Japanese)
T. Amemiya, K. Hirota, M. Hirose, "OBOE: Oboe-like Braille interface for Outdoor Environment", In Proc. of 9th International Conference
on Computers Helping People with Special Needs (ICCHP 2004), pp. 498-505, 2004.
OBOE: input device
OBOE = Oboe-like Braille interface for Outdoor Environment