Creation of Haptic Virtual Reality

1. Introduction


Although recent progress of information technology has remarkably augmented the quality of communication, haptic (tactile and force) information is not incorporated in the contemporary communication media. The information content handled in the communication system is essentially for humans so that it should be optimized to the form humans can receive efficiently. All the sensation channels they have, the five senses, need to be engaged in communication to fully exploit the information channels of humans. Communication system for haptic channel has long been under development for several decades to extend mostly the function of our ’hand.’ However, haptic communication system is not necessarily available yet for ordinary people who work in a broad field since it is not applicable to general tasks.

Human haptic sensation is a sensor of mechanical information around the body and it enables manipulation of physical environment based on obtained information. The sensing is directed to both environmental mechanical interaction and inner status of own body. The haptic sensation consists of tactile and proprioceptive sensations each of which has own receptors. Information acquired by these receptors is integrated in central nerve system to consistently perceive physical status of both the body and the environment. Then the perceived information is provided to perform physical functionality of manipulation of environment based on actuation of the body. Exerted force, motion induced, and interaction at the surface are sensed and controlled based on operator’s intention. Thus the haptic sensation inevitably involves the control of body and environment. It is generated with a complex construct based on diverse interaction.

2. Haptics and communication

2.1. Communication of living organism

Communication occurs everywhere in the living organism and it is indispensable for its control. The communication related to the living organism is classified into two; an inner communication and an outer communication where the density of information flow differs. The former is for control of the body and the latter is for physical environment and interpersonal interaction. Communication technology for haptics usually focuses on the latter.

Communication is viewed as multilayered structure in its functional levels such as semantic (symbolic) interaction, outer manipulation, inner control, and basic signal levels. Haptic information is in more control side than symbolic side. It is received basically, then sent out to make a dynamic loop of cybernetic organization. Haptic information discussed in the later section is more in basic reception level since high quality haptic information has not been presented yet on a communication terminal specifically in the sense of tactile one.

2.2. Level and function of haptic communication

Haptic information as other kinds is discussed in several different levels and functions.

Basic signal level

Contact status and force exerted between the body and the environment obtained by mechanoreceptors. It is a raw data for other levels. Not only tactile sensation at the contact surface but proprioception due to the force exerted at the surface.

Inner control level

Kinetic status of the human body monitored by somatic sensation. The body is controlled by real-time feedback of this status information. It is a basis of manipulation of environmental objects. Some parts of this information are obtained by removing environmental status from the interaction between the body and the environment. The environmental status is considered to be removed based on the consistency of information of the operator’s body.

Outer manipulation level

Information on the continuous manipulation of environmental objects based on inner control. It is obtained by removing the information of inner control level. It involves the information of the body so that the boundary of inner/outer information is ambiguous. Accuracy of the information depends also on the model of the object the operator has.

Semantic level

Symbolic treatment of lower level information to other people as in a language. Haptic interactions such as a tap, a push, a pull, a scratch, a stroke, a twist, etc. have their own meaning in the specific context and in more general cultural conventions. Both language equivalent and proper representation are involved. Lexical range is small beyond comparison with a general language.

3. Implementation and use of haptic display

Haptic information terminal is not established even in the sense of the basic signal level. It is still in the early stage towards high quality presentation. Some developments in this course specifically focused on tactile sensation are presented in this and later sections.

3.1. Implementation of haptic display

Haptic display is to be built to represent both tactile and proprioceptive sensations. Either channel is not easy to implement since spatiotemporal and kinetic range of haptic display is very large. Tactile sensation arouses based on a very small stimulus; an order of mm change is easily detected. Receptors in the skin cover all parts of the body that has a complicated shape which changes its geometry very rapidly. The range and velocity of kinetic motion are broad in the three-dimensional real space. So far many types of haptic display have been proposed, almost all of which are intended to impart limited range stimulation to a single channel.

Three displays introduced in the present paper focus mainly on producing a realistic tactile sensation to the fingertip that is the most sensitive part of the body surface and the principal channel in manipulation of objects. Integration of tactile and proprioceptive presentations was examined with the tactile and force(proprioceptive) compound haptic displays. Both displays are among those with the highest resolution and reality ever developed. They are introduced in the next section.

3.2. Use of haptic display 

Haptic displays are applied to the tasks that are difficult to perform only with the visual and auditory channels. Essential parts of the functionality presented are the following.

  • Feel mechanical status of remote/virtual objects. Weight, hardness, surface texture, shape, and the fitness to the hand (body part) are felt. (Heat property accompanies the touch.)
  • Manipulate it spatially.
    • Remote control or manipulation of real objects in the real space.
    • Model control or manipulation of virtual objects in a virtual environment.
  • Haptic representation of non-haptic data.
    Mapping from untouchable space to the haptic space.
Research prototypes built by the author are in the first stage to create tactile sensation and its consistent rendering with a force reflection. The next section describes three basic designs of haptic display.

4. Design of displays for skin and force sensation

The skin sensation is generated chiefly by the stimulus added from the environment on the surface of a skin. The stimulation can be applied through either mechanical or electrical stimulation. Mechanical stimulation produces more localized and natural sensation of touch than electrical stimulation which activates peripheral nerves under the skin and often causes a pain. The authors have developed mechanical stimulation haptic displays which have a vibratory pin array contacted to the fingertip. Each pin is independently controlled in its amplitude and frequency of vibration. This design is targeted to produce texture sensation that we receive when we touch the surfaces with relatively small geometrical variation. Roughness (or smoothness) is the typical sensation that we experience with such a variation.

Three displays, a high density desktop type (TextureDisplay2R), and force reflection types (TextureExplorer, TD44FX) are described in the following sections.









4.1. TextueDisplay2R---a high-definition, wide- band tactile display

TextureDisplay2R shown in Fig. 1 is designed specifically for high definition tactile sensation. The display has a dense window in which fifty vibratory pins are arranged in a 5 x 10 array with a 2-mm inter-pin distance [1]. Each pin is driven by a piezoelectric actuator. The amplitude is controlled in 256 levels by changing the duty ratio of the PWM power output. The peak amplitude as a function of vibration frequency is shown in Fig. 2. Around 280 Hz is there a resonant frequency that is set there to optimize the amplitude at 250 Hz where skin sensation indicates highest sensitivity [2]. The highest output frequency is 410 Hz where peak amplitude maintains above 10 mm, and the lowest frequency is 40 Hz where another sort of mechanoreceptor exhibits high sensitivity [3].

Haptic texture presentation technique has not been established in its psychophysical aspects as well as the procedure for stimulation from the surface data of real objects to spatiotemporal control and the mechanical implementation of the display. The authors have introduced a new method in which variable-frequency stimulation is imparted based both on psychophysical property of tactile sensation and the control parameters of amplitude (intensity), frequency, and duration. Figure 3 shows the relation between the vibration frequency and the sensation magnitude. Based on the relation any intensity across frequency is properly presented. Mapping from sensation of real surface texture to the control space of the display in general is a challenge. The author has explored a simple mapping between a single frequency and several specific textures. In addition, mixture patterns of two or more frequencies are investigated to give a variety of sensations. Figure 4 shows samples used in the display evaluation. Subtle difference between these textures were sufficiently recognized by the subjects, and the matching test with real texture patches exhibited very high correct ratio, which indicated high quality of presented textures.

4.2. TextureExplorer

The TextureExplorer, shown in Fig. 5 [4], is a compound haptic display consisting of a vibratory pin-array and the Phantom (1.5AG) force display that is able to produce both cutaneous and proprioceptive stimulations. During exploration on a surface of a virtual object, the user can perceive stimulus variation within a surface at the fingerpad as well as reflection force from the object at the hand. This unique design simulates a real exploration of a surface by the finger, in which the skin contacts the object at multiple points in a surface area with different intensity of stimuli under a constraint force from the surface.

The system hardware consists of three parts, a visual display, a tactile stimulator and a force display, respectively. For visual presentation, a stereoscopic view is provided by an 80-inch rear projection screen using shutter glasses. A 3-D motion sensor (FASTRAK, Polhemus Inc.) is attached to the glasses to enable head tracking. For tactile stimulation, a vibratory pin-array display (Fig. 6) is attached to the stylus of the PHANToM with a handle to grip by the palm. The 3-degree of freedom (DOF) rotation of the pin-array is measured by another motion sensor. Ten pins synchronously deliver the three-dimensional distribution of stimulus at a limited area of the fingerpad. The force display imparts the information related to shape, texture and stiffness of the object based on the mode to the finger and palm simultaneously with cutaneous pin stimulation.

4.3. TD44FX (TextureDisplay44 force extension)

One more degree of freedom of force presentation was added in design of TD44FX shown in Fig. 7. Two small force displays (Phantom Omni) were incorporated to drive a pin-array that has forty-four contact pins for a fingertip. Each force display has three-DOF positioning. The position of the pin stimulator is controlled up to five DOF which produces moment of force during touch to the object surface. A three dimensional surface can be explored with more natural force feedback including a torque occurs from friction between the virtual finger and objects.

5. Conclusions

The displays described here are still under development to make a hi-fidelity terminal for haptic communication. The stage is in the basic signal level to create high quality virtual touch sensation that is equivalent to the real experience. A part of this objective has been achieved although much left to be improved, for example, in coding of texture information in the vocabulary of display devices, and in elucidation of texture perception of humans as a basis for the coding.

Aside from cutaneous texture presentation, force presentation to the hand implemented here and by researchers in this field is also limited to reduced degrees of freedom. In that case, outer manipulation level and inherent inner control level have not realized at the acceptable level for general users. TextureExplorer and TD44FX designed to produce both cutaneous and force information show a method to compensate for the lack of feedback DOF. The benefit of tactile sensation was remarkable in object perception with reduced stiffness for stability.


Part of the present research was performed under the support of the special research fund of TMIT and the Grant-in-Aid for Scientific Research. The author would like to thank contributors of this study in the lab: Mariko Yamada, Shigeki Arata, Masashi Shiratori, and project support members.


[1] Y. Ikei, M. Yamada, and S. Fukuda. “A new design of haptic texture display–texturedisplay2– and its preliminary evaluation,“ Proc. of IEEE Virtual Reality 2001, pp.21–28 (2001).

[2] H. Oyama, Y. Ikei, “Tactile sensation measurement for the design of a vibratory haptic display,” Proc. of ICAT2004, pp. 389-396 (2004).

[3] S. Arata, Y. Ikei, “Wideband vibratory stimulation on the TextureDisplay2R,” Proc. of EuroHaptics 2006, pp. 625-627, (2006).

[4] Y. Ikei, M. Shiratori, “TextureExplorer: A tactile and force display for virtual textures,” Proc. of Haptic symposium, pp. 327-334, (2002).

(A part of this text was publised at Universal Communication Symposium 2007)

Yasushi Ikei @Tokyo Metro Univ 2007

Return To Home




Fig. 1. TextureDisplay2R (TD2R)



Fig. 2. Peak amplitude of TD2R as
a function of drive frequency. Natural
frequency was set around 280 Hz to
optimize frequency band at 250 Hz



Fig. 3. Equal sensation magnitude curves


Fig. 4. Texture data displayed on TD2R


Fig. 5. TextureExplorer consisting of a vibratory cutaneous stimulator and a force display.


Fig. 6. Pin stimulator of 10 contact pins driven by bimorph piezoelectric actuator. Intricate body was fabricated by stereolithography


Fig. 7. TD44FX: Texture Display with 44-pin stimulator driven by two force displays.