Haptic Environment for Designing Human Interface of Virtual Mechanical Products

F.Kimura1(1), N.Yamane1
1
Department of Precision Machinery Engineering
The University of Tokyo, Tokyo, Japan

Abstract
For efficient and high-quality design of mechanical products, it is now possible to use virtual product models
as a basis for functional simulation and verification. However it is not easy to evaluate human operational
interfaces of such virtual products only by graphical interfaces. In this paper, a method is proposed to
simulate human operational interfaces by use of generic haptic devices. Based on behavioural simulation of
virtual product models, appropriate information for haptic interaction is generated, and machine operational
interfaces are simulated by use of generic haptic devices. For evaluating the effectiveness of the proposed
method, assessment of virtual mechanical switching operations has been performed.

Keywords:
Virtual Prototyping, Haptic Interface, Simulation

1 INTRODUCTION behaviour can be represented. Based on such

In recent years, a development process of mechanical
products has been renovated by comprehensive usage of
computer support technology for products and associated
processes. It is called virtual product development
process, and most of the product functionality can be
evaluated based on computer simulation of product
behaviour. By eliminating tedious trial and error
processes in physical prototyping, a product development
process can be made shorter and efficient.
However, the design of human interfaces for machines
cannot be easily performed by a virtual approach.
Sensory interaction with machines is very delicate, and
cannot be easily evaluated by simple methods. Typical
examples are the driving feel of automobiles, or easy
manipulation of audio/visual equipments. Physical mock-
ups are normally required for verifying user interface
design of such products.
An objective of this research is to realize a haptic
environment for developing realistic user interfaces for
newly designed products based on virtual mechanical
models, and to eliminate the need of making cumbersome
physical mock-ups. Probably imitation of true shapes and
movable objects of product interfaces directly
manipulated by users may be necessary for avoiding
psychological discrepancy. For example, a cockpit
arrangement is modelled by a mock-up, but the actual
feel of steering operations may be given, not by a real
steering mechanism, but by a simulated virtual
mechanism. By such installation, high-quality human user
interface design can be made by flexibly changing
behaviour characteristics of associated virtual machine
products.
The approach proposed above has become possible due
to the advances of virtual product design based on
product modelling[1,2]. For touch-and-feel evaluation of
human interface of machine products, very precise
sensation of interface device behaviour is essential, such
as stiffness, stickiness, smoothness, vibration, etc. By
sophisticated modelling of components and their
assembly in products, various kinds of disturbances are
simulated, and, in addition to nominal behaviour, realistic
technology, product deterioration during product usage
has been modelled, and used as a basis for life cycle
maintenance planning[3,4].
Virtual environments are extensively studied by VR
(Virtual Reality) research[5]. Various sophisticated visual,
audio and tactile devices have been developed. However,
the research effort has been focused on virtual interfaces,
and not much effort has been spent on simulation of
physical machines behind the virtual environment.
Various kinds of haptic devices and interfaces have been
studied[6,7,8,9]. They are focusing on rapid response,
and, as a consequence, behavioural modelling of objects
is necessarily simple. The approach on virtual gearshift
applications[10,11] is similar to the objective of this
research. The virtual gearshift research is focusing on
designing good gear shift feel before going into the
technical design of gear shift mechanisms. This paper
aims to support design of virtual mechanical products
which gives good feel for human users. For evaluation by
users, a haptic environment is constructed, which mimics
real physical interfaces of machines.
2 DESIGNING HUMAN INTERFACE
A basic concept for virtual product development and
human user interface design is discussed.
2.1 Fusion of real and virtual objects
At the start of a virtual product creation process, some
components are completely newly designed, and virtual
models must be constructed. Other components are
reused from the existing design. Therefore a product
consists of a mixture of virtual models and real objects.
According to the progress of the development, virtual
objects are gradually replaced by real ones, and all the
components are represented by real objects at the end of
the development. Such flexible management of virtual
and real objects is one of the key issues. Appropriate
interfaces are required to connect and to integrate virtual
and real components, as shown in Figure 1.
If product functionality can be totally evaluated based on
established theory, functional evaluation, in principle, can

Annals of the CIRP Vol. 55/1/2006

be performed in the virtual world. But, human interface is
an important component in product design. Human users
should be involved in product development from the start,
and real human interface should be integrated for
evaluation by users. Such human interfaces can be
considered to be similar to the interface between real and
virtual components, as shown in Figure 1. Front-end
human interfaces may use real objects during the design,
and evolve from generic to dedicated ones.
Real Virtual
Component Component
Interface Interface
Human
Virtual
Interface:
Front-End Component
Human
Interface
Interface
Figure 1: Virtual/real fusion and human interface.
2.2 Human interface construction
Based on the above approach, consistent product
development can be carried out, which integrates and
satisfies human interaction requirements. Human
interfaces are constructed as shown in Figure 2. Human
interfaces can be considered as virtual machines. At the
start of the development, it is simply a combination of real
object front-end, sensors and actuators. The front-end of
an interface may be a true real object which enables
realistic sensations. Gradually simulation parts are
reduced, and real components are increased.
Real Object Interface
Front-End (Actuator,
Sensor, etc.)
(a) Start of the design: (b) End of the design:
Generic interface Dedicated interface
Figure 2: Evolution of human interface.
Based on such systematic development, all the human
interactions, such as operation behaviour and feeling, can
be captured by sensing at the interface, and reactive
actions are given digitally to human users. Therefore
human interface design can be quantified, and rational
design methods can be derived. This is the most effective
result by using this approach.
Interface characteristics include many items, such as
visual displays, tactile sensing, motion, vibration, heat,
etc. In the following discussion, haptic interaction is
treated in detail.
3 VIRTUAL MECHANICAL PRODUCT
A precise behaviour modelling of mechanical products is
essential for realizing virtual human interfaces. Based on
simulation results, necessary information for driving virtual
interfaces is extracted. A generic modelling structure of
virtual mechanical products is shown in Figure 3.
A product model consists of four types of sub-models:
component, assembly, behaviour and function models.
Each sub-model is semantically connected. A component
is an individual part. Assembly relations represent various
assembling constraints, such as fixed or movable, etc.
Associated with each assembling relations or constraints,
appropriate behaviour analysis procedures are linked. By
giving some boundary/initial conditions, behaviour of
products is simulated, and results are represented in a
behaviour model. Function models are application-
dependent. Based on given functional requirements,
model behaviour is interpreted to fullfil the requirements
or not.
Functional Function Interface
Requirement
Information
Behaviour
Behaviour
Disturbance Assembly Analysis
Manufacture
Usage, etc.
Component
Figure 3: Modelling with disturbance.
For user interface design applications, subtle evaluation
of product behaviour becomes important. For example,
feeling of manipulating automobile steering depends on
rotational behaviour of a steering wheel, such as smooth,
sticky, rattling, etc. Such precise behaviour appears not
only from nominal product structure, but also from small
disturbances on nominal structures.
There are many causes for such disturbances: parts
manufacturing errors, material variations, assembly
errors, deterioration during usage, etc. It is possible to
simulate the effects of such small disturbances by giving
small model variations to a nominal model. Some
disturbances can be represented by changing model
parameters, and some disturbances must be represented
by changing geometrical shape of parts or parts
assembly/contact conditions. Preliminary experiments
have been done for simulating parts deterioration of
mechatronics products[12,13]. A similar method has been
applied to an example described in Section 4.
In many cases, simulation becomes very complicated,
and cannot be done in real time. For linking to human
interfaces, behaviour simulation data is required in real
time, so special arrangements are mandatory. In practical
cases, types of interaction by human users are
categorized according to required operation modes.
Therefore it is possible to make a set of categorized
simulations beforehand, and to generate required
simulation behaviour by interpolation of category
parameters and sensitivity analysis. The idea has been
tested for examples described in Section 4.
4 PROTOTYPE EXPERIMENT
For verifying the concept described in the previous
sections, a prototype system for realizing haptic
environments have been constructed. By taking an
example of electric switch design, appropriate human feel
is evaluated for operating the switch.
4.1 Prototype system haptic interface. By the feeling of users, a CAD model is
modified, and simulation is repeated. Modified product
The prototype system aims at realizing a general system parameters are re-evaluated, and appropriate product
platform for evaluation of human operational interfaces of attributes are determined without relaying on physical
small mechatronics devices, such as switches, buttons, or prototyping.
dials. It is expected that, based on this prototyping
experiment, a comprehensive system for haptic design
Plunger
environment can be elaborated.
Push-Rod
The architecture of the prototype system is shown in
Figure 4. It consists of three main components: a product See-Saw-Block
modeller, a behaviour simulator and a haptic interface. A Ball-End Contact
product modeller accepts basic models of target
mechanical products from CAD systems, extends model Movable Contact
representations for behaviour simulation, as described in Fixed Contact
section 3, and generates appropriate finite element
models. A behaviour simulator is a system framework for
(a) Switch Components
integrated analysis of product behaviour by use of various
simulation software, such as dynamics, elastic/plastic
behaviour, contact phenomenon, etc. A haptic interface
generates appropriate physical behaviour of target
products based on input from a human operator and
associated simulated product behaviour from a simulator.
The basic construction of the prototype system is general.
According to the required applications, appropriate CAD
systems, simulation software and haptic devices can be
integrated. In this case study, a commercial 3D CAD
system[14], and a general purpose nonlinear dynamic
finite element analysis software package[15] are used.
As a haptic device, also a commonly used force feedback
device is adopted[16].

(b) CAD Model (c) Mesh Model

Front-End Simulation CAD Figure 5: Switch as a target product.
(Haptic,etc.)
Operating Release Free
Position Position Position
Behaviour
Generic Analysis Product
Interface (Real time Modeller
interface)
Software Platform

Figure 4: A prototype system for human interface design.
4.2 Switch as a target product
A miniature pushbutton switch[17] is taken as an
example, as shown in Figure 5 (a)(b). With each push
operation of a plunger, contact states of a switch alternate
by an alternate positioning of a see-saw block which
pushes and changes switch contacts.
For a comfortable switching operation, careful design of
plunger stroke movements and reaction forces is
necessary. As shown in Figure 6, following product
parameters are known to be important[18]:
- Force: Operating Force(OF), Total travel Force(TTF),
Release Force(RF)
- Position: Free Position(FP), Operating Position(OP),
Total Travel Position(TTP), Release Position(RP)
- Travel: Pre-Travel(PT), Over-Travel(OT), Total-
Travel(TT)
In the conventional development processes, those
parameters are carefully adjusted by trial and error based
on physical prototyping. Our aim is to model the total
behaviour of switch operations in computer, and to get
such product parameters by simulation. Then such
product parameters are shown to human users through a
Total Travel
Position
Figure 6: Product parameters for a switch.
4.3 Simulation of switching behaviour
For the evaluation of product parameters, it is necessary
to perform a comprehensive simulation of the behaviour
of a switch. From a CAD model of a switch, a finite
element model, as shown in Figure 5(c), is generated for
analysis by nonlinear dynamic analysis software.
Each component of a switch is modelled as an elastic
body. A plunger and a push-rod are connected by a
rotational constraint. Friction contacts between a push-rod
and a see-saw block are treated by an iterative method.
Other connections and contacts are appropriately
modelled.
An example of switching behaviour is shown in Figure 7.
Push velocity of a switch button or a plunger is set as
2.3mm/s to 9.2mm/s. A profile of the reaction force
against push distance is very important for human feeling
of operating a switch. By changing the stiffness of a
movable contact spring, a different profile can be
computed, as shown in Figure 8.
 products under disturbance condition. By changing model
parameters and disturbances, delicate difference of
human feel can be tested, and appropriate human
interface can be designed. The prototype software
Reaction platform is flexible, and can be applied to other
Force(N) applications.
2.5

2 6 REFERENCES
押し込み速度 [1] Krause, F.-L., Kimura, F., Kjellberg, T., Lu, S.C.-Y.,
反力（N）

1.5 2.3mm/s 1993, Product Modelling, Annals of the CIRP,

4.6mm/s 42/2:695-706.
1 9.2mm/s
[2] Kimura, F., 1993, Product and Process Modelling as
Push a Kernel for Virtual Manufacturing Environment,
0.5
Velocity Annals of CIRP, 42/1, 147-150.
0 [3] van Houten, F.J.A.M., Tomiyama, T., Salomons,
0
0.37
0.74
1.1
1.47
1.84
2.21

2.94
2.58

3.31
3.68
4.05
4.42

O.W., 1998, Product Modelling for Model-Based

Maintenance, Annals of CIRP, 47/1, 123-128.
Push Distance(mm)
押し込み量（mm）
[4] van Houten, F.J.A.M., Kimura, F., 2000, The Virtual
Maintenance System: A Computer-Based Support
Figure 7: Simulation result of a switch (case 1). Tool for Robust Design, Product Monitoring, Fault
Diagnosis and Maintenance Planning, Annals of
Reaction CIRP, 49/1, 91-94.
Push押し込み速度4.6mm/s
Velocity 4.6mm/s [5] Lu, S.C.-Y., Shpitalni, M., Bar-Or, R., Gadh, R.,
Force(N)
4 1999, Virtual and Augmented Reality Technologies
3.5 for Product Realization, Annals of CIRP, 48/2, 471-
Stiffness
元のモデルa 496.
3
Stiffness
ばねの強さ2倍 b [6] Kim, Y.J., Otaduy, M.A., Lin, M.C., Monocha, D.,
2.5
反 力 （N）

2003, Six-Degree-of-Freedom Haptic Display using

2
Incremental and Localized Computations, Presence,
1.5 12/3, 277-295.
1 [7] Berkelman, P.J., Hollis, R.L., 2000, Lorentz
0.5 Magnetic Levitation for Haptic Interaction: Device
0 Design, Performance, and Integration with Physical
Simulations, Int. Journal of Robotics Research, 19/7,
0.28
0.55

1.93
2.21

3.31
3.59
3.86
0

0.83
1.1
1.38
1.66

2.48
2.76
3.04

4.14
4.42

Push Distance
押し込み量（mm）(mm)
Figure 8: Simulation result of a switch (case 2).
4.4 Evaluation by haptic environment
Simulation results, as shown in Figure 7 and 8, are given
to a haptic interface via real-time interpolation module. As
a prototype experiment, a commercial haptic device is
used for this case study. By detecting movements of
button pushing operations by human users, position and
velocity information is fed to the haptic device. Then,
based on an impedance control principle, appropriate
forces are given back to users according to the results of
simulation.
Preliminary evaluation of human sensation during the
switch operation is performed, and qualitative coincidence
between virtual evaluation and physical evaluation is
confirmed. As an experimental arrangement, a front-end
of a haptic device is connected to a normal button of a
switch. By this, visual and psychological discrepancy for
virtual evaluation is avoided.
5 SUMMARY
A concept of real/virtual fusion and human interface
construction is discussed. A haptic environment is
considered in this conceptual framework, and a prototype
system for designing push-button switches has been
developed. Preliminary experimental results show the
effectiveness of the proposed approach.
Human interface information for a haptic device is
generated by precise behavioural simulation of target
644-677.
[8] Hasegawa, S., Ishii, M., Koike, Y., Sato.,M., 2003,
Real-time Rigid Body Simulation based on
Volumetric Penalty Method, Proc. Haptic Interfaces
for Virtual Environment and Teleoperator Systems,
326-332.
[9] Love, L., Book, W., 1995, Contact Stability Analysis
of Virtual Walls, Proc. Dynamic Systems and Control
Division, ASME, 689-694.
[10] van Houten, F.J.A.M., 2001, The Use and
Development of Haptic Devices and Virtual Reality
as Engineering Tools, Proc. The 34th CIRP
International Seminar on Manufacturing Systems,
275-283.
[11] Tideman, M., van der Voort, M.C., van Houten,
F.J.A.M., 2004, Design and Evaluation of a Virtual
Gearshift Application, Proc. IEEE Intelligent Vehicle
Symposium, 465-470.
[12] Kimura, F., Hata, T., Suzuki, H., 1998, Product
Quality Evaluation Based on Behaviour Simulation
of Used Products, Annals of CIRP, 47/1, 119-122.
[13] Kimura, F., Hata, T., Kobayashi, N., 2004,
Reliability-Centered Maintenance Planning based on
Computer-Aided FMEA, CIRP Journal of
Manufacturing Systems, 33/5, 477-482.
[14] Pro/Engineer, Parametric Technology,
http://www.ptc.com/
[15] SAMCEF Field, Samtech, http://www.samcef.com/
[16] PHANToM GHOST, SensAble Technologies,
http://www.sensable.com/
[17] Nihon Kaiheiki, http://www.nikkaiswitches.com/
[18] OMRON, http://www.omron.co.jp/ (Japanese)