Keywords

1 Introduction

As a result of recent advances in three dimensional (3D) video technology and stereo sound systems, virtual reality (VR) has become a familiar part of people’s lives. Concurrent with these advances has been a wealth of research on touch interface technology [1], and educators have begun exploring ways to incorporate teaching tools utilizing touch properties in their curriculums [2, 3]. However, when used as teaching tools, it is important that a touch interface provide a “feel” that is as close to reality as possible. This will make replacing familiar teaching tools with digital media incorporating VR seem more attractive. For example, various learning support systems that utilize virtually reality (VR) technology [4] are being studied. Examples include a system that utilizes a stereoscopic image and writing brush display to teach the brush strokes used in calligraphy [5, 6], the utilization of a robot arm with the same calligraphy learning system [7], a system that uses a “SPIDAR” haptic device to enable remote calligraphy instruction [8], and systems that analyze the learning process involved in piano instruction [9] or in the use of virtual chopsticks [10].

The advantages of a system that uses virtual reality (VR) are that the software program can be changed to permit various types of technical training to be performed with a single device, and that the work environment can also be changed easily. Another advantage is that a network can be used to allow multiple users to train at different remote locations.

However, with a VR space connected via the Internet or other network, as a result of network latency and packet loss [11], as well as differing amounts of information, the data transmission times for various sensory operations will not necessarily be the same. An example of this phenomenon is the lag between video and sound in a network teleconference system. In an environment where latency exists, such a system cannot be said to be suitable as a technical training system, and this is a problem when using a VR system.

In this study, we also created and evaluated a soldering training system.

2 Evaluation of a Soldering Test System

To facilitate the passing down of technical skills, various operations have been analyzed and the application of those analyses is being investigated. Soldering work by skilled workers and unskilled workers is also being analyzed. (1) For workers having a certain amount of experience, there is diversity of right wrist motions. (2) For beginners, various soldering iron insertion angles and motions of each wrist, and a tendency for instability are observed. (3) For skilled workers, the soldering iron insertion angle and wrist motions are stable, and soldering is completed in nearly a single operation.

On the basis of the above, the soldering iron insertion angle, wrist motion stability, and the timing with which to remove the soldering iron are suggested to be three operation characteristics. These are shown in Table 1.

Table 1. Soldering work evaluation items and their characteristic values [13]
Full size table

2.1 Experiment Overview

In this study, we used a PHANToM Omni Device (Sensable Technologies) as our haptic device. It was attached to a control computer (CPU: Intel® Core™i5-4430 [3.00 GHz], RAM:8.00 GB, OS:64bit) running Open-Haptics™ toolkit v3.0 as the control program [12].

We began by modeling images of the surface texture for notebook and other paper types using friction experiments. When creating friction via the haptic display, it was first necessary to determine what level of friction was discernible.

We conducted both dynamic and static friction experiments, during which we measured the threshold for frictional force and points of subjective equality. Five male test subjects, approximately 20–21years of age, participated in both experiments.

2.2 Overview of the Soldering Operation

In soldering, the substrate warming time interval and the timing with which to remove the soldering iron are entirely heat-dependent. According on the type of solder and soldering iron, various combinations exist and it would be difficult to categorize them all. Solder types include both lead solder and lead-free solder, and although lead-free solder is most widely used at present, lead solder is used in this system. Typical lead solder having a composition ratio of Sn63 %:Pb37 % is used, and the melting point for this composition ratio is 183°C. A suitable junction temperature is 60 to 70°C greater than the solder melting point, and therefore an appropriate temperature for the soldering work is assumed to be around 250°C. Accordingly, when the soldering iron is set to the appropriate temperature of 350°C, the soldering time is approximately 3 s. Setting the removal timing to within approximately 3 s prevents the soldering iron from contacting the substrate for too long, and is thought to prevent soldering defects. If the solder did not completely melt, the soldering iron should be removed once and then the soldering operation repeated. Here, a temperature-adjustable soldering iron is used and a temperature of 350°C is assumed.

2.3 System Overview

  1. 1.

    Two haptic devices are connected so that they can be operated simultaneously in the virtual space. Thus, two devices can be operated and used as the solder and soldering iron.

  2. 2.

    By reading in a bitmap image and determining the width and height dimensions, red–green–blue (RGB) values corresponding to coordinate points are stored in an array. When preparing a graphic image, those values are mapped to position a floor and the substrate within the 3D space.

  3. 3.

    A training mode and practice mode are provided for beginners and people who are unfamiliar with the system. In the training mode, soldering operations are explained one by one in stages, and while learning about the operating method, the user also gains an understanding of how to determine the solder insertion timing and the insertion angle. By controlling the haptic device so as to forcibly retract it from the substrate, the user learns the timing for removing the soldering iron. Additionally, in order to stabilize the user’s wrists, control is implemented to forcibly secure the haptic device in place.

In the practice mode, pressure is constantly applied in the gravity direction to the haptic device to reproduce a gravity space and leads and lands that approximate those of actual work. This is shown in Fig. 1.

Fig. 1.
figure 1figure 1

Practice mode (execution screen)

Full size image
  1. 4.

    The viewing perspective can be changed by multiplying the model view matrix to allow movement and scaling.

  2. 5.

    The depth dimension can be difficult to ascertain while watching a virtual space on a screen and operating the haptic devices at hand. Therefore, the positional coordinates of the cursor (pen tip) in the screen are acquired and projected as a point onto the x-z plane.

  1. 6.

    As a method for acquiring the angle, the acquisition of vertical angle information of the pen tip of the haptic device itself was considered; however, vertical motion depends not only on the pen tip, but also on the arm itself, and so a suitable angle could not be obtained with this method. Therefore, we improved the accuracy by reading in the angle at the location of the cursor (pen tip) in the space. A 3 × 3 matrix that specifies a cursor transform in world coordinates can be obtained, and is used to convert the angle to a Euler angle. Since the rotation is around the Z-Y-Z axes sequentially, the equation of the rotation matrix is expressed as follows.

$$ {\text{R = }}\left( {\begin{array}{*{20}c} {{ \cos }\upalpha{ \cos }\upbeta{ \cos }\upgamma - { \sin }\upalpha{ \sin }\upgamma} & { - { \cos }\upalpha{ \cos }\upbeta{ \cos }\upgamma - { \sin }\upalpha{ \cos }\upgamma} & {{ \cos }\upalpha{ \sin }\upbeta} \\ {{ \sin }\upalpha{ \cos }\upbeta{ \cos }\upgamma + { \cos }\upalpha{ \sin }\upgamma} & { - { \sin }\upalpha{ \cos }\upbeta{ \sin }\upgamma + { \cos }\upalpha{ \cos }\upgamma} & {{ \sin }\upalpha{ \sin }\upbeta} \\ { - { \sin }\upbeta{ \cos }\upalpha} & {{ \sin }\upbeta{ \sin }\upgamma} & {{ \cos }\upbeta} \\ \end{array} } \right) $$
(1)

From here, the Euler angle α will be

$$ \upalpha \, = { \arctan }\left( {{\text{R}}\left[ { 2, 3} \right]/{\text{R}}\left[ { 1, 3} \right]} \right), $$
(2)

and so the angle can be obtained (α: angle around the z-axis). The basic posture while soldering is with arms parallel to the substrate, and so the Z axial direction in the coordinate axis in OpenGL is not considered.

  1. 7.

    To express the sinking feeling when solder melts, rather than being depicted as a simple cube, the substrate is depicted as a cube formed from multiple superimposed flat sheets. The sinking feeling is expressed by having the haptic device pass through a single sheet when a certain amount force is applied.

3 Evaluation

The ease of operation was evaluated. Five test subjects performed a five-level evaluation ranging from 1 (Poor) to 5 (Good). There were 7 items to be evaluated, and these and the results are shown in Table 2.

Table 2. System evaluation [units: number of people]
Full size table

From the results of a survey, we learned that the present challenges are the stabilization of the wrists, representation of gravity, and the sensation of diminishing solder. The issue of wrist stabilization is difficult to improve because, with control from the haptic device, there exists an area for which stabilization is not possible. On the other hand, for the representation of gravity and sensation of diminishing solder, the representation can be changed in the software program and further experimental investigation is needed in the future.

4 Concluding Remarks

We learned that the present challenges are the stabilization of the wrists, representation of gravity, and the sensation of diminishing solder. The issue of wrist stabilization is difficult to improve because, with control from the haptic device, there exists an area for which stabilization is not possible. On the other hand, for the representation of gravity and sensation of diminishing solder, the representation can be changed in the software program and further experimental investigation is needed in the future.