Background & Summary

The complexity of the human hand, with 25 main degrees of freedom (DoF) controlled by more than 30 muscles, provides the required ability to perform activities of daily living (ADLs). Understanding the relationship between hand movements and forearm muscular activation during ADLs is challenging, and may be useful for several applications such as improving control of prosthetic hands1, developing more realistic hand models2, or improving hand rehabilitation methods3. Nevertheless, for these purposes a large amount of kinematic and muscular activation data is needed. In this sense, several researchers4 have pointed out the importance of high-quality open-access datasets of grasping data, while also highlighting the need to compile, classify and standardize these data.

Product designers and ergonomists can also benefit from these databases. Product ergonomics assessment is usually limited to studying qualitative kinematic parameters such as grasp type used or hand contacting areas, and grip strength5. Analysing kinematic and muscular activation parameters such as median or extreme postures used by participants with varied characteristics when performing different tasks with products with different characteristics would provide important information regarding the effect of product design characteristics, contributing to the design of more inclusive products.

Although some datasets of hand kinematics during task performance are available in the literature6,7,8,9,10, as well as forearm EMG datasets, very few datasets exist with simultaneously recorded kinematics and EMG11,12,13,14,15, and they present several weaknesses. The main weakness is that the tasks recorded are not representative of ADLs11,12,14,15, being mainly grasping movements or static hand postures. Moreover, the kinematic data are sometimes presented as raw data instead of anatomical angles11, or have been collected with methods that are not the most reliable12.

To tailor a dataset representative of human hand behaviour (therefore allowing to characterise human hand function from a clinical point of view, to apply artificial intelligence techniques or to provide data to product designers, among others), datasets must contemplate a wide range of tasks representative of ADLs, elementary actions (holding, twisting, pouring, etc.) and grasp types, as several studies evidence that the task performed affects hand posture16. Another key aspect is considering products with different design characteristics, as aspects such as handle diameters have been observed to affect functional range of motion17, and aspects such as additional handles, thickened handles or bent handles have been found to produce specific effects in extreme postures and ranges of motion18. Similarly, handle diameter, weight and object position have been observed to affect forearm muscle activity19,20,21. Furthermore, several studies evidence an effect on grasp type selected when using products with different design characteristics22,23,24. Apart from these effects of the product itself, a representative sample of participants has to be recruited, considering participants’ gender, age and hand length. Finally, some technical aspects have to be considered, such as continuous recording of data (especially in ADLs requiring product manipulation, where the complexity of the tasks makes hand posture more variable), using the most appropriate units of measure depending on the purpose of the data collected, or providing data for both the dominant and non-dominant hands.

In this paper we present the MOVMUS-UJI Dataset, which contains a total of 4186 recordings with hand anatomical angles and forearm muscular activation while performing activities of daily living, along with tagged characteristics of products used, tasks performed and participants. Data are presented as MATLAB structures, accompanied with a custom data visualization application, ERGOMOVMUS, to ease the use of the data for ergonomists. The main contribution of this dataset compared to others is its synchronization of hand kinematics and forearm muscle activation with tagged information regarding the subject’s and task’s characteristics, and product design features, allowing more specific data analyses focusing on certain task/product characteristics. One of the strengths of the dataset is the variety of products used (105 products) and the variety of tasks performed requiring different elementary actions (161 different tasks, divided into 614 elementary tasks), and different grasp types, performance heights and product orientations. It is also worth mentioning that the sample of participants was selected so as to be representative of the healthy adult population (with a controlled proportion of ages and genders). Another important strength is the presentation of data, both available as a MATLAB/GNU Octave data structure (.mat) containing the full continuous recordings, and also a statistical summary of the recordings (5th, 50th and 95th percentiles) in a spreadsheet file (in.xlsx or.ods) accompanied with a custom data visualization application. The data structure may be useful for users experienced in coding, having a lot of potential for applications such as machine learning, as it contains complete continuous recordings. The statistical summary of these recordings, along with the data visualization application, provides a more intuitive way to access statistical data. This can be particularly valuable for specialists such as clinicians or product designers when characterizing healthy human hand behaviour or optimizing product ergonomics, among other applications. This dataset is accompanied with a usage guide, which contains detailed information regarding the environment, tasks, objects, data acquisition system, visualization application and file structure details. The data presented herein follow standard rules: kinematic data are provided as anatomical angles following the ISB sign criteria25, and muscular activity is presented as a value between 0–1 proportional to participants’ maximum voluntary contraction (MVC).

Methods

Study participants

The study consisted of three phases (A, B and C), with 26 right-handed participants (13 males, 13 females) participating in each phase. Only 22 participants took part in all the phases of the experiment, so that the total amount of participants recruited was 30. The mean age was 30.81 ± 11.17 years in phase A and 31.19 ± 11.18 years in phases B and C. Inclusion criteria were gender parity in overall data, right-handedness, age between 20 and 55 years and no reported upper limb pathologies. Recruitment was performed through print advertisements and social media. All participants gave written informed consent before the experiments, which were performed in accordance with the principles of the Declaration of Helsinki. Approval was granted by the Research Ethics Committee with Human Beings (formerly Deontology Committee) of Universitat Jaume I (Spain), reference number CD/31/2019.

Acquisition setup

Motion capture equipment

Kinematic data were acquired using two instrumented gloves (CyberGlove Systems LLC): a CyberGlove II on the right hand and a CyberGlove III on the left hand (Fig. 1a). Each of these gloves has 18 strain gauges, allowing to determine the anatomical angles of the underlying joints. The angle rotated by each joint with respect the reference posture (hands resting flat on a table, with the fingers and thumb together, and the middle fingers aligned with the forearms, see Fig. 1b) is then obtained from these signals, according to a previously validated calibration protocol26.

Fig. 1
figure 1

(A) Participant equipped with Biometrics sEMG electrodes (LE230) and CyberGlove data gloves. (B) Hand reference posture for kinematics recording.

Full size image

sEMG equipment

Muscle activity was recorded using a Biometrics DataLITE wireless recording unit (Biometrics Ltd.) at a sampling frequency of 1000 Hz. Seven integral dry reusable wireless sEMG electrodes (LE230) were used for each forearm. Electrodes were placed in the centre of seven representative spot areas of the right forearm, according to previous work27, and were set out in longitudinal direction (Figs. 1, 2), following the SENIAM recommendations28. To locate these seven spot areas, a Tubigrip with a grid defining the spots location was put on the subject’s forearm, placing it using five easily identifiable anatomical landmarks27 (Fig. 2). These seven spots were chosen to be representative of all available muscle activity of the whole forearm, according to previous works27.

Fig. 2
figure 2

(A) Grid and spot areas selected for the sEMG recordings. (B) Five anatomical landmarks used to draw the grid. (C) Tubigrip with the grid defining the spots location. The signals from these seven spots are related to seven different movements according to previous work27. Spot 1: wrist flexion and ulnar deviation (WF_UD); spot 2: wrist flexion and radial deviation (WF_RD); spot 3: digit flexion (DF); spot 4: thumb extension and abduction/adduction (TM); spot 5: finger extension (FE); spot 6: wrist extension and ulnar deviation (WE_UD); spot 7: wrist extension and radial deviation (WE_RD).

Full size image

Environment

The tasks were performed in a laboratory environment that simulated home areas. The scenarios (Fig. 3) consisted of: a table and shelves at different heights (Scenario 1), a table (Scenario 2), a door with exchangeable handles (Scenario 3), a sink (Scenario 4), a desk (Scenario 5), a lock on a panel (Scenario 6), a screw in a wooden wall (Scenario 7) and electrical sockets and turning buttons on a panel (Scenario 8).

Fig. 3
figure 3

Different scenarios of the experiment. Scenarios: a table and shelves at different heights (1), a table (2), a door with exchangeable handles (3), a sink (4), a desk (5), a lock on a panel (6), a screw in a wooden wall (7) and electrical sockets and turning buttons on a panel (8).

Full size image

Products

The 105 products used in the recorded tasks were selected to be representative of the wide variety of products commonly used to perform ADLs. Thus, different product types were selected: bottles, cans, jars, houseware, food packaging, cutlery, assistive devices, self-care and cleaning products, among others. Furthermore, they were selected to cover different weights, dimensions and section shapes (further information regarding their characteristics can be found in the guide attached to the dataset). All the products used were real, except in the task of peeling a cucumber (the cucumber was replaced with a 3D printed cucumber) and in tasks requiring cutting food (play dough was used instead of food). The location of the products in each scenario is detailed in the guide attached to the dataset. Figure 4 shows an overview of the products used.

Fig. 4
figure 4

Overview of the products used to perform the tasks.

Full size image

Acquisition protocol

The main dimensions of participants’ hands were measured before wearing the instrumented gloves. Furthermore, participants’ forearm hair was removed and skin cleaned with alcohol before placing the sEMG sensors in the forearm. Then, a kinematic reference posture (hands lying flat on a table with fingers and thumbs close together, with middle fingers aligned with forearms) was recorded before performing all the experiment tasks, and was considered zero for all the anatomical angles.

To normalize muscular activation signal, seven records of maximum voluntary contraction (MVC) were performed: flexion and extension of the fingers, flexion and extension of the wrist, ulnar and radial deviation of the wrist, and pronation of the forearm (Fig. 5). Participants were asked to take a comfortable posture and exert maximum effort without the help of muscles other than those of the forearm and hand.

Fig. 5
figure 5

Seven MVC records for the normalization of the muscle activity signal. From left to right: flexion and extension of the fingers, flexion and extension of the wrist, ulnar and radial deviation of the wrist, and pronation of the forearm.

Full size image

Signal synchronisation

The CyberGlove kinematic data and sEMG records were synchronised using custom data acquisition software especially designed for this purpose, matching the initial and final instants of each record. This acquisition software was developed in C++, synchronising glove and sEMG by using the SDK libraries of the CyberGlove and Biometrics devices.

Recorded tasks

The experiment was performed in three different phases: A, B and C. In phase A, the tasks performed only required using the right hand with cylindrical, lateral pinch and lumbrical grasp types. In phase B, the tasks only required using the right hand with oblique, special pinch, hook, intermediate and five-finger pinch grasp types. Tasks in phase C required using both hands (bimanual tasks) with all the previously mentioned grasp types. Order of tasks was randomized for each participant. One complete task was performed during each continuous run of data recording, during which the operator marked specific events that were later used to separate the task’s component elementary tasks, e.g. holding, pouring, twisting, etc. Therefore, phase A was composed of 79 tasks (342 elementary tasks), phase B consisted of 33 tasks (128 elementary tasks), and phase C of 49 tasks (144 elementary tasks). Further information regarding the elementary tasks considered in each recording can be found in the guide attached to the dataset. An example for one task can be seen in Table 1.

Table 1 Elementary tasks into which task T = 111 is divided.
Full size table

The participants were given clear instructions about how to perform each task, and they were told to start and end the task in the same posture: hands lying relaxed at both sides of the body for tasks performed in a standing posture, and hands lying relaxed on the table when sitting (initial posture for each task is specified in the guide attached to the dataset). While carrying out each task, the operator marked specific events that were later used to separate the different elementary tasks.

For each task, participants were given clear instructions on the elementary tasks to be performed and on the grasp types to be used by each hand. Table 1 illustrates the instructions given to participants as well as labelling of tasks and elementary tasks in the dataset.

Signal processing

Angles calculation

Joint angles were computed from raw glove data following the calibration protocol proposed in previous works26. This protocol includes the determination of gains and also some corrections to avoid cross-coupling effects for specific anatomical angles. The joint angles obtained according to this protocol are shown in Fig. 6:

Fig. 6
figure 6

Recorded anatomical angles. Nomenclature: _F for flexion (in yellow), _A for abduction (in turquoise); digits 1 to 5. Joints: IP (interphalangeal joint), PIP (proximal interphalangeal joints), MCP (metacarpophalangeal joints), CMC (carpometacarpal joints), PalmArch (palmar arch resulting from flexion/extension of carpometacarpal joints of ring and little fingers), WR (wrist).

Full size image

sEMG signal processing

To compute muscle activity, sEMG records were first normalised by dividing them by the maximal values from any record (7 MVCs and all tasks) measured for each subject. In this way, sEMG activity from the same spot of different individuals or sEMG activity between different spots can be compared. Afterwards, the normalised recordings were resampled to 100 Hz to present synchronized data with angles.

Data splitting

Each task recorded was separated into different elementary tasks as detailed in the dataset guide by using the labelling performed by the operator while recording the data.

Filtering

Kinematic data were filtered with a 2nd-order two-way low pass Butterworth filter with a cut-off frequency of 5 Hz. The sEMG data were filtered with a fourth-order bandpass filter between 25–500 Hz, rectified, filtered by a fourth-order low-pass filter at 8 Hz, and smoothed by Gaussian smoothing29.

Outliers and missing signal during recordings

A total of 4186 tasks (15964 elementary tasks) were recorded across all the participants and experiments. Nevertheless, signal was missing during some recordings. Tasks with missing signal were removed, leaving 15823 elementary tasks in the final dataset. Outliers were erased considering the active and functional range of motion of the joints measured in a previous study30. Specifically, the 50th percentile of each joint for each elementary task was calculated and those values higher or lower than the mean range of motion values of each joint plus three times its standard deviation were eliminated. During this analysis, a malfunction of the glove gauge recording right palmar arch flexion was detected after performing the experiments. Therefore, to ensure that all data are correct, data of this DoF are not reported in the dataset. In the end, 55 outliers were deleted: 5 cases in CMC1_F, 30 cases in IP1_F, 11 cases in PIP2_F, 7 cases in PIP3_F, 1 case in PIP4_F and 1 case in PIP5_F. Outliers in the thumb joint may be due to a poor fit of the gauge in that finger. Outliers in PIP joints are negative values (around −20°) and may be due also to a poor fit of these gauges for a specific subject, since most cases were found in the same subject, who had a small hand length.

Data Records

Data files

Data are presented as several MATLAB structures stored in.mat files: KIN_EMG_DATA.mat, PARTICIPANT_DATA.mat, TASK_DATA.mat and PRODUCT_DATA.mat. The data are accompanied by a guide (.pdf), which provides more detailed information regarding the data series as well as the environment, tasks, products and data acquisition system. The dataset is publicly available to all research community at the Zenodo open repository31.

KIN_EMG_DATA

Substructure containing all kinematic and sEMG data recorded, classified by task, elementary task and participant. For the kinematic data, the sign criteria for each joint motion were defined as follows:

PIP(2-5)_F, IP1_F, MCP(1-5)_F: Flexion + / Extension -

MCP(2-3, 3-4, 4-5)_A: Fingers separated + / Fingers together -

PalmArch: Flexion + /Extension –

CMC1_F: Flexion + /Extension – (See Fig. 7)

CMC1_A: Abduction + /Adduction - (See Fig. 7)

WR_F: Flexion + / Extension -

WR_A: Ulnar deviation + /Radial deviation –

Note that movement of thumb CMC joint is complex, and nomenclature used in literature to define these movements is varied32,33. We adopted the one used by Brand and Hollister33.

Fig. 7
figure 7

Carpometacarpal joint motion.

Full size image

For the muscle activity, normalised signal for the seven representative spot areas (Fig. 2) were presented ordered from Spot 1 to 7 (columns 1 to 7 in case of right hand, columns 8 to 14 in case of left hand).

PARTICIPANT_DATA

Substructure containing information of the participants recruited (age, gender, hand and forearm length, height, and phases in which the participant was recruited).

TASK_DATA

Substructure containing information regarding task characteristics (product grasped/manipulated with each hand, grasp span, use of both hands, product orientation, action performed and height). Table 2 details the variables, their meanings and codification.

Table 2 Fields in TASK_DATA substructure.
Full size table
Fig. 8
figure 8

Product orientations considered. 1 = Vertical, 2 = Horizontal transverse, 3 = Horizontal longitudinal. In the special case of dishes and bowls, without a clear grasping axis, “horizontal transversal” orientation was considered. When product orientation was not controlled, it was classified as “4 = other”.

Full size image

PRODUCT_DATA

Substructure containing information regarding product characteristics. Table 3 details the variables, their meaning and codification.

Table 3 Fields in PRODUCT_DATA substructure.
Full size table
Fig. 9
figure 9

Example of products classified in each section shape group.

Full size image

Custom data visualization application (ERGOMOVMUS)

A custom data visualization application for ergonomics assistance (ERGOMOVMUS) has been developed using the MATLAB GUIDE environment, in order to easily select different fields regarding participants, task and product characteristics previously detailed. The application is provided with the dataset available at Zenodo31, and has several spreadsheets in OOXML Transitional and OpenDocument formats (in.xlsx/.ods) attached: KINEMATIC_DATA.xlsx and sEMG_DATA.xlsx contain a statistical summary of the full recordings of the dataset consisting of 5th, 50th and 95th percentiles of each joint kinematics and each spot muscular activation for each subject and elementary task; PARTICIPANTS_DATA.xlsx, TASK_DATA.xlsx and PRODUCT_DATA.xlsx contain the same information as the homonym .mat files of the dataset. The application allows visualizing and saving in an OOXML Transitional format (.xlsx) spreadsheet both kinematic and forearm muscular activation data corresponding to selectable specific characteristics of participants, tasks and products. A screenshot of the main window of the application can be seen in Fig. 10.

Fig. 10
figure 10

Screenshot of the main menu of ERGOMOVMUS.

Full size image

Technical Validation

Data acquisition

Before and after each experiment phase, participants were asked to randomly move their hands, in order to check that all the gauges were shown active on the virtual model of the CyberGlove software.

After recordings, the number of labels used to divide each task into elementary tasks was checked to be correct, ensuring that no labels were missing. Furthermore, as detailed in previous sections, in order to avoid possible unexpected signal values all kinematic and sEMG data collected were filtered.

Statistical descriptive analysis of data collected

In order to validate data, 95th, 50th and 5th percentiles of joint angles and muscular activation for each participant and elementary task were plotted using box-and-whisker graphs. Kinematic data from two grasp types requiring markedly different postures (cylindrical and special pinch) were plotted for comparison (Fig. 11). Analogously, sEMG data during transportation tasks of products belonging to weight groups 0g–500g and 2500g–3000g were plotted (Fig. 12). It can be observed that the hand posture obtained for each grasp type is different, and values of joint angles are within the ranges obtained in previous studies measuring hand functional range of motion30,34,35 or datasets providing hand kinematic data during the performance of different ADLs6,7,13. Furthermore, the muscular activation obtained was higher during transportation tasks of products belonging to weight group 2500g–3000g, and the obtained values are aligned with those reported in previous studies where surface electromyography (sEMG) was collected during the performance of a diverse set of ADLs using products with different weights7,13.

Fig. 11
figure 11

Box and whisker plots of 95th, 50th and 5th percentiles of kinematic data collected in tasks requiring cylindrical and special pinch grasp types. Joints and movements labelled as explained in Fig. 6. Forearm spots labelled as explained in the main text. Note that right palmar arch data is not available and does not appear in the plots.

Full size image
Fig. 12
figure 12

Box and whisker plots of 95th and 50th percentiles of sEMG data collected in transportation tasks using products with weight ≤500 g and 2500g-300g. Forearm spots labelled as explained in the main text.

Full size image

Limitations

The use of instrumented gloves may imply some loss of dexterity during the performance of tasks requiring fine manipulation. Nonetheless, this loss of dexterity may not produce a significant effect on mean postures, ranges of motion or motion synergies. Muscle activation has been obtained using surface EMG, which does not allow focusing on a specific muscle, and only from seven specific areas of the forearm where only extrinsic hand muscles are present.

Usage Notes

These data can be used for several applications, such as machine learning, healthy hand characterization or product design ergonomics. The main strengths of this dataset for these potential uses are its provision of synchronized data of hand kinematics and forearm muscle activation with tagged information regarding participants and task characteristics, and product design features, allowing more specific analyses. Another strength is the variety of products used (105 products) and tasks performed, requiring different elementary actions, grasp types, performance heights and product orientations (161 different tasks, divided into 614 elementary tasks). Finally, the presentation of these data, both as a MATLAB/GNU Octave data structure (.mat) and through a custom data visualization application, allows its usability in several applications.

It has to be considered that real food was not used to perform the tasks in order to prevent the gloves from getting stained or wet (all products are appropriately tagged with the corresponding substitute material in the dataset guide file). Therefore, tasks involving food were simulated and might be performed in a slightly different way than when performed with real food.

Finally, it has to be mentioned that velocity of performance of the tasks might be slightly affected by the loss of dexterity and touch sensitivity resulting from the use of the instrumented gloves36.