Motion limitations of non-contact photoplethysmography due to the optical and topological properties of skin

Six participants were recruited for the experiment (four male, two female, of mixed ethnicity, all older than 18 years), and the recordings (photographs and video) were repeated three times. Six regions of the skin were investigated. All participants, after being informed about what data was to be collected from them and the reasons for the study itself, consented to having anonymised data collected, analysed and published. The study was approved by the University of Nottingham's Research Ethics Committee (reference number: 2014–140).

To extract a PPG from the data, a rectangular region of interest (ROI) was selected. The ROI size and position could be modified as a function of time in order to simulate panning. A single value was extracted from each frame of the video by averaging the monochromatic pixel values within the chosen window; these values were then processed and plotted as a time-varying signal. An example of a PPG from one subject with no simulated panning motion is shown in figure 2.

Zoom In Zoom Out Reset image size

In order to characterize the effect of a panning motion accurately, 10 s recordings, of 500 frames, were taken with the participants' hand held stationary (as far as was possible). Artificial motion was then digitally introduced by moving the ROI linearly across a single frame of the video (the first frame) at a chosen speed; a single frame was chosen, instead of all frames in the video, to prevent the inclusion of the PPG. A simple linear motion was chosen as this would allow for direct comparison between the spatial frequencies of the skin surface and the temporal frequencies of typical heart-rates.

Under these controlled conditions, two main components existed in the signal: the PPG, whose frequency and phase was assumed to be constant throughout the frame, and the changes in intensity within the image as a result of the panning motion meaning that different regions of skin filled the ROI. This assumption, regarding the phase of the PPG, was based on the lack of an observed phase shift over the recorded video frames, and that at 50 frames per second with a 100 mm frame width, a phase shift would only be visible if the pulse-wave-velocity (PWV) was slower than 5 ms⁻¹ whereas typically, PWVs exceed this (Koivistoinen et al 2007). Due to the non-homogeneous nature of the skin, the variation due to the motion was not likely to be negligible.

The slow changes (low spatial frequencies) visible in figure 3 are due to the macro structure of the hand that is its general curvature and physical make-up. The higher spatial frequencies are due to the micro structures of the skin; the skin's cellular structure, its variation in pigmentation, and vascular networks beneath the surface will all contribute. This illustrates the difference in amplitude between a PPG (as obtained earlier, see figure 2(c)) and the variation in reflected intensity from the skin due to position. It is clear that the amplitude of the spatial variations would dominate if any motion occurred.

When remotely detecting PPGs from participants, motion can manifest itself in two different but related ways. Firstly, 'random' movement that occurs naturally during activity will cause the established 'motion artefact' by adding an uncorrelated signal to that which is detected. However, a second component may exist that is correlated to the PPG: the optical ballistocardiogram (Ratan 2004). This signal is created when the pulsations of the heart cause a mechanical effect on the body due to the redistribution of blood, effectively displacing regions being tested by small (but potentially detectable) amounts; the mechanical effect is known as the ballistocardiogram (BCG) and is itself a method of extracting vital signs (Alametsä et al 2008). When optically measuring the skin, this displacement can manifest itself as a pulsatile signal similar to the PPG. It is evident in figure 3, for example, that a BCG could be produced with very little movement by observing a single pixel (the two points A and B are spaced apart in distance by less than 2 mm, yet have a difference in intensity that is an order of magnitude greater than the PPG). This BCG could, depending on the movement, either add or subtract to the actual photoplethysmographic signal. The large positive change in intensity from point A to point B can be negated by looking at (and averaging) multiple neighbouring pixels which have the opposite gradient. Whilst ballistocardiographic effects are not considered further in this document, an important message is that a clean PPG signal may have its origins in ballistocardiographic motion.

The spatial frequency of the skin surface is not relevant when there is no motion present as the ROI will encompass the same pixels. However, as the motion (speed) is increased, the spatial variations (spatial frequency) are perceived as a temporal frequency scaled by the speed. For example, with a traversal velocity of 1 mm s⁻¹, the graph in figure 4 could be interpreted as a temporal FFT; i.e. the horizontal 'cycles per mm' would become 'cycles per second' (Hz). If the traversal velocity were to double, the frequency components would scale by the same amount (a component originally at x Hz would stretch to 2x Hz, etc), moving the more dominant low-frequency components into a typical heart-rate range (figure 5).

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

From this analysis, it is clear that the lower spatial frequency components (being more dominant) have a more significant effect on the PPG than the higher spatial frequency components. Increasing the ROI area and averaging the contained pixel values creates a simple low-pass spatial filter that removes the higher spatial frequency components (see figure 4). When motion is present, this filter appears as an equivalent temporal filter.

To reduce the effect of motion artefacts, therefore, either the area of the region under test must be increased, or the motion speed decreased. Although the latter is not often controllable, a relationship between the required area of the ROI for a desired PPG-to-artefact (signal-to-noise) ratio, and the motion velocity (for a panning motion) can be established. However, as each camera model (and type) will have different optical and electrical (noise) characteristics, using an absolute method of characterising the effect of motion on a signal quality would not provide a reproducible result. Because of this, a relative method of signal characterisation was used.

To compare the 'current' sample (of a specific area and speed) and the reference sample (the theoretical best area and speed), a standard correlation technique (Pearson) was deployed. In this case, the 'best case' scenario for obtaining a PPG (no motion, maximum possible area) was correlated with the baseline signal, but with a variation in one or both of the ROI area and traversal velocity. A direct comparison was then determined between the effects of the area and the ROI traversal velocity on the quality of the PPG as described by the correlation coefficient.

Three features of the plot in figure 6 are of particular interest. Firstly, the correlation in the 'area dimension' shows that a reduced ROI area results in a lower signal quality, as expected. It must be noted in this example, however, that the correlation values that exists when there is no motion present (speed  =  0 mm s⁻¹) still reduce for smaller areas. This is most likely due to the fact that a small amount of motion was present in the recordings.

Zoom In Zoom Out Reset image size

Secondly, not surprisingly, increasing the motion velocity for any given ROI area decreases the signal correlation. Although the relationship between the speed and the correlation is more complex than the previous relationship, the general trend follows a quadratic law (figure 7). The 'ripples' that are visible in figure 6 are due to the spatial frequency components in the sample not being uniformly distributed. In other words, some temporal frequency components that exist at a certain motion velocity have a detrimental effect on the PPG quality; hence local troughs. Conversely, some have similar properties to a PPG signal, effectively and erroneously 'enhancing' the calculated quality (the local peaks). A change in ROI area affects the quality predictably, however, due to the random nature of the skin's texture and structure, the relationship is less well defined for changes in velocity.

Zoom In Zoom Out Reset image size

Finally, for a given correlation coefficient value, a relationship between the ROI area and velocity can be constructed. Initially, it would be reasonable to think that to compensate for an increase in ROI velocity; its area could be increased by a proportionate amount. However, there are clear limits; increasing the area of detection (for any region on a body) cannot indefinitely improve the PPG quality, since eventually, the larger structures of the body such as skin creases and the edge of the body will dominate and the quality will no longer increase. If, for example, due to an increase in area, the ROI were to extend near to or beyond the 'edge' of the subject's skin (where the optical normal will diverge from the camera's until the skin is no longer completely within the ROI), the quality will decrease.

For a given ROI area (A), the quality Q is negatively proportional to the square of the traversal velocity (V): $Q\propto -{{V}^{2}}$. This is clearly visible in figure 7. Although less clear in the figures, for a given traversal velocity, the quality is inversely (and negatively) proportional to the ROI area: $Q\propto -{{A}^{-1}}$ (this is obtained from the coefficients of the curves in figure 7).

Thus an equation (1) can be constructed to link the area, velocity and resulting quality. Despite being linked to the correlation coefficient mentioned earlier, a negative quality value (Q) has no meaning and represents zero 'quality'.

$$\begin{eqnarray}&&Q=1-k\times \frac{V{{\,}^{2}}}{A}\end{eqnarray} \tag{ 1 }$$

where k is a constant dependant of the system.

With the assumption that the ability to extract a PPG from a signal containing artefacts is limited by the PPG-to-artefact ratio, the above equation can be used to calculate a relative area required to reduce a motion artefact by a fixed amount.

For example, in figure 7, if a 'quality' (correlation) of 0.8 is deemed satisfactory with an ROI velocity of 0.25 mm s⁻¹ and area 42 mm², then to counter a decrease in PPG-to-artefact ratio when the velocity increases by a factor of 2 to 0.5 mm s⁻¹, the area would have to be increased to 168 mm² (=42 mm${{}^{2}}\times {{2}^{2}}$). If the velocity were to increase again by the same factor (to 1.0 mm s⁻¹), then the area would need to be increased to 676 mm² (=168 mm${{}^{2}}\times {{2}^{2}}$); although this is not possible to achieve with the current set-up.

All previous results have concentrated on a single region: the palm. Within the experiment, six regions were investigated. The following section analyses the difference in artefact susceptibility between the six regions.

In figure 8, points where the spatial frequency responses intersect the mean PPG amplitude were determined and their frequencies are summarised in table 1. The frequencies of occurrence are a good indication of how 'good' the area is in relation to the artefacts that are produced. For example, regions with a low mean intersection frequency allow for faster ROI traversal velocities before the dominant low spatial frequency components overlap the typical heart-rate region (see figure 5). A low standard deviation of spatial frequencies represents a more reliable region to extract a PPG from when multiple participants are studied; i.e. the variation between participants is minimal.

Zoom In Zoom Out Reset image size

The position of each of the six regions in table 1 can be easily explained by the topology of the skin. The region with the largest mean and standard deviation (the cheek) also has the largest curvature due to its relatively small area. Both the ventral and dorsal forearms score 'best' with low means and standard deviations. The arm has relatively little curvature (along its length) resulting in much lower spatial frequencies.

The ability to extract a PPG signal from the skin is also dependent on several factors relating to the blood flow beneath the surface of the skin. These include capillary density and blood perfusion which can vary between different regions. Regions such as the forehead, cheeks and palms provide larger PPG signal amplitudes compared to other areas enabling a higher PPG-to-artefact ratio (Hertzman 1938). Unfortunately, according to this research, neither of the forearm sites (ventral and dorsal) provide particularly strong PPGs. The forehead and palm however, do provide stronger signals, and so despite being regions that allow for larger artefacts to be generated, are often chosen for experiments.