Pressing the Flesh: Sensing Multiple Touch and Finger Pressure on Arbitrary Surfaces

Abstract

This paper identifies a new physical correlate of finger pressure that can be detected and measured visually in a wide variety of situations. When a human finger is pressed onto a hard object the flesh is compressed between two rigid surfaces: the surface of the target object and the fingernail. This forces blood out of the vessels in the fingertip, changing its colour slightly, but systematically. The effect is visible to the naked eye and can be measured using techniques from computer vision. As measurements are made of properties of the hand, and not the target surface, multiple-touch and pressure sensing can be added to a range of surfaces - including opaque, transparent, smooth, textured and non-planar examples - without modification of the underlying physical object. The proposed approach allows touch sensing to be fitted to surfaces unsuitable for previous technologies, and objects which cannot be altered, without forfeiting the extra range of expression of pressure sensitivity. The methods involved are simple to set up and low cost, requiring only a domestic-quality camera and a typical computer in order to augment a surface. Two systems which exploit this cue to generate a response to pressure are presented, along with a case study of an interactive art installation contructed using the resulting technology. Initial experiments are reported which suggest that visual monitoring of finger colour will support recogntion of push events.