Filling gaps in visual motion for target capture

Introduction

A remarkable challenge our brain must face constantly when interacting with the environment is represented by ambiguous and, at times, even missing sensory information. This is particularly compelling for visual information, the main sensory system we rely upon to gather cues about the external world. The issue can be exemplified by typical situations occurring in the sports field. Let’s imagine a scene from a soccer game, such as that illustrated in Figure 1: a goalkeeper dives to save the goal from an opponent’s long distance shot, with the ball being hidden by other players that occlude the goalkeeper view. Despite the temporary disappearance of the ball from his field of view, the goalkeeper may be able to intercept the shot successfully and save the goal. Apparently, the goalkeeper has been able to predict where and when to intercept the shot by “guessing” the motion of the ball through the periods of absent visual motion information. In the wilderness, similar mechanisms may be important for survival. Predators, for example, are able to catch the escaping prey even though visual obstacles like trees and bushes may hide it from view, underlining the potential evolutionary pressure for the development of predictive mechanisms, which might compensate for the lack of sensory information. This ability appears to be acquired rather early during post-natal development in humans. Von Hofsten and colleagues have shown, in fact, that 4–5-months-old infants can anticipate where and when an occluded moving object will reappear (see Figure 2), and their predictions may reflect a representation of object velocity during the occlusion (Rosander and von Hofsten, 2000; von Hofsten et al., 2007).

Ample debate exists in the literature about the putative mechanisms that may account for the ability of predicting the objects’ visual motion through periods of transient disappearance. An experimental paradigm that has been commonly adopted to investigate this issue is the prediction motion (PM) task whereby subjects estimate the time of arrival of a hidden visual target at a specific location indicated by the experimenter. One early hypothesis to explain behavioral responses to PM tasks involved a clocking mechanism by which observers may initially estimate the time to contact (TTC) using on-line optical variables, and then, at the time of object’s disappearance, they may use a clocking process to count down time until the estimated TTC equals zero (Tresilian, 1995). While cognitive clocking mechanisms may account for PM responses to approaching objects along the line of sight (looming stimuli), they have been shown to generalize poorly to other behavioral situations involving, for example, lateral motion of the visual target (Lyon and Waag, 1995; DeLucia and Liddell, 1998). It has been, then, hypothesized that the brain adopts motion extrapolation mechanisms involving internal or cognitive representations of the object’s visible motion (Schiff and Oldak, 1990; Lyon and Waag, 1995). There is evidence that visual extrapolation can be sustained by a combination of several, perhaps interdependent, processes such as imagery, corollary eye movement signals, and attentional shifts. For example, observers may imagine the object continuing its motion after disappearance, and then respond as the imagined object gets to the target. Interestingly, a recent fMRI study has shown that visual motion imagery engages specific activations in occipital cortex, hMT/V5+ and IPL, regions involved in visual motion processing (Kaas et al., 2010). Oculomotor tracking of the invisible target may also reflect mental imagery processing, providing additional information to the representation of the occluded target trajectory (Huber and Krist, 2004). Moreover, motion imagery may be associated with visuospatial attention shifts (de’Sperati and Deubel, 2006). Therefore, visual extrapolation processes may also involve shifting an attentional “spotlight” (Lyon and Waag, 1995; Kerzel, 2003a,b). Further evidence that visual motion extrapolation may demand additional cortical processing and attentional resources comes from a series of experiments, which showed that absence of visual information may prolong the latency to suppress an anticipatory movement to intercept a moving target and affect the corticospinal excitability (Marinovic et al., 2010, 2011).

A fundamental question to understand the nature of the visual extrapolation processes concerns, however, the type of information potentially contributing to the internal representation of the hidden target motion built by the CNS. In this respect, much insight has been gained, over the last few decades, from studies examining the oculomotor and the manual interceptive behavior.

Evidence from Oculomotor Studies

Experimental paradigms involving transient disappearance of the visual target have been used extensively to study the predictive component of pursuit and saccadic eye movements. For example, when visual feedback is transiently removed during ocular pursuit, it has been shown that extra-retinal input may continue to drive the smooth pursuit, albeit at a reduced gain (Mitrani and Dimitrov, 1978; Morris and Lisberger, 1987; Pola and Wyatt, 1997). Continuation of smooth pursuit in the absence of a visual target is believed to be under volitional control and mediated by the subject’s intention, as it is driven by the expectancy of the target to reappear (Becker and Fuchs, 1985; Bennett and Barnes, 2003; Madelain and Krauzlis, 2003; Barnes and Collins, 2008). In particular, a series of seminal studies carried out by Bennett, Barnes and collaborators shed some light on the variables the oculomotor control processes might take into account to maintain ocular tracking of hidden targets. By imposing a systematic change in the target velocity during the invisible portion of the trajectory, Bennett and Barnes (2004) were able to show that recovery in eye velocity after the loss of visual feedback was scaled and hence predictive of the upcoming target velocity. They proposed that predictive changes of eye velocity were the result of scaled modifications of an internal gain signal. In a later study (Bennett and Barnes, 2005), the same authors used target blanking of variable duration to examine whether the timing of this anticipatory recovery could be influenced by the duration of the target disappearance. They found that the recovery of the eye velocity was indeed timed to the target disappearance, perhaps as result of reactivation of variable gain mechanisms acting on the visuomotor signals, which drive ocular pursuit. Moreover, when subjects were required to pursue accelerating targets that underwent transient occlusions and were presented in either random or blocked order with respect to the acceleration conditions, they showed anticipatory smooth pursuit prior to target motion onset, scaled to the velocity generated by the target acceleration, at least during blocked presentations. This scaling in the eye velocity prior to the target reappearance led to the interpretation that the oculomotor control system might use a target velocity representation to drive predictive pursuit (Bennett and Barnes, 2006a).

Similarly, by using an occluded onset pursuit paradigm, Collins and Barnes (2006) found that smooth pursuit started progressively earlier with respect to the target appearance as the occlusion interval increased, and with increasingly higher pursuit gains by the time the target appeared. The prolongation of the anticipatory smooth pursuit throughout the occlusion was interpreted again as reflecting storage of velocity information in the form of working memory (Collins and Barnes, 2006).

Both pursuit and saccadic control use predictive information about target motion during extrapolation of occluded targets. In fact, subjects can correct for the error in eye position due to the decrease in pursuit gain, by producing saccades that place the eye ahead of the extrapolated position of the occluded object (Bennett and Barnes, 2006b). Further evidence for interaction between pursuit and saccadic system in the ocular tracking of occluded targets was provided by a study of Orban de Xivry et al. (2006), who found that eye velocity at target reappearance was influenced by expected target velocity, whereas saccades reflected the expected change of target position (see also de Brouwer et al., 2001).

Besides target velocity, also target acceleration information appears to be used by the oculomotor control system for the extrapolation of occluded targets. Bennett et al. (2007) adopted an experimental paradigm where target’s position and velocity during the occlusion and at reappearance could not be predicted without extracting the acceleration of the target before its disappearance. They found that smooth and saccadic eye movements discriminated between different levels of acceleration after an initial exposure of at least 500 ms (Bennett et al., 2007). Despite the fact that this result was obtained by using non-ecological experimental conditions, its significance may have some bearing also with respect to more natural situations where moving objects, because of the effects of external forces (such as gravity), do not move at constant velocity.

Visual extrapolation mechanisms driving eye movements can also take into account dynamic directional changes of the target motion, as shown by the results of experiments where subjects had to track targets along two-dimensional paths. For example, Mrotek and Soechting instructed subjects to track targets, which moved initially along a straight path and then followed the arc of a circle, just before disappearing behind an occluder (Mrotek and Soechting, 2007). When the target re-emerged after following the curvilinear path through the occlusion, gaze behavior indicated that subjects were able to predict the curvilinear target motion through the occlusion, compatible with the idea that extrapolation of occluded targets may include expectations about the time-varying behavior of the target motion.

Further evidence along these lines comes from a study, which investigated smooth pursuit and saccadic responses during occlusion of targets moving along circular paths (Orban de Xivry et al., 2008). In this study, subjects pursued visual targets moving along counterclockwise circular paths that could vary randomly across trials in both radius and frequency. The target, initially visible for at least a half-circle, underwent three successive periods of occlusion ranging from 0.4 to 1 s alternated with visible periods lasting between 0.8 and 1.5 s. Analysis of smooth pursuit responses showed that, during target occlusions, the smooth pursuit heading direction could predict accurately the heading of the target. This result was best explained by a dynamic memory model that included time-varying information about target velocity during the occlusion period, suggesting that predictive pursuit was driven by a time-varying internal representation of target motion. In addition, it was found that the predictive smooth pursuit system influenced the amplitude of the predictive saccades but not vice versa, underlining the complexity of the interactions between the pursuit and the saccadic system. Besides maintaining smooth pursuit in the absence of the visual stimulus, internal representations based on expectancy of predictable target motion can also drive smooth eye deceleration, direction reversal, and subsequent acceleration (Kveraga et al., 2001).

Dynamic internal representations driving eye movements do not incorporate only short-term visual information acquired before target’s disappearance, but they may also reflect cognitive factors and memory of target motion accumulated with previous experience. This idea was tested directly by a series of experiments, in which accelerative and decelerative target motions were presented in either random or blocked-order, and catch trials with different target acceleration were interleaved randomly between blocked-order trials. It was found that the recovery of the smooth pursuit following the target’s occlusion was better scaled in blocked-order than in random-order trials. Particularly, in blocked-order trials, the reduced smooth pursuit gain during occlusion was compensated by modulation of the saccade amplitude, such that the total eye displacement matched well the target displacement. In catch trials, experimental subjects could show scaled responses to the unexpected change in target acceleration, albeit they also exhibited transfer effects from the preceding blocked-order trial. Overall, these findings were interpreted as evidence that short-term predictions derived from online information can be combined with long-term memorized information from previous trials to generate accurate predictions of the occluded target trajectories (Bennett et al., 2010b). Similar predictive mechanisms, in effect, have been shown to account also for smooth pursuit initiation and oculomotor responses to sudden perturbations of the target motion (Kowler and Steinman, 1981; Kowler et al., 1984; Tabata et al., 2008).

By simulating naturalistic conditions, recent studies of oculomotor behavior have indicated that long-term information might include also experience-based models of physical interactions between the visual target and the environment. In a series of experiments, Diaz et al. created an immersive virtual racquetball environment, which allowed them to study both eye and hand interceptive movements made by unskilled players to hit virtual balls that were projected in a parabolic arc consistent with the effects of gravity and bounced on ground before arrival (Diaz et al., 2013a). The ball trajectories were manipulated parametrically such that balls varied in the launch point, the location of the bounce, and the location where they passed the observer. In addition, balls could differ in elasticity, and thus could follow different trajectories after the bounce even when they had the same pre-bounce trajectories. Consistent with the idea that predictions based on internal models of the physical properties of the environment may represent a significant component of oculomotor control, Diaz and colleagues found that pre-bounce saccadic movements accounted for changes due to both ball velocity and elasticity and predicted the time and location of the ball after the bounce. Moreover, subjects’ oculomotor behavior was not affected by a 100 ms period of visual occlusion of the ball’s trajectory immediately after the bounce, reinforcing the idea that memorized information about the dynamic properties of the moving object is incorporated in the oculomotor plan (Diaz et al., 2013b).

In the same vein, target motion congruent with the natural motion of an object rolling on the ground elicits faster anticipatory pursuit responses compared with incongruent motion, suggesting that information about natural object kinematics acquired through daily-life experience is taken into account by oculomotor control processes (Souto and Kerzel, 2013). In this respect, gravity represents a major factor since it affects invariably the motion of falling objects by imposing a constant accelerative force. While most of the evidence in favor of the idea that the brain has internalized information about gravity effects for predictive control of motor behavior comes from manual interceptive studies (which we will review in the next section), there is recent evidence that also the oculomotor control may exploit such a-priori knowledge. Delle Monache et al. (2015) analyzed the oculomotor behavior of head-fixed subjects. Subjects shifted a mouse cursor to intercept computer-simulated ballistic trajectories, which could be occluded for variable intervals before interception. The law of motion of the trajectories could be either congruent with Earth’s gravity (1 g) or perturbed with weightlessness (0 g) or hypergravity (2 g) effects. Analysis of the overall oculomotor behavior during the manual interception task and of isolated periods of smooth pursuit of the ball trajectories showed that eye movements depended significantly on the target law of motion and occlusion, in ways compatible with the view that predictive mechanisms based on implicit knowledge of gravity effects may contribute to oculomotor control (Delle Monache et al., 2015).

In sum, the idea emerging from reviewing these studies is that control of predictive eye movements to overcome missing visual information may take advantage of an internal representation of the target motion, which combines time-varying information (target velocity, acceleration) derived from vision of the target motion prior to its disappearance and prior knowledge of the target motion derived from heuristics and from internalization of invariant physical characteristics of the environment.

Evidence from Manual Interception Studies

Hand interceptive actions represent another particularly interesting experimental model to study visual extrapolation mechanisms. These actions require accurate estimation of the time and place for object interception, achieved through a fine interplay of feedback and anticipatory mechanisms to compensate for visuomotor delays, which, depending on the complexity of interceptive action, may make unusable the last 100–200 ms of visual information about the target motion (see Regan and Gray, 2000; Zago et al., 2008, 2009; Merchant et al., 2009 for reviews on the psychophysics and neurophysiology of interception). Thus, spatial and temporal aspects of the interceptive action might reflect predictions of the objects’ motion, especially when transient occlusions of the interceptive target or degradation of the visual motion information, such as apparent motion stimuli, are introduced within the experimental paradigm.

Interception of Occluded Targets

One main issue that has been investigated by using visual occlusion paradigms relates, for example, to the early work of Whiting and collaborators who varied systematically onset and duration of visual occlusions of parabolic ball trajectories to determine potentially critical viewing time windows for successful interception (Whiting et al., 1970; Sharp and Whiting, 1974). These original studies have been replicated recently by using up-do-date technologies and it was found that the interceptive performance deteriorates dramatically only when vision is occluded from 150 to 200 ms prior to the movement onset, albeit the probability of successful interception may vary, to some extent, also with the overall length of time that the target is made visible (Marinovic et al., 2009; López-Moliner et al., 2010). For batting movements, however, precise timing of the interceptive actions may depend on whether visual information is available during the swinging movement (Brenner et al., 2014). In effect, there is also evidence that sudden changes in target direction could be estimated and the motion of the target extrapolated accurately with short viewing times comprised between 50–150 ms, suggesting that, under certain experimental conditions, even relatively small viewing times may allow accurate interception (Mrotek et al., 2004). Besides the extent of the visible motion, timing accuracy in motion extrapolation tasks has also been found to depend on the spatial properties of the target, such as its size. Target size effects seem to be strongly modulated by the target speed, according to the velocity transposition principle by which larger targets are perceived to be slower (Sokolov et al., 1997; Sokolov and Pavlova, 2003).

Target velocity, in fact, appears to be a major parameter of object motion taken into account by visual extrapolation processes for the timing of interceptive actions. Experiments in which a target moved across a computer screen with different velocity profiles and could be occluded either early or late in the trajectory, showed that subjects are able to use target velocity information acquired during the early part of visible trajectory to build a velocity representation of the occluded portion of the target motion and control the timing of the interceptive action (Dubrowski et al., 2000). The idea that internal representations of the target motion are built during the visible portion of the target motion and then used to update or predict the future trajectory is supported also by experiments that examined the effects on the temporal accuracy and the kinematics of interceptive movements of visual occlusions occurring after a sudden target velocity change. Compatible with this view, reduction of visual exposure affected movement variability, but not the velocity of the arm or the directional trend of timing errors (Teixeira et al., 2006).

Use of the velocity information available before target occlusion for guiding the interceptive action may be somewhat reminiscent of the control mechanisms underlying ocular pursuit of occluded targets described in the previous section. In effect, some analogies with the smooth pursuit system have emerged from a study, which examined the effects of occluding different portions of ball trajectories approaching from different directions on the lateral hand movements to catch the balls (Dessing et al., 2009). When the late portion of the ball trajectories was occluded, interceptive movements showed a significant spatial bias, which could be interpreted as a result of incomplete motion extrapolation. In particular, the authors hypothesized a reduction of the velocity gain similar to that observed in the smooth pursuit system (Dessing et al., 2009). Conversely, another study, which tried to disentangle the use of position and velocity information for the control of the interception position, suggested that target velocity might not contribute directly to the spatial estimates of the interceptive action (Brouwer et al., 2002). In this study, subjects hit visual targets moving at constant speed and disappearing after variable time intervals within either a stationary or a moving background. Hitting positions after the target disappearance denoted the use of velocity information only indirectly, perhaps via a speed-dependent misperception of position, implying also differential control of spatial and temporal estimates of the interceptive action (Brouwer et al., 2002).

Moreover, studies that involved more complex target motion patterns have indicated that visual extrapolation might include additional parameters besides target speed. For example, analysis of the initial direction of interception movements to hidden circular and oval trajectories pointed out that extrapolation of target motion was based on target position, velocity and curvature (Soechting and Flanders, 2008). Along the same lines, when subjects intercepted targets whose speed could be governed by three different laws of motion (i.e., constant speed, a power law relation between speed and curvature, or a sum of sinusoids), the initial direction of the interceptive movement could be predicted by assuming that target position was extrapolated using a combination of target velocity and distance from the finger to the target (Soechting et al., 2009). The studies mentioned thus far dealt mostly with motion of the interceptive target along the fronto-parallel plane. Conversely, for target motion approaching the observer (i.e., looming stimuli), classical views based on Gibson’s ecological approach have argued that TTC information can be derived directly from optical variables, like tau, that is, the ratio between the size of the object’s image on the retina and its expansion rate (Lee and Reddish, 1981). As mentioned in a previous section, when an occluder interrupts the flow of optical information, TTC would be counted down by relying on internal clocks (Tresilian, 1995). However, this view has been challenged by DeLucia (2004), who used computer-generated objects that approached the observer and were partially concealed by either stationary or moving occluders of variable size and shape. Results indicated that both types of occluders affected TTC judgments, suggesting that TTC estimates of partially occluded approaching objects is not based on tau but may involve information about heuristics and invariants.

In this respect, a growing body of research over the past decade has explored the possibility that, in addition to short-term memory of the target motion parameters, visual extrapolation for the control of interceptive actions may take into account both short and long-term representations derived from past history, cognitive cues or from internalized knowledge of the physical characteristics of the environment.

It has been shown, for example, that in PM tasks the target velocity of previous trials can affect current velocity estimates, suggesting that subsequently encoded velocity representations can be partially blended (Makin et al., 2008). Long-term representations of the target motion could be derived also from abstract categorical information, as indicated by a later study from the same research group, reporting that performance in a PM task was influenced by the color of the target, which was associated with different distributions of target velocities (Makin et al., 2009a).

One interesting aspect, which we also mentioned in the previous section, concerns the idea that internal models of physical characteristics of the visual environment can be built over experience or extensive practice and used to predict and intercept the motion of an object. This may be crucial to overcome limitations inherent to the visual system, like its poor sensitivity for accelerating motion despite the fact that in nature objects’ motion is often accelerated by gravity or decelerated by frictional forces (Werkhoven et al., 1992; Watamaniuk and Heinen, 2003). For interceptive actions of free-falling objects performed under full visual guidance, there is plenty of experimental evidence that TTC estimates reflect a combination of visual motion cues with prior knowledge of gravity effects on the object’s motion (see Zago and Lacquaniti, 2005; Zago et al., 2008, 2009; Lacquaniti et al., 2013 for reviews). The extent to which vertical accelerations could be extrapolated through brief occlusions was investigated by experiments that presented virtual targets descending on a blank screen accelerated by gravity, decelerated by reversed gravity, or at constant speed (Zago et al., 2010). Targets could disappear for a brief, variable period prior to arrival or could remain visible throughout their trajectory (see Figure 3A). Subjects’ response timing, during occluded trials, denoted better interceptive performance with 1 g targets compared to 0 g and −1 g targets (Figure 3B), arguing in favor of the idea that prediction of occluded vertical motion incorporates expectation of gravity effects (Zago et al., 2010).

Evidence in support of this view emerged also from a study, which evaluated the anticipatory muscle tensing (AMT) responses recorded from the biceps and flexor carpi radialis during catching of free-falling objects. Subjects were allowed full vision of the falling object in some trials, while in other trials they were instructed to keep their eyes closed. Moreover, in randomly selected trials the object stopped suddenly 12 cm above the hand, so to dissociate the effect of impact anticipation from the reflexive tactile response associated with the impact and determine the time scale of AMT control. The study found that the time-lags of the AMT responses were sufficiently long (121 ms, on average) to include acceleration in the anticipation of the falling object motion (Vishton et al., 2010). Predictive control based on acceleration timing information has been reported also with respect to the initiation of batting movements when subjects hit balls dropped from different heights and partially concealed, even though prospective control mechanisms could be involved in guiding the bat to ball interception (Katsumata and Russell, 2012).

The finding that movement initiation when intercepting free falling objects takes into account gravity acceleration information can be also seen within the probabilistic framework defined by the work of Faisal and Wolpert (2009). In this study, subjects had either to estimate the landing location of parabolic trajectories or intercept the trajectory at a pre-defined point by moving a virtual paddle on the PC screen. In separate experiments, either sensory or motor aspects of the task were manipulated by varying the portion of visible trajectory and the time allowed to move the virtual paddle, respectively. It was reported that subjects’ decisions when to start moving were statistically near optimal, provided individual sensory and motor uncertainties. In this respect, a-priori information about gravity acceleration could be seen as a factor reducing the sensory variability.

The relative contribution of various information sources to the visual motion extrapolation processes underlying the control of the interceptive action has been examined more extensively by a recent study, which manipulated a computer-generated visual environment representing a baseball game (Bosco et al., 2012). Subjects intercepted simulated fly-ball trajectories, which were either fully visible or occluded for variable time intervals before ball landing (Figure 4A). The natural ball motion could be randomly perturbed during the descending limb of the parabolic trajectory with effects of either weightlessness or increased gravity, or it could remain unaltered. In addition, to examine the contribution of previous visual experience with the perturbed trajectories to the interception of the occluded targets, the order of visible and occluded sessions was permuted among subjects. Interception of fully visible targets denoted the combination of servo-controlled and predictive processes, whereas when intercepting occluded targets subjects relied mostly on predictive mechanisms. Ball motion predictions were based, however, on different information depending on previous visual exposure with the perturbed trajectories (see Figure 4B). Subjects without prior experience of the perturbed trajectories showed interceptive errors consistent with predictive processes based on a-priori knowledge of gravity. Instead, subjects who had been exposed to the fully visible trajectories showed interceptive responses to the hidden targets compatible with the idea that implicit knowledge of the perturbed motion was also taken into account for the extrapolation of occluded trajectories. This latter result is in agreement with another recent finding, which indicated that subjects can learn, after relatively short practice, to extrapolate and intercept, through periods of visual occlusion, complex targets’ motion resulting from arbitrary accelerations (de Rugy et al., 2012). Thus, at least within the context of the experimental conditions used by these two studies, internal models of arbitrary accelerations can be developed rapidly to afford visual extrapolation and interception of targets in absence of visual feedback. Furthermore, the evidence that a-priori knowledge of gravity is a primary factor taken into account for the interception of occluded projectile trajectories goes along with results of a perceptual study, which investigated the possible representational analogue of the projectile trajectory. In a series of experiments, subjects observed visual targets moving horizontally, either in the leftward or rightward direction, and vanishing after a variable distance along the motion trajectory. Subjects were asked to indicate the position of the vanishing target after a variable retention delay. Moreover, to control for oculomotor factors, in different experimental trials subjects were either constrained to ocular fixation or they could move their eyes freely. It was found that the horizontal and vertical components of the positional displacement in the subjects’ responses followed a different trend with respect to the length of the retention interval and to the oculomotor behavior. The perceptual localization of the vanishing position was shifted forward in the horizontal direction of motion with retention delays up to 150 ms and remained constant or decreased at longer delays. However, the degree of the horizontal displacement depended significantly on whether subjects held fixation or pursuit the visual target. Conversely, the vertical displacement of the perceptual localization was very small and constant for retention delays up to 150 ms, whereas it increased downward at constant rate with longer retention intervals, irrespective of the constrained or unconstrained ocular behavior, perhaps reflecting internalization of gravity information (De Sá Teixeira et al., 2013).

The idea that extrapolation processes underlying both perceptual and visuomotor responses might have access to an internal representation of gravity has been tested also within the context of inclined plane motion by a recent study carried out by La Scaleia et al. (2014). In these experiments, a ball rolled down an incline with different kinematics depending on the starting position and the slope of the incline. In different experiments, participants intercepted the ball as it fell off the incline, or they had to imagine that the ball kept moving after it was stopped at the end of the incline and either intercept it or draw with their hand in air the imagined ball trajectory. Although the most accurate behavioral performance was found with fully visible ball motion and haptic feedback of the hand-ball impact, also when participants intercepted or drew the imaginary trajectory, global aspects of the target motion, such as its path, speed and arrival time could be estimated well (see Figure 5). These results might imply that the paths and kinematics of balls rolling down inclined planes were extrapolated using both visual information and internal representations of the target motion (La Scaleia et al., 2014).

Finally, visual extrapolation processes could be influenced by the familiarity with specific environmental conditions, which facilitates the interpretation of the underlying physical events. For example, time-to-contact estimates of approaching objects may be affected greatly by identity/familiarity information about the object, in addition to the retinal image expansion rates (Hosking and Crassini, 2010). Experiments in which participants intercepted targets sliding down either an inclined plane or a tautochrone (i.e., the cycloid curve along which a point mass slides to the bottom under gravity in the same time regardless of the starting position) examined this possibility more in depth (Mijatović et al., 2014). Although gravity acceleration affected the targets similarly in both cases, the inclined plane represented a familiar situation, whereas the tautochrone did not (Figure 6A). In these experiments, the gravity field could be congruent with either natural gravity or reversed gravity and the target motion could be occluded from view for variable time intervals before interception. Time shifts consistent with a-priori assumptions of natural gravity between the interceptive responses to natural and non-natural targets were found only for the familiar condition of the inclined plane but not for the tautochrone (see Figure 6B), suggesting that target motion extrapolation depends on integration of high-level cues about trajectory familiarity with lower-level target kinematics information (Mijatović et al., 2014).

Interception of Targets in Apparent Motion

In addition to the target occlusion paradigms we considered so far, there are other cases in which the kinematic information about a target displacement is incomplete or degraded. One such condition is represented by a target that suddenly disappears and reappears at a different position, so called “target jump”. The mechanisms involved in the interception of a jumping target will not be covered here, because they have been reviewed extensively elsewhere (Battaglia-Mayer et al., 2014). In this section, instead, we consider the case of intercepting long-range apparent motion (LAM). LAM is typically generated by flashing a stationary target in sequence at different locations along the path, with a wide spatial and temporal separation between successive locations. There is still no consensus as to whether LAM represents a high-level motion process distinct from low-level real motion (RM) or whether both RM and LAM result from the activation of spatio-temporal correlators by appropriate spatial and temporal combinations of the stimuli (see for instance, Adelson and Bergen, 1985; Watson and Ahumada, 1985; Cavanagh, 1992; Lu and Sperling, 2001). Irrespective of the specific mechanism involved, the spatio-temporal properties of LAM differ widely from those of RM (Gregory and Harris, 1984). Therefore, it is of interest to compare the interception mechanisms involved with LAM and RM.

Georgopoulos and colleagues studied the manual interception of targets moving at constant speed along a circle in RM or LAM in man and monkey (for a review, see Merchant and Georgopoulos, 2006). Subjects shifted a cursor to catch the target within a given interception zone. The spatial angular error increased linearly with target speed and was larger for LAM than RM (Port et al., 1996; Merchant et al., 2003). There was a similar trend of timing errors for both RM and LAM: subjects tended to be early with slowly moving targets, and late with fast moving targets. Overall, the results showed that there are similarities in performance between RM and LAM, but the performance is somewhat degraded in LAM.

Manual interception of accelerating/decelerating LAM was studied by Maffei et al. (2010). Here, a target moved up and down at 1 g, 0 g, or −1 g and randomized initial speeds, and subjects had to press a button to intercept the sphere at the expected time of arrival at destination (Figure 7A). Because the target was never flashed at the arrival point in LAM, subjects had to extrapolate the stimulus traversing the vacant space in order to intercept the target. The resulting pattern of absolute timing errors was very similar for LAM and RM, with significantly smaller errors for 1 g and 0 g targets than −1 g targets (Figure 7B).

Overall, the studies reviewed here on the interception of occluded and apparent motion targets, have contributed to identify critical information the CNS may rely upon when filling the gaps in either degraded or missing visual information. Interestingly, some of the evidence presented in this section appears congruent with that obtained by analysis of the oculomotor behavior under similar experimental conditions, in that short-term low-level information about target kinematics may be combined with long-term cognitive associations derived from either external cues or familiarity conditions and internalized information about physical invariants. In particular, interceptive actions requiring haptic interactions with the visual target may make more compelling the use of a-priori knowledge about invariant physical forces acting on the object, such as the gravity acceleration.

Do Common Predictive Signals Drive Oculomotor and Interceptive Behavior?

As just noted above, strong commonalities with respect to the type of information contributing to the visual motion extrapolation processes emerged by examining the oculomotor and the manual interception literature (see Table 1 for a summary). Although these commonalities might imply that interceptive and oculomotor control share interactively predictive signals about the target motion, the issue is still controversial. On the one hand, psychophysical studies have raised the possibility that predictive information could be weighted differentially, or could even be entirely different (Wexler and Klam, 2001; Eggert et al., 2005). Moreover, based on analyses of the optical variables accounting for oculomotor and interceptive responses in a motion prediction task, Benguigui and Bennett (2010) argued for no functional relationship and independent control of ocular pursuit and interceptive timing. In later experiments, the same authors analyzed oculomotor and interceptive responses in a PM task using accelerating objects and found a general bias in the estimation of the reappearance position of the occluded objects that could be only in part explained by the oculomotor behavior during the occlusion. This led the authors to conclude that eye movements during occlusion may contribute, but do not uniquely specify information for accurate spatial or temporal estimation in the manual task (Bennett and Benguigui, 2013).

On the other hand, there is also evidence that, even though independent control of arm and ocular responses may take place, concurrent arm movements can improve the gain of the smooth ocular pursuit during transient occlusions (Bennett et al., 2012). Analogously, the metrics of saccadic responses to memorized targets can be influenced significantly by the metrics of accompanying arm movements to the same targets, suggesting that signals related to the metrics of limb movements influence, at a programming stage, those controlling saccade metrics (Kattoulas et al., 2008). Further evidence in favor of shared predictive signals between oculomotor and manual interceptive control comes from a study which examined manual interceptive responses to occluded visual targets while subjects either fixated a designated point of the visual scene or tracked the visual target continuously (Bennett et al., 2010a). The study showed that eye movements could facilitate accurate TTC estimation in the interception task, indicating that pursuit of the visible portion of the object motion could be crucial in providing both extra-retinal and retinal signals to the predictive processes for the control of the interceptive timing (Bennett et al., 2010a). The importance of extra-retinal signals derived from eye movements for visual motion prediction has been remarked also by experiments testing the perceptual judgments made by subjects on whether a disappearing visual target would hit or miss a stationary goal. Results indicated again that motion direction predictions were significantly better when subjects pursued the target than when they maintained fixation (Spering et al., 2011).

Analogous differences were reported by Makin and Poliakoff (2011), who tested subjects with both a perceptual (a two-alternative forced choice task with respect to the time of target reappearance after the occlusion) and a PM task (subjects indicated with a button press the time of target reappearance after the occlusion). The two tasks could be performed under conditions that either enforced ocular fixation or allowed tracking the visual target. Subjects showed significantly different performance in both tasks under fixation compared to free eye movement conditions. More interestingly, eye movements during the occlusion were related to the subjects’ responses, suggesting that overlapping systems govern eye movements and perceptual/motor judgments on visual motion extrapolation tasks (Makin and Poliakoff, 2011). Along these lines, a recent study investigated the relationships between oculomotor behavior and interceptive performance while subjects intercepted, by acting on a computer mouse, projectile-like trajectories that could be either fully visible or occluded for variable time intervals before the interceptive event (Delle Monache et al., 2015). Interceptive errors were related by means of multivariate regression models to a number of oculomotor variables indicating where subjects directed their gaze throughout the trial, how many gaze shifts occurred during the trial, how stable was the oculomotor behavior close to the interception event, and how accurately subjects tracked either the target trajectory or the mouse cursor just prior to the interceptive response. Interestingly, when subjects intercepted the occluded targets, smaller interceptive errors were associated with more accurate tracking of the target trajectories and with a more stable oculomotor behavior prior to the interceptive response, supporting further the idea that efferent oculomotor information could contribute to the predictive estimates of the target motion necessary for the interceptive action.

Putative Neural Substrates

With respect to the brain regions potentially involved in visual motion extrapolation processes, both monkey neurophysiology and human neuroimaging studies attribute a critical role to the posterior parietal cortex, particularly to the lateral intraparietal sulcus (LIP) in monkeys and to the posterior part of the human intraparietal sulcus (IPS), which is considered as the human homologue of the simian LIP. Strong evidence has come primarily from a series of electrophysiological studies in monkeys carried out by Assad and collaborators. By comparing the activity of parietal neurons during full vision, target motion occlusion and visual target blinks, it was found that parietal neurons retain a sustained activity during the occlusion condition that might represent inferred target motion in the absence of visual input or motor output (Assad and Maunsell, 1995). Eskandar and Assad (1999) investigated this issue further by comparing the activity of LIP, medial superior temporal area (MST) and medial intraparietal area (MIP) neurons while monkeys used a joystick to guide a spot on a computer display from the center of the screen to peripheral targets. In some trials, vision of the moving spot was occluded. They used information theory analysis to determine the type of information transmitted by the neurons in the three cortical areas, and found that LIP had the highest percentage of neurons whose activity represented inferred motion of the spot, while MST and MIP activity was best related to visual motion and hand motion, respectively (see Figure 8A). However, neurons in MST and MIP could also transmit information about inferred motion. This idea has received further support by neuroimaging and electroencephalographic studies in humans that have reported consistent activations of the posterior parietal cortex, particularly of the IPS, during oculomotor and manual tasks requiring visual extrapolation of occluded targets (Lencer et al., 2004; Olson et al., 2004; Ogawa and Inui, 2007; Shuwairi et al., 2007; O’Reilly et al., 2008; Beudel et al., 2009; Makin et al., 2009b).

In addition to the extrapolation of occluded targets, the role of posterior parietal cortex (area 7a) has been investigated with respect to the manual interception of constant speed, circular RM and LAM, and compared with that of the primary motor cortex in the monkey (Merchant et al., 2004a,b,c, 2005). Stimulus-related activity has been found to prevail in area 7a and hand-related activity in the motor cortex (see also Merchant et al., 2001). Moreover, neural activity was selectively associated with the stimulus angle during RM, whereas it was tightly correlated to the time-to-contact in the LAM condition, particularly in the motor cortex (see Merchant and Georgopoulos, 2006).

Premotor areas also appear to be involved consistently in visual extrapolation processes. For example, in studies where the experimental paradigm involved the production of an oculomotor response, such as a saccade to the moving target or a smooth pursuit movement, activity within the frontal eye-fields (FEF) during the periods of target occlusion showed activity compatible with this idea (Fukushima et al., 2002, 2008; Barborica and Ferrera, 2003, 2004; Xiao et al., 2007; Ferrera and Barborica, 2010). Similarly, human FEF, along with the prefrontal cortex, the angular gyrus and parieto-insular vestibular cortex (PIVC), seem to be engaged during maintenance of ocular tracking in the absence of a visual target (Nagel et al., 2006). Moreover, when tasks involve a manual response and/or temporal predictions, activations of human supplementary motor areas (SMA and pre-SMA) and, to a lesser extent, the premotor cortex, have been reported in relation to transient occlusions of visual targets (Ogawa and Inui, 2007; Shuwairi et al., 2007; O’Reilly et al., 2008; Beudel et al., 2009). Instead, the evidence that visual motion areas may also contribute to visual motion extrapolation processes is more controversial. For example, while Assad’s studies failed to show consistent activity related to target motion during visual occlusion in monkey area MST, Thier et al. have shown activity in the same area (but not in MT) related to the motion of “imaginary” targets (Ilg, 2003; Ilg and Thier, 2003). Also in human fMRI studies, activation of area hV5/MT+ during visual target occlusion has not been consistently found. However, the finding reported by Olson et al. (2004) that activity in hV5/MT+ was elevated during the occlusion period with a similar trend as that observed in the IPS may be suggestive of the possibility that hV5/MT+ does not exert a purely sensory function, but may also contribute to predictive processes. This could be consistent with other fMRI evidence that hV5/MT+ activity may be modulated by stimulus predictability and with results of transcranial magnetic stimulation studies showing that that hV5/MT+ may be involved in the processing of temporal predictions for target interception, as well as in motion prediction processes underlying apparent motion perception (Bosco et al., 2008; Alink et al., 2010; Vetter et al., 2013). In addition, visual motion extrapolation processes appear to involve also subcortical structures like the cerebellum, as suggested by neuroimaging studies that reported cerebellar or basal ganglia activations along with cortical areas (O’Reilly et al., 2008; Beudel et al., 2009). Direct electrophysiological evidence, in this respect, comes from Purkinje cells recordings in the D2 zone of the cat cerebellum, which showed that Purkinje cells maintain simple spike tonic activity during transient disappearance of a moving visual target, possibly reflecting the use of an internal model based on memory of the previous target motion (Cerminara et al., 2009).

Visual motion extrapolation in complex perceptual and motor tasks may require coordinated activity in many of these brain regions (see Figure 8B). In fact, functional connectivity analyses carried out by the study of O’Reilly et al. (2008) obtained good indication that temporal predictions resulting from visual extrapolation processes may be subtended by interplay between subcortical structures (such as, the cerebellum and the basal ganglia) with a parieto-frontal cortical network, including IPS, the FEF and V5/MT+.

Finally, a specific role of the posterior insula and lingual gyrus for encoding gravity effects during interception of both linear RM and LAM has been revealed by fMRI (Maffei et al., 2010; Figure 9A). These regions seem to extract and process visual gravitational motion across a wide range of spatial and temporal frequencies (Maffei et al., 2010). In addition, a higher-order motion region located in the inferior parietal lobule close to the temporo-parietal junction (so called HM-IPL region) showed a preference for visual gravitational motion in LAM but not RM (Figure 9B). In previous studies, HM-IPL was described as a substrate for high-level motion processing based on salience, being engaged by color-salient, isoluminant gratings and by quartet-display LAM at 7 Hz (Claeys et al., 2003). These imaging findings fit with patients’ studies indicating that IPL lesions may impair LAM perception (Battelli et al., 2001).

In sum, visual motion extrapolation processes appear to be consistently reflected by activity in the IPS, with several other cortical and subcortical brain areas being engaged depending on the nature of the behavioral task (purely perceptual, oculomotor or manual interception) and of the visual motion stimulus (smooth or apparent motion).

Conclusions

The present article reviewed experimental evidence, drawn mostly from the oculomotor and the manual interception literature, concerning the nature of visual motion extrapolation processes, which might allow efficient motor interactions with the environment when visual feedback is lacking or degraded. We concluded that several kinds of information contribute to visual motion extrapolation, including not only short term memory of the target kinematics, but also long-term information derived from heuristics, cognitive cues and internal representations of the physical properties of the visual environment. In particular, we discussed a growing body of evidence indicating that predictions of objects’ motion in a natural environment might be based on presupposed knowledge of gravity effects. Finally, with respect to the putative neural substrates, a critical role in maintaining a neural representation of the invisible target motion appears to be played by the intraparietal cortex together with other cortical and subcortical areas that may be engaged differentially depending on the nature of the behavioral task.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The work was supported by the Italian University Ministry (PRIN grant) and the Italian Space Agency (SLINK, COREA and ARIANNA grants). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References