The role of attention processes in near transfer of cognitive skills

https://doi.org/10.1016/S1041-6080(01)00038-3Get rights and content

Abstract

We tested hypotheses about the respective roles of working memory, perceptual speed, knowledge, and attention disengagement processes in skill transfer errors that resemble einstellung effects or strong-but-wrong slips. Correlational evidence was consistent with the hypothesis that these errors are primarily due to the inability of learners to disengage attention from expected or familiar operations. The data also supported the notion that error proneness during near transfer generalizes across different skills. Contrary to expectations, working memory capacity had little or no relationship to either transfer errors or attention disengagement ability. Results are discussed with respect to skill training and individual differences in skill performance ability.

Introduction

The research presented in this article addressed the transfer of cognitive skills. Most research on cognitive skills has addressed questions about the acquisition of skill or the nature of performance improvements that develop with practice. Yet, understanding skill transfer may be just as important in most real-world training contexts. Training is only effective if it transfers to the new processing demands that are required in subsequent applications. Most skills cannot be trained in an instructional setting such that learners are exposed to all performance conditions that will eventually be encountered. Instead, it is more typical that a learner receives skill training in a relatively narrow set of conditions. Examples of cognitive skills that necessarily must be taught using a limited number of training instances include math problem solving, aircraft piloting, comprehension skills used in reading, and diagnostic skills used in medicine and technical specialties. In fact, it is difficult to think of complex skills in which a learner can be trained on all possible variations that will eventually be encountered.

In the study of skill transfer, it is sometimes useful to distinguish near from distant transfer of the learned material (Salomon & Perkins, 1989). As the name implies, near-transfer conditions refer to skill performance demands that differ in small ways from performance demands during the acquisition of the skill. For example, a learner may have acquired skill in multicolumn addition through exposure to hundreds of different practice items. Inevitably, the learner will encounter new instances of such addition problems, and performance on these would be considered near transfer of the learned skill. Conversely, distant transfer refers to the other end of the continuum describing differences between training and transfer demands. For example, if the learner who practiced multicolumn addition is next presented with multicolumn subtraction problems, this requires new operations (e.g., borrowing instead of carrying) rather than new instances of old operations. As noted by Salomon and Perkins (1989), there is no formal definition or metric for transfer distance, but it is nonetheless a useful concept in understanding different transfer phenomena.

Past research on problem solving and skill transfer has suggested that when transfer from learning can be demonstrated, it almost always occurs under near-transfer conditions. Anderson et al. have reported several detailed demonstrations of positive transfer in skills in which specific processing components overlapped. These demonstrations occurred in relatively complex skills such as typing in word processors (Singley & Anderson, 1985), computer programming (McKendree & Anderson, 1987), and geometry problem solving (Lovett & Anderson, 1994). In contrast, it is far less common to see compelling evidence for the transfer of general skills or heuristics (i.e., distant skill transfer). While some examples can be found in the research literature, it is commonly acknowledged that general transfer is at best difficult to demonstrate (Singley & Anderson, 1989). Instead, current views of skill learning and transfer more typically emphasize the specificity of cognitive skills.

The infrequency of compelling distant-transfer demonstrations contradicts early theories in education, which held that the study of subjects such as Latin and geometry would produce transfer to general learning ability (Woodrow, 1927). Instead, the bulk of the evidence for skill transfer is more consistent with Thorndike's early theory of transfer referred to as the theory of identical elements Thorndike, 1903, Thorndike & Woodworth, 1901. That is, learned skills tend to transfer to new skills as a function of the number of overlapping processing components. Variants and extensions of this principle have been described more recently in the context of procedural and implicit memory theory Kolers & Roediger, 1984, Morris et al., 1977, Singley & Anderson, 1989. The term transfer-appropriate processing has been used recently to describe the phenomenon (Morris et al., 1977). When processing demands during transfer share enough components with some prior performance (i.e., near-transfer conditions), positive transfer typically results.

Given that positive transfer in skill performance is found primarily under near-transfer conditions, it is somewhat paradoxical that negative transfer may also be found under such “optimal” conditions. Probably the best known example of negative skill transfer was in the work of Luchins (1942) who demonstrated the einstellung effect in problem solving (also referred to as the set effect). He had participants solve a series of arithmetic computation problems involving the manipulation of volumes in hypothetical water jugs. The object was to measure a specified quantity of water using various combinations of three jugs (A, B, and C) with known volumes (e.g., obtain 100 units when A=21, B=127, and C=3 units, respectively). When learners were exposed to a series of water jug problems that all required the same sequence of water jug manipulations to obtain the correct answer (e.g., Jug B−Jug A−2×Jug C), many participants were incapable of switching to a new operation sequence needed to solve transfer problems. The new problems are best described as near transfer because they looked nearly identical to the training problems. However, exposure to the restricted range of training problems produced negative transfer; those with no training did better than those with training that required only one algorithm.

The demonstration of negative transfer by Luchins could be dismissed, because empirical evidence for this type of transfer effect is relatively rare (Singley & Anderson, 1989). On the other hand, people often complain of interference when faced with modifying a familiar skill (e.g., learning a new word processor after years of using a different one; driving an automobile in the United Kingdom after learning to drive in the US). It is possible that negative transfer occurs with some regularity under certain real-world conditions, and it could be of great importance in settings where performance errors are critical. Earlier research conducted by the Army Air Force's Aviation Psychology Program (Fitts, 1947) demonstrated such a phenomenon in experiments using simulated airplane control knobs. Negative transfer errors (termed “habit interference”) were found when the position of control knobs was changed. The researchers concluded that changing cockpit controls from one plane to another would produce interference that could result in critical errors.

Norman (1981), Reason (1990), and Reason and Mycielska (1982) have proposed taxonomies of human error that occur during everyday work and leisure activities. Both taxonomies are extensive and multifaceted. However, one common theme that runs through some categories of performance slips corresponds to the einstellung phenomenon. Reason introduced the term strong-but-wrong errors to describe certain performance slips that occur in procedures or skills that are highly practiced. In essence, these are the application of incorrect but well-learned procedures, typically under near-transfer conditions. For example, when a cook intends to double a highly familiar recipe that is not normally doubled, it is difficult to overcome the tendency to use the routine measurements for the ingredients.

Although the demonstrations of einstellung-like effects have been infrequent following Luchins original work, there have been some. Woltz, Bell, Kyllonen, and Gardner (1996) and Woltz, Gardner, and Bell (2000) presented evidence of negative transfer within a sequential computation skill. The task, Number Reduction, was derived from a learning test originally used by Thurstone and Thurstone (1941). Participants applied two simple rules to reduce three- or four-digit numbers to single digits. Problems required a short sequence of rule applications. When participants were trained on only half of the possible rule sequences, errors increased dramatically when new sequences of the original rules were introduced. The magnitude of errors increased rather than decreased as a function of more training.

Both the Luchins' water jug evidence and our number reduction evidence are clear examples of negative transfer in cognitive skills under near-transfer conditions. Furthermore, they both correspond to the strong-but-wrong class of real-world errors described by Norman (1981) and Reason (1990). However, little is known about the cognitive processes underlying these skill performance errors. In Luchins' work and in our own, not all participants showed negative transfer. For those that did, it is unclear whether they represented the skill differently in memory or whether the transfer errors reflected cognitive failures in perception (i.e., encoding the new problems as different from the old ones), working memory (i.e., monitoring performance for possible errors or effectively adapting to new task demands), attention (i.e., redirecting attention from expected processes to new ones after task differences are detected), or some combination of these. The current research attempted to address these questions.

To investigate the cognitive processes underlying near-transfer skill performance errors, we used an individual differences approach. We tested whether measures from different processing domains were more important than the others in predicting individual differences in skill transfer errors.

Correlational evidence is typically ambiguous with respect to causal inferences. However, given the current questions, this methodology has certain advantages over the alternative of experimental manipulations. Introducing experimental manipulations during transfer to assess the role of working memory and attention processes could dramatically change the nature of the skill (e.g., introducing additional memory loads or secondary task demands). With such manipulations, it is difficult to address questions of near-transfer processes, because the manipulations themselves make the transfer more distant, even if nothing else in the skill task changes. Our research questions pertain to cognitive processes underlying errors that occur in highly practiced skills when very small variations in the skill task are introduced. Correlating performance under such transfer conditions with cognitive process measures seems less intrusive to the skilled performance processes than introducing additional task manipulations. In addition, correlational evidence following the same logic has been quite instrumental in testing other cognitive processes and capacity-related theoretical issues Ackerman, 1990, Cantor & Engle, 1993, Daneman & Carpenter, 1980, Daneman & Carpenter, 1983, Engle et al., 1992, Engle et al., 1991, Just & Carpenter, 1992, Kyllonen & Christal, 1990.

There is little prior evidence regarding individual differences in skill transfer performance, especially near-transfer performance. Goska and Ackerman (1996) recently hypothesized ability differences related to different types of skill transfer. Based partly on earlier work by Sullivan (1964), reviewed by Goska and Ackerman and evidence from their own skill acquisition research, they proposed that general cognitive ability would be important to transfer performance only when transfer is distant. Evidence from two correlation studies was consistent with these predictions. General ability predicted transfer performance best when training did not expose participants to all task operations required in transfer. Thus, more distant transfer appeared to place heavier demands on general cognitive capacity or ability than did near transfer. However, this research did not address the issue of the cognitive demands under conditions that produce negative near transfer.

Some correlational evidence regarding negative transfer has been reported recently. Bell, Gardner, and Woltz (1997) reported two studies that investigated personality and ability characteristics that might be related to near-transfer errors. Two main conclusions were drawn. First, self-report measures of error proneness Broadbent et al., 1982, Reason & Mycielska, 1982 were not related to near-transfer errors that correspond to the strong-but-wrong description. Second, measures of working memory capacity were predictive of the near-transfer errors. Bell et al. (1997) reported correlations of r=.44 and .46 between two working memory error measures and near-transfer performance errors.1 However, in subsequent work with larger samples, Bell (1997) found lower but still statistically significant correlations of r=.23 under the most similar near-transfer condition and r=.34 under a more dissimilar near-transfer condition. Thus, working memory ability correlated with the near-transfer skill performance, but the magnitude of this relationship was variable and may depend on transfer task characteristics (e.g., transfer distance).

In this research, we addressed three primary questions. First, we addressed the generalizability of skilled performance errors by individual performers across skill tasks. If error proneness is idiosyncratic to particular learners in particular tasks, then the construct has little theoretical or practical importance. However, if the same people who make errors after extensive practice in one skill are also prone to make errors in other skills, it suggests a general cognitive mechanism that is thus far not understood. Previous correlation studies used only a single skill task, so existing data do not speak to this issue. In the current research, we provided participants with moderate amounts of practice on two cognitive skills. As noted earlier, Number Reduction had been used in several previous skill learning studies and produced negative transfer errors Bell et al., 1997, Kyllonen & Stephens, 1990, Woltz et al., 1996, Woltz et al., 2000. The second skill task required the use of Boolean transformations known as logic gates. Logic gates, which have their origins in electrical engineering, have also been used in previous research on cognitive skill acquisition Carlson et al., 1989, Carlson & Yaure, 1990, Kyllonen & Stephens, 1990. Participants must interpret symbols that transform multiple digital inputs to a single output. In both skill tasks, participants performed training trials followed by near-transfer trials that introduced new processing sequences of the original rules or operations. The correlation of transfer errors in these two skills was used to assess the generalizability of error proneness in cognitive skills.

The second question we addressed was whether errors in skill performance, especially in near-transfer conditions, were related to individual differences in working memory capacity and attention shifting ability. Although prior evidence about working memory and skill errors was somewhat mixed, we hypothesized that error prevention would be enhanced by additional cognitive resources for monitoring performance even after extensive practice. We hypothesized that attention-switching ability would be important for a potentially different reason. When transfer stimuli are perceived to differ in some small feature from more familiar training stimuli, correct performance should depend on the ability to disengage attention from a heavily used procedure and shift to a less frequently used procedure. This disengagement process may simply depend on the availability of general processing resources (i.e., working memory) or may depend on specific attentional abilities. If both working memory and attention measures predict transfer errors, the question of whether they account for unique or overlapping variance in errors becomes important. Some theoretical descriptions of working memory suggest that attention control constructs may be central to overall capacity. Yet, attention and working memory are treated as separate constructs in much of the research literature, and as noted here, their hypothesized roles in error prevention are potentially different.

To address these questions, we included two measures of working memory capacity and four measures of attention shifting ability. The working memory measures were derived from tasks used in prior research that showed reasonably high correlations with other working memory measures Woltz, 1988, Woltz & Shute, 1993 or had high loadings on general working memory factors (Kyllonen & Christal, 1990). The attention tasks fell into two categories. Two tasks represented variations of the Stroop paradigm, which is generally thought to assess the ability to overcome strong automated processes MacLeod, 1991, Stroop, 1935. One task was a computerized version of the original color–word Stroop paradigm. The other was a number string Stroop task based on work by Morton (1969). In the latter task, participants were shown a string of one, two, or three digits (e.g., 222, 33, 1, etc.). They responded with the length of the string, which on most trials conflicted with the number value of the digits in the string.

The second category of attention tasks represented variations of the Posner attention disengagement paradigm (Posner, Snyder, & Davidson, 1980). In this paradigm, all trials begin with a cue that informs the participant as to the type of stimulus to be presented. On most trials (e.g., 80%), the cue is valid and allows the participant to focus attention on the requirements of the upcoming stimulus. On the remaining 20% of the trials, the cue is invalid, and when the stimulus appears, the participant must disengage or shift attention from the expected or cued trial condition to that which was actually presented. Although the original Posner paradigm pertained to visual attention shifts in space, our tasks were designed to measure attention shifts within more complex cognitive processes. One task required participants to shift between two comparison processes applied to word pairs (meaning and font comparisons). The other task required participants to shift between evaluating local and global features of numeric stimuli (Filoteo et al., 1994). Large digits were presented, and each large digit was comprised of an arrangement of small digits. Participants were cued to evaluate either the large digit (75% of the time) or the small digit (25% of the time).

The third question we asked pertained to more traditional ability constructs: perceptual speed and verbal knowledge. Perceptual speed was thought to be relevant in that the avoidance of einstellung or strong-but-wrong errors must begin with the perception that something in the stimulus is different from the familiar training stimuli. Thus, we included two perceptual speed measures that required rapid detection of features. String Comparison required participants to decide if two briefly exposed digit strings were identical or differed by one digit. Similarly, Finding Vowels exposed a string of letters briefly, and participants had to decide if a previously specified vowel was contained in the string. Both measures were modeled after perceptual speed tests in the French Kit of Cognitive Abilities (Ekstrom, French, Harman, & Dermen, 1976).

Two verbal knowledge measures were included in the study to represent an alternative, crystallized ability or knowledge explanation for individual differences in skill performance errors. It could be that individual differences in skill performance during transfer are simply expressions of general learning and knowledge differences that manifest in most performance tasks. For example, in these skill tasks, general verbal knowledge measures may be related to the participants' ability to acquire declarative understanding of the skill task and its requirements, and this may affect performance accuracy more than any other cognitive attribute. Such an explanation reveals little about cognitive processes in error making but must be contrasted with the process hypotheses to ensure that this is not a more valid explanation of the data. Two verbal knowledge tests were included in the study: Vocabulary and General Information. The vocabulary items were drawn from the French Kit (Ekstrom et al., 1976), and the general information items were drawn from the Air Force Cognitive Abilities Measurement (CAM) battery (Kyllonen, 1995).

Section snippets

Participants

Participants in this study were 141 University of Utah undergraduate and graduate students. Approximately half of the sample participated in the study for extra credit in Department of Educational Psychology courses. The remaining participants responded to campus advertisements and were paid US$5 per hour for their participation. Six participants were eliminated from the sample due to extremely high error rates and unrealistically fast responses on the tasks. Of the 135 remaining students, 50

Results

Prior to presenting the correlation analyses that tested the primary hypotheses, we present mean data by trial condition for the skill tasks. In addition, mean data for the cognitive process measures are summarized. Response latency values for each individual were computed as the median latency for correct trials within the designated trial type of each task, unless otherwise noted. The p value for all statistical tests was set at .05.

Discussion

The main findings of this research can be summarized as follows. First, we demonstrated substantial negative near transfer in the form of performance errors in two sequential cognitive skills. The nature of these errors bears close resemblance to errors in Luchins' (1942) einstellung demonstrations and to more recent descriptions of real-world slips that have been described as capture or strong-but-wrong errors Norman, 1981, Reason, 1990. Second, those participants who made more near-transfer

Acknowledgements

Research reported in this article was supported by Grant F49620-97-1-0219 from the U.S. Air Force Office of Scientific Research. The authors wish to thank Phillip Ackerman for helpful suggestions made on an earlier draft.

References (66)

  • M.K Singley et al.

    The transfer of text-editing skill

    International Journal of Man Machine Studies

    (1985)
  • P.D Tomporowski et al.

    Sustained attention and intelligence

    Intelligence

    (1990)
  • D.J Woltz et al.

    Individual difference in repetition priming and its relationship to declarative knowledge acquisition

    Intelligence

    (1993)
  • P.L Ackerman

    Individual differences in skill learning: an integration of psychometric and information processing perspectives

    Psychological Bulletin

    (1987)
  • P.L Ackerman

    Determinants of individual differences during skill acquisition: cognitive abilities and information processing

    Journal of Experimental Psychology General

    (1988)
  • P.L Ackerman

    A correlational analysis of skill specificity: learning, abilities, and individual differences

    Journal of Experimental Psychology: Learning Memory, and Cognition

    (1990)
  • P.L Ackerman

    Predicting individual differences in complex skill acquisition: dynamics of ability determinants

    Journal of Applied Psychology

    (1992)
  • A Baddeley

    Working memory or working attention?

  • Bell, B. G. (1997). The effects of anxiety on skilled performance errors. Unpublished doctoral dissertation. University...
  • P.M Bentler

    EQS structural equations program manual

    (1995)
  • D.E Broadbent et al.

    The cognitive failures questionnaire (CFQ) and its correlates

    British Journal of Clinical Psychology

    (1982)
  • J Cantor et al.

    Working-memory capacity as long-term memory activation: an individual-differences approach

    Journal of Experimental Psychology: Learning Memory, and Cognition

    (1993)
  • R.A Carlson et al.

    Practice schedules and subgoal instantiation in cascaded problem solving

    Journal of Experimental Psychology: Learning Memory, and Cognition

    (1996)
  • R.A Carlson et al.

    Practice and working memory effects in building procedural skill

    Journal of Experimental Psychology: Learning Memory, and Cognition

    (1989)
  • R.A Carlson et al.

    Practice schedules and the use of component skills in problem solving

    Journal of Experimental Psychology: Learning Memory, and Cognition

    (1990)
  • M Daneman et al.

    Individual differences in integrating information between and within sentences

    Journal of Experimental Psychology: Learning Memory, and Cognition

    (1983)
  • F.N Dempster et al.

    Interference and inhibition in cognition and behavior: unifying themes for educational psychology

    Educational Psychology Review

    (1999)
  • R.B Ekstrom et al.

    Kit of factor-referenced cognitive tests

    (1976)
  • R.W Engle et al.

    Individual differences in working memory and comprehension: a test of four hypotheses

    Journal of Experimental Psychology: Learning Memory, and Cognition

    (1992)
  • R.W Engle et al.

    Individual differences in working memory for comprehension and following directions

    Journal of Educational Research

    (1991)
  • J.V Filoteo et al.

    Abnormally rapid disengagement of covert attention to global and local stimulus levels may underlie the visuoperceptual impairment in Parkinson's patients

    Neuropsychology

    (1994)
  • P.M Fitts

    Psychological research on equipment design

    (1947)
  • E.A Fleishman

    On the relation between abilities, learning, and human performance

    American Psychologist

    (1972)
  • Cited by (21)

    • Long-term persistence of sort strategy in free classification

      2009, Acta Psychologica
      Citation Excerpt :

      The prototypical example of this is perhaps the water jug Einstellung problems used to study the perseveration of mental set (e.g., Luchins, 1942; Schultz, Stone, & Christie, 1997). Cognitive rigidity has also been demonstrated in a sequential number reduction task (Woltz, Gardner, & Bell, 2000), the sequential application of logic gates (Woltz, Gardner, & Gyll, 2000), and in conditional and cumulative risk judgments (McCloy, Beaman, Morgan, & Speed, 2007). A further example of cognitive inflexibility is provided by studies of functional fixedness (e.g., Duncker, 1945).

    • From models to rules: Mechanization of reasoning as a way to cope with cognitive overloading in combinatorial problems

      2004, Acta Psychologica
      Citation Excerpt :

      Although most replications have involved problems identical or very similar to Luchins' jar problems (e.g., Ferrante, 1988; Lippman, 1994, 1996; Luchins & Luchins, 1996; McKelvie, 1990; cf. Ohlsson, 1992), the effect has been observed on some less-known problems, involving mazes (Luchins, 1942), alphabet mazes ( Cowen, Wiener, & Hess, 1953), and anagrams (Rees & Israel, 1935); einstellung problems have been extensively used as a measure of the “rigidity” construct within the psychology of personality and individual differences (cf. Schultz & Searleman, 2002). In recent years, Woltz, Bell, Kyllonen, and Gardner (1996), Woltz, Gardner, and Bell (2000), and Woltz, Gardner, and Gyll (2000) reported evidence of einstellung-like effects in tasks requiring different serial computation skills. Mechanization effects correspond to the strong-but-wrong class of real-world errors described by Norman (1981) and Reason (1990) (cf. Woltz et al., 2000): in essence, these are the application of incorrect but well-learned procedures to tasks similar to the ones originally used to learn those procedures.

    View all citing articles on Scopus
    View full text