Elsevier

Academic Radiology

Volume 24, Issue 6, June 2017, Pages 762-767
Academic Radiology

Resident Education
The Effect of Teaching Search Strategies on Perceptual Performance

https://doi.org/10.1016/j.acra.2017.01.007Get rights and content

Rationale and Objectives

Radiology expertise is dependent on the use of efficient search strategies. The aim of this study is to investigate the effect of teaching search strategies on trainee's accuracy in detecting lung nodules at computed tomography.

Materials and Methods

Two search strategies, “scanning” and “drilling,” were tested with a randomized crossover design. Nineteen junior radiology residents were randomized into two groups. Both groups first completed a baseline lung nodule detection test allowing a free search strategy, followed by a test after scanning instruction and drilling instruction or vice versa. True positive (TP) and false positive (FP) scores and scroll behavior were registered. A mixed-design analysis of variance was applied to compare the three search conditions.

Results

Search strategy instruction had a significant effect on scroll behavior, F(1.3) = 54.2, P < 0.001; TP score, F(2) = 16.1, P < 0.001; and FP score, F(1.3) = 15.3, P < 0.001. Scanning instruction resulted in significantly lower TP scores than drilling instruction (M = 10.7, SD = 5.0 versus M = 16.3, SD = 5.3), t(18) = 4.78, P < 0.001; or free search (M = 15.3, SD = 4.6), t(18) = 4.44, P < 0.001. TP scores for drilling did not significantly differ from free search. FP scores for drilling (M = 7.3, SD = 5.6) were significantly lower than for free search (M = 12.5, SD = 7.8), t(18) = 4.86, P < 0.001.

Conclusions

Teaching a drilling strategy is preferable to teaching a scanning strategy for finding lung nodules.

Introduction

Perceptual errors account for a substantial part of misdiagnoses in radiology (1) and can be related to the search behavior of the observer (2). For educational purposes, it is important to identify which visual search patterns are most effective and to investigate if teaching search strategies improves perception.

Visual search characteristics that are related to expertise and high performance have been identified in various radiology perception tasks (3). For example experts tend to fixate on abnormalities faster 4, 5, 6 and need less time and a smaller number of eye fixations to inspect the image 7, 8. These characteristics derive from experience, and they lack an underlying structure that can be taught to novices.

Some specific visual search patterns are found to be related to high performance 4, 9, 10, 11. Most patterns apply to visual search in X-rays, such as chest X-rays or mammography. Two visual search types are distinguished for searching chest computed tomography (CT) images: “scanners” and “drillers” (11). Scanners tend to visually search a single slice, before scrolling further through the stack, whereas drillers focus their eyes on one quadrant of the lung fields and quickly scroll through the stack in depth before moving to another quadrant. Drillers outperformed scanners with respect to higher true positive rates and a larger lung coverage (11). One interesting finding was that, when given the option to search freely, more experienced readers tend to select “drilling” as a search pattern (the more effective pattern), suggesting it might be a pattern that has, consciously or unconsciously, evolved through instruction or practice. The relationship between search patterns and experience has been noted in several other studies 4, 9, 10, 11, although it is unknown if experts unconsciously adopt these patterns or deliberately chose or had acquired one, as a strategy.

Teaching junior trainees to use expert search strategies may not necessarily be effective. First, learning the strategy may not be easy, particularly given that most experts acquire their behaviors after years of practice. Second, the improvement in perceptual performance that comes with experience is probably due to multiple factors. Knowledge gained and feedback received are known to be critical factors in developing visual expertise 12, 13, 14, 15. Therefore, it is not evident that learners' perceptual performance will improve simply by using the search strategies of experts. However, some perceptual tasks, such as finding lung nodules on chest CT scans, do not depend on a large knowledge base, and therefore teaching a search strategy may improve detection. Experimental studies may be beneficial to determine if search patterns can be taught to junior observers, and if this can improve perceptual performance.

The aim of this research study is twofold: (1) to investigate if drilling and scanning search strategies can be taught to junior radiology trainees, and (2) to compare the effect of teaching each search strategy on trainee's perception accuracy of lung nodule detection. We hypothesized that junior radiology trainees could adopt a new search strategy after instruction and that the use of a drilling strategy would improve the trainees' perceptual performance compared to a scanning strategy.

Section snippets

Design

An experimental study was conducted to compare the effect of two teaching methods on perceptual performance. A randomized crossover design was chosen to adjust for individual variation in performance, differences in search strategies prior to any search strategy instruction, and possible differences in search behavior due to the sequence of the search strategy instructions. The design is illustrated in Figure 1.

Study Population and Procedure

Over a 3-month period, 19 (70%) first and second-year radiology trainees of a US

Participants

After randomization, 9 participants were assigned to group A and 10 to group B, of which 4 in each group were second-year residents.

Test Performance

Reliabilities of the tests were acceptable: Cronbach's α were 0.74, 0.82, and 0.84 for Tests 1, 2, and 3, respectively. Average P values were 0.49, 0.39 and 0.48, respectively.

Scroll Behavior

Median number of runs and scrolling time for each search strategy are provided in Table 2.

There was no significant effect of study group or year of residency on the number of runs or time.

Discussion

Search strategy instruction had a significant effect on both scroll behavior and perceptual performance. The scanning instruction decreased the number of long scroll movements and scrolling time. The drilling instruction did not alter scroll behavior significantly. However, the majority of participants reported already using some kind of drilling strategy at their free search. Perceptual performance following drilling search instructions outperformed performance following scanning search

Conclusion

Search strategy instruction can influence scroll behavior and perceptual performance of junior radiology residents completing a lung nodule detection task. In junior trainees, a drilling strategy yields a better perceptual performance than a scanning strategy. Teaching a scanning strategy further decreases the perceptual performance of junior radiology residents below their baseline performance. Teaching a drilling strategy for detecting lung nodules in chest CT scans is therefore preferable.

References (18)

There are more references available in the full text version of this article.

Cited by (7)

  • How to Read an Abdominal CT: Insights from the Visual and Cognitive Sciences Translated for Clinical Practice

    2022, Current Problems in Diagnostic Radiology
    Citation Excerpt :

    Even with the advent of AI, you will still need to be able to search with your own prepared and informed intelligence. Acquiring perceptual (visual) fluency typically requires years of training and experience, but there is evidence that the acquisition of this skill can be accelerated through nonverbal, implicit learning processes, such as image immersion.41,55 Acquisition can be accelerated, for instance, through trainings on 60 images viewed 3 times each day for 4 days, and 1280 images for 52 minutes.7

  • A Review of Perceptual Expertise in Radiology-How it develops, How we can test it, and Why humans still matter in the era of Artificial Intelligence

    2020, Academic Radiology
    Citation Excerpt :

    Eckstein et al. found higher detectability for calcifications in single slice images and relatively improved detection of masses in volumetric imaging (39). Using saliency maps, Wen et al. demonstrated that the great majority of radiologists (42) use dynamic (motion based) information when interpreting cross-sectional imaging (43). Both Wen and Eckstein studies lend support to the notion that observer performance in 2D search might not generalize to volumetric search tasks (39,43), as the 2 types of search entail different perceptual tasks.

  • Improving Novice Radiology Trainees’ Perception Using Fine Art

    2017, Journal of the American College of Radiology
    Citation Excerpt :

    To illustrate this, recent studies have shown that radiology-specific perception training can improve the detection of lung nodules. Van der Gijp et al [9] taught two groups of residents different perception strategies for detecting lung nodules. They found that residents were more likely to correctly identify lung nodules in a more efficient manner with one of the perception methods.

View all citing articles on Scopus
View full text