Exploiting Eye Movements for Evaluating Reviewer's Performance in Software Review

Hidetake UWANO
Masahide NAKAMURA
Akito MONDEN
Ken-ichi MATSUMOTO

Publication
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E90-A    No.10    pp.2290-2300
Publication Date: 2007/10/01
Online ISSN: 1745-1337
DOI: 10.1093/ietfec/e90-a.10.2290
Print ISSN: 0916-8508
Type of Manuscript: PAPER
Category: Reliability, Maintainability and Safety Analysis
Keyword: 
software review,  human factor,  eye movement,  experimental evaluation,  system development,  

Full Text: PDF(705.2KB)>>
Buy this Article


Summary: 
This paper proposes to use eye movements to characterize the performance of individuals in reviewing software documents. We design and implement a system called DRESREM, which measures and records eye movements of document reviewers. Based on the eye movements captured by eye tracking device, the system computes the line number of the document that the reviewer is currently looking at. The system can also record and play back how the eyes moved during the review process. To evaluate the effectiveness of the system we conducted an experiment to analyze 30 processes of source code review (6 programs, 5 subjects) using the system. As a result, we have identified a particular pattern, called scan, in the subject's eye movements. Quantitative analysis showed that reviewers who did not spend enough time on the scan took more time to find defects on average.

open access publishing via