On Deriving Adaptation Rule Confidence from the Rule Generation Process

Abstract

Previous case-based reasoning research makes a compelling case for the importance of CBR systems determining the system’s confidence in its conclusions, and has developed useful analyses of how characteristics of individual cases and the case base as a whole influence confidence. This paper argues that in systems which perform case adaptation, an important additional indicator for solution confidence is confidence in the adaptations performed. Assessing confidence of adaptation rules may be particularly important when knowledge-light methods are applied to generate adaptations automatically from the case base, giving the opportunity to improve performance by astute rule selection. The paper proposes a new method for calculating rule confidence for automatically-generated adaptation rules for regression tasks, when the rules are generated by the common “difference heuristic” method of comparing pairs of cases in a case base, and a method for confidence-influenced selection of cases to adapt. The method is evaluated in four domains, showing performance gains over baseline methods and case based regression without using confidence knowledge.