A comparative study of think-aloud and critical decision knowledge elicitation methods

The knowledge elicitation phase of expert system development continues to be identified as a major hurdle in construction of comprehensive, functional expert systems (Berry, 1987; Hoffman, 1987). One of the major difficulties is that expert knowledge is multifaceted. The expert has both explicit, objective knowledge and knowledge that is more implicitly understood. Such tacit knowledge is among the most difficult for experts to articulate on their own. Reliance on standard interview methods or unstructured think-aloud protocols may inadvertently bias the knowledge engineer into focusing on those aspects of the task that can be reasonable well represented within if/then, rule-based systems. The systems that result may lack critical components of expert knowledge. It is therefore essential that knowledge elicitation methods be sensitive to the contributions made by tacit knowledge and perceptual learning (Berry, 1987; Collins, Green, & Draper, 1985; Klein, Calderwood, & MacGregor, in press).