AGREEMENT AND RELIABILITY ASSESSMENTS FOR PERFORMANCE OF SENSORY DESCRIPTIVE PANEL

ABSTRACT In the present paper, two indices, agreement coefficient and reliability coefficient, are introduced and proposed to measure quality of ratings data given by a sensory descriptive panel. Agreement coefficient describes variation of ratings for a single target (product), whereas reliability coefficient reflects discrimination ability of a descriptive panel for a sample of products. The two indices, which profoundly describe inherent characteristics of ratings data, are based on classical reliability theory and recent advances in psychological measurement. In the paper, estimates and tests for the two indices are provided. Confidence intervals for the indices are derived. A space with the two indices as dimensions, which is referred as rating quality space, gives visual presentation of performances of a descriptive panel or individual panelists. Numerical examples illustrate the use and interpretation of the indices.