Attribute R&R (Repeatability & Reproducibility) is used to evaluate your ability to consistently and precisely measure subjective characteristics such as taste or aroma.
Whenever you make a subjective measurement you have to be able to understand how much of the variability in your data comes from the product you’re measuring and how much comes from the people and the methods you use to make the measurement. And you want to understand and minimize the amount of variability that comes from that measurement system.
This is equally true whether you’re doing objective tests like chemical analysis of the amount of flavor compounds in a product, or subjective tests like Sensory evaluation of the flavour or aroma of the product.
If the variability between the appraisers is small we know that any conclusions we reach when we compare one product to another are reliable. However, if the variability between our appraisers increases, we may have to conclude that we are not able to reliably distinguish between different products that are reasonably similar to each other. For instance, we may not be able to statistically distinguish between one of our products and a competitor’s product. So we may make business decisions based on unreliable data.
That’s why it’s important to understand how much variability there is in our Sensory evaluations.
We can also use the information to take remedial action to decrease the level of variability. The results of this type of analysis can tell us whether all our appraisers are well trained or they need to have “calibration sessions” to become more consistent, or if they don’t have the same understanding of the “vocabulary” and so they aren’t describing the Sensory characteristics of products the same way. Maybe most of our appraisers are well trained and consistent with each other but one or two of them need to be retrained to prevent them from introducing too much variability into our measurements.
The Attribute R&R program enables us to determine how much of the variability in our Sensory measurements is due to the people analyzing the samples.
The Attribute R&R program automates the most time-consuming and error-prone tasks:
- Fully automate the tedious tasks of giving random codes to the samples and randomizing the order of presentation of the samples to the panelists
- Generate questionnaires for each panelist in the same order as the samples will be presented
- Capture the data directly from the questionnaires to avoid transcription error
- “De-randomize” the data to ensure that each value is correctly assigned to the appropriate data element in the statistical table
The Attribute R&R program determines the level of variability between appraisers and also helps identify how to reduce the level of variability.
One of the benefits of using a statistical approach is that we take subjectivity out of the decision making. We can quantify the true level of variability and compare it with the level of precision we need to be able to make good business decisions rather than just having a feeling or a belief that our variability is either acceptable or too large.
Further information on identifying and dealing with outlying replicates here
Examples of Attribute R&R charts (Confidence Limits, Fleiss’ Kappa Values, and Kendall’s Correlation):
Precision Scientific Software 