What Is Attribute Agreement Analysis?

Definition

Attribute Agreement Analysis is a measurement system study for pass/fail or graded inspection decisions. It checks whether appraisers agree with themselves, with each other and with a known standard.

Attribute Agreement Analysis in Simple Terms

A Gauge R&R study works for measured values such as a diameter. Many checks are not measured: a visual inspector decides good or bad, or a go/no-go gauge fits or does not. Attribute agreement analysis is the equivalent study for those decisions and is part of MSA as described in the AIAG MSA manual.

How It Works

  1. Choose about 30 to 50 parts, including good, bad and borderline ones.
  2. Have an expert set the reference decision for each part.
  3. Two or three appraisers inspect each part at least twice, in random order, without seeing earlier results.
  4. Calculate agreement within each appraiser, between appraisers and against the reference, usually with percent agreement and the kappa statistic.
Kappa valueUsual interpretation (AIAG)
0.75 or moreGood to excellent agreement
0.40 to 0.75Marginal
Below 0.40Poor agreement

Example

A painted bumper line has three inspectors judging surface defects. In a 40-part study, one inspector agrees with the reference only 70 percent of the time and passes several parts with dirt inclusions. The team adds boundary samples and a lighting standard, retrains, and the repeat study shows kappa above 0.9 for all three.

Common Mistakes

  • Using only obvious good and bad parts, which makes agreement look better than it is.
  • Letting appraisers know which part they are looking at.
  • Not checking against a reference, so a team can agree with each other and still be wrong.

The OperaHive calibration management software keeps Gauge R&R and measurement system analysis records with the gauge history.

FAQ

Frequently Asked Questions

Still unsure? Ask our team.

When should I use attribute agreement analysis instead of Gauge R&R?

Use it when the result is a category, such as pass/fail or a defect grade. Use Gauge R&R when the result is a measured number.

How many parts do I need?

AIAG suggests around 50 parts for an attribute study, though many plants use 30 or more with a good mix of borderline parts.

What is attribute agreement analysis?

Attribute agreement analysis is a study that checks how consistently inspectors classify parts, such as pass or fail, and how well they agree with a known reference answer.

How do you perform an attribute agreement analysis?

Select parts with known status, including borderline ones, have two or three appraisers judge each part two or three times in random order, then compare results within, between and against the standard.

What is a good result in attribute agreement analysis?

A kappa of 0.75 or higher is generally seen as good agreement and above 0.9 as excellent. Below 0.4 shows poor agreement that needs action.

What is kappa in attribute agreement analysis?

Kappa is a statistic that measures agreement after removing the agreement expected by chance. A value of 1 means perfect agreement and 0 means no better than chance.

What is the difference between within and between appraiser agreement?

Within appraiser agreement shows whether one inspector gives the same answer each time. Between appraiser agreement shows whether different inspectors give the same answer.

What is an attribute agreement analysis example?

Three inspectors each judge 50 painted panels twice. If one inspector often changes their answer on the same panel, retraining or clearer defect samples are needed.

Is attribute agreement analysis required by IATF 16949?

IATF 16949 requires measurement system analysis for systems in the control plan, which includes attribute inspection. Attribute agreement analysis is the usual method.

How do you improve attribute agreement results?

Use clear boundary samples, improve lighting and inspection aids, write precise defect definitions and retrain the inspectors who disagree most.

Put Attribute Agreement Analysis Into Practice

Book a demo and see how OperaHive handles this on your own parts, processes and documents.