Kappa coefficient

The Kappa coefficient can be thought of as a way of quantifying the amount of disagreement between raters. It takes into account both chance agreement (agreement that would be expected by chance alone) and real agreement (agreement that is not due to chance alone). The coefficient is equal to one if the raters are in agreement, and less than or equal to zero if there is no agreement.