Cohen's Kappa for more than two categories

Question

I have data set of teacher's evaluation rated by 4 different raters. The teacher's are evaluated for 13 different categories for example (Interaction with students, lesson delivery etc). All of the 4 raters are rating teachers in 13 categories from 1 to 5.

I want to find the agreement level between observers using cohen's kappa. I know how to compute kappa for one category only but I am confused how can we do that for different categories? Do I have to compute it for each category separately? or is there any other method?

For computing kappa I am using STATA. Here's an example of how my data looks like. Can't share the original data.

Carl · Answer

From kappa - Stata "kap (second syntax) and kappa calculate the kappa-statistic measure when there are two or more (nonunique) raters and two outcomes, more than two outcomes when the number of raters is fixed, and more than two outcomes when the number of raters varies. kap (second syntax) and kappa produce the same results; they merely differ in how they expect the data to be organized."

So, yes you can. Also, this other answer may help you with interpretation.

So, yes you can. Also, this other answer may help  you with interpretation.

Cohen's Kappa for more than two categories

One Answer

Add your own answers!

Ask a Question