Možnosti, problémy a odporúčania pri meraní zhody medzi posudzovateľmi - deskriptívny prístup

• Authors: Lucia Kočišová

Title variants

EN

Options, problems and guidelines for measuring interrater agreement - a descriptive approach

• Languages of publication: SK

Abstracts

EN

Interrater agreement is one way to establish reliability (and also validity) in social science research. The traditionally preferred method of measuring interrater agreement is the descriptive approach owing to its simplicity. This approach is also associated with a number of different agreement indices, which makes it difficult to select the right index. This article summarises theoretical definition on the prevailing approach used to measure interrater agreement (in both quantitative and qualitative research). From a practical point of view, the article focuses on the possibilities of measuring agreement by using percent agreement, the kappa coefficient, and the AC1 coefficient. A more detailed description of the indices explains how to define, calculate, and interpret them and the problems associated with their use. The indices are then discussed in comparison. Although underestimated and criticised, percent agreement may be a good indicator of interrater agreement. Several paradoxes accompany the use of the kappa coefficient, which is only possible under certain conditions. The appropriate alternative to it is the AC1 coefficient. The article concludes with a summary of recommendations for improving the quantification of interrater agreement.

Keywords

EN

interrater agreement; agreement index; percent agreement; kappa coefficient; AC1 coefficient

Discipline

• SOCIOLOGY: SOCIOLOGY

• Publisher: Institute of Sociology, Czech Academy of Sciences
• Journal: Sociologický časopis. Czech Sociological Review
• Year: 2025
• Volume: 61
• Issue: 3
• Pages: 277-300
• Document type: ARTICLE

Contributors

author

Lucia Kočišová

• Sociologický časopis, redakce, Sociologický ústav AV ČR, v.v.i., Jilská 1, 110 00 Praha 1, Czech Republic