Comparison of Mantel–Haenszel test with IRT procedures for DIF detection and effect size estimation for dichotomous items

• Authors: Bartosz Kondratek , Magdalena Grudniewska
• Languages of publication: EN

Abstracts

EN

The article compares two methods used to detect differential item functioning (DIF) of dichotomously scored items: a nonparametric solution based on the Mantel–Haenszel procedure (MH) and a parametric IRT approach with a likelihood ratio test. A Monte Carlo experiment was performed in order to evaluate performance of both statistics in various conditions of DIF uniformity. Results confirmed the theoretical prediction that the MH test has greater statistical power in detecting uniform DIF than the likelihood ratio test and less power than the LR test in cases of non-uniform DIF. Apart of examining statistical power of the test, specific measures of DIF effect size were compared: MH D–DIF and three measures of P–DIF expressed on the item easiness scale.

Keywords

EN

differential item functioning; Mantel–Haenszel test; item response theory

• Publisher: Instytut Badań Edukacyjnych
• Journal: Edukacja
• Year: 2014
• Issue: 5(130)
• Pages: 92–111

Dates

published

2014-02-06

Contributors

author

Bartosz Kondratek

• Educational Research Institute

author

Magdalena Grudniewska

• Instytut Badań Edukacyjnych

References

• Agresti, A. (2002). Categorical data analysis. New Jersey: John Wiley & Sons.
• Dorans, N. J. and Holland, P. W. (1993). DIF detection and description: Mantel–Haenszel and standardization. In P. W. Holland and H. Wainer (eds.), Differential item functioning (pp. 35–66). Hillsdale, NJ: Lawrence Earlbaum.
• Glas, C. A. (2010). Preliminary manual of the software program Multidimensional Item Response Theory (MIRT). Enschede: University of Twente.
• Kondratek, B. (2012). Bias of IRT observed score equating under NEAT design. Poster presented at the conference Modern Modelling Methods, Storrs, Connecticut.
• Lord, F. M. (1983). Statistical bias in maximum likelihood estimators of item parameters. Psychometrika, 48(3), 425–435.
• Lord, F. M. and Novick, M. R. (1968). Statistical theories of mental test scores. Reading, Massachusetts: Addison–Wesley.
• Mantel, N. and Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22(4), 719–748
• Monahan, P. O., McHorney, C. A., Stump, T. E. and Perkins, A. J. (2007). Odds ratio, delta, ETS classification, and standardization measures of DIF magnitude for binary logistic regression. Journal of Educational and Behavioral Statistics, 32(1), 92–109.
• Penfield, R. D. and Camilli, G. (2007). Differential item functioning and item bias. In C. R. Rao and S. Sinharay (eds.), Handbook of statistics, Vol. 26. Psychometrics (pp. 125–167). New York, NY: Elsevier.
• Radhakrishna, S. (1965). Combination of results from several 2 × 2 contingency tables. Biometrics, 21(1), 86–98.
• Swaminathan, H. and Rogers J. H. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27(4), 361–370.
• Thissen, D., Steinberg, L. and Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland and H. Wainer (eds.), Differential Item Functioning (pp. 67–113). Hillsdale, NJ: Lawrence Earlbaum.
• Wainer, H. (1993). Model-based standardized measurement of an items differential impact. In P. W. Holland and H. Wainer (eds.), Differential Item Functioning (pp. 255–276). Hillsdale, NJ: Lawrence Earlbaum.
• Woolf, B. (1955). On estimating the relation between blood group and disease. Annals of Human Genetics, 19(4), 251–253.
• Zieky, M. (1993). Practical questions in the use of DIF statistics in test development. In P. W. Holland and H. Wainer (eds.), Differential Item Functioning (pp. 337–348). Hillsdale, NJ: Lawrence Earlbaum.
• Zieky, M. (2003). A DIF primer. Princeton, NJ: Educational Testing Service.

Notes

EN

http://www.edukacja.ibe.edu.pl/images/numery/2014/5-6-kondratek-grudniewska-comparison-of-mh-and-irt.pdf

Identifiers

ISSN

0239-6858