Sex differences in guessing and item omission

• Authors: Karolina Świst , Paulina Skórska , Maciej Koniewski , Aleksandra Jasińska-Maciążek
• Languages of publication: EN

Abstracts

EN

Guessing and item omission may be regarded as risk-taking or risk-avoidance strategies – sex specific adaptations to testing situations. In this article, these phenomena were analysed by (a) percentage of omissions by sex, (b) negative binomial regression to asses sex differences in the number of omissions, (c) c-DIF analysis using IRT-LR test and (d) linear regression using item attributes, to assess whether the c-parameter is sex differentiated by the percentage of omits (controlling item difficulty). The data set analysed were from the 2012–2014 Polish lower-secondary schools final exams, comprising tests in maths, language, science and humanities. Contrary to the vast body of literature, boys omitted items slightly more frequently than girls. Possible explanations of this finding – specific to the Polish examination system – were provided. The hypothesis of a higher c-parameter for boys did not find strong support from this study. It was shown that the c-parameter should not only be interpreted as resulting from item non-omission. This supports the modern concept of the c-parameter as a consequence not only of random guessing, but also problem solving, creative guessing or cheating.

Keywords

EN

differences; guessing; item omission; c-DIF

• Publisher: Instytut Badań Edukacyjnych
• Journal: Edukacja
• Year: 2015
• Issue: 3(134)
• Pages: 48–62

Dates

published

2015-09-30

Contributors

author

Karolina Świst

• Educational Research Institute

author

Paulina Skórska

• Educational Research Institute

author

Maciej Koniewski

• Educational Research Institute

author

Aleksandra Jasińska-Maciążek

• Educational Research Institute

References

• Angoff, W. A. and Schrader, W. B. (1981). A study of alternative methods for equating rights scores to formula scores (ETS RR-81-8). Princeton: Educational Testing Service.
• Arnett, J. (1992). Reckless behavior in adolescence: a developmental perspective. Developmental Review, 12, 339–373.
• Ben-Shakhar, G. and Sinai, Y. (1991). Gender differences in multiple-choice tests: the role of differential guessing tendencies. Journal of Educational Measurement, 28(1), 23–25.
• Bielinski, J. and Davison, M. L. (2001). A sex difference by item difficulty interaction in multiple-choice mathematics items administered to national probability samples. Journal of Educational Measurement, 38(1), 51–77.
• Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord and M. R. Novick (eds.), Statistical theories of mental test scores (pp. 17–20). Reading: Addison-Wesley.
• Boughton, K. A. and Yamamoto, K. (2007). A hybrid model for test speededness. In M. von Davier and C. H. Carstensen (eds.), Multivariate and mixture distribution Rasch models (pp. 147–156). New York: Springer.
• Byrnes, J. P. (1998). The nature and development of decision-making: a self-regulation model. Hillsdale: Erlbaum.
• Byrnes, J. P., Miller, D. C. and Schafer, W. D. (1999). Gender differences in risk taking: a meta-analysis. Psychological Bulletin, 125(3), 367–383.
• Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.
• Coombs, C. H., and Lehner, P. E. (1981). Evaluation of two alternative models of a theory of risk: I. Are moments of distributions useful in assessing risk? Journal of Experimental Psychology: Human Perception and Performance, 7(1), 1110–1123.
• DeMars, C. E., Bashkov, B. M. and Socha, A. B. (2013). The role of gender in test-taking motivation under low-stakes conditions. Research & Practice in Assessment, 8(2), 69–82.
• Finch, W. H. and French, B. F. (2014). The impact of group pseudo-guessing parameter differences on the detection of uniform and nonuniform DIF. Psychological Test and Assessment Modeling, 56(1), 25–44.
• Hambleton, R. K., Swaminathan, H. and Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park: Sage.
• Han, K. T. (2012). Fixing the c parameter in the three-parameter logistic model. Practical Assessment, Research & Evaluation, 17(1), 1–24.
• Hilbe, J. M. (2011). Negative binomial regression (2nd ed.). Cambridge–New York: Cambridge University Press.
• Holland, P. W. (1990). The Dutch identity: a new tool for the study of item response models. Psychometrika, 55(1), 5–18.
• Kahneman, D., and Tversky, A. (1979). Prospect theory: an analysis of decision under risk. Econometrica, 47(2), 263–291.
• Kelling, G. W., Zirkes, R. and Myerowitz, D. (1976). Risk as value: a switch of set hypothesis. Psychological Reports, 38, 655–658.
• Kolen, M. (1981). Comparison of traditional and item response theory methods for equating tests. Journal of Educational Measurement, 18(1), 1–11.
• Long, J. S. (2001). Regression models for categorical dependent variables using Stata. College Station: Stata Press.
• Lopes, L. L. (1987). Between hope and fear: the psychology of risk. In L. Berkowitz (ed.), Advances in experimental social psychology (vol. 20, pp. 255–295). New York: Academic Press.
• Lord, F. M. (1974). Estimation of latent ability and item parameters when there are omitted responses. Psychometrika, 39(2), 247–264.
• Lord, F. M. (1975). Evaluation with artificial data of a procedure for estimating ability and item characteristic curve parameters. (Research Bulletin RB-75-33). Princeton: Educational Testing Service.
• Lord, F. M. (1980). Applications of item response theory to practical testing problems. London Routledge.
• Maris, G. and Bechger, T. (2009). On interpreting the model parameters for the three parameter logistic model. Measurement: Interdisciplinary Research and Perspectives, 7(2), 75–88.
• Meijer, R. R. (guest ed.). (1996a). Person fit research: theory and applications. Applied Measurement In Education [Special Issue], 9(1).
• Meijer, R. R. (1996b). Person-fit research: an introduction. Applied Measurement in Education, 9(1), 3–8.
• Pekkarinen, T. (2014). Gender differences in strategic behaviour under competitive pressure: evidence on omission patterns in university entrance examinations. IZA Discussion Paper No. 8018. Retrieved from http://ftp.iza.org/dp8018.pdf
• Penner, A. (2003). International gender X item difficulty interactions in mathematics and science achievement tests. Journal of Educational Psychology, 95(3), 650–655.
• Rao, C. R. (1973). Linear statistical inference and its applications (2nd ed). New York: John Wiley and Sons.
• San Martin, E. Pino, G. del and De Boeck, P. (2006). IRT models for ability-based guessing. Applied Psychological Measurement, 30(3), 183–203.
• Slakter, M. J. (1967). Risk taking on objective examinations. American Educational Research Journal, 4(1), 31–43.
• Slakter, M. J. (1968a). The effect of guessing strategy on objective test scores. Journal of Educational Measurment, 5(3), 217–221.
• Slakter, M. J. (1968b). The penalty for not guessing. Journal of Educational Measurement, 5(3), 141–144.
• Slakter, M. J., Koehler, R. A., Hampton, S. H. and Grennell, R. L. (1971). Sex, grade level, and risk taking on objective examinations. The Journal of Experimental Educational, 39(3), 65–68.
• Swineford, F. (1941). Analysis of a personality trait. Journal of Educational Psychology, 32(6), 438–444.
• Teresi, J. A., Ocepek-Welikson, K., Kleinman, M., Cook, K. F., Crane, P. K., Gibbons, L. E., Morales, L. S., Orlando-Edelen, M. and Cell, D. (2007). Evaluating measurement equivalence using the item response theory log-likelihood ratio (IRTLR) method to assess differential item functioning (DIF): applications (with illustrations) to measures of physical functioning ability and general distress. Quality of Life Research, 16(Supplement 1), 43–68.
• Thissen, D. (2001). IRTLRDIF v2.0b: software for the computation of the statistics involved in item response theory likelihood-ratio tests for differential item functioning. 2.0b ed. Chapel Hill: L.L. Thurstone Psychometric Laboratory, University of North Carolina.
• Thissen, D., Steinber, L. and Wainer, H. (1988). Use of item response theory in the study of group difference in trace lines. In H. Wainer and H. Braun (eds.), Test Validity (pp. 147–170). Hillsdale: Erlbaum.
• Von Schrader, S. and Ansley, T. (2006). Sex differences in the tendency to omit items on multiple-choice tests: 1980–2000. Applied Measurement in Education, 19(1), 41–65.
• Wald, A. (1943). Tests of statistical hypothesis concerning several parameters when the number of observations is large. Transactions of the American Mathematical Society, 54, 426–482.
• Wilson, M. and Daly, M. (1985), Competitiveness, risk taking and violence: the young male syndrome, Ethology and Sociology, 6(1), 59–73.
• Zhu, D. and Thompson, T. D. (1995). Gender and ethnic differences in tendencies to omit responses on multiple-choice tests using number-right scoring. (ERIC Document Reproduction Service No. ED 382 689). Retrieved from http://files.eric.ed.gov/fulltext/ED382689.pdf
• Ziller, R. C. (1957a). A measure of gambling response-set in objective tests. Psychometrika, 22(3), 289–292.
• Ziller, R. C. (1957b). Vocational choice and utility for risk. Journal of Counseling Psychology, 4(1), 61–64.
• Zuckerman, M. (1991). Psychobiology of personality. Cambridge: Cambridge University Press.

Notes

http://www.edukacja.ibe.edu.pl/images/numery/2015/3-3-swist-i-in-sex-differences.pdf

Identifiers

ISSN

0239-6858