COMPUTER-BASED (CBT) VS. PAPER-BASED (PBT) TESTING: MODE EFFECT, RELATIONSHIP BETWEEN COMPUTER FAMILIARITY, ATTITUDES, AVERSION AND MODE PREFERENCE WITH CBT TEST SCORES IN AN ASIAN PRIVATE EFL CONTEXT

• Authors: Khoshsima Hooshang , Toroujeni Seyyed Morteza Hashemi , Thompson Nathan , Ebrahimi Mohammad Reza
• Languages of publication: EN

Abstracts

EN

The current study was conducted to investigate whether test scores of Iranian English as Foreign Language (EFL) learners were equivalent across CBT and PBT modes, with 58 intermediate learners studying at a private language academy located in Behshahr city in northern Iran. Moreover, test takers’ computer familiarity, attitudes, aversion, and testing mode preference were regarded as the potential issues to influence CBT test scores. Data were collected using CBT and PBT versions of Nelson Proficiency Multiple-Choice Tests and Computer Aversion, Attitudes, and Familiarity Index (CAAFI) questionnaire as well as a simple testing administration mode preference question. The participants produced similar scores across modes, although they insignificantly outperformed on the CBT version. Additionally, analysis of the overall scores on the CAAFI and mode preference question obtained from CBT testing session indicated no statistically significant correlation between computer familiarity, attitude, aversion, and mode preference variables and test takers’ CBT scores. The qualitative findings of this study obtained by semi-structured interview revealed that most of the participants showed high preference and more advantages for CBT over PBT to rationalize why they preferred this mode of testing.

Keywords

EN

Computer-Based Testing; testing administration mode; computer familiarity; attitudes and aversion; testing administration mode preference

• Publisher: IATEFL Poland Computer Special Interest Group & University of Nicosia
• Journal: Teaching English with Technology
• Year: 2019
• Volume: 19
• Issue: 1
• Pages: 86-101

Contributors

author

Khoshsima Hooshang

• Chabahar Maritime University

author

Toroujeni Seyyed Morteza Hashemi

• Chabahar Maritime University

author

Thompson Nathan

• Assessment Systems Corporation

author

Ebrahimi Mohammad Reza

• Gonabad University

References

• Alakyleh, A. S. (2018). Evaluating the comparability of (PPT) (CBT) by implementing the compulsory Islamic Culture Course Test in the University of Jordan. International Journal of Assessment Tools in Education, 5(1), 176-186. DOI: 10.21449/ijate.370494.
• Al-Amri, S. (2009). Computer-Based Testing vs. Paper-Based Testing: Establishing the Comparability of Reading Tests through the Revolution of a New Comparability Model in a Saudi EFL Context. Unpublished Doctor of Philosophy in Linguistics thesis. Colchester: University of Essex.
• American Educational Research Association, (2014). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association.
• Balogun, A. G., & Olanrewaju, A. S. (2016). Role of computer self-efficacy and gender in computer-based test anxiety among undergraduates in Nigeria. Psychological Thought, 9(1), 58-66. https://doi.org/10.5964/psyct.v9i1.160
• Boevé, A. J., Meijer, R. R, Albers, C. J, Beetsma, Y., & Bosker, R. J. (2015). Introducing computer-based testing in high-stakes exams in higher education: results of a field experiment. PLOS ONE, 10(12) e0143616. DOI:10.1371/journal.pone.0143616.
• Chen, G., Cheng, W., Chang, T.-W., Zheng, X., & Huang, R. (2014). A comparison of reading comprehension across the paper, computer screens, and tablets: Does tablet familiarity matter? Journal of Computers in Education, 1(2-3), 213-225.
• Corlett-Rivera, K., & Hackman, T. (2014). E-book usage and attitudes in the humanities, social sciences, and education. Portal: Libraries and the Academy, 14(2), 255-286.
• Dammas, A. H. (2016). Investigate students’ attitudes toward the Computer Based Test (CBT) at chemistry course. Archives of Business Research, 4(6), 58-71.
• Daniels, L. M., & Gierl, M. J. (2017). The impact of immediate test score reporting on university students’ achievement emotions in the context of computer-based multiple-choice exams. Learning, and Instruction, 52, 27-35. http://dx.doi.org/10.1016/j.learninstruc.2017.04.001
• Fowler, W. A., & Coe, N. (1976). Nelson English Language Tests. Ontario: Thomas Nelson and Sons Ltd.
• García Laborda, J., & Alcalde Penalver, E. (2018). Constraining issues in face-to-face and Internet-based language testing. Journal for Educators, Teachers, and Trainers, 9(2), 47-56.
• The International Test Commission (2006) International Guidelines on Computer-Based and Internet-Delivered Testing, International Journal of Testing, 6(2), 143-171, DOI: 10.1207/s15327574ijt0602_4 http://dx.doi.org/10.1207/s15327574ijt0602_4.
• Jeong, H. (2014). A comparative study of scores on computer-based tests and paper-based tests. Behavior & Information Technology, 33(4), 410-422.
• Khoshsima, H., & Hashemi Toroujeni, S. M. (2017a). Transitioning to an alternative assessment: Computer-Based Testing and key factors related to testing mode. European Journal of English Language Teaching, 2(1), 54-74. http://dx.doi.org/10.5281/zenodo.268576.
• Khoshsima, H., & Hashemi Toroujeni, S. M. (2017b). Comparability of Computer-Based Testing and Paper-Based Testing: Testing mode effect, testing mode order, computer attitudes and testing mode preference. International Journal of Computer (IJC), 24(1), 80-99. http://ijcjournal.org/index.php/InternationalJournalOfComputer/article/view/825/4188.
• Khoshsima, H., & Hashemi Toroujeni, S.M. (2017h). Computer-Based Testing: Score Equivalence and Testing Administration Mode Preference in a Comparative Evaluation Study. International Journal of Emerging Technologies in Learning (iJET), 12(10). https://doi.org/10.3991/ijet.v12i10.6875.
• Kirsch, I., Jamieson, J., Taylor, C., & Eignor, D. (1998). Computer Familiarity among TOEFL Test Takers (TOEFL Research Report 59). Princeton, NJ: Educational Testing Service.
• Mangen, A., Bente, R. W., & Kolbjørn, B. (2013). Reading linear texts on paper versus computer screen: Effects on reading comprehension. International Journal of Educational Research, 58, 61-68.
• Mastuti, E., & Handoyo, S. (2017). Effects of individual differences on the performance in computer-based test (CBT). Advances in Social Science, Education, and Humanities Research: Proceedings of the 3rd ASEAN Conference on Psychology, Counselling, and Humanities (ACPCH, 2017). Malang: University of Muhammadiyah. http://dx.doi.org/10.2991/acpch-17.2018.44.
• McDonald, A. S. (2002). The impact of individual differences on the equivalence of computer-based and paper-and-pencil educational assessments. Computers & Education, 39(3), 299-312. https://doi.org/10.1016/S0360-1315(02)00032-5.
• Mizrachi, D. (2015). Undergraduates’ academic reading format preferences and behaviors. The Journal of Academic Librarianship, 41(3), 301-311. http://dx.doi.org/10.1016/j.acalib.2015.03.009.
• Phillips, D. (2001). Longman Complete Course for the TOEFL Test: Preparation for the Computer and Paper Tests. White Plains, NY: Longman.
• Sangmeister, J. (2017). Commercial competence: Comparing test results of paper and pencil versus computer-based assessments. Empirical Research in Vocational Education and Training, 9(3). DOI: 10.1186/s40461-017-0047-2.
• Schulenberg, S. E. (2002). The development of computer aversion, attitudes, and familiarity index (CAAFI). Dissertation Abstracts International, 62(12), 5978B, (UMI No. 3037841).
• Schulenberg, S. E., & Melton, A. M. A. (2008). The computer aversion, attitudes, and familiarity index (CAAFI): A validity study. Computers in Human Behavior, 24(6), 2620-2638. https://doi.org/10.1016/j.chb.2008.03.002.
• Schulenberg, S. E., Yutrzenka, B. A., & Gohm, C. L. (2006). The computer aversion, attitudes, and familiarity index (CAAFI): A measure for the study of computer-related constructs. Journal of Educational Computing Research, 34(2), 129-146. https://doi.org/10.2190/45B4-GMH7-GEQB-T1H1.
• Washburn, S., Herman, J., & Stewart, R. (2017). Evaluation of performance and perceptions of electronic vs. multiple-choice paper exams. Advances in Physiology Education, 41(4), 548-555. DOI: 10.1152/advan.00138.2016.
• Yurdabakan, I., & Uzunkavak, C. (2012). Primary school students’ attitudes toward computer-based testing and assessment in turkey. Turkish Online Journal of Distance Education, 13(3), 177-188.