Do we know what we are asking? Individual and group cognitive interviews

• Authors: Miroslav Popper , Magda Petrjánošová
• Languages of publication: EN

Abstracts

EN

The paper deals with cognitive interview, a method for pre-testing survey questions that is used in pilot testing to develop new measures and/or adapt ones in foreign languages. The aim is to explore the usefulness of the method by looking at two questionnaires measuring anti-Roma prejudice. The first, the Stereotype Content Model (SCM), contains questions that are dominantly used to test two dimensions of social perceptions of various groups: warmth and competence. The second, Interventions for Reducing Prejudice against Stigmatized Minorities (INTERMIN) consists of the items most frequently used in contact research to measure attitudes, social distance, anxiety, trust and behavioural intentions towards outgroups. Two rounds of cognitive interviews were held on both questionnaires to verbally evaluate participants’ understanding and/or interpretation of the draft questions. The first round was attended by university students, while the second round (with improved versions of the questionnaires) was done with high school students, as they are the target group for planned interventions based on the contact paradigm. The paper explains the problems/difficulties the participants had answering some of the questions and our attempts at improving the questionnaires. The problems can be grouped around six issues: The first two deal with the strategies participants used to answer our questions – whom exactly did they have in mind when answering the questionnaires and whose viewpoint did they represent in their answers. The next four problems are around nuances in the formulations of our questions and generally have to do with how the participants interpreted our questions – they concern assumptions that distinct items were logically interconnected, the period of time and locality referred to in our questions, translation and transferability of meanings from one language to another and double negation.

Keywords

EN

cognitive interview; questionnaire testing; SCM; INTERMIN; measuring stereotypes and prejudice

• Publisher: Sciendo
• Journal: Human Affairs
• Year: 2015
• Volume: 26
• Issue: 3
• Pages: 253-270

Dates

published

2016-09-01

online

2016-07-25

Contributors

author

Miroslav Popper

• Institute of Applied Psychology, Faculty of Social and Economic Sciences, Comenius University in Bratislava, Mlynské luhy 4, 821 05 Bratislava, Slovakia

author

Magda Petrjánošová

• Institute for Research in Social Communication, Slovak Academy of Sciences, Dúbravská cesta 9,841 04 Bratislava 4, Slovakia

References

• Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71(2), 287-311.[WoS]
• Blair, J., & Brick, P. D. (2010). Methods for the analysis of cognitive interviews. Section on survey research methods – JSM 2010.
• Bogardus, E. S. (1925). Measuring social distances. Journal of Applied Sociology, 9, 299-308.
• Bradburn, N. M. (2006). Understanding the question-answer process. Statistics Canada, 30(1), 5-15.
• Cuddy, A. J. C, Fiske, S. T., & Glick, P. (2007). The BIAS map: Behaviors from intergroup affect and stereotypes. Journal of Personality and Social Psychology, 92(4), 631-648.
• Cuddy, A. J. C, Fiske, S. T., & Glick, P. (2008). Warmth and competence as universal dimensions of social perception: The stereotype content model and the BIAS Map. Advances in Experimental Social Psychology, 40, 61-149.[WoS]
• Cuddy, A. J. C., Fiske, S. T., Kwan, V. S. Y., Glick, P., Demoulin, S., Leyens, J.P., ..., Ziegler, R. (2009). Stereotype content model across cultures: Towards universal similarities and some differences. British Journal of Social Psychology, 48(1), 1-33.[WoS]
• Fiske, S. T. (1998). Stereotyping, prejudice, and discrimination. In D. T. Gilbert, S. T. Fiske, & G. Lindzey (Eds.), Handbook of social psychology (4th ed., Vol. 2, pp. 357-411). New York: McGraw -Hill.
• Fiske, S. T., Cuddy, A. J. C., Glick, P., & Xu, J. (2002). A model of (often mixed) stereotype content: Competence and warmth respectively follow from perceived status and competition. Journal of Personality and Social Psychology, 82(6), 878-902.
• Fiske, S. T., & North, M. S. (2014). Measures of stereotyping and prejudice: Barometers of bias. In G. J. Boyle, D. H. Saklofske, & G. Matthews (Eds.), Measures of personality and social psychological constructs (pp. 684-716). London: Academic Press.
• Furnham, A., & Steele, H. (1993). Measuring locus of control: A critique of general, children’s, health- and work-related locus of control questionnaires. British Journal of Psychology, 84(4), 443-479.
• Gray, D., & Durrheim, K. (2013). Collective rights and personal freedoms: A discursive analysis of participant accounts of authoritarianism. Political Psychology, 34(4), 631-648.[WoS]
• Gutek, B. A, Murphy, R. O., & Douma, B. (2004). A review and critique of the sexual experiences questionnaire (SEQ). Law and Human Behavior, 28(4), 457-482.
• Helfrich, H. (1986). On linguistic variables influencing the understanding of questionnaire items. In A. Angleitner, & J. S. Wiggins (Eds.), Personality Assessment via Questionnaires (pp. 178-188). Berlin – Heidelberg: Springer.
• Kanovský, M. (2016). Robustné štatistické metódy v sociálnych vedách [Robust statistical methods in social sciences]. Bratislava: Slovenská asociácia sociálnej antropológie.
• Lášticová, B. (2006). Identification with large scale social categories: A social psychology perspective. Sociológia – Slovak Sociological Review, 38(6), 546-561.
• Lášticová, B., & Findor, A. (2016). Developing explicit measures of stereotypes and anti-Roma prejudice in Slovakia: Conceptual and methodological challenges. Human Affairs, 26(3), 233-252.
• Mann, A., & Ruppeldtová, S. (2009). Pohodlné stereotypy o Rómoch [Indolent Roma stereotypes]. Britské listy, 23.02.2009
• Mušinka, A., & Matlovičová, K. (2015). Atlas rómskych komunít na Slovensku 2013 ako pramenná databáza pre analýzu situácie Rómov na Slovensku a jeho potenciál pre ďalšie výskumy a analýzy. In T. Podolinská, & T. Hrustič (Eds.), Čierno-biele svety: Rómovia v majoritnej spoločnosti na Slovensku [Atlas of Roma Communities in Slovakia 2013 as a source database for analysis of Roma situation in Slovakia and its potential for further research and analysis]. Bratislava: VEDA, Ústav etnológie Slovenskej akadémie vied.
• Presser, S., Couper, M. P., Lessler, J. T., Martin, E., Rothgeb, J. M., & Singer, E. (2004). Methods for testing and evaluating survey questions. Public Opinion Quarterly, 68(1), 109-130.
• Ryan, K., Gannon-Slater, N., & Culberstone, M. J. (2012). Improving survey methods with cognitive interviews in small- and medium-scale evaluations. American Journal of Evaluation, 33(3), 414-430.[WoS]
• Tam, T., Hewstone, M., Kenworthy, J., & Cairns, E. (2009). Intergroup trust in Northern Ireland. Personality and Social Psychology Bulletin, 35(1), 45-59.[WoS]
• Tourengeau, R., & Bradburn, N. M. (2010). The psychology of survey response. In J. D. Wright & P. V. Marsden (Eds.), Handbook of survey research (pp. 315-346).West Yorkshire: Emerald Group.
• Turner, R. N., West, K., & Christie, Z. (2013). Out-group trust, intergroup anxiety, and out-group attitude as mediators of the effect of imagined intergroup contact on intergroup behavioural tendencies. Journal of Applied Social Psychology, 43(S2), 196-205.[WoS]
• Willis, G. B. (1999). Cognitive interviewing. A “how to” guide, from the short course “Reducing Survey Error through Research on the Cognitive and Decision Processes in Surveys”. Paper presented at the meeting of the American Statistical Association.
• Willis, G. B., & Artino, A. R. (2013). What do our respondents think we’re asking? Using cognitive interviewing to improve medical education surveys. Journal of Graduate Medical Education, September, 353-356.

Identifiers

DOI

10.1515/humaff-2016-0023