Creating Overall Measurements of Unidimensional Constructs for Comparative Research: A Methodological Study

• Authors: Charalampi Anastasia , Tsouparopoulou Eva , Tsiganou Joanna , Michalopoulou Catherine
• Languages of publication: EN

Abstracts

EN

Standardization of measurement is a prerequisite for cross-national and/or overtime comparative analyses. However, there are instances in the literature where the validation of constructs resulted in producing scales or subscales defined differently from the proposed theoretical structure and across countries. In this paper, we propose an empirical methodology that provides standardized overall measurements of unidimensional constructs to be used in cross-national and overtime comparative research. Initially, the inclusion of items for further analyses is investigated at country level and overtime. The common items are to define the overall measurements and their structure is validated. Based on the Confirmatory factor analyses results, their psychometric properties are assessed. To demonstrate the implementation of the suggested methodology and facilitate practical applications, we use the human values measurements included in the European Social Survey questionnaire for Southern Europe, 2002-2018. Moreover, in order to show how these measurements may be used in further analyses, their association to subjective life satisfaction, happiness and general health are also presented.

Keywords

EN

comparative research; confirmatory factor analysis; validity; reliability; European Social Survey

• Publisher: Institute of Philosophy and Sociology, Polish Academy of Sciences
• Journal: ASK. Research and Methods
• Year: 2021
• Volume: 30
• Issue: 1
• Pages: 25-56

Contributors

author

Charalampi Anastasia

author

Tsouparopoulou Eva

author

Tsiganou Joanna

author

Michalopoulou Catherine

References

• Bartholomew, David J., Fiona Steele, Irini Moustaki, and Jane Galbraith. 2008. Analysis of multivariate social science data. Chapman & Hall/CRC. https://doi.org/10.1201/b15114
• Bollen, Kenneth A. 1989. Structural equations with latent variables. Wiley. https://doi.org/10.1002/9781118619179
• Brown, Timothy A. 2015. Confirmatory factor analysis for applied research (2nd edition). The Guilford Press. Carey, Siobhan (Ed.) 2000. Measuring adult literacy: The International Adult Literacy Survey (IALS) in the European context. Office for National Statistics.
• Chang, Sea-Jin, Arjen van Witteloostuijn, and Lorraine Eden. 2010. “From the Editors: Common method variance in international business research.” Journal of International Business Studies 41(2): 178–184. https://doi.org/10.1057/jibs.2009.88
• Charalampi, Anastasia. 2018. The importance of items’ level of measurement in investigating the structure and assessing the psychometric properties of multidimensional constructs (Doctoral dissertation). Panteion University of Social and Political Sciences.
• Charalampi, Anastasia, Catherine Michalopoulou, and Clive Richardson. 2019. “Determining the structure and assessing the psychometric properties of multidimensional scales constructed from ordinal and pseudo-interval items.” Communications in Statistics – Case Studies, Data Analysis and Applications 5(1): 26–38. https://doi.org/10.1080/23737484.2019.1579683
• Charalampi, Anastasia, Catherine Michalopoulou, and Clive Richardson. 2020. “Validation of the 2012 European Social Survey measurement of wellbeing in seventeen European countries.” Applied Research in Quality of Life 15(1): 73–105. https://doi.org/10.1007/s11482-018-9666-4
• Chin, Weng Yee, Edmond P. Choi, Kit T. Y. Chan, and Carlos K. H. Wong 2015. “The psychometric properties of the Center for Epidemiologic Studies depression scale in Chinese primary care patients: Factor structure, construct validity, reliability, sensitivity and responsiveness.” PLoS ONE 10(8): e0135131. https://doi.org/10.1371/journal.pone.0135131
• Cieciuch, Jan, and Eldad Davidov. 2012. “A comparison of the invariance properties of the PVQ-40 and the PVQ-21 to measure human values across German and Polish samples.” Survey Research Methods 6(1): 37–48.
• Clark, Lee A., and David Watson. 1995. “Constructing validity: Basic issues in objective scale development.” Psychological Assessment, 7(3): 309–319. https://doi.org/10.1037/1040-3590.7.3.309
• D’Agostino, Ralph B., Albert Belanger and Ralph B. Jr. D’Agostino, 1990. “A suggestion for using powerful and informative tests of normality.” The American Statistician 44(4): 316–321. https://doi.org/10.1080/00031305.1990.10475751
• Davidov, Eldad. 2008. “A cross-country and cross-time comparison of the human values measurements in the second Round of the European Social Survey.” Survey Research Methods 2(1): 33–46.
• Davidov, Eldad, Peter Schmidt, and Shalom H. Schwartz. 2008. “Bringing values back in the adequacy of the European Social Survey to measure values in 20 countries.” Public Opinion Quarterly 72(3): 420–445. https://doi.org/10.1093/poq/nfn035
• Ehling, Manfred. 2003. “Harmonising data in official statistics.” Pp. 17–31 in Advances in cross-national comparison: A European working book for demographic and socio-economic variables, edited by J. H. P. Hoffmeyer-Zlotnik and C. Wolf. Kluger Academic-Plenum. https://doi.org/10.1007/978-1-4419-9186-7_2
• Enders, Craig K. 2010. Applied Missing Data Analysis. The Guilford Press.
• Ertanir, Beyhan, Christian Rietz, Ulrike Graf, and Wassilis Kassis. 2021. “A crossnational validation of the shortened version of the Adolescent Stress Questionnaire
• (ASQ-S) among adolescents from Switzerland, Germany and Greece.” Frontiers in Psychology 12: 1–11. https://doi.org/10.3389/fpsyg.2021.619493
• European Social Survey. n.d. The human values scale: Findings from the European Social Survey. Retrieved from: http://www.europeansocialsurvey.org/docs/findings/ESS_Findings_HVS.pdf
• European Social Survey Round 1 Data. 2002. Data File Edition 6.6. Norwegian Centre for Research Data, Norway – Data Archive and distributor of ESS data for ESS ERIC.
• European Social Survey Round 2 Data. 2004. Data File Edition 3.6. Norwegian Centre for Research Data, Norway – Data Archive and distributor of ESS data for ESS ERIC.
• European Social Survey Round 4 Data. 2008. Data File Edition 4.5. Norwegian Centre for Research Data, Norway – Data Archive and distributor of ESS data for ESS ERIC.
• European Social Survey Round 5 Data. 2010. Data File Edition 3.4. Norwegian Centre for Research Data, Norway – Data Archive and distributor of ESS data for ESS ERIC.
• European Social Survey Round 6 Data. 2012. Data File Edition 2.4. Norwegian Centre for Research Data, Norway – Data Archive and distributor of ESS data for ESS ERIC.
• European Social Survey Round 8 Data. 2016. Data File Edition 2.1. Norwegian Centre for Research Data, Norway – Data Archive and distributor of ESS data for ESS ERIC.
• European Social Survey Round 9 Data. 2018. Data File Edition 2.0. Norwegian Centre for Research Data, Norway – Data Archive and distributor of ESS data for ESS ERIC.
• Fornell, Claes, and David F. Larcker. 1981. “Evaluating structural equation models with unobservable variables and measurement error.” Journal of Marketing Research 18(1): 39–50. https://doi.org/10.1177/002224378101800104
• Harkness, Janet A., Ana Villar, and Brad Edwards. 2010. “Translation, adaptation and design.” Pp. 117–140 in Survey methods in multinational, multiregional, and multicultural contexts, edited by J.A. Harkness, M. Braun, B. Edwards, T.P. Johnson, L.P. Lyberg, P.Ph. Mohler, B.E. Pennell, and T.W. Smith (Eds.). Wiley. https://doi.org/10.1002/9780470609927.ch7
• Hellström, Amanda, Peter Hagell, Anders Broström, Martin Ulander, Annemarie I. Luik, Colin A. Espie, and Kristofer Ǻrestedt. 2019. “A classical test theory evaluation of the sleep condition indicator accounting for the ordinal nature of item response data.” PLoS ONE 14(3): e0213533. https://doi.org/10.1371/journal.pone.0213533
• Hu, Li-tze, and Peter M. Bentler. 1999. “Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives.” Structural Equation Modeling 6(1): 1–55. https://doi.org/10.1080/10705519909540118
• Kish, Leslie. 1994. “Multi-population survey designs: Five types with seven shared aspects.” International Statistical Review 62(2): 167–186. https://doi.org/10.2307/1403507
• Lilleoja, Laur, and Willem E. Saris. 2014. “Testing a new operationalization of the basic values on Estonian-and Russian-speaking subpopulations in Estonia.” Social Indicators Research 116(1): 153–172. https://doi.org/10.1007/s11205-013-0272-4
• Lilleoja, Laur, and Willem E. Saris. 2015. “Does correction for measurement error have an effect on the structure and comparability of basic human values?” Survey Research Methods 9(3): 169–187.
• Marsh, Herb W., Kit-Tai Hau, and Zhonglin Wen. 2004. “In search of golden rules: Comment on hypotheses-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings.” Structural Equation Modeling 11(3): 320–341. https://doi.org/10.1207/s15328007sem1103_2
• Michalopoulou, Catherine. 2016. “Statistical internationalism: From Quetelet’s census uniformity to Kish’s cross-national sample survey comparability.” Statistical Journal of the IAOS 32(4): 545–554. https://doi.org/10.3233/SJI-160960
• Michalopoulou, Catherine. 2017. “Likert scales require validation before application – Another cautionary tale.” Bulletin de Méthodologie Sociologique 134: 5–23. https://doi.org/10.1177/0759106317693786
• Michalopoulou, Catherine, and Maria Symeonaki. 2017. “Improving Likert scale raw scores interpretability with k-means clustering.” Bulletin de Méthodologie Sociologique 135: 101–109. https://doi.org/10.1177/0759106317710863
• Nunnally, Jum C., and Ira H. Bernstein. 1994. Psychometric theory. McGraw-Hill.
• Raykov, Tenko. 2007. “Reliability if deleted, not ‘alpha if deleted’: Evaluation of scale reliability following component deletion.” British Journal of Mathematical and Statistical Psychology 60(2): 201–216. https://doi.org/10.1348/000711006X115954
• Sagin, Lilach, and Shalom H. Schwartz. 2000. “Value priorities and subjective well-being: Direct relations and congruity effects.” European Journal of Social Psychology 30(2): 177–198. https://doi.org/10.1002/(SICI)1099-0992(200003/04)30:2<177::AIDEJSP982>3.0.CO;2-Z
• Schmitt, Thomas A. 2011. “Current methodological considerations in exploratory and confirmatory factor analysis.” Journal of Psychoeducational Assessment 29(4): 304–322. https://doi.org/10.1177/0734282911406653
• Schwartz, Shalom H. n.d. “Computing scores for the 10 human values.” Retrieved from https://www.europeansocialsurvey.org/docs/methodology/ESS_computing _human_values_scale.pdf
• Schwartz, Shalom H. 1992. “Universals in the content and structure of values: Theory and empirical tests in 20 countries.” Pp. 1–65 in Advances in experimental social psychology, edited by M. Zanna. Academic Press. https://doi.org/10.1016/S0065-2601(08)60281-6
• Schwartz, Shalom H. 2012. An overview of the Schwartz theory of basic values. Online readings in psychology and culture, Unit 2. Retrieved from http://scholarworks.gvsu.edu/orpc/vol2/iss1/11. https://doi.org/10.9707/2307-0919.1116
• Tabachnick, Barbara G., and Linda S. Fidell. 2007. Using multivariate statistics. Pearson Allyn & Bacon.
• The ESS Sampling Expert Panel. 2016. Sampling guidelines: Principles and implementation for the European Social Survey. ESS ERIC Headquarters. Retrieved from http://www.europeansocialsurvey.org/
• Thompson, Bruce. 2005. Exploratory and confirmatory factor analysis: Understanding concepts and applications. American Psychological Association. https://doi.org/10.1037/10694-000
• West, Stephen G., John F. Finch, and Patrick J. Curran. 1995. “Structural equation models with nonnormal variables: Problems and remedies.” Pp. 56–75 in Structural equation modeling: Concepts, issues, and applications, edited by R. H. Hoyle. Sage.
• Zieger, Matthias, and Dirk Hagemann. 2015. “Editorial: Testing the unidimensionality of items.” European Journal of Psychological Assessment 31(4): 231–237. https://doi.org/10.1027/1015-5759/a000309