Can we measure working memory via the Internet? The reliability and factorial validity of an online n-back task

• Authors: Konrad Kulikowski , Katarzyna Potasz-Kulikowska
• Languages of publication: EN

Abstracts

EN

The aim of this study was to check whether an online n-back task conducted in the uncontrolled environment of the Internet can yield valid and reliable data. For this purpose, 169 participants completed an online n-back task with n1, n2 and n3 blocks on their home computers. The results have shown acceptable reliability for overall accuracy and reaction time indices across n1, n2, n3 blocks, as well as for reaction time indices for each n block. Unacceptable reliability has been found for separate n levels accuracy indices and for response bias indices. Confirmatory factor analysis has revealed that, among 8 proposed measurement models, the best fit for the data collected is a model with two uncorrelated factors: accuracy consisting of n1, n2, n3 indices and reaction time consisting of n2, n3 indices. The results of this study have demonstrated for the first time that a reliable administration of online n-back task is possible and may therefore give rise to new opportunities for working memory research.

Keywords

EN

n-back; working memory; online; reliability; validity

• Publisher: Committee for Psychological Sciences, Polish Academy of Sciences
• Journal: Polish Psychological Bulletin
• Year: 2016
• Volume: 47
• Issue: 1
• Pages: 51-61

Dates

published

2016-04-01

online

2016-05-14

Contributors

author

Konrad Kulikowski

• Institute of Psychology, Jagiellonian University

author

Katarzyna Potasz-Kulikowska

• Department of Neurology, Jagiellonian University Medical College

References

• Ackerman, P. L., Beier, M. E., & Boyle, M. O. (2005). Working memory and intelligence: The same or different constructs? Psychological Bulletin, 131, 30-60.[Crossref]
• Alcorn, M., & Morris, G. L. (1996). P300 correlates of inspection time. Personality and Individual Differences, 20, 619-627.[Crossref]
• Attridge, N., Noonan, D., Eccleston, C., & Keogh, E. (2015). The disruptive effects of pain on n-back task performance in a large general population sample. Pain.[Crossref]
• Baddeley, A. (2012). Working Memory: Theories, Models, and Controversies. Annual Review of Psychology, 63, 1-29.[Crossref]
• Baddeley, A. D., & Hitch, G. (1974). Working Memory. Psychology of Learning and Motivation, 8, 47-89.[Crossref]
• Barchard, K. A., & Williams, J. (2008). Practical advice for conducting ethical online experiments and questionnaires for United States psychologists. Behavior Research Methods, 40(4), 1111-1128. doi:10.3758/BRM.40.4.1111[Crossref]
• Byrne, B. (2009). Structural equation modeling with AMOS: Basic concepts, applications, and programming. Uta.Fi. doi:10.4324/9781 410600219[Crossref]
• Cohen, J. D., Peristein, W. M., Braver, T. S., Nystrom, L. E., Noll, D. C., Jonides, J., & Smith, E. E. (1997). Temporal dynamics of brain activation during a working memory task. Nature, 386, 604-608.
• Colom, R., Escorial, Shih, P. C., & Privado, J. (2007). Fluid intelligence, memory span, and temperament difficulties predict academic performance of young adolescent, Personality and Individual Differences, 42, 1503-1514.
• Conway, A. R., Kane, M. J., Bunting, M. F., Hambrick, D. Z., Wilhelm, O., & Engle, R. W. (2005). Working memory span tasks: A methodological review and user’s guide. Psychonomic Bulletin & Review, 12, 769-786.[Crossref]
• Daneman, M., & Merikle, P. M. (1996). Working memory and language comprehension: A metaanalysis. Psychonomic Bulletin & Review, 3, 422-433.[Crossref]
• De Clercq, Crombez, Buysse, & Roeyers (2003). A simple and sensitive method to measure timing accuracy. Behavior Research Methods, Instruments and Computers, 35, 109-115.[Crossref]
• De Dreu, C. K. W., Nijstad, B., Baas, M., Wolsink, I., & Roskes, M. (2012). Working memory benefits creative insight, musical improvisation, and original ideation through maintained task-focused attention. Personality and Social Psychology Bulletin, 38, 656-669.[Crossref]
• Domínguez, U., Martín-Rodríguez, J. F., & León-Carrión, J. (2015). Executive n-back tasks for the neuropsychological assessment of working memory. Behavioural Brain Research, 292(9), 167-173. doi:10.1016/j.bbr.2015.06.002.[Crossref]
• Dougherty, M. R. P., & Hunter, J. (2003). Probability judgment and subadditivity: the role of working memory capacity and constraining retrieval. Memory & Cognition, 31(6), 968-982.[Crossref]
• Dougherty, M. R., Hamovitz, T., & Tidwell, J. W. (2015). Reevaluating the effectiveness of n-back training on transfer through the Bayesian lens: Support for the null. Psychonomic Bulletin & Review. doi:10.3758/s13423-015-0865-9[Crossref]
• Eurostat (2015). Information society statistics. Retrieved from: http://ec.europa.eu/eurostat/web/information-society/data/main-tables [access date: 16.08 2014]
• Fornell, C., & Larcker, D. F. (1981). Evaluating Structural Equation Models with Unobservable Variables and Measurement Error, Journal of Marketing Research, 18(1), 39-50.[Crossref]
• Forns, J., Esnaola, M., López-Vicente, M., Suades-González, E., Alvarez- -Pedrerol, M., Julvez, J., Grellier, J., Sebastián-Gallés, N., & Sunyer, J. (2014). The n-back test and the attentional network task as measures of child neuropsychological development in epidemiological studies. Neuropsychology, 28(4), 519-529. doi:10.1037/ neu0000085[Crossref]
• Foster, J. L., Shipstead, Z., Harrison, T. L., Hicks, K. L., Redick, T. S., & Engle, R. W. (2014). Shortened complex span tasks can reliably measure working memory capacity. Memory & Cognition, 43(2), 226-236. doi:10.3758/s13421-014-0461-7[Crossref]
• Friedman, N. P., Miyake, A., Corley, R. P., Young, S. E., DeFries, J. C., & Hewitt, J. K. (2006). Not All Executive Functions Are Related to Intelligence. Psychological Science, 17(2), 172-179.[Crossref]
• Friedman, N. P., Miyake, A., Young, S. E., Defries, J. C., Corley, R. P., & Hewitt, J. K. (2008). Genetic in Origin. Journal of Experimental Psychology, 137(2), 201-225. doi:10.1037/0096-3445.137.2.201[Crossref]
• Goozee, R., Reinders, A. A., Handley, R., Marques, T., Taylor, H., O’Daly, O., McQueen, G., Hubbard, K., Mondelli, V., Pariante, C., & Dazzan, P. (2015). Effects of aripiprazole and haloperidol on neural activation during the n-back in healthy individuals: A functional MRI study. Schizophrenia Research. doi:10.1016/j.schres.2015.02.023[Crossref]
• Gosling, S. D., & Mason, W. (2015). Internet Research in Psychology. Annual Review of Psychology 66, 877-902.[Crossref]
• Guastello, S. J., Reiter, K., Malon, M., Timm, P., Shircel, A., & Shaline J. (2015). Catastrophe models for cognitive workload and fatigue in N-back tasks. Nonlinear Dynamics Psychology, and Life Sciences, 19(2), 173-200.
• Hockey, A., & Geffen, G. (2004). The concurrent validity and test-retest reliability of a visuospatial working memory task. Intelligence, 32(6), 591-605. doi:10.1016/j.intell.2004.07.009[Crossref]
• Houben, K., & Wiers, R. W. (2008). Measuring implicit alcohol associations via the Internet: validation of Web-based implicit association tests. Behavior Research Methods, 40(4), 1134-1143. doi:10.3758/ BRM.40.4.1134[Crossref]
• Jacola, L. M., Willard, V. W., Ashford, J. M., Ogg, R. J., Scoggins, M. A., Jones, M. M., Wu, S., & Conklin, H. M. (2014). Clinical utility of the N-back task in functional neuroimaging studies of working memory. Journal of Clinical and Experimental Neuropsychology, 36(8), 875-886. doi:10.1080/13803395.2014.953039[Crossref]
• Jaeggi, S. M., Studer-Luethi, B., Buschkuehl, M., Su, Y. F., Jonides, J., & Perrig, W. J. (2010a). The relationship between n-back performance and matrix reasoning - implications for training and transfer. Intelligence, 38(6), 625-635. doi:10.1016/j.intell.2010.09.001[Crossref]
• Jaeggi, S. M., Buschkuehl, M., Perrig, W. J., & Meier, B. (2010b). The concurrent validity of the N-back task as a working memory measure. Memory, 18(4), 394-412. doi:10.1080/09658211003702171[Crossref]
• Kane, M. J., Conway, A. R. A., Miura, T. K., & Colflesh, G. J. H. (2007). Working memory, attention control, and the N-back task: a question of construct validity. Journal of Experimental Psychology. Learning, Memory, and Cognition, 33(3), 615-622. doi:10.1037/0278-7393.33.3.615[Crossref]
• Kirchner, W. K. (1958). Age differences in short-term retention of rapidly changing information. Journal of Experimental Psychology, 55, 352-358.[Crossref]
• König, C., Bühner, M., & Mürling, G. (2005). Working memory, fluid intelligence, and attention are predictors of multitasking performance, but polychronicity and extraversion are not. Human Performance, 18(3), 243-266.[Crossref]
• Kraut, R., Olson, J., Banaji, M., Bruckman, A., Cohen, J., & Couper, M. (2004). Psychological Research Online: Report of Board of Scientific Affairs’ Advisory Group on the Conduct of Research on the Internet. American Psychologist, 59(2), 105-117.[Crossref]
• Linck, J., Osthus, P., Koeth, J. T., & Bunting, M. F. (2014). Working memory and second language comprehension and production: A metaanalysis. Psychonomic Bulletin & Review, 21, 861-883.[Crossref]
• Medow, M. S., Sood, S., Messer, Z., Dzogbeta, S., Terilli, C., & Stewart J. M. (2014). Phenylephrine alteration of cerebral blood fl ow during orthostasis: effect on n-back performance in chronic fatigue syndrome. Journal of Applied Physiology, 117(10), 1157-1164. doi:10.1152/japplphysiol.00527.2014[Crossref]
• Miller, K. M., Price, C. C., Okun, M. S., Montijo, H., & Bowers, D. (2009). Is the N-back task a valid neuropsychological measure for assessing working memory? Archives of Clinical Neuropsychology, 24(7), 711-717. doi:10.1093/arclin/acp063[Crossref]
• Nosek, B. A., Banaji, M. R., & Greenwald, A. G. (2002). E-Research: Ethics, Security, Design, and Control in Psychological Research on the Internet. Journal of Social Issues, 58(1), 161-176.[Crossref]
• Oberauer, K., Schulze, R., Wilhelm, O., & Süss, H.-M. (2005). Working memory and intelligence their correlation and their relation: comment on Ackerman, Beier, and Boyle (2005). Psychological Bulletin, 131(1), 61-65; author reply 72-75.[Crossref]
• Oberauer, K., Süß, H.-M., Wilhelm, O., & Wittman, W. (2003). The multiple faces of working memory: Storage, processing, supervision, and coordination. Intelligence, 31, 167-193.[Crossref]
• Pérez, A. I., Paolieri, D., Macizo, P., & Bajo, T. (2014). The role of working memory in inferential sentence comprehension. Cognitive Processing, 15(3), 405-413.[Crossref]
• Philip, N. S., Sweet, L. H., Tyrka, A. R., Carpenter, S. L., Albright, S. E., Price, L. H., & Carpenter, L. L. (2015). Exposure to childhood trauma is associated with altered n-back activation and performance in healthy adults: implications for a commonly used working memory task. Brain Imaging Behavior.
• Ragland, J. D., Turetsky, B. I., Gur, R. C., Gunning-Dixon, F., Turner, T., Schroeder, L., Chan, R., & Gur, R. E. (2002). Working memory for complex figures: an fMRI comparison of letter and fractal n-back tasks. Neuropsychology, 16(3), 370-379. doi:10.1037/0894-4105.16.3.370[Crossref]
• Raykov, T. (1997). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21(2), 173-184.[Crossref]
• Raz, S., Bar-Haim, Y., Sadeh, A., & Dan, O. (2012). Reliability and Validity of the Online Continuous Performance Test Among Young Adults. Assessment, 21(1), 108-118. doi:10.1177/1073191112443409[Crossref]
• Redick, T. S., & Lindsey, D. R. B. (2013). Complex span and n-back measures of working memory: a meta-analysis. Psychonomic Bulletin & Review, 20(6), 1102-1113. doi:10.3758/s13423-013-0453-9[Crossref]
• Salminen, T., Frensch, P., Strobach, T., & Schubert, T. (2015). Age-specific differences of dual n-back training. Neuropsychology, development, and cognition. Section B, Aging, neuropsychology and cognition, 13, 1-22. doi:10.1080/13825585.2015.1031723[Crossref]
• Salthouse, T. A., Atkinson, T. M., & Berish, D. E. (2003). Executive functioning as a potential mediator of age-related cognitive decline in normal adults. Journal of Experimental Psychology. General, 132(4), 566-594. doi:10.1037/0096-3445.132.4.566[Crossref]
• Sánchez-Fernández, J., Muñoz-Leiva, F., & Montoro-Ríos, F. J. (2012). Improving retention rate and response quality in Web-based surveys. Computers in Human Behavior, 28(2), 507-514. doi:10.1016/j. chb.2011.10.023[Crossref]
• Schmiedek, F., Hildebrandt, A., Lövdén, M., Lindenberger, U., & Wilhelm, O. (2009). Complex span versus updating tasks of working memory: the gap is not that deep. Journal of Experimental Psychology. Learning, Memory, and Cognition, 35(4), 1089-1096. doi:10.1037/a0015730[Crossref]
• Schmiedek, F., Lövdén, M., & Lindenberger, U. (2014). A task is a task is a task: putting complex span, n-back, and other working memory indicators in psychometric context. Frontiers in Psychology, 5, 1-8. doi:10.3389/fpsyg.2014.01475[Crossref]
• Shelton, J. T., Elliott, E. M., Hill, B., Calamia, M. R., & Gouvier, W. (2009). A comparison of laboratory and clinical working memory tests and their prediction of fluid intelligence. Intelligence, 37(3), 283-293. doi:10.1016/j.intell.2008.11.005 [Crossref]
• Snodgrass, J. G., & Corwin, J. (1988). Pragmatics of measuring recognition memory: applications to dementia and amnesia. Journal of Experimental Psychology: General, 117(1), 34-50. doi:10.1037/0096-3445.117.1.34[Crossref]
• Sörqvist, P. (2010). The role of working memory capacity in auditory distraction: a review. Noise & Health, 12, 217-224.
• Szmalec, A., Verbruggen, F., Vandierendonck, A., & Kemps, E. (2011). Control of interference during working memory updating. Journal of Experimental Psychology. Human Perception and Performance, 37, 137-151.[Crossref]
• Unsworth, N., Brewer, G., & Spillers, G. J. (2012). Working memory capacity and retrieval from long-term memory: the role of controlled search. Memory & Cognition, 41(2), 242-254.[Crossref]
• Van de Weijer-Bergsma, E., Kroesbergen, E. H., Prast, E. J., & Van Luit, J. E. H. (2014). Validity and reliability of an online visualspatial working memory task for self-reliant administration in school-aged children. Behavior Research Methods, 47(3), 708-719. doi:10.3758/s13428-014-0469-8[Crossref]
• Van Leeuwen, M., van den Berg, S. M., Hoekstra, R. A., & Boomsma, D. I. (2007). Endophenotypes for intelligence in children and adolescents. Intelligence, 35(4), 369-380. doi:10.1016/j.intell.2006.09.008[Crossref]
• Wilhelm, O., Hildebrandt, A., & Oberauer, K. (2013). What is working memory capacity, and how can we measure it? Frontiers in Psychology, 4, 1-22. doi:10.3389/fpsyg.2013.0043[Crossref]

Identifiers

DOI

10.1515/ppb-2016-0006