Evaluating Artificial Models of Cognition

• Authors: Marcin Miłkowski
• Languages of publication: EN

Abstracts

EN

Artificial models of cognition serve different purposes, and their use determines the way they should be evaluated. There are also models that do not represent any particular biological agents, and there is controversy as to how they should be assessed. At the same time, modelers do evaluate such models as better or worse. There is also a widespread tendency to call for publicly available standards of replicability and benchmarking for such models. In this paper, I argue that proper evaluation of models does not depend on whether they target real biological agents or not; instead, the standards of evaluation depend on the use of models rather than on the reality of their targets. I discuss how models are validated depending on their use and argue that all-encompassing benchmarks for models may be well beyond reach.

Keywords

EN

modeling; mechanism; mechanistic models; scaffolding; explanatory focus; idealization

• Publisher: Sciendo
• Journal: Studies in Logic, Grammar and Rhetoric
• Year: 2015
• Volume: 40
• Issue: 1
• Pages: 43-62

Dates

published

2015-03-01

online

2015-04-10

Contributors

author

Marcin Miłkowski

• Institute of Philosophy and Sociology, Polish Academy of Sciences

References

• Barandiaran, X. E., & Chemero, A. (2009). Animats in the modeling ecosystem. Adaptive Behavior, 17(4), 287-292.[Crossref][WoS]
• Beer, R. D., &Williams, P. L. (2009). Animals and animats:Why not both iguanas? Adaptive Behavior, 17(4), 296-302.[WoS][Crossref]
• Braitenberg, V. (1984). Vehicles, experiments in synthetic psychology. Cambridge, MA: MIT Press.
• Broadbent, D. E. (1958). Perception and communication. Oxford: Pergamon Press.
• Cleeremans, A., & French, R. M. (1996). From chicken squawking to cognition: Levels of description and the computational approach in psychology. Psychologica Belgica, 36, 1-28.
• Craver, C. F. (2007). Explaining the brain. Oxford: Oxford University Press.
• Cummins, R. (1983). The nature of psychological explanation. Cambridge, MA: MIT Press.
• Dawson, M. R. W. (2004). Minds and machines: Connectionism and psychological modeling. Malden, MA: Blackwell.
• Dennett, D. C. (1981). Reflections on D. R. Hofstadter’s “The Turing Test: A coffeehouse conversation”. In D. R. Hofstadter & D. C. Dennett (Eds.), The mind’s I (pp. 69-95). New York: Bantam Books.
• Dennett, D. C. (1991). Real patterns. Journal of Philosophy, 88(1), 27-51.[Crossref]
• Eliasmith, C., Stewart, T. C., Choo, X., Bekolay, T., DeWolf, T., Tang, C., & Rasmussen, D. (2012). A large-scale model of the functioning brain. Science, 338(6111), 1202-1205. doi:10.1126/science.1225266.[Crossref]
• Farrell, S., & Lewandowsky, S. (2010). Computational models as aids to better reasoning in psychology. Current Directions in Psychological Science, 19(5), 329-335.[WoS]
• Fodor, J. A. (1968). Psychological explanation: An introduction to the philosophy of psychology. New York: Random House.
• Frijda, N. H. (1967). Problems of computer simulation. Behavioral Science, 12(1), 59-67.[Crossref][PubMed]
• Glennan, S. S. (2002). Rethinking mechanistic explanation. Philosophy of Science, 69(S3), S342-S353.
• Glymour, C. (1987). Android epistemology and the frame problem: Comments on Dennett’s “Cognitive wheels.” In Z. W. Pylyshyn (Ed.), The robot’s dilemma: Frame problem in Artificial Intelligence (pp. 65-75). Norwood: Ablex.
• Godfrey-Smith, P. (2008). Models and fictions in science. Philosophical Studies, 143(1), 101-116.[WoS]
• Hinton, G. E., & Nowlan, S. J. (1987). How learning can guide evolution. Complex Systems, 1, 495-502.
• Koehn, P. (2010). Statistical machine translation. Cambridge: Cambridge University Press.
• Krohs, U. (2008). How digital computer simulations explain real-world processes. International Studies in the Philosophy of Science, 22(3), 277-292.
• Kuhn, T. S. (1970). The structure of scientific revolutions. Chicago: University of Chicago Press.
• Lewandowsky, S. (1993). The rewards and hazards of computer simulations. Psychological Science, 4(4) (July), 236-243.[Crossref]
• Levins, R. (1966). The strategy of model building in population biology. American Scientist, 54(4), 421-431.
• Levy, A. (2012).Models, fictions, and realism: Two packages. Philosophy of Science, 79(5) (November 19), 738-748.[Crossref][WoS]
• McClelland, J. L., Rumelhart, D. E. & PDP Research Group (Eds.) (1986). Parallel Distributed Processing: Explorations in the microstructures of cognition, Vol. 2: Psychological and biological models. Cambridge, MA: MIT Press.
• Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81-97.[Crossref][PubMed]
• Miłkowski, M. (2013). Explaining the computational mind. Cambridge, MA: MIT Press.
• Montebelli, A., Lowe, R., Ieropoulos, I., Melhuish, C., Greenman, J., & Ziemke, T. (2010). Microbial fuel cell driven behavioral dynamics in robot simulations. In H. Fellermann et al. (Eds.), Artificial Life XII: Proceedings of the Twelfth International Conference on the Synthesis and Simulation of Living Systems (pp. 749-756). Cambridge, MA: MIT Press. Available at https://mitp-web2.mit.edu/sites/default/files/titles/alife/0262290758chap133.pdf [Accessed November 10, 2011].
• Neisser, U. (1963). The imitation of man by machine: The view that machines will think as man does reveals misunderstanding of the nature of human thought. Science, 139(3551): 193-197.
• Newell, A., Shaw, J. C., & Simon, H. A. (1960). A variety of intelligent learning in a general problem solver. In M. C. Yovits & S. Cameron (Eds.), Selforganizing systems: Proceedings of an interdisciplinary conference (pp. 153-189). Oxford: Pergamon Press.
• Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.
• Oaksford, M., Chater, N., & Larkin, J. (2000). Probabilities and polarity biases in conditional inference. Journal of Experimental Psychology. Learning, Memory, and Cognition, 26(4) (July), 883-99.
• Di Paolo, E., Noble, J., & Bullock, S. (2000). Simulation models as opaque thought experiments. In M. Bedau, J. McCaskill, N. Packard & S. Rasmussen (Eds.), The Seventh International Conference on Artificial Life (pp. 497-506). Cambridge, MA: MIT Press.
• Piccinini, G., & Craver, C. F. (2011). Integrating psychology and neuroscience: Functional analyses as mechanism sketches. Synthese, 183(3) (March 11), 283-311. doi:10.1007/s11229-011-9898-4.[Crossref][WoS]
• Pinker, S., & Prince, A. (1988). On language and connectionism: Analysis of a parallel distributed processing model of language acquisition. Cognition, 23, 73-193.[Crossref]
• Santos, D., Sangbae, K., Spenko, M., Parness, A., & Cutkosky, M. (2007). Directional adhesive structures for controlled climbing on smooth vertical surfaces. In Proceedings 2007 IEEE International Conference on Robotics and Automation, 1262-1267. IEEE.
• Sanz, R., & Hern´andez, C. (2010). Autonomy, intelligence and animat mesmerization. In C. Hern´andez, J. Gómez & R. Sanz (Eds.), From brains to systems: Preprints of the BICS 2010 conference on brain-inspired cognitive systems (pp. 256-270).Madrid. Preprint available at http://tierra.aslab.upm.es/events/BIC2010/documents/BICS-2010-Preprints-complete.pdf.
• Simon, H. A. (1996). The sciences of the artificial. Cambridge, MA: MIT Press.
• Sperling, G. (1960). The information available in brief visual presentations. Psychological Monographs: General and Applied, 74 (11), 1-29.
• Súarez, M. (2003). Scientific representation: Against similarity and isomorphism. International Studies in the Philosophy of Science, 17(3) (October 1), 225-244. doi:10.1080/0269859032000169442.[Crossref]
• Súarez, M., (Ed.) (2009). Fictions in science: Philosophical essays on modeling and idealization. Vol. 4. New York: Routledge.
• Sun, R. (2009). Theoretical status of computational cognitive modeling. Cognitive Systems Research, 10(2), 124-140.
• Tolman, E. C. (1939). Prediction of vicarious trial and error by means of the schematic sowbug. Psychological Review, 46(4), 318-336.[Crossref]
• Webb, B. (2009). Animals versus animats: Or why not model the real iguana? Adaptive Behavior, 17(4) (July 28), 269-286.[WoS][Crossref]
• Weber, B. H., & Depew, D. J. (Eds.) (2003). Evolution and learning: The Baldwin effect reconsidered. Cambridge, MA: MIT Press.
• Weisberg, M. (2013). Simulation and similarity: Using models to understand the world. New York: Oxford University Press.
• Zeigler, B. (1976). Theory of modelling and simulation. New York: Wiley.

Identifiers

DOI

10.1515/slgr-2015-0003