MEASURING THE NON-COOPERATION OF PLAYERS – A LOEBNER CONTEST CASE STUDY

• Authors: Łupkowski Paweł

Title variants

PL

Pomiar niekooperatywnych zachowań graczy – przykład konkursu Loebnera

• Languages of publication: EN

Abstracts

EN

The main aim of this paper is to present a technique designed to assess the level of non-cooperativeness of players in the Loebner Contest (thereafter I will refer to the contest as LC). My tool here will be the measure of the Degrees of Non-Cooperation (DNC) in a dialogue proposed by Brian Plüss (2010, 2011). This technique is based on the identification of the set of non-cooperative features (NCFs) appearing in a given dialogue type. Plüss proposes such a set for the domain of political debates, whereas in this paper, I will propose my own set to use with the LC dialogues. In the first section I will introduce the basic rules and ideas behind the LC. The second section contains the results of an empirical study of four LC dialogues using the DNC measure: description of the study sample, types of NCFs used, a discussion of the annotation reliability and DNC measures for players are covered in this section. In the summary I will concern future applications of the presented approach.

Keywords

EN

TURING TEST; LOEBNER CONTEST; STRATEGY; NON-COOPERATION DEGREE MEASURE

• Publisher: Polskie Towarzystwo Badania Gier (Games Research Association of Poland)
• Journal: Homo Ludens
• Year: 2013
• Issue: 1(5)
• Pages: 13-22
• Document type: ARTICLE

Contributors

author

Łupkowski Paweł

• Paweł Łupkowski, Uniwersytet im. Adama Mickiewicza, Instytut Psychologii, Zakład Logiki i Kognitywistyki, ul. A. Szamarzewskiego 89, 60-568 Poznań, Poland

• Paweł Łupkowski, Uniwersytet im. Adama Mickiewicza, Instytut Psychologii, Zakład Logiki i Kognitywistyki, ul. A. Szamarzewskiego 89, 60-568 Poznań, Poland

References

• Asher, N., Lascarides, A., Lemon, O., Guhe, M., Rieser, V., Muller, P., Afantenos, S., Benamara, F., Vieu, L., Denis, P., Paul, S., Keizer, S., Degremont, C. (2012). Modelling Strategic Conversation: the STAC Project. Proceedings of 16th Workshop on the Semantics and Pragmatics of Dialogue (Seinedial). Paris.
• Gamer, M., Lemon, J., Singh, I.F.P. (2012). irr: Various Coefficients of Interrater Reliability and Agreement. R package version 0.84. Online: <http://CRAN.R-project.org/package=irr>.
• Carletta, J. (1996). Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22(2), 249-254.
• Loebner, H. (2009). How to Hold a Turing Test Contest. [In:] R. Epstein, G. Roberts, G. Beber (eds.), Parsing the Turing Test. Philosophical and Methodological Issues in the Quest for the Thinking Computer (p. 173-179). Springer.
• Łupkowski, P., Wiśniewski, A. (2011). Turing interrogative games. Minds and Machines, 21(3), 435-448.
• Plüss, B. (2010). Non-Cooperation in Dialogue. [In:] Proceedings of the ACL 2010 Student Research Workshop (p. 1-6). Uppsala, Sweden, 13 July 2010.
• Plüss, B., Piwek, P., Power, R. (2011). Modelling Non-Cooperative Dialogue: the Role of Conversational Games and Discourse Obligations. [In:] SemDial 2011: Proceedings of the 15th Workshop on the Semantics and Pragmatics of Dialogue (p. 212-213). Los Angeles, California, 21-23 September 2011.
• Turing, A.M. (1950). Computing machinery and intelligence. Mind, LIX (236), 443-455.
• Viera, A.J., Garrett, J.M. (2005). Understanding Interobserver Agreement: The Kappa Statistic, Family Medicine, 37(5), 360-363.