The main aim of this paper is to present a technique designed to assess the level of non-cooperativeness of players in the Loebner Contest (thereafter I will refer to the contest as LC). My tool here will be the measure of the Degrees of Non-Cooperation (DNC) in a dialogue proposed by Brian Plüss (2010, 2011). This technique is based on the identification of the set of non-cooperative features (NCFs) appearing in a given dialogue type. Plüss proposes such a set for the domain of political debates, whereas in this paper, I will propose my own set to use with the LC dialogues. In the first section I will introduce the basic rules and ideas behind the LC. The second section contains the results of an empirical study of four LC dialogues using the DNC measure: description of the study sample, types of NCFs used, a discussion of the annotation reliability and DNC measures for players are covered in this section. In the summary I will concern future applications of the presented approach.
In this paper the idea of Loebner contest as a practical implementation of the Turing test is presented. The Brian Plüss’ measure of the degrees of non-cooperation in a dialogue is applied to the dialogues of the Loebner contest. The proposal of a typology of non-cooperative features in the contest’s dialogues is discussed and the reliability of annotation with the use of this typology of features is analysed. The degrees of non-cooperation of judges and programs for the Loebner contest (editions 2009 – 2012) are presented and discussed. On the basis of the results the role of a judge and the strategies used by programs are discussed for the contest and the Turing test.
JavaScript is turned off in your web browser. Turn it on to take full advantage of this site, then refresh the page.