A Comparison of Fake News Detecting and Fact-Checking AI Based Solutions

• Authors: Školkay Andrej , Filin Juraj
• Languages of publication: EN

Abstracts

EN

Scientific objective of this paper is to analyse how advanced are Artificial Intelligence (AI) tools to fight successfully information disorder. More specifically, this is an overview and ranking on existing tools based on AI in this specific area. Research method is comparative analytics. We compare the most developed and publicly available fake-news detecting and fact-checking AI based solutions (intelligent machines). The comparison is based on two key parameters: accuracy and comprehensiveness. Results and conclusions: Analyse show that a third of the examined AI systems are, in terms of comprehensiveness, in the top category, while the majority are in the medium category. As far as accuracy is concerned, very few AI machine developers are interested in providing further details about their products and functionalities for studies such as ours which raises suspicions about their actual performance. Surprisingly, one of the most discussed AI systems among EU leaders seems to actually belong to the least developed. Cognitive value: There is a need for a larger and more detailed study with involvement of AI specialists who would be able, and allowed, to test all available AI machines with their key features and functionalities.

Keywords

EN

artificial intelligence; comparison; fact-checking; fake news; testing

• Publisher: Uniwersytet Warszawski. Wydział Dziennikarstwa, Informacji i Bibliologii
• Journal: Studia Medioznawcze
• Year: 2019
• Issue: 4
• Pages: 365-383

Contributors

author

Školkay Andrej

• School of Communication and Media, Bratislava

author

Filin Juraj

• School of Communication and Media, Bratislava

References

• Alaphilippe, A., Gizikis, A., Hanot, C., & Bontcheva, K. (2019). Automated Tackling of Disinformation. Retrieved from http://www.europarl.europa.eu/RegData/etudes/STUD/2019/624278/EPRS_STU(2019)624278_EN.pdf
• Babakar, M., & Moy, W. (2016). The State of Automated Factchecking. How to Make Factchecking Dramatically More Effective with Technology We Have Now. Full Fact. Retrieved from https://fullfact.org/media/uploads/full_fact-the_state_of_automated_factchecking_aug_2016.pdf
• Carriço, G. (2018). The EU and Artifi cial Intelligence: A Human-Centred Perspective. European View, 2018, 17(1): 29–36. DOI: 0.1177/1781685818764821
• Cellan-Jones, R. (2014, December 2). Stephen Hawking Warns Artifi cial Intelligence Could End Mankind, BBC. Retrieved from http://www.bbc.com/news/technology-30290540
• Chinnappa, M. (2017). We Are All in This Together, British Journalism Review, 28(3), 50-55. DOI: 10.1177/0956474817730769
• Craft, S., Ashley, S. & Maksl, A. (2017). News Media Literacy and Conspiracy Theory Endorsement. Communication and the Public, 2(4): 388–401. DOI: 10.1177/2057047317725539
• Faggella, D. (2018, May 14). How to Assess an Artifi cial Intelligence Product or Solution (Even if You’re Not an AI Expert). Retrieved from https://www.techemergence.com/how-to-assess-an-artificialintelligence-product-or-solution-for-non-experts/
• Funke, D. (2018, February 23). Report: There are 149 Fact-Checking Projects in 53 Countries. That’s a New High. Retrieved from https://www.poynter.org/news/report-there-are-149-fact-checking-projects-53-countries-thats-new-high
• Ghafourifar, A. (2017). How AI is Winning the War Against Fake News. Retrieved from https://venturebeat.com/2017/06/11/how-ai-is-winning-the-war-against-fake-news/
• Graves, L. (2018, February). Understanding the Promise and Limits of Automated Fact-Checking. Factsheet, Reuters Institute for the Study of Journalism. Retrieved from https://reutersinstitute.politics.ox.ac.uk/sites/default/fi les/2018-02/graves_factsheet_180226%20FINAL.pdf
• Hassan, N., Li, Ch., & Tremayne, M. (2015). Detecting Check-worthy Factual Claims in Presidential Debates. Proceedings of the 24th ACM International Conference on Information and Knowledge Management: 1835–1838.
• Hassan, N., Zhang, G., Arslan, F., Caraballo, J., Jimenez, D., Gawsane, S.,… Shohedul, H. (2017). ClaimBuster: The First-ever End-to-end Fact-checking System. Proceedings of the VLDB Endowment, 10(12): 1945–1948.
• Marr, B. (2017). The Biggest Challenges Facing Artificial Intelligence (AI) In Business And Society. Retrieved from https://www.forbes.com/sites/bernardmarr/2017/07/13/the-biggest-challenges-facingartificial-intelligence-ai-in-business-and-society/3/#37119b357b6
• Özgöbek, Ö., & Gullain, J.A. (2017). Towards an Understanding of Fake News. Norwegian Big Data Symposium NOBIDS:35-42.
• Orlowski, A. (2017, January 2). Artifi cial Intelligence’ Was 2016’s Fake News. Retrieved from http://www.theregister.co.uk/2017/01/02/ai_was_the_fake_news_of_2016/?page=1
• Pavleska, T., Školkay, A., Zankova, B., Ribeiro, N., & Bechmann, A. (2018, February). Performance Analysis of Fact-Checking Organizations and Initiatives in Europe: A Critical Overview of Online Platforms Fighting Fake News. Retrieved from http://compact-media.eu/fake-news/
• Popat, K., Mukherjee, S., Strötgen, J., & Weikum, G. (2016). Credibility Assessment of Textual Claims on the Web. Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, 2173–2178. ACM. DOI: 10.1145/2983323.2983661
• Prateek, J. (2017). Artificial Intelligence with Python. Packt Publishing.
• Rony, M.M.U., Hassan, N., & Yousuf, M. (2018). BaitBuster: A Clickbait Identifi cation Framework. Proceedings of the 32nd AAAI Conference on Artifi cial Intelligence. (AAAI–18).
• Rony, M.M.U., Hassan, N., & Yousuf, M. (2017). BaitBuster: Destined to Save You Some Clicks. Proceedings of the 2017 Computation+Journalism Symposium.
• Rony, M.M.U., Hassan, N., & Yousuf, M. (2017). Diving Deep into Clickbaits: Who Use Them to What Extents in Which Topics with What Effects?. Proceedings of 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining.
• Russell, S., Dewey, D., & Tegmark, M. (2015). Research Priorities for Robust and Beneficial Artificial Intelligence. Association for the Advancement of Artificial Intelligence: 105–114.
• Shotter, J. (1997, May). Artifi cial Intelligence and the Dialogical, American Behavioral Scientists, 40(6): 813–827.
• Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017, September 3). Fake News Detection on Social Media: A Data Mining Perspective. Retrieved from https://arxiv.org/pdf/1708.01967.pdf
• Su, D., Zhang, H., Chen, H., Yi, J., Chen, Pin-Yu., & Gao, Y. (2018). Is Robustness the Cost of Accuracy? A Comprehensive Study on the Robustness of 18 Deep Image Classifi cation Models. Retrieved from http://openaccess.thecvf.com/content_ECCV_2018/html/Dong_Su_Is_Robustness_the_ECCV_2018_paper.html
• Vlachos, A., & Riedel, S. (2014). Fact Checking: Task Defi nition and Dataset Construction. Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science:18–22.
• Wang, W.Y. (2017, May 1). Liar, Liar Pants on Fire: A New Benchmark Dataset for Fake News Detection. Retrieved from https://arxiv.org/pdf/1705.00648.pdf
• Wood, C. (2016, August 1). What Is Artifi cial Intelligence? Government Technology. Retrieved from http://www.govtech.com/computing/What-Is-Artificial-Intelligence.html
• Zhao, Z., Resnick, P., & Mei, Q. (2015). Enquiring Minds: Early Detection of Rumors in Social Media from Enquiry Posts. Proceedings of the 24th International Conference on World Wide Web: 1395–1405.

Identifiers

DOI

10.33077/uw.24511617.ms.2019.4.187