Why We Should Have Seen That Coming: Comments on Microsoft's tay "Experiment," and Wider Implications

Document Type

Peer-Reviewed Article

Publication Date

9-2017

Abstract

In this paper we examine the case of Tay, the Microsoft AI chatbot that was launched in March, 2016. After less than 24 hours, Microsoft shut down the experiment because the chatbot was generating tweets that were judged to be inappropriate since they included racist, sexist, and anti-Semitic language. We contend that the case of Tay illustrates a problem with the very nature of learning software (LS is a term that describes any software that changes its program in response to its interactions) that interacts directly with the public, and the developer's role and responsibility associated with it. We make the case that when LS interacts directly with people or indirectly via social media, the developer has additional ethical responsibilities beyond those of standard software. There is an additional burden of care. Why we should have seen that coming: comments on Microsoft's tay "experiment," and wider implications.

Comments

ACM SIGCAS Computers and Society, Newsletter: http://www.sigcas.org/newsletter

DOI

10.1145/3144592.3144598

Link to Full Text


Share

COinS