Machine learning models are commonly used to detect toxicity in online conversations. These models are trained on datasets annotated by human raters and require relatively large datasets. We explore how raters' self-described identities impact how they annotate toxicity in online comments. We first define the concept of Specialized Rater Pools: rater pools formed based on raters' self-described identities, rather than at random. We found that rater identity is a statistically significant factor in how raters will annotate toxicity for identity-related annotations and point to how smaller datasets can be used to improve efficiency.
Tesh (Nitesh) Goyal leads research at the intersection of AI and Safety at Google Deepmind.
His work at Google has led to the launch of ML based tools like SynthID to enable AI Literacy, AIStudio and MakerSuite to enable creatives for leveraging AI to bring their ideas to life, Harassment Manager to empower targets of online harassment, ML based moderation to reduce online toxic content production on platforms like OpenWeb, and multiple NLP based tools that reduce biased sensemaking.
He received his MSc in Computer Science from UC, Berkeley and RWTH Aachen, prior to receiving his PhD from Cornell University in Information Science.
His research has been supported by the German Govt. Fellowship, and National Science Foundation. Frequently collaborating with industry (Google Research, Yahoo Labs, HP Labs, Bloomberg Labs), he has published in top-tier HCI conferences and journals (CHI, CSCW, JASIST, ICTD, ICIC and Ubicomp/IMWUT), received three best paper honorable mention awards (CHI, CSCW) and his work is frequently covered in the press.
Tesh has served multiple roles in the Organization Committee for ACM SIGCHI conferences (most recently as the Tech Program Chair at CHI 2023). Tesh serves on the ACM SIGCHI Steering Committee, as appointed Adjunct Professor at NYU Computer Science Department, and as ACM Distinguished Speaker.