Eddie Ungless

Eddie Ungless

Looking for work A collection of my thoughts on the topics of ethics, fairness and social bias in NLP and beyond.

Use of Reddi-rty Data

#WIP - This blog will be expanded as I develop my ideas

cn: mental health stigma

Two recent papers, and my own increasing use of the site, have led me to question the sanity of using Reddit data to train or finetune language models (something which is sure to increase now free access to Twitter data has been so cruelly ripped out from under our feet - won’t somebody think of the language models?). The first paper proposes a toolkit to measure the safety of dialogue systems1 (and in doing so provides a really thorough exploration of potential harms). What I find interesting is their observation that in over half of DialoGPTs responses to offensive content the model begins “I don’t know why you’re being downvoted”, clearly parroting a popular Reddit turn of phrase. It is truly wild to me that over half of the responses include this phrase. Somewhere in the process of creating suitable training data, the views of reddit’s devils advocates have been allowed far too much weight.

As an aside, it’s honestly a bit dissapointing that ChatGPT refuses to weigh in on AITA (r/AmItheAsshole/) questions.

Screenshot of conversation with ChatGPT where it responds to "AITA for refusing to stop cooking and eating in my kitchen because of my roommate's girlfriend's "condition." by saying "As an AI language model, I cannot make a judgment call on your actions or intentions. However, I can provide you with some insights and considerations to help you make an informed decision."

The second paper that caught my eye attempts to source personality modelling training data from novel places2. Whilst I understand the aims (to some extent… I’m not sure what the benefits of creating a model that mimics disordered behaviour is) I am concerned about the use of r/psychopath, r/sociopath and r/antisocial (alongside TV show scripts) as a source of unlabeled data to pretrain a model on. TV representations of psychopathy/ antisocial personality disorder are famously skewed (most people diagnosed with ASPD are not cunning and hyper intelligent). And whilst some may be using the reddit threads to express their true feelings as people with ASPD, there is no doubt a lot of trolling, confusion, individuals who are anti-social in ways unrelated to a diagnosable personality disorder, questions for the community etc. It’s very very messy data.

Don’t downvote me.

  1. Emily Dinan, Gavin Abercrombie, A. Bergman, Shannon Spruit, Dirk Hovy, Y-Lan Boureau, and Verena Rieser. 2022. SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4113–4133, Dublin, Ireland. Association for Computational Linguistics. ↩

  2. Yair Neuman, Vladyslav Kozhukhov, and Dan Vilenchik. 2023. Data Augmentation for Modeling Human Personality: The Dexter Machine. arXiv:2301.08606 [cs]. ↩

Updated: