Diyi Yang

Recent Preprints

• Show, Don't Tell: Aligning Language Models with Demonstrated Feedback

  Omar Shaikh*, Michelle Lam*, Joey Hejna*, Yijia Shao, Michael Bernstein, Diyi Yang

  arXiv:2406.00888, 2024 [pdf]

• Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers

  Chenglei Si, Diyi Yang, Tatsunori Hashimoto

  arXiv:2409.04109, 2024 [pdf]

• Social Skill Training with Large Language Models

  Diyi Yang*, Caleb Ziems*, William Held*, Omar Shaikh*, Michael S Bernstein, John Mitchell

  arXiv:2404.04204, 2024 [pdf]

• Design2Code: How Far Are We From Automating Front-End Engineering

  Chenglei Si*, Yanzhe Zhang*, Zhengyuan Yang, Ruibo Liu, Diyi Yang

  arXiv:2403.03163, 2024 [pdf]

Publications [Google Scholar] [Show Selected] [Show All]

• DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph

  Zhehao Zhang, Jiaao Chen, Diyi Yang

  NeurIPS, 2024 [pdf]

• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action

  Yijia Shao, Tianshi Li, Weiyan Shi, Yanchen Liu, Diyi Yang

  NeurIPS D\&B, 2024 [pdf]

• Demystifying Verbatim Memorization in Large Language Models

  Jing Huang, Diyi Yang, Christopher Potts

  EMNLP, 2024 [pdf]

• CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies

  Weiyan Shi, Ryan Li, Yutong Zhang, Caleb Ziems, Chunhua yu, Raya Horesh, Rogério Abreu de Paula, Diyi Yang

  EMNLP Findings, 2024 [pdf]

• Unintended Impacts of LLM Alignment on Global Representation

  Michael J Ryan, William Held, Diyi Yang

  ACL, 2024 [pdf]

• Multi-Level Feedback Generation with Large Language Models for Empowering Novice Peer Counselors

  Alicja Chaszczewicz, Raj Sanjay Shah, Ryan Louie, Bruce A Arnow, Robert Kraut, Diyi Yang

  ACL, 2024 [pdf]

• Social Intelligence Data Infrastructure: Structuring the Present and Navigating the Future

  Minzhi Li, Weiyan Shi, Caleb Ziems, Diyi Yang

  ACL Findings, 2024 [pdf]

• Dynamic LLM-Agent Network: An LLM-agent Collaboration Framework with Agent Team Optimization

  Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, Diyi Yang

  COLM, 2024. [pdf]

• Rehearsal: Simulating Conflict to Teach Conflict Resolution

  Omar Shaikh, Valentino Chai, Michele J. Gelfand, Diyi Yang, Michael S. Bernstein

  CHI, 2024. [pdf]

• Grounding or Guesswork? Large Language Models Are Presumptive Grounders

  Omar Shaikh, Kristina Gligorić, Ashna Khetan, Matthias Gerstgrasser, Diyi Yang, Dan Jurafsky

  NAACL, 2024. [pdf]

• DyVal: Graph-informed Dynamic Evaluation of Large Language Models

  Kaijie Zhu, Jiaao Chen, Jindong Wang, Neil Zhenqiang Gong, Diyi Yang, Xing Xie

  ICLR, 2024. [pdf]

• Training socially aligned language models on simulated social interactions

  Ruibo Liu, Ruixin Yang, Chenyan Jia, Ge Zhang, Diyi Yang, Soroush Vosoughi

  ICLR, 2024. [pdf]

• Using Large Language Models in Psychology

  Dorottya Demszky*, Diyi Yang*, David S. Yeager*, Christopher J. Bryan, Margarett Clapper, Susannah Chandhok, Johannes C. Eichstaedt, Cameron Hecht, Jeremy Jamieson, Meghann Johnson, Michaela Jones, Danielle Krettek-Cobb, Leslie Lai, Nirel JonesMitchell, Desmond C. Ong, Carol S. Dweck, James J. Gross, James W. Pennebaker

  Nature Reviews Psychology, 2023. [pdf]

• Can Large Language Models Transform Computational Social Science?

  Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, Diyi Yang

  Computational Linguistics, 2023. [pdf]

• Unlearn What You Want to Forget: Efficient Unlearning for LLMs

  Jiaao Chen, Diyi Yang

  EMNLP, 2023. [pdf]

• CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations

  Myra Cheng, Tiziano Piccardi, Diyi Yang

  EMNLP, 2023. [pdf]

• DADA: Dialect Adaptation via Dynamic Aggregation of Linguistic Rules

  Yanchen Liu, Will Held, Diyi Yang

  EMNLP, 2023. [pdf]

• Is ChatGPT a General-Purpose Natural Language Processing Task Solver?

  Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, Diyi Yang

  EMNLP, 2023. [pdf]

• CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation

  Ella Li, Taiwei Shi, Caleb Ziems, Min-Yen Kan, Nancy F. Chen, Zhengyuan Liu, Diyi Yang

  EMNLP, 2023. [pdf]

• Impressions: Visual Semiotics and Aesthetic Impact Understanding

  Julia Kruk, Caleb Ziems, Diyi Yang

  EMNLP, 2023. [pdf]

• Generating and Evaluating Tests for K-12 Students with Language Model Simulations: A Case Study on Sentence Reading Efficiency

  Eric Zelikman, Wanjing Anya Ma, Jasmine Elizabeth Tran, Diyi Yang, Jason D Yeatman, Nick Haber

  EMNLP, 2023. [pdf]

• Understanding Black Content Creator Experiences on TikTok

  Camille Harris, Amber Gayle Johnson, Sadie Palmer, Diyi Yang, Amy Bruckman

  CSCW, 2023. [pdf]

  Recognition for Contribution to Diversity and Inclusion Award

• Multi-VALUE: A Framework for Cross-Dialectal English NLP

  Caleb Ziems, William Held, Jingfeng Yang, Jwala Dhamala, Rahul Gupta, Diyi Yang

  ACL, 2023. [pdf][website]

• NormBank: A Knowledge Bank of Situational Social Norms

  Caleb Ziems, Jane Dwivedi-Yu, Yi-Chia Wang, Alon Halevy, Diyi Yang

  ACL, 2023. [pdf]

• On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning

  Omar Shaikh, Hongxin Zhang, William Held, Michael S. Bernstein, Diyi Yang

  ACL, 2023. [pdf]

• Forgotten Knowledge: Examining the Citational Amnesia in NLP

  Janvijay Singh, Mukund Rungta, Diyi Yang, Saif M. Mohammad

  ACL, 2023. [pdf]

  Best Paper Honorable Mention

• Human-in-the-loop Abstractive Dialogue Summarization

  Jiaao Chen, Mohan Dodda, Diyi Yang

  ACL (Findings), 2023. [pdf]

• Task Agnostic Dialect Adapters for English

  William Held, Caleb Ziems, Diyi Yang

  ACL (Findings), 2023. [pdf]

• Modeling Cross-Cultural Pragmatic Inference with Codenames Duet

  Omar Shaikh, Caleb Ziems, William Held, Aryan J. Pariani, Fred Morstatter, Diyi Yang

  ACL (Findings), 2023. [pdf]

• Parameter-Efficient Fine-Tuning Design Spaces

  Jiaao Chen, Aston Zhang, Xingjian Shi, Mu Li, Alex Smola, Diyi Yang

  ICLR, 2023. [pdf]

• Shapley Head Pruning: Identifying and Removing Interference in Multilingual Transformers

  William Held, Diyi Yang

  EACL, 2023. [pdf]

• Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraints

  Albert Lu, Hongxin Zhang, Yanzhe Zhang, Xuezhi Wang, Diyi Yang

  EACL (Findings), 2023. [pdf]

• Metrics for Peer Counseling: Triangulating Success Outcomes for Online Therapy Platforms

  Tony Wang, Haard K Shah, Raj Sanjay Shah, Yi-Chia Wang, Robert Kraut, Diyi Yang

  SIGCHI, 2023. [pdf]

• An Empirical Survey of Data Augmentation for Limited Data Learning in NLP

  Jiaao Chen, Derek Tam, Colin Raffel, Mohit Bansal, Diyi Yang

  TACL. [pdf]

• Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond

  Amir Feder, Katherine A Keith, Emaad Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood-Doughty, Jacob Eisenstein, Justin Grimmer, Roi Reichart, Margaret E Roberts, Brandon M Stewart, Victor Veitch, Diyi Yang

  TACL. [pdf]

• Geographic Citation Gaps in NLP Research

  Mukund Rungta, Janvijay Singh, Saif M. Mohammad, Diyi Yang

  EMNLP, 2022. [pdf]

• When FLUE Meets FLANG: Benchmarks and Large Pretrained Language Model for Financial Domain

  Raj Sanjay Shah, Kunal Chawla, Dheeraj Eidnani, Agam Shah, Wendi Du, Sudheer Chava, Natraj Raman, Charese Smiley, Jiaao Chen, Diyi Yang

  EMNLP, 2022. [pdf]

• Robustness of Demonstration-based Learning Under Limited Data Scenario

  Hongxin Zhang, Yanzhe Zhang, Ruiyi Zhang, Diyi Yang

  EMNLP, 2022. [pdf]

• A Sketch Is Worth a Thousand Words: Image Retrieval with Text and Sketch

  Patsorn Sangkloy, Wittawat Jitkrittum, Diyi Yang, James Hays

  ECCV, 2022. [pdf]

• Modeling Motivational Interviewing Strategies On Online Peer-to-Peer Counseling Platforms

  Raj Sanjay Shah, Faye Holt, Shirley Anugrah Hayati, Aastha Agrawal, Yi-Chia Wang, Robert Kraut, Diyi Yang

  CSCW, 2022. [pdf]

• SUBS: Subtree Substitution for Compositional Semantic Parsing

  Jingfeng Yang, Le Zhang, Diyi Yang

  NAACL, 2022. [pdf]

• TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understanding

  Le Zhang, Zichao Yang, Diyi Yang

  NAACL, 2022. [pdf]

• Explaining Toxic Text via Knowledge Enhanced Text Generation

  Rohit Sridhar, Diyi Yang

  NAACL, 2022. [pdf]

• Measure and Improve Robustness in NLP Models: A Survey

  Xuezhi Wang, Haohan Wang, Diyi Yang

  NAACL, 2022. [pdf]

• Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models

  Tianlu Wang, Rohit Sridhar, Diyi Yang, Xuezhi Wang

  NAACL (Findings), 2022. [pdf]

• Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models

  Jingfeng Yang, Haoming Jiang, Qingyu Yin, Danqing Zhang, Bing Yin, Diyi Yang

  NAACL (Findings), 2022. [pdf]

• Exploring the Role of Grammar and Word Choice in Bias Toward AAVE in Hate Speech Classification

  Camille Harris, Matan Halevy, Ayanna Howard, Amy Bruckman, Diyi Yang

  FAccT, 2022. [pdf]

• VALUE: Understanding Dialect Disparity in NLU

  Caleb Ziems, Jiaao Chen, Camille Harris, Jessica Anderson, Diyi Yang

  ACL, 2022. [pdf] [slides]

• Inducing Positive Perspectives with Text Reframing

  Caleb Ziems, Minzhi Li, Anthony Zhang, Diyi Yang

  ACL, 2022. [pdf]

  Outstanding Paper Award

• Continual Sequence Generation with Adaptive Compositional Modules

  Yanzhe Zhang, Xuezhi Wang, Diyi Yang

  ACL, 2022. [pdf]

• Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems

  Caleb Ziems, Jane A. Yu, Yi-Chia Wang, Alon Halevy, Diyi Yang

  ACL, 2022. [pdf]

• Focus on the Action: Learning to Highlight and Summarize Jointly for Email To-Do Items Summarization

  Kexun Zhang, Jiaao Chen, Diyi Yang

  ACL (Findings), 2022. [pdf]

• FairytaleQA: An Authentic Dataset for Narrative Comprehension

  Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Jia-Jun Li, Nora Bradford, Branda Sun, Tran Bao Hoang, Yisi Sang, Yufang Hou, Xiaojuan Ma, Diyi Yang, Nanyun Peng, Zhou Yu, Mark Warschauer

  ACL, 2022. [pdf]

• GNN is a Counter? Revisiting GNN for Question Answering

  Kuan Wang, Yuyu Zhang, Diyi Yang, Le Song, Tao Qin

  ICLR, 2022. [pdf]

• Will AI Console Me when I Lose my Pet? Understanding Perceptions of AI-Mediated Email Writing

  Yihe Liu, Anushk Mittal, Diyi Yang, Amy Bruckman

  CHI, 2022. [pdf]

• Pretty Princess vs. Successful Leader: Gender Roles in Greeting Card Messages

  Jiao Sun, Tongshuang Wu, Yue Jiang, Ronil Awalegaonkar, Xi Victoria Lin and Diyi Yang

  CHI, 2022. [pdf]

  Best Paper Honorable Mention

• Linguistic Characterization of Divisive Topics Online: Case Studies on Contentiousness in Abortion, Climate Change, and Gun Control

  Jacob Beel, Tong Xiang, Sandeep Soni, Diyi Yang

  ICWSM 2022. [pdf]

• Simple Conversational Data Augmentation for Semi-supervised Abstractive Dialogue Summarization

  Jiaao Chen, Diyi Yang

  EMNLP, 2021. [pdf]

• Latent Hatred: A Benchmark for Understanding Implicit Hate Speech

  Mai ElSherief*, Caleb Ziems*, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Munmun De Choudhury, Diyi Yang

  EMNLP, 2021. [pdf]

• To Protect and To Serve? Analyzing Entity-Centric Framing of Police Violence

  Caleb Ziems, Diyi Yang

  EMNLP (Findings), 2021. [pdf]

• RECAST: Enabling User Recourse and Interpretability of Toxicity Detection Models with Interactive Visualization

  Austin P Wright, Omar Shaikh, Haekyu Park, Will Epperson, Muhammed Ahmed, Stephane Pinel, Duen Horng (Polo) Chau, Diyi Yang

  CSCW, 2021. [pdf]

• Evaluating the Effectiveness of Deplatforming as a Moderation Strategy on Twitter

  Shagun Jhaver, Christian Boylston, Diyi Yang, Amy Bruckman

  CSCW, 2021. [pdf]

  Best Paper Honorable Mention

• Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Framework

  Matan Halevy, Camille Harris, Amy Bruckman, Diyi Yang, Ayanna Howard

  ACM conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO), 2021. [pdf]

  Best Student Paper Award

• Understanding the Usage of Online Media for Parenting from Infancy to Preschool At Scale

  Yujia Gao, Jinu Jang, Diyi Yang

  SIGCHI, 2021. [pdf]

  Best Paper Honorable Mention

• HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalizability

  Jiaao Chen, Dinghan Shen, Weizhu Chen, Diyi Yang

  ACL, 2021. [pdf]

• A Dataset for Understanding Disfluencies in Question Answering

  Aditya Gupta, Jiacheng Xu, Shyam Upadhyay, Diyi Yang, Manaal Faruqui

  ACL (Findings), 2021. [pdf]

• Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs

  Jiaao Chen, Diyi Yang

  NAACL, 2021. [pdf]

• Continual Learning for Text Classification with Information Disentanglement Based Regularization

  Yufan Huang, Yanzhe Zhang, Jiaao Chen, Xuezhi Wang and Diyi Yang

  NAACL, 2021. [pdf]

• Personalized Response Generation via Generative Split Memory Network

  Yuwei Wu, Xuezhe Ma,Diyi Yang

  NAACL, 2021. [pdf]

• The Importance of Modeling Social Factors of Language: Theory and Practice

  Dirk Hovy, Diyi Yang

  NAACL, 2021. [pdf]

• Weakly-Supervised Hierarchical Models for Predicting Persuasive Strategies in Good-faith Textual Requests

  Jiaao Chen, Diyi Yang

  AAAI, 2021. [pdf]

• Tuiteamos o pongamos un tuit? Investigating the Social Constraints of Loanword Integration in Spanish Social Media

  Ian Stewart, Diyi Yang, Jacob Eisenstein

  The Society for Computation in Linguistics (SCiL), 2021. [pdf]

• Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization

  Jiaao Chen, Diyi Yang

  EMNLP, 2020. [pdf] [news]

• Local Additivity Based Data Augmentation for Semi-supervised NER

  Jiaao Chen*, Zhenghui Wang*, Ran Tian, Zichao Yang and Diyi Yang

  EMNLP, 2020. [pdf]

• Planning and Generating Natural and Diverse Disfluent Texts as Augmentation for Disfluency Detection

  Jingfeng Yang, Zhaoran Ma, and Diyi Yang

  EMNLP, 2020. [pdf]

• ToTTo: A Controlled Table-To-Text Generation Dataset

  Ankur P. Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das

  EMNLP, 2020. [pdf]

• Examining the Ordering of Rhetorical Strategies in Persuasive Requests

  Omar Shaikh, Jiaao Chen, Jon Saad-Falcon, Polo Chau and Diyi Yang

  EMNLP (Findings), 2020. [pdf][news]

• MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification

  Jiaao Chen, Zichao Yang, and Diyi Yang

  ACL, 2020, [pdf]

• Characterizing Collective Attention via Descriptor Context: A Case Study of Public Discussions of Crisis Events

  Ian Stewart, Diyi Yang, and Jacob Eisenstein

  ICWSM, 2020, [pdf]

• Automatically Neutralizing Subjective Bias in Text

  Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang

  AAAI, 2020, oral [pdf]

• Successful Online Socialization: Lessons from the Wikipedia Education Program

  Ang Li, Zheng Yao, Diyi Yang, Chinmay Kulkarni, Rosta Farzan, Robert Kraut

  CSCW, 2020 [pdf]

  Best Paper Honorable Mention

• Multi-level Modeling of Social Roles in Online Micro-lending Platforms

  Lu Sun, Robert Kraut, and Diyi Yang

  CSCW, 2019 [pdf]

• Modeling Persuasive Strategies via Semi-Supervised Neural Nets on Crowdfunding Platforms

  Diyi Yang*, Jiaao Chen*, Zichao Yang, Dan Jurafsky, and Eduard Hovy

  NAACL, 2019, oral [pdf]

• Seekers, Providers, Welcomers, and Storytellers: Modeling Social Roles in Online Health Communities

  Diyi Yang, Robert Kraut, Tenbroeck Smith, Elijah Mayfield, and Dan Jurafsky

  CHI, 2019 [pdf] [supplement]

  Best Paper Honorable Mention

• The Channel Matters: Self-disclosure, Reciprocity and Social Support in Online Cancer Support Groups

  Diyi Yang, Zheng Yao, Joseph Seering, and Robert Kraut

  CHI, 2019 [pdf]

  Best Paper Honorable Mention

• Persuading Teammates to Give: Systematic versus Heuristic Cues for Soliciting Loans

  Diyi Yang and Robert Kraut

  CSCW, 2018. [pdf]

• Identifying Semantic Edit Intentions from Revisions in Wikipedia

  Diyi Yang, Aaron Halfaker, Robert Kraut and Eduard Hovy

  EMNLP, 2017. [pdf] [data]

• Commitment of Newcomers and Old-timers to Online Health Support Communities

  Diyi Yang, Robert Kraut and John Levine

  CHI, 2017. [pdf]

• Who does What: Editor Role Identification in Wikipedia

  Diyi Yang, Aaron Halfaker, Robert Kraut and Eduard Hovy

  ICWSM, 2016. [pdf]

  Best Paper Honorable Mention

  In the news: [CMU LTI], [Wikimedia Newsletter]

• Hierarchical Attention Networks for Document Classification

  Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola and Eduard Hovy

  NAACL, 2016. [pdf]

• Exploring the Effect of Student Confusion in Massive Open Online Courses

  Diyi Yang, Robert Kraut and Carolyn Rose

  Journal of Educational Data Mining (JEDM). [pdf]

• Humor Recognition and Humor Anchor Extraction

  Diyi Yang, Alon Lavie, Chris Dyer and Eduard Hovy

  EMNLP 2015, oral. [pdf]

• Weakly Supervised Role Identification in Teamwork Interactions

  Diyi Yang, Miaomiao Wen, Carolyn Rosé

  ACL 2015, oral. [pdf]

• That's So Annoying!!!: A Lexical and Frame-Semantic Embedding Based Data Augmentation Approach to Automatic Categorization of Annoying Behaviors using #petpeeve Tweets

  William Yang Wang, Diyi Yang

  EMNLP 2015. [pdf] [data]

  Notable Data Set Award

• Incorporating Word Correlation Knowledge into Topic Modeling

  Pengtao Xie, Diyi Yang, Eric Xing

  NAACL, 2015. [pdf]

• Local Implicit Feedback Mining for Music Recommendation

  Diyi Yang, Tianqi Chen, Weinan Zhang, Qiuxia Lu, Yong Yu

  RecSys 2012. [pdf]

Workshops and Posters

• Personalized Response Generation with Tensor Factorization

  Zhenghui Wang, Lingxiao Luo, Diyi Yang

  The GEM workshop at ACL 2021. [pdf]

• The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics

  The GEM Team

  The GEM workshop at ACL 2021. [pdf]

• Putting Humans in the Natural Language Processing Loop: A Survey

  Zijie Wang, Dongjin Choi, Shenyu Xu, Diyi Yang

  HCI+NLP workshop at EACL 2021. [pdf]

• This is a Problem, Don’t You Agree? Framing and Bias in Human Evaluation for Natural Language Generation

  Stephanie Schoch, Diyi Yang, Yangfeng Ji

  Workshop on Evaluating NLG Evaluation at INLG 2020. [pdf]