Lead AI Research Scientist, Zoom Ph.D., Computer Science Department, Stanford University M.S., Statistics Department, Stanford University B. Eng., Computer Science & Technology Department, Tsinghua University Email: A [at] B, where A=zcg.stanford and B is gmail.com [ Google Scholar | Books | Publications | Patents | Tutorials and Workshops | Talks ]

Bio

 I am a Lead AI Research Scientist at Zoom. My research directions are 1) Knowledge-enhanced Language Model, 2) Few-shot and Prompt Learning, 3) Responsible AI, and 4) Multimodal Learning. I develop state-of-the-art deep learning technologies for both research and business applications. My previous work has achieved first places in multiple NLP competitions, including achieving human parity in CommonsenseQA, HellaSwag and CoQA, 1st places in CommonGen, FEVER, ARC and SQuAD v1.0. Before Zoom, I was Principal Research Scienest at Microsoft. I got my PhD in Computer Science Department, Stanford University and Bachelor in Computer Science from Yao Class, Tsinghua University.

News

 Sep. 21, 2023: 2 papers accepted at NeurIPS 2023.

 May 5, 2023: Prof. Diyi Yang from Stanford University and I gave the tutorial on Summarization of Dialogues and Conversations At Scale at EACL 2023.

 May 1, 2023: 3 papers accepted at ACL 2023.

 Apr. 24, 2023: 1 paper accepted at ICML 2023.

 Feb. 28, 2023: 2 papers accepted at CVPR 2023.

 Feb. 27, 2023: I gave the talk "How We Achieved Human Parity in CommonsenseQA – Fusing Knowledge into Language Models" at Singapore Management University. [ Slides ]

 Feb. 27, 2023: We gave the tutorial on Knowledge-Augmented Methods for Natural Language Processing at WSDM 2023. [ Website | Video ]

 Feb. 23, 2023: 1 paper accepted at TACL.

 My recent work covers:

• Prompt optimization in a gradient descent style [ Paper ]

• Leverage GPT-4 to build SoTA NLG evaluator [ Paper ]

• The "Impossible Triangle" of pre-trained language models [ Paper ]

• Integrative multimodal learning framework i-Code [ i-Code v1 | i-Code v2 | i-Code v3 | i-Code Doc | i-Code Studio ]

Books

	Machine Reading Comprehension: Algorithm and Practice (Chinese Edition) 《机器阅读理解：算法与实践》 Chenguang Zhu China Machine Press (机械工业出版社) , 2020.03 Top 5 Favorite IT Books (Artificial Intelligence) in 2020 by 51CTO.com [ Link ] [ Amazon.com | China-pub | jd.com | dangdang.com | tmall.com | Amazon.cn ] [ GitHub Code ]
	Machine Reading Comprehension: Algorithm and Practice Chenguang Zhu Elsevier, 2021.04 [ Amazon.com | Google Books | Barnes & Noble ] [ GitHub Code ]

Publication

2023

• Any-to-Any Generation via Composable Diffusion
  Zineng Tang, Ziyi Yang, Chenguang Zhu, Michael Zeng, Mohit Bansal
  Conference on Neural Information Processing Systems (NeurIPS), New Orleans, Louisiana, USA, 2023.
  [ arXiv | Blog | Demo ]
  

• Sparse Modular Activation for Efficient Sequence Modeling
  Liliang Ren, Yang Liu, Shuohang Wang, Yichong Xu, Chenguang Zhu, ChengXiang Zhai
  Conference on Neural Information Processing Systems (NeurIPS), New Orleans, Louisiana, USA, 2023.
  [ arXiv ]
  

• UniSumm: Unified Few-shot Summarization with Multi-Task Pre-Training and Prefix-Tuning
  Yulong Chen, Yang Liu, Ruochen Xu, Ziyi Yang, Chenguang Zhu, Michael Zeng, Yue Zhang
  Association for Computational Linguistics (ACL), Toronto, Canada, 2023.
  [ arXiv | Code ]
  

• Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization
  Pengcheng He, Baolin Peng, Liyang Lu, Song Wang, Jie Mei, Yang Liu, Ruochen Xu, Hany Hassan Awadalla, Yu Shi, Chenguang Zhu, Wayne Xiong, Michael Zeng, Jianfeng Gao, Xuedong Huang
  Association for Computational Linguistics (ACL), Toronto, Canada, 2023.
  [ arXiv ]
  

• APOLLO: A Simple Approach for Adaptive Pre-training of Language Models for Logical Reasoning
  Soumya Sanyal, Yichong Xu, Shuohang Wang, Ziyi Yang, Reid Pryzant, Wenhao Yu, Chenguang Zhu, Xiang Ren
  Association for Computational Linguistics (ACL), Toronto, Canada, 2023.
  Conference on Neural Information Processing Systems (NeurIPS), Workshop on Distribution Shifts: Connecting Methods and Applications, New Orleans, Louisiana, USA, 2022.
  [ arXiv ]
  

• Global Selection of Contrastive Batches via Optimization on Sample Permutations
  Vin Sachidananda, Ziyi Yang, Chenguang Zhu
  The 40th International Conference on Machine Learning (ICML), 2023.
  [ arXiv ]
  

• Unifying Vision, Text, and Layout for Universal Document Processing
  Zineng Tang, Ziyi Yang, Guoxin Wang, Yuwei Fang, Yang Liu, Chenguang Zhu, Michael Zeng, Cha Zhang, Mohit Bansal
  Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, 2023. (Highlight paper, acceptance rate: 2.5%)
  [ arXiv ]
  

• Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles
  Shuquan Ye, Yujia Xie, Dongdong Chen, Yichong Xu, Lu Yuan, Chenguang Zhu, Jing Liao
  Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, 2023. (Highlight paper, acceptance rate: 2.5%)
  [ arXiv ]
  

• MACSum: Controllable Summarization with Mixed Attributes
  Yusen Zhang, Yang Liu, Ziyi Yang, Yuwei Fang, Yulong Chen, Dragomir Radev, Chenguang Zhu, Michael Zeng, Rui Zhang
  Transactions of the Association for Computational Linguistics (TACL), 2023.
  [ arXiv ]
  

• Generate rather than Retrieve: Large Language Models are Strong Context Generators
  Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, Meng Jiang
  International Conference on Learning Representations (ICLR), Kigali, Rwanda, 2023.
  [ arXiv | Code ]
  

• i-Code: An Integrative and Composable Multimodal Learning Framework
  Ziyi Yang, Yuwei Fang, Chenguang Zhu, Reid Pryzant, Dongdong Chen, Yu Shi, Yichong Xu, Yao Qian, Mei Gao, Yi-Ling Chen, Liyang Lu, Yujia Xie, Robert Gmyr, Noel Codella, Naoyuki Kanda, Bin Xiao, Yuan Lu, Takuya Yoshioka, Michael Zeng, Xuedong Huang
  The 37th AAAI Conference on Artificial Intelligence (AAAI), Washington DC, USA, 2023.
  [ arXiv | AI科技评论 | Synced Review | MarkTechPost ]

• Fine-Tuning Language Models with Advantage-Induced Policy Alignment
  Banghua Zhu, Hiteshi Sharma, Felipe Vieira Frujeri, Shi Dong, Chenguang Zhu, Michael I. Jordan, Jiantao Jiao
  arXiv preprint arXiv: 2306.02231, 2023.
  

• In-Context Demonstration Selection with Cross Entropy Difference
  Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu
  arXiv preprint arXiv: 2305.14726, 2023.
  

• PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents
  Simeng Sun, Yang Liu, Shuohang Wang, Chenguang Zhu, Mohit Iyyer
  arXiv preprint arXiv: 2305.14564, 2023.
  

• i-Code Studio: A Configurable and Composable Framework for Integrative AI
  Yuwei Fang, Mahmoud Khademi, Chenguang Zhu, Ziyi Yang, Reid Pryzant, Yichong Xu, Yao Qian, Takuya Yoshioka, Lu Yuan, Michael Zeng, Xuedong Huang
  arXiv preprint arXiv: 2305.13738, 2023.
  [ Demo ]
  

• LMGQS: A Large-scale Dataset for Query-focused Summarization
  Ruochen Xu, Song Wang, Yang Liu, Shuohang Wang, Yichong Xu, Dan Iter, Chenguang Zhu, Michael Zeng
  arXiv preprint arXiv: 2305.13086, 2023.
  

• InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT
  Yichong Xu, Ruochen Xu, Dan Iter, Yang Liu, Shuohang Wang, Chenguang Zhu, Michael Zeng
  arXiv preprint arXiv: 2305.13083, 2023.
  

• i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data
  Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Mei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang
  arXiv preprint arXiv: 2305.12311, 2023.
  

• Small Models are Valuable Plug-ins for Large Language Models
  Canwen Xu, Yichong Xu, Shuohang Wang, Yang Liu, Chenguang Zhu, Julian McAuley
  arXiv preprint arXiv: 2305.08848, 2023.
  

• Automatic Prompt Optimization with “Gradient Descent” and Beam Search
  Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, Michael Zeng
  arXiv preprint arXiv: 2305.03495, 2023.
  [ Synced Review ]

• G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
  Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, Chenguang Zhu
  arXiv preprint arXiv: 2303.16634, 2023.
  

• How Does In-Context Learning Help Prompt Tuning?
  Simeng Sun, Yang Liu, Dan Iter, Chenguang Zhu, Mohit Iyyer
  arXiv preprint arXiv: 2302.11521, 2023.
  

2022

• Towards A Unified Multi-Dimensional Evaluator For Text Generation
  Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji and Jiawei Han
  Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022.
  [ arXiv ]
  

• A Unified Encoder-Decoder Framework with Entity Memory
  Zhihan Zhang, Wenhao Yu, Chenguang Zhu, Meng Jiang
  Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022.
  [ arXiv ]
  

• Retrieval Augmentation for Commonsense Reasoning: A Unified Approach
  Wenhao Yu, Chenguang Zhu, Zhihan Zhang, Shuohang Wang, Zhuosheng Zhang, Yuwei Fang and Meng Jiang
  Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022.
  [ arXiv ]
  

• Empowering Language Models with Knowledge Graph Reasoning for Open-Domain Question Answering
  Ziniu Hu, Yichong Xu, Wenhao Yu, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Kai-Wei Chang and Yizhou Sun
  Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022.
  Best Paper Award at SoCalNLP 2022 Symposium
  [ arXiv ]
  

• ParaTag: A Dataset of Paraphrase Tagging for Fine-Grained Labels, NLG Evaluation, and Data Augmentation
  Shuohang Wang, Ruochen Xu, Yang Liu, Chenguang Zhu, Michael Zeng
  Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022.
  

• Leveraging Locality in Abstractive Text Summarization
  Yixin Liu, Ansong Ni, Linyong Nan, Budhaditya Deb, Chenguang Zhu, Ahmed H. Awadallah, Dragomir Radev
  Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022.
  [ arXiv ]
  

• Automatic Rule Induction for Efficient Semi-Supervised Learning
  Reid Pryzant, Ziyi Yang, Yichong Xu, Chenguang Zhu, Michael Zeng
  Findings of Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022.
  [ arXiv ]
  

• Narrate Dialogues for Better Summarization
  Ruochen Xu, Chenguang Zhu, Michael Zeng
  Findings of Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022.
  

• AdaPrompt: Adaptive Model Training for Prompt-based NLP
  Yulong Chen, Yang Liu, Li Dong, Shuohang Wang, Chenguang Zhu, Michael Zeng, Yue Zhang
  Findings of Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022.
  [ arXiv ]
  

• Unsupervised Summarization with Customized Granularities
  Ming Zhong, Yang Liu, Suyu Ge, Yuning Mao, Yizhu Jiao, Xingxing Zhang, Yichong Xu, Chenguang Zhu, Michael Zeng, Jiawei Han
  Findings of Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022.
  [ arXiv ]
  

• Task Compass: Scaling Multi-task Pre-training with Task Prefix
  Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu, Michael Zeng
  Findings of Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022.
  [ arXiv | Code ]
  

• Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
  Zhenhailong Wang, Manling Li, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Derek Hoiem, Shih-Fu Chang, Mohit Bansal, Heng Ji
  Conference on Neural Information Processing Systems (NeurIPS), New Orleans, Louisiana, USA, 2022.
  [ arXiv ]
  

• REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answering
  Yuanze Lin, Yujia Xie, Dongdong Chen, Yichong Xu, Chenguang Zhu, Lu Yuan
  Conference on Neural Information Processing Systems (NeurIPS), New Orleans, Louisiana, USA, 2022.
  [ arXiv ]
  

• Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
  Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang
  The 31st International Joint Conference on Artificial Intelligence (IJCAI), Vienna, Austria, 2022.
  [ arXiv | Code | Blog ]
  [ Human parity in CommonsenseQA leaderboard, 2021.11.12 ]

• Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
  Shuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu, Siqi Sun, Ruochen Xu, Chenguang Zhu, Michael Zeng
  Association for Computational Linguistics (ACL), Dublin, Ireland, 2022.
  [ arXiv | CSDN ]

• KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering
  Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng
  Association for Computational Linguistics (ACL), Dublin, Ireland, 2022.
  [ arXiv ]

• Leveraging Visual Knowledge in Language Tasks: An Empirical Study on Intermediate Pre-training for Cross-Modal Knowledge Transfer
  Woojeong Jin, Dong-Ho Lee, Chenguang Zhu, Jay Pujara, Xiang Ren
  Association for Computational Linguistics (ACL), Dublin, Ireland, 2022.
  

• DYLE: Dynamic Latent Extraction for Abstractive Long-Input Summarization
  Ziming Mao, Chen Henry Wu, Ansong Ni, Yusen Zhang, Rui Zhang, Tao Yu, Budhaditya Deb, Chenguang Zhu, Ahmed H. Awadallah, Dragomir Radev
  Association for Computational Linguistics (ACL), Dublin, Ireland, 2022.
  [ arXiv ]

• Summ^N: A Multi-Stage Summarization Framework for Long Input Dialogues and Documents
  Yusen Zhang, Ansong Ni, Ziming Mao, Chen Henry Wu, Chenguang Zhu, Budhaditya Deb, Ahmed H. Awadallah, Dragomir Radev, Rui Zhang
  Association for Computational Linguistics (ACL), Dublin, Ireland, 2022.
  [ arXiv ]

• Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts
  Wenhao Yu, Chenguang Zhu, Lianhui Qin, Zhihan Zhang, Tong Zhao, Meng Jiang
  Findings of Association for Computational Linguistics (ACL), Dublin, Ireland, 2022.
  [ arXiv ]

• Leveraging Knowledge in Multilingual Commonsense Reasoning
  Yuwei Fang, Shuohang Wang, Yichong Xu, Ruochen Xu, Siqi Sun, Chenguang Zhu, Michael Zeng
  Findings of Association for Computational Linguistics (ACL), Dublin, Ireland, 2022.
  [ arXiv ]

• Dict-BERT: Enhancing Language Model Pre-training with Dictionary
  Wenhao Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng, Meng Jiang
  Findings of Association for Computational Linguistics (ACL), Dublin, Ireland, 2022.
  [ arXiv ]

• End-to-End Segmentation-based News Summarization
  Yang Liu, Chenguang Zhu, Michael Zeng
  Findings of Association for Computational Linguistics (ACL), Dublin, Ireland, 2022.
  [ arXiv ]
  

• CLIP-Event: Connecting Text and Images with Event Structures
  Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, Shih-Fu Chang
  Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, Louisiana, USA, 2022. (Oral)
  [ arXiv ]
  

• An Empirical Study of Training End-to-End Vision-and-Language Transformers
  Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang, Lijuan Wang, Chenguang Zhu, Nanyun (Violet) Peng, Zicheng Liu, Michael Zeng
  Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, Louisiana, USA, 2022.
  [ arXiv ]
  

• A Survey of Knowledge-Enhanced Text Generation
  Wenhao Yu, Chenguang Zhu, Zaitang Li, Zhiting Hu, Qingyun Wang, Heng Ji, Meng Jiang
  ACM Computing Surveys (Impact factor: 10.282)
  [ arXiv ]
  

• JAKET: Joint Pre-training of Knowledge Graph and Language Understanding
  Donghan Yu*, Chenguang Zhu*, Yiming Yang, Michael Zeng
  (*: Equal contribution)
  36th AAAI Conference on Artificial Intelligence (AAAI), 2022.
  [ arXiv ]

• DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization
  Ming Zhong, Yang Liu, Yichong Xu, Chenguang Zhu, Michael Zeng
  36th AAAI Conference on Artificial Intelligence (AAAI), 2022.
  [ arXiv ]

• FAST: Improving Controllability for Text Generation with Feedback Aware Self-Training
  Junyi Chai, Reid Pryzant, Victor Ye Dong, Konstantin Golobokov, Chenguang Zhu, Yi Liu
  arXiv preprint arXiv: 2210.03167, 2022.
  

• Impossible Triangle: What’s Next for Pre-trained Language Models?
  Chenguang Zhu, Michael Zeng
  arXiv preprint arXiv: 2204.06130, 2022.
  [ 机器之心 | AI科技评论 ]
  

2021

• Want To Reduce Labeling Cost? GPT-3 Can Help
  Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu and Michael Zeng
  Findings of Empirical Methods in Natural Language Processing (EMNLP), Punta Cana, Dominican Republic, 2021.
  [ arXiv ]
  

• Sentence-Permuted Paragraph Generation
  Wenhao Yu, Chenguang Zhu, Tong Zhao, Zhichun Guo, Meng Jiang
  Empirical Methods in Natural Language Processing (EMNLP), Punta Cana, Dominican Republic, 2021.
  [ arXiv ]
  

• Injecting Entity Types into Entity-Guided News Generation
  Xiangyu Dong*, Wenhao Yu*, Chenguang Zhu and Meng Jiang
  (*: Equal contribution)
  Empirical Methods in Natural Language Processing (EMNLP), Punta Cana, Dominican Republic, 2021.
  [ arXiv ]
  

• An Exploratory Study on Long Dialogue Summarization: What Works and What's Next
  Yusen Zhang*, Ansong Ni*, Tao Yu, Rui Zhang, Chenguang Zhu, Budhaditya Deb, Asli Celikyilmaz, Ahmed Hassan Awadallah and Dragomir Radev
  (*: Equal contribution)
  Findings of Empirical Methods in Natural Language Processing (EMNLP), Punta Cana, Dominican Republic, 2021.
  [ arXiv ]
  

• Modeling Entity Knowledge for Fact Verification
  Yang Liu, Cenguang Zhu, Michael Zeng
  Fact Extraction and VERification Workshop (FEVER) in Empirical Methods in Natural Language Processing (EMNLP), Punta Cana, Dominican Republic, 2021.
  

• RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems
  [ Leaderboard | arXiv ]
  Baolin Peng, Chunyuan Li, Zhu Zhang, Chenguang Zhu, Jinchao Li, Jianfeng Gao
  Association for Computational Linguistics (ACL), Bangkok, Thailand, 2021.
  

• Fusing Context Into Knowledge Graph for Commonsense Reasoning
  Yichong Xu*, Chenguang Zhu*, Ruochen Xu, Yang Liu, Michael Zeng, Xuedong Huang
  (*: Equal contribution)
  Findings of Association for Computational Linguistics (ACL), Bangkok, Thailand, 2021.
  [ arXiv ]

• Retrieval Enhanced Model for Commonsense Generation
  Han Wang, Yang Liu, Chenguang Zhu, Linjun Shou, Ming Gong, Yichong Xu, Michael Zeng
  Findings of Association for Computational Linguistics (ACL), Bangkok, Thailand, 2021.
  [ arXiv | 1st place on CommonGen leaderboard, 2021.01.13]

• Leveraging Lead Bias for Zero-shot Abstractive News Summarization
  Chenguang Zhu, Ziyi Yang, Robert Gmyr, Michael Zeng, Xuedong Huang
  The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), Montreal, Canada, 2021.
  [NIPS 2020 Self-Supervised Learning Workshop version | NIPS Poster | arXiv | Talk ]
  

• Enhancing Factual Consistency of Abstractive Summarization
  Chenguang Zhu, William Hinthorn, Ruochen Xu, Qingkai Zeng, Michael Zeng, Xuedong Huang, Meng Jiang
  North American Chapter of the Association for Computational Linguistics (NAACL), Mexico City, Mexico, 2021.
  [ arXiv | Predictions | Talk ]
  

• MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization
  Chenguang Zhu*, Yang Liu*, Jie Mei and Michael Zeng
  (*: Equal contribution)
  North American Chapter of the Association for Computational Linguistics (NAACL), Mexico City, Mexico, 2021.
  [ arXiv | Hugging Face | GitHub | Talk ]

• SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding
  Yu-An Chung*, Chenguang Zhu*, Michael Zeng
  (*: Equal contribution)
  North American Chapter of the Association for Computational Linguistics (NAACL), Mexico City, Mexico, 2021.
  [ arXiv ]
  

• Filtered Inner Product Projection for Multilingual Embedding Alignment
  Vin Sachidananda, Ziyi Yang, Chenguang Zhu
  International Conference on Learning Representations (ICLR), Vienna, Austria, 2021.
  [ arXiv | Code ]

• Data Augmentation for Spoken Language Understanding via Pretrained Models
  Baolin Peng^∗, Chenguang Zhu^∗, Michael Zeng, Jianfeng Gao
  INTERSPEECH, Brno, Czechia, 2021.
  (*: Equal contribution)
  [ arXiv ]

• MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  Yixin Nie*, Linjie Li*, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang (*: Equal contribution)
  arXiv preprint arXiv: 2112.04453, 2021.

• Does Knowledge Help General NLU? An Empirical Study
  Ruochen Xu*, Yuwei Fang*, Chenguang Zhu, Michael Zeng
  (*: Equal contribution)
  arXiv preprint arXiv: 2109.00563, 2021.

2020

• TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising
  [ arXiv ]
  Ziyi Yang*, Chenguang Zhu*, Robert Gmyr, Michael Zeng, Xuedong Huang, Eric Darve
  (*: Equal contribution)
  Findings of Empirical Methods in Natural Language Processing (EMNLP), 2020.
  

• A Hierarchical Network for Abstractive Meeting Summarization with Cross-Domain Pretraining
  [ arXiv | Talk | Code ]
  Chenguang Zhu*, Ruochen Xu*, Michael Zeng, Xuedong Huang
  (*: Equal contribution)
  Findings of Empirical Methods in Natural Language Processing (EMNLP), 2020.
  

• Few-shot Natural Language Generation for Task-Oriented Dialog
  [ arXiv | Code & Demo ]
  Baolin Peng, Chenguang Zhu, Chunyuan Li, Xiujun Li, Jinchao Li, Michael Zeng, Jianfeng Gao
  Findings of Empirical Methods in Natural Language Processing (EMNLP), 2020.
  

• Mixed-Lingual Pre-training for Cross-lingual Summarization
  [ arXiv ]
  Ruochen Xu*, Chenguang Zhu*, Yu Shi, Michael Zeng, Xuedong Huang
  (*: Equal contribution)
  Asia-Pacific Chapter of the Association for Computational Linguistics (AACL), Suzhou, China, 2020.
  

• Boosting Naturalness of Language in Task-oriented Dialogues via Adversarial Training
  [ Talk ]
  Chenguang Zhu
  Special Interest Group on Discourse and Dialogue (SIGdial), Boise, Idaho, 2020.

• Text Summarization
  文本摘要：浓缩的才是精华
  Chenguang Zhu, Nanshan Zeng
  Communications of the China Computer Federation (CCCF), 《中国计算机学会通讯》，Jul. 2020.

• Accelerating Real-Time Question Answering via Question Generation
  Yuwei Fang, Shuohang Wang, Zhe Gan, Siqi Sun, Jingjing Liu, Chenguang Zhu
  arXiv preprint arXiv: 2009.05167, 2020.

• Meta Dialogue Policy Learning
  Yumo Xu, Chenguang Zhu, Baolin Peng, Michael Zeng
  arXiv preprint arXiv: 2006.02588, 2020.

2019

• Multi-task Learning for Natural Language Generation in Task-Oriented Dialogue
  [ Poster ]
  Chenguang Zhu, Michael Zeng, Xuedong Huang
  Empirical Methods in Natural Language Processing (EMNLP), Hong Kong, China, 2019.

• Parameter-free Sentence Embedding via Orthogonal Basis
  [ Code | Slides | Talk ]
  Ziyi Yang, Chenguang Zhu, Weizhu Chen
  Empirical Methods in Natural Language Processing (EMNLP), Hong Kong, China, 2019.
  

• Embedding Imputation with Grounded Language Information
  [ Poster | Code ]
  Ziyi Yang, Chenguang Zhu, Vin Sachidananda, Eric Darve
  Association for Computational Linguistics (ACL), Florence, Italy, 2019.

• Learning to Attend On Essential Terms: An Enhanced Retriever-Reader Model for Open-domain Question Answering
  [ Code | Poster ]
  Jianmo Ni, Chenguang Zhu, Weizhu Chen, Julian McAuley.
  North American Chapter of the Association for Computational Linguistics (NAACL), Minneapolis, USA, 2019.

• SIM: A Slot-Independent Neural Model for Dialogue State Tracking
  [ Talk at Stanford HAI OVAL | Poster ]
  Chenguang Zhu, Michael Zeng, Xuedong Huang
  Special Interest Group on Discourse and Dialogue (SIGdial), Stockholm, Sweden, 2019.

• Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarization
  [ arXiv | Poster ]
  Beliz Gunel, Chenguang Zhu, Michael Zeng, Xuedong Huang
  Conference on Neural Information Processing Systems (NeurIPS), Knowledge Representation & Reasoning Meets Machine Learning (KR2ML workshop), Vancouver, Canada, 2019.

• Machine Reading Comprehension: How to Make Computer Understand Articles
  机器阅读理解：如何让计算机读懂文章
  Chenguang Zhu
  Communications of the China Computer Federation (CCCF), 《中国计算机学会通讯》，Feb. 2019.

2018

• FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension
  [ arXiv | Code | Poster ]
  Hsin-Yuan Huang, Chenguang Zhu, Yelong Shen, Weizhu Chen.
  International Conference on Learning Representations (ICLR), Vancouver, Canada, 2018.

• SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering
  [ Code ]
  Chenguang Zhu, Michael Zeng, Xuedong Huang.
  arXiv preprint arXiv: 1812.03593, 2018.

2017

• Measuring the Pulse of a City Via Taxi Operation: A Case Study
  Chenguang Zhu, Balaji Prabhakar.
  Transportation Research Board 96th Annual Meeting (TRB), Washington, D.C., 2017.

• Reducing Inefficiencies in Taxi Systems
  Chenguang Zhu, Balaji Prabhakar.
  56th IEEE Conference on Decision and Control (CDC), Melbourne, Australia, 2017.

2015

• Reducing Road Congestion Through Incentives: A Case Study
  Chenguang Zhu, Jia Shuo Yue, Chinmoy V. Mandayam, Deepak Merugu, Hossein Karkeh Abadi, Balaji Prabhakar.
  Transportation Research Board 94th Annual Meeting (TRB), Washington, D.C., 2015.
  Featured on The New York Times, The Wall Street Journal, International Business Times, Ars Technica and Stanford News

• Polling One's Friends: A Graph Theoretic View
  Chenguang Zhu, Hossein Karkeh Abadi, Balaji Prabhakar.
  53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2015.

Earlier

• Information Diffusion and External Influence in Networks
  Seth A. Myers, Chenguang Zhu, Jure Leskovec.
  ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2012

• A Novel Click Model and Its Applications to Online Advertising
  Zeyuan Allen Zhu, Weizhu Chen, Tom Minka, Chenguang Zhu, Zheng Chen.
  ACM International Conference on Web Search and Data Mining (WSDM), 2010

• A General Magnitude-Preserving Boosting Algorithm for Search Ranking
  Chenguang Zhu, Weizhu Chen, Zeyuan Allen Zhu, Gang Wang, Dong Wang, Zheng Chen.
  ACM Conference on Information and Knowledge Management (CIKM), 2009

• Inverse Time Dependency in Convex Regularized Learning
  Zeyuan Allen Zhu, Weizhu Chen, Chenguang Zhu, Gang Wang, Haixun Wang, Zheng Chen.
  IEEE International Conference of Data Mining (ICDM), 2009 Best Student Paper Award Runner-Up

• P-packSVM: Parallel Primal Gradient Descent Kernel SVM
  Zeyuan Allen Zhu, Weizhu Chen, Gang Wang, Chenguang Zhu, Zheng Chen.
  IEEE International Conference of Data Mining (ICDM), 2009

Patents

• Systems and Methods for Facilitating Integrative, Extensible, Composable, and Interpretable Deep Learning (US 20230229960 A1)
  Chenguang Zhu, Lu Yuan, Yao Qian, Yu Shi, Nanshan Zeng, Xuedong Huang

• Automated Meeting Minutes Generator (US 20230205985 A1)
  Chenguang Zhu, Yu Shi, William Isaac Hinthorn, Nanshan Zeng, Ruochen Xu, Liyang Lu, Xuedong Huang

• Synthetic data generation for training of natural language understanding models (US 11508360 B2)
  Baolin Peng, Chenguang Zhu, Chunyuan Li, Xiujun Li, Jinchao Li, Nanshan Zeng, Jianfeng Gao

• Generation Of Optimized Knowledge-based Language Model Through Knowledge Graph Multi-alignment (US 20220230625 A1)
  Chenguang Zhu, Nanshan Zeng.

• Generation Of Optimized Spoken Language Understanding Model Through Joint Training With Integrated Knowledge Language Module (US 20220230628 A1)
  Chenguang Zhu, Nanshan Zeng.

• Generation Of Optimized Spoken Language Understanding Model Through Joint Training With Integrated Acoustic Knowledge-speech Module (US 20220230629 A1)
  Chenguang Zhu, Nanshan Zeng.

• Using machine comprehension to answer a question (US 20190156220)
  Chenguang Zhu, Hsin-Yuan Huang, Pengcheng He, Weizhu Chen, Yelong Shen, Zheng Chen.

• Conversational Virtual Assistant (US 20180232376)
  Chenguang Zhu, Weizhu Chen, Jianwen Zhang, Xuedong Huang, Zheng Chen.

• Caching Content Addressable Data Chunks for Storage Virtualization (US 20140280664)
  Sudipta Sengupta, Chenguang Zhu, Chun Ho Cheung, Jin Li, Abhishek Gupta.

Tutorials and Workshops

•  EACL 2023 Tutorial: Summarization of Dialogues and Conversations At Scale, May 2023.

•  WSDM 2023 Tutorial: Knowledge-Augmented Methods for Natural Language Processing, Feb. 2023. [ Video ]

•  AAAI 2023 Workshop: The Workshop on Knowledge Augmented Methods for NLP (KnowledgeNLP-AAAI’23), Feb. 2023.

•  ACL 2022 Tutorial: Knowledge-Augmented Methods for Natural Language Processing, May, 2022. [ Video ]

Mentoring

I am very fortunate to have mentored and worked with talented students.

• Vin Sachidananda from Stanford University. I am the co-advisor of EE PhD student Vin and our work on embedding alignment and imputation have been published in ICLR 2021 and ACL 2019.

• Wenhao Yu (2021 summer), University of Notre Dame. Our three papers on language generation were published in EMNLP 2021 and ACL 2022. Our paper on dictionary-boosted language model was published in ACL 2022.

• Han Wang (2020 winter), New York University. Our paper on common sense language generation was published in ACL 2021.

• Donghan Yu (2021 summer and 2020 summer), Carnegie Mellon University. Our two papers on language models boosted by knowledge graph were published in AAAI 2022 and ACL 2022.

• Yu-An Chung (2020 summer), MIT. Our paper on speech-text co-pretraining was published in NAACL 2021.

• Yumo Xu (2020 spring), University of Edinburgh.

• Ziyi Yang (2018 summer and 2019 summer), Stanford University. Our paper on word embedding was published in ACL 2019. Our paper on sentence embeddings was published in EMNLP 2019. Our paper on unsupervised text summarization was published in EMNLP 2020. Our paper on zero-shot news summarization was published in SIGIR 2021. Our paper on multilingual embedding alignment was published in ICLR 2021.

• Beliz Gunel (2019 summer), Stanford University. Our paper on knowledge-boosted text summarization was published in NeurIPS 2019 workshop of KR2ML.

• Jianmo Ni (2018 summer), UC San Diego. Our paper on machine reading comprehension was published in NAACL 2019.

• Hsin-Yuan Huang (2017 summer), Caltech. Our paper on machine reading comprehension was published on ICLR 2018.

Talks

• How We Achieved Human Parity in CommonsenseQA – Fusing Knowledge into Language Models" at Singapore Management University, 2023.2. [ Slides ]

• Tutorial on Knowledge-Augmented Methods for Natural Language Processing at WSDM 2023 in Singapore, 2023.2. [ Website | Video ]

• Panelist at Deep Learning on Graphs Workshop for Natural Language Processing (DLG4NLP at NAACL), Seattle, 2022.7. [ Recording (The panel starts at 2:00:30) ]

• Tutorial on Knowledge-Augmented Methods for Natural Language Processing at ACL 2022, 2022.5. [ Website | Video ]

• How We Achieved Human Parity in CommonsenseQA – Fusing Knowledge into Language Models" at NLP seminar at Stanford University, 2022.3. [ Slides ]

• How We Achieved Human Parity in CommonsenseQA – Fusing Knowledge into Language Models" at NLP and AI seminar at Georgia Tech, 2022.2. [ Slides ]

• How We Achieved Human Parity in CommonsenseQA – Fusing Knowledge into Language Models. BLISS seminar (Berkeley Laboratory for Information and System Sciences) at UC Berkeley, 2022.2. [ Slides ]

• Fusing Knowledge into Language Model. Machine Learning / Duolingo Seminar at School of Computer Science, Carnegie Mellon University, 2021.10. [ Slides ]

• Panelist at SIGDial 2021 Special Session on Summarization of Dialogues and Multi-Party Meetings (SummDial 2021), Virtual, 2021.7

• Knowledge Graph and Its Applications in NLP. Seminar at Department of Computer Science and Engineering, University of Notre Dame, 2020.09

• Machine Reading Comprehension

• Future of Computing Forum, Tsinghua University, 2020.05 [Video]

• AI Express, Microsoft Online, 2020.05 [Video]

• Research Progress in Task-Oriented Dialogue. First Open Virtual Assistant Workshop, Stanford University, 2019.10 [Video]

• SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering. Stanford NLP Seminar, Stanford University, 2019.01

• FusionNet: Fusing with Fully Aware Attention in Machine Reading Comprehension

• Stanford Platform Lab Seminar, Stanford University, 2018.02

• Guest lecture at EE392K, Stanford University, 2018.02

Professional Services

• Area Chair: AACL 2022 (Generation Track)
  
• Program Committee: ACL 2019 (Question Answering Track)
  

Awards

• ACM International Collegiate Programming Contest (ICPC), World Finals 2012: 13th place (Representing Stanford University), UPE First Solution Award [ Photo at Award Ceremony ]

• Winner of Stanford Local Programming Contest, 2010, 2011

• Best Student Paper Award Runner-Up at IEEE International Conference of Data Mining (ICDM), 2009

Miscellaneous

• National Champion in US National Table Tennis Championships U2000 Division D, Dec. 2015

• Second Runner-up in men's singles table tennis match of Tsinghua University