Sang Michael Xie

I am a Computer Science Ph.D. student studying Machine Learning at Stanford University, advised by Percy Liang and Tengyu Ma.
I was a Student Researcher at Google Brain, working with Adams Wei Yu, Hieu Pham, and Quoc Le. I am grateful to be a FY2019 NDSEG Fellow. Previously, I received my B.S. with departmental honors and M.S. in Computer Science from Stanford in 2017, where I am grateful to have worked with Stefano Ermon on machine learning methods for sustainability, particularly in poverty mapping using satellite imagery.

My research interests are in generalization to unseen inputs and tasks (robustness to distribution shifts, extrapolation), learning with unlabeled data and limited supervision (transfer learning, semi-supervised learning, unsupervised learning), and applications/motivations from real-world robustness problems and NLP.

[Music] Email: xie AT cs.stanford.edu

Papers

2023

DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining [Paper]
arXiv, 2023.

Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy Liang, Quoc Le, Tengyu Ma, Adams Wei Yu
Data Selection for Language Models via Importance Resampling [Paper] [Data and Code]
arXiv, 2023.

Sang Michael Xie, Shibani Santurkar, Tengyu Ma, Percy Liang
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models [Paper]
International Conference on Machine Learning (ICML), 2023. Oral

Hong Liu, Sang Michael Xie, Zhiyuan Li, Tengyu Ma
Reward Design with Language Models [Paper]
International Conference on Learning Representations (ICLR), 2023.

Minae Kwon, Sang Michael Xie, Kalesha Bullard, Dorsa Sadigh

2022

Holistic Evaluation of Language Models [Paper]
arXiv, 2022.

Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D Manning, Christopher Ré, Diana Acosta-Navas, Drew A Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, Yuta Koreeda
Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation [Paper]
International Conference on Machine Learning (ICML), 2022. Long talk

Kendrick Shen*, Robbie Jones*, Ananya Kumar*, Sang Michael Xie*, Jeff Z. HaoChen, Tengyu Ma, Percy Liang
An Explanation of In-context Learning as Implicit Bayesian Inference [Paper] [Code] [Video] [Blog]
International Conference on Learning Representations (ICLR), 2022.

Sang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu Ma
Extending the WILDS benchmark for Unsupervised Adaptation [Paper] [Code]
International Conference on Learning Representations (ICLR), 2022. Oral

Shiori Sagawa*, Pang Wei Koh*, Tony Lee*, Irena Gao*, Sang Michael Xie, Kendrick Shen, Ananya Kumar, Weihua Hu, Michihiro Yasunaga, Henrik Marklund, Sara Beery, Etienne David, Ian Stavness, Wei Guo, Jure Leskovec, Kate Saenko, Tatsunori Hashimoto, Sergey Levine, Chelsea Finn, Percy Liang

2021

On the Opportunities and Risks of Foundation Models [Paper]

arXiv, 2021

Bommasani et al.

Robustness section: Sang Michael Xie, Ananya Kumar, Rohan Taori, Tony Lee, Pang Wei Koh, Shiori Sagawa, Tatsu Hashimoto

Reasoning section: Yuhuai Wu, Frieda Rong, Hongyu Ren, Sang Michael Xie, Xuechen Li, Andy Shih, Drew A. Hudson, Omar Khattab

Adaptation section: Xiang Lisa Li*, Eric Mitchell*, Sang Michael Xie, Xuechen Li, Tatsunori Hashimoto

Theory section: Aditi Raghunathan, Sang Michael Xie, Ananya Kumar, Niladri Chatterji, Rohan Taori, Tatsunori Hashimoto, Tengyu Ma
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning [Paper][Code]
NeurIPS, 2021. Spotlight

Colin Wei, Sang Michael Xie, Tengyu Ma
Ensembles and Cocktails: Robust Finetuning for Natural Language Generation [Paper]
NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications , 2021

John Hewitt*, Xiang Lisa Li*, Sang Michael Xie*, Ben Newman, Percy Liang
No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets [Paper]

ICML Workshop on Uncertainty & Robustness in Deep Learning, 2021.

Fahim Tajwar, Sang Michael Xie, Ananya Kumar, Percy Liang
Composed Fine-Tuning: Freezing Pre-Trained Denoising Autoencoders for Improved Generalization [Paper] [Code]

International Conference on Machine Learning (ICML), 2021. Long talk

Sang Michael Xie, Tengyu Ma, Percy Liang
WILDS: A Benchmark of in-the-Wild Distribution Shifts [Paper] [Website]

International Conference on Machine Learning (ICML), 2021. Long talk

Pang Wei Koh*, Shiori Sagawa*, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard L. Phillips, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, Percy Liang
In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness [Paper] [Code]

International Conference on Learning Representations (ICLR), 2021.

Sang Michael Xie*, Ananya Kumar*, Robbie Jones*, Fereshte Khani, Tengyu Ma, Percy Liang
Automated detection of skin reactions in epicutaneous patch testing using machine learning [Paper]

British Journal of Dermatology (BJD), 2021.

Warren Chan, R. Srivastava, N. Damaraju, H. Do, G. Burnett, J. MacFarlane, Sang Michael Xie, J.K. Chen, G. Honari, K.Y. Sarin

2016-2020

Understanding and Mitigating the Tradeoff Between Robustness and Accuracy [Paper][Video]

International Conference on Machine Learning (ICML), 2020.

Aditi Raghunathan*, Sang Michael Xie*, Fanny Yang, John C. Duchi, Percy Liang
Weakly supervised deep learning for segmentation of remote sensing imagery [Paper]

Remote Sensing, 2020.

Sherrie Wang, William Chen, Sang Michael Xie, George Azzari, David Lobell
Adversarial Training Can Hurt Generalization [Paper]

ICML Workshop on Identifying and Understanding Deep Learning Phenomena, 2019.

Aditi Raghunathan*, Sang Michael Xie*, Fanny Yang, John C. Duchi, Percy Liang
Reparameterizable Subset Sampling via Continuous Relaxations [Paper]

International Joint Conferences on Artificial Intelligence (IJCAI), 2019.
Sang Michael Xie, Stefano Ermon
Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance. [Paper]

Neural Information Processing Systems (NeurIPS), 2018.

Neal Jean*, Sang Michael Xie*, Stefano Ermon
Incorporating Spatial Context and Fine-grained Detail from Satellite Imagery to Predict Poverty [Paper]

Jae Hyun Kim, Michael Xie, Neal Jean, Stefano Ermon
Combining Satellite Imagery and Machine Learning to Predict Poverty. [Video, Maps, Media, and Links] [Paper] [Code] [Top of Mind radio show segment]

Science, 2016.

Neal Jean*, Marshall Burke*, Michael Xie, William Davis, David Lobell, Stefano Ermon
Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping. [Paper] [Oral Presentation] [Stanford Report] [NYTimes]

Association for the Advancement of Artificial Intelligence (AAAI), 2016. Oral Presentation, NVIDIA Global Impact Award Finalist, Scientific American 10 World Changing Ideas of 2016

Michael Xie, Neal Jean, Marshall Burke, David Lobell, Stefano Ermon
Mapping Poverty with Satellite Imagery. [Paper]

Honors Thesis for B.S. with Honors

Michael Xie

Teaching

Course Assistant for CS 324: Understanding and Developing Large Language Models, Winter 2022
Course Assistant for CS 229: Machine Learning, Spring 2022
Section Leader for ENGR 40M: Intro to Making: What is EE, Winter 2015

Service

I co-organized the ICLR 2023 Workshop on Mathematical and Empirical Understanding of Foundation Models (ME-FoMo) with Ananya Kumar, Tiffany Vlaar, Yamini Bansal, Mathilde Caron, Tengyu Ma, Hanie Sedghi, Aditi Raghunathan, and Percy Liang.
I have reviewed for NeurIPS (2019, 2020, 2021, 2022), ICML (2020, 2022, 2023), ICLR (2021, 2022), IEEE SatML 2023, NeurIPS 2022 RobustSeq Workshop, NeurIPS 2022 DistShift Workshop, ICML 2022 First Workshop on Pre-Training, ICML 2022 Principles of Distribution Shift (PODS) workshop, the NeurIPS 2021 Workshop on Distribution Shifts (DistShift), the Workshop on Computer Vision for Global Challenges (CV4GC) at CVPR 2019.