Sang Michael Xie

2024

A Survey on Data Selection for Language Models [Paper] [Survey Paper List]

Alon Albalak, Yanai Elazar, Shayne Longpre, Nathan Lambert, Xinyi Wang, Niklas Meunnighoff, Bairu Hou, Liangming Pan, Haewon Jeong, Colin Raffel, Shiyu Chang, Tatsunori Hashimoto, William Yang Wang

arXiv preprint 2024

Connect Later: Improving Fine-tuning for Robustness with Targeted Augmentations [Paper] [Code]

Helen Qu, Sang Michael Xie

International Conference on Machine Learning (ICML) 2024

Meta-Designing Quantum Experiments with Language Models [Paper]

Sören Arlt, Haonan Duan, Felix Li, Sang Michael Xie, Yuhuai Wu, Mario Krenn

NeurIPS Machine Learning and the Physical Sciences Workshop 2024

ICML AI for Science: Scaling in AI for Scientific Discovery Workshop 2024

2023

DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining [Paper] [Code] [Blog]

Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy Liang, Quoc Le, Tengyu Ma, Adams Wei Yu

Conference on Neural Information Processing Systems (NeurIPS) 2023 Spotlight

BayLearn 2023 Oral

Data Selection for Language Models via Importance Resampling [Paper] [Data and Code]

Sang Michael Xie, Shibani Santurkar, Tengyu Ma, Percy Liang

Conference on Neural Information Processing Systems (NeurIPS) 2023

Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models [Paper]

Hong Liu, Sang Michael Xie, Zhiyuan Li, Tengyu Ma

International Conference on Machine Learning (ICML) 2023 Oral

Reward Design with Language Models [Paper] [Code]

Minae Kwon, Sang Michael Xie, Kalesha Bullard, Dorsa Sadigh

International Conference on Learning Representations (ICLR) 2023

2022

An Explanation of In-context Learning as Implicit Bayesian Inference [Paper] [Code] [Video] [Blog]

Sang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu Ma

International Conference on Learning Representations (ICLR) 2022

Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation [Paper]

Kendrick Shen*, Robbie Jones*, Ananya Kumar*, Sang Michael Xie*, Jeff Z. HaoChen, Tengyu Ma, Percy Liang

International Conference on Machine Learning (ICML) 2022 Long talk

Extending the WILDS benchmark for Unsupervised Adaptation [Paper] [Code]

Shiori Sagawa*, Pang Wei Koh*, Tony Lee*, Irena Gao*, Sang Michael Xie, Kendrick Shen, Ananya Kumar, Weihua Hu, Michihiro Yasunaga, Henrik Marklund, Sara Beery, Etienne David, Ian Stavness, Wei Guo, Jure Leskovec, Kate Saenko, Tatsunori Hashimoto, Sergey Levine, Chelsea Finn, Percy Liang

International Conference on Learning Representations (ICLR) 2022 Oral

Holistic Evaluation of Language Models [Paper]

Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D Manning, Christopher Ré, Diana Acosta-Navas, Drew A Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, Yuta Koreeda

TMLR 2022

2021

Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning [Paper] [Code]

Colin Wei, Sang Michael Xie, Tengyu Ma

Conference on Neural Information Processing Systems (NeurIPS) 2021 Spotlight

Composed Fine-Tuning: Freezing Pre-Trained Denoising Autoencoders for Improved Generalization [Paper] [Code]

Sang Michael Xie, Tengyu Ma, Percy Liang

International Conference on Machine Learning (ICML) 2021 Long talk

In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness [Paper]

Sang Michael Xie*, Ananya Kumar*, Robbie Jones*, Fereshte Khani, Tengyu Ma, Percy Liang

International Conference on Learning Representations (ICLR) 2021

WILDS: A Benchmark of in-the-Wild Distribution Shifts [Paper] [Website]

Pang Wei Koh*, Shiori Sagawa*, Henrik Marklund, Sang Michael Xie, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard L. Phillips, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, Percy Liang

International Conference on Machine Learning (ICML) 2021 Long talk

Ensembles and Cocktails: Robust Finetuning for Natural Language Generation [Paper]

John Hewitt*, Xiang Lisa Li*, Sang Michael Xie*, Ben Newman, Percy Liang

NeurIPS 2021 Workshop on Distribution Shifts 2021

No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets [Paper]

Fahim Tajwar, Sang Michael Xie, Ananya Kumar, Percy Liang

ICML Workshop on Uncertainty & Robustness in Deep Learning 2021

On the Opportunities and Risks of Foundation Models [Paper]

Bommasani et al.
Robustness section: Sang Michael Xie, Ananya Kumar, Rohan Taori, Tony Lee, Pang Wei Koh, Shiori Sagawa, Tatsu Hashimoto
Reasoning section: Yuhuai Wu, Frieda Rong, Hongyu Ren, Sang Michael Xie, Xuechen Li, Andy Shih, Drew A. Hudson, Omar Khattab
Adaptation section: Xiang Lisa Li*, Eric Mitchell*, Sang Michael Xie, Xuechen Li, Tatsunori Hashimoto
Theory section: Aditi Raghunathan, Sang Michael Xie, Ananya Kumar, Niladri Chatterji, Rohan Taori, Tatsunori Hashimoto, Tengyu Ma

arXiv 2021

Automated detection of skin reactions in epicutaneous patch testing using machine learning [Paper]

Warren Chan, R. Srivastava, N. Damaraju, H. Do, G. Burnett, J. MacFarlane, Sang Michael Xie, J.K. Chen, G. Honari, K.Y. Sarin

British Journal of Dermatology (BJD) 2021

Combining Satellite Imagery and Machine Learning to Predict Poverty [Paper] [Video/Maps/Media/Links] [Code] [Top of Mind radio interview]

Neal Jean*, Marshall Burke*, Michael Xie, William Davis, David Lobell, Stefano Ermon

Science 2016

Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping [Paper] [Stanford Report] [NYTimes]

Michael Xie, Neal Jean, Marshall Burke, David Lobell, Stefano Ermon

Association for the Advancement of Artificial Intelligence (AAAI) 2016 Oral; NVIDIA Global Impact Award Finalist; Scientific American 10 World Changing Ideas of 2016

Mapping Poverty with Satellite Imagery [Paper]

Michael Xie

Honors Thesis for B.S. with Honors 2017