Beidi Chen

I am currently a postdoc researcher at Stanford working with Dr. Chris Ré. I received my Ph.D. in Computer Science from Rice University under the supervision of Dr. Anshumali Shrivastava in 2020. I received my B.S. from University of California, Berkeley in 2015. My mentors were Dr. Sara Alspaugh, Dr. Kaifei Chen and my advisor was Dr. Randy Katz.

My research focuses on large-scale machine learning. Specifically, I design and optimize randomized algorithms (algorithm-hardware co-design) to accelerate large machine learning systems for real-world problems.

I am on the 2021-2022 job market.

NEWS:

Research

Binhang Yuan, Yongjun He, Jared Quincy Davis, Tianyi Zhang, Tri Dao, Beidi Chen, Percy Liang, Christopher Ré, Ce Zhang. " Decentralized Training of Foundation Models in Heterogeneous Environments". In Submission.

Jue Wang, Binhang Yuan, Luka Rimanic, Yongjun He, Tri Dao, Beidi Chen, Christopher Ré, Ce Zhang. " Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees". In Submission.

Tri Dao, Beidi Chen, Nimit Sohoni, Arjun Desai, Michael Poli, Jessica Grogan, Alexander Liu, Aniruddh Rao, Atri Rudra, Christopher Ré. " Monarch: Expressive Structured Matrices for Efficient and Accurate Training". ICML 2022. (Long Talk) Outstanding Poster Award at the EfficientML Bay Area meetup). [ code ][ poster ]

Beidi Chen*, Tri Dao*, Kaizhao Liang, Jiaming Yang, Zhao Song, Atri Rudra, Christopher Ré. " Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models". In Proceedings of International Conference on Learning Representations, ICLR 2022 (Spotlight). [video][ code ]

Zichang Liu, Zhaozhuo Xu, Alan Ji, Junyan Zhang, Jonathan Li, Beidi Chen, Anshumali Shrivastava. "HALOS: Hashing Large Output Space for Cheap Inference". In Proceedings of the 5rd Conference on Machine Learning and Systems, MLSys 2022.

Beidi Chen*, Tri Dao*, Eric Winsor, Zhao Song, Atri Rudra, Christopher Ré. " Scatterbrain: Unifying Sparse and Low-rank Attention Approximation ". In Neural Information Processing Systems, NeurIPS 2021. [ code ]

Beidi Chen, Zichang Liu, Binghui Peng, Zhaozhuo Xu, Jonathan Lingjie Li, Tri Dao, Zhao Song, Anshumali Shrivastava, Christopher Re. "MONGOOSE: A Learnable LSH Framework for Efficient Neural Network Training". In Proceedings of International Conference on Learning Representations, ICLR 2021. (Oral) [video][ code]

Zhaozhuo Xu, Beidi Chen, Chaojian Li, Weiyang Liu, Le Song, Yingyan Lin, and Anshumali Shrivastava. " Locality Sensitive Teaching". In Neural Information Processing Systems, NeurIPS 2021.

Shabnam Daghaghi, Nicholas Meisburger, Mengnan Zhao, Beidi Chen, Tharun Medini, and Anshumali Shrivastava. "A Tale of Two Efficient and Informative Negative Sampling Distributions". In Proceedings of International Conference on Machine Learning, ICML 2021. (Long Talk)

Tharun Medini, Beidi Chen, Anshumali Shrivastava. "SOLAR: Sparse Orthogonal Learned and Random Embeddings". In Proceedings of International Conference on Learning Representations, ICLR 2021.

Beidi Chen, Weiyang Liu, Animesh Garg, Zhiding Yu, Anshumali Shrivastava, Jan Kautz, Anima Anandkumar. "Angular Visual Hardness". In Proceedings of the 37th International Conference on Machine Learning, ICML 2020. [Video]

Beidi Chen, Tharun Medini, James Farwell, Sameh Gobriel, Charlie Tai, Anshumali Shrivastava. "SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems". In Proceedings of the 3rd Conference on Machine Learning and Systems, MLSys 2020.

Beidi Chen, Yingchen Xu, and Anshumali Shrivastava. "LGD: Fast and Accurate Stochastic Gradient Estimation". In Neural Information Processing Systems, NeurIPS 2019. (LSH-Sampling Breaks the Computational Chicken-and-Egg Loop in Adaptive Stochastic Gradient Estimation in ICLR 2018 Workshop)

Beidi Chen, M. Sadegh Riazi, Anshumali Shrivastava, DanWallach, Farinaz Koushanfar. "Sub-linear Privacy-preserving Search with Untrusted Server and Semi-honest Parties". Manuscript.

Beidi Chen, Anshumali Shrivastava. " Revisiting Winner Take All (WTA) Hashing for Sparse Datasets". In Proceedings of the 34th Conference in Uncertainty in Artificial Intelligence, UAI 2018.

Beidi Chen, Anshumali Shrivastava, Rebecca C. Steorts. "Unique Entity Estimation with Application to the Syrian Conflict". The Annals of Applied Statistics 12.2 (2018). (Also Won IISA 2018 Best Student Paper in Applied Statistics with this paper.)

Kaifei Chen, Siyuan He, Beidi Chen, John Kolb, Randy H. Katz, David E. Culler. "BearLoc: A Composable Distributed Framework for Indoor Localization Systems". In Proceedings of the 2015 Workshop on IoT challenges in Mobile and Industrial Systems, IoT-Sys@MobiSys 2015, pages 7-12, May. 2015. Florence, Italy.

S. Alspaugh, Beidi Chen, Jessica Lin, Archana Ganapathi, Marti Hearst, and Randy Katz. "Analyzing Log Analysis: An Empirical Study of User Log Mining". In Proceedings of the 28th Large Installation System Administration Conference, LISA 2014. (Best Student Paper)

Experience

Research Intern

Microsoft Research, Redmond, WA
May 2019 - Present

Machine Learning Scientist Intern

Nvidia, Santa Clara, CA

Feb 2019 - May 2019

Applied Scientist Intern

A9.com, Palo Alto, CA

May 2018 - Dec 2018

Applied Scientist Intern

Amazon Web Services, Inc., Palo Alto, CA
March 2017 - June 2017

Software Engineering Intern

Apple Inc., Cupertino, CA

Feb 2015 - Aug 2015

Research and Development Intern

VMware Inc., Palo Alto, CA
May 2014 - Aug 2014

Software Engineering Intern

Broadcom Corporation, Sunnyvale, CA

May 2013 - Aug 2013

Education

Stanford University

Postdoc
Computer Science
August 2020 - Present

Rice University

Doctor of Philosophy
Computer Science
August 2015 - May 2020

University of California, Berkeley

Bachelor of Science
Electrical Engineering and Computer Science
August 2011 - May 2015

Wuhan Foreign Languages School

High School
Sep 2008 - June 2011

Honors and Awards

  • MIT EECS Rising Stars 2021
  • UIUC EECS Rising Stars 2019
  • IISA 2018 Best Student Paper Award
  • Ken Kennedy Institute for Information Technology 2017 Fellowship
  • USENIX LISA 2014 Best Student Paper Award
  • ANITA BORG INSTITUTE GHC Scholarship 2014
  • Qualcomm Undergraduate Experiences in Science & Technology Scholar 2013

Invited Talks

LightOn AI Meetup [webpage]

VMware ML

TWIML AI Podcast [video]

Microsoft Machine Translation Group

Microsoft Rearch Talks [video]

Record Linkage workshop at CIMAT [webpage]

IISA2018 Conference

JSM 2018 Topic-Contributed Session

Activity and Service

Teaching

TA in Comp530 (Database System Implementation)
Rice University, Spring 2018

TA in Comp382 (Reasoning about Algorithms)
Rice University, Fall 2017, 2019

TA in Comp330 (Tools and Models For Data Science)
Rice University, Fall 2016, 2018

TA in EE122 (Introduction to Communication Networks)
UC Berkeley, Spring 2013, 2014

TA in EE40 (Introduction to Microelectronic Circuits)
UC Berkeley, Fall 2014

Service

Reviewer for Science Advances, NeurIPS, ICML, ICLR, AISTATS, AAAI, UAI, AISTATS

Activity

Stanford CS Undergraduate Mentoring Program

Women in Science and Engineering at Stanford

Member of CSters at Rice University

Member of Association of Women in EECS (AWE) at UC Berkeley