Research Interests

My research interests focus on algorithm and system design for machine learning in memory-constrained settings. Modern machine learning / deep learning models are memory-intensive for the deployment in datacenters and on edge devices. To enable memory-efficient training and inference with strong statistical performance, I develop and analyze algorithm/system for memory-efficient compressed ML. I recently worked on training and inference using compressed word embeddings, low precision kernel approximation features and model sparsity.

My previous works also cover large-scale machine learning systems, such as asynchronous deep neural network training systems at supercomputer scale (DL on the Cori Supercomputer). On the application side of machine learning, I am interested in and have been working on NLP and computer vision topics, including machine reading comprehension (the SQuAD dataset) and visual scene understanding.


PipeMare: Asynchronous Pipeline Parallel DNN Training
Bowen Yang, Jian Zhang, Jonathan Li, Chris Aberger, Chris De Sa, and Chris Ré.
arXiv preprint arXiv:1910.05124, Sepetember 2019.

High-accuracy Low-precision Training
Chris De Sa, Megan Leszczynski, Jian Zhang, Alana Marzoev, Chris Aberger, Kunle Olukotun, and Chris Ré.
arXiv preprint arXiv:1803.03383, March 2018.
[PDF] [Blog]


On the Downstream Performance of Compressed Word Embeddings
Avner May, Jian Zhang, Tri Dao, Christopher Ré.
Neural Information Processing Systems (NeurIPS), Vancouver, Canada, December 2019. (Spotlight, 3% acceptance)

Low-precision Random Fourier Features for Memory-constrained Kernel Approximation
Jian Zhang*, Avner May*, Tri Dao, Christopher Ré. (* equal contribution)
International Conference on Artificial Intelligence and Statistics (AISTATS), Naha, Japan, April 2019

YellowFin and the Art of Momentum Tuning
Jian Zhang, Ioannis Mitliagkas.
SysML Conference (SysML), Stanford, USA, April 2019
[PDF] [Blog] [Talk]

Training with Low-precision Embedding Tables
Jian Zhang, Jiyan Yang, Hector Yuen.
Workshop on Systems for ML and Open Source Software at NeurIPS, Montreal, Canada, December 2018

Analysis of the Time-to-accuracy Metric and Entries in the DAWNBench Deep Learning Benchmark
Cody Coleman, Daniel Kang, Deepak Narayanan, Luigi Nardi, Tian Zhao, Jian Zhang, Peter Bailis, Kunle Olukotun, Chris Ré, Matei Zahari.
Workshop on Systems for ML and Open Source Software at NeurIPS, Montreal, Canada, December 2018

Exploring the Utility of Developer Exhaust
Jian Zhang, Max Lam, Stephanie Wang, Paroma Varma, Luigi Nardi, Kunle Olukotun, Christopher Ré.
Workshop on Data Management for End-to-End Machine Learning (DEEM), Houston, USA, June 2018

DAWNBench: An End-to-end Deep Learning Benchmark and Competition
Cody Coleman, Deepak Narayanan, Daniel Kang, Tian Zhao, Jian Zhang, Luigi Nardi, Peter Bailis, Kunle Olukotun, Chris Ré, Matei Zahari.
SysML Conference (SysML), Stanford, US, February 2018

Peta-scale Deep Learning: Supervised and Semi-supervised classification for scientific data
Thorsten Kurth, Jian Zhang, Nadathur Satish, Ioannis Mitliagkas, Evan Racah, Md. Mostofa Ali Patwary, Tareq Malas, Narayanan Sundaram, Wahid Bhimji, Mikhail Smorkalov, Jack Deslippe, Mikhail Shiryaev, Srinivas Shridharan, Prabhat, Pradeep Dubey.
Supercomputing (SC), Denver, USA, November 2017.

SQuAD: 100,000+ Questions for Machine Comprehension of Text
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, Percy Liang.
Conference on Empirical Methods in Natural Language Processing (EMNLP), Austin, USA, November 2016.
Best resource paper award.
[PDF] [Project Page]

Parallel SGD: When does averaging Help?
Jian Zhang, Christopher De Sa, Ioannis Mitliagkas, Chris Ré.
OptML Workshop at ICML, New York, US, June 2016

Higher-order Inference for Multi-class Log-supermodular Models
Jian Zhang, Josip Djolonga, Andreas Krause.
International Conference on Computer Vision (ICCV), Santiago, Chile, December 2015.

Message Passing Inference for Large Scale Graphical Models with High Order Potentials
J. Zhang, Alexander G. Schwing, Raquel Urtasun.
Neural Information Processing Systems (NIPS), Montreal, Quebec, Canada, December 2014.
[PDF] [Project Page]

Estimating the 3D Layout of Indoor Scenes and Its Clutter from Depth Sensors
Jian Zhang, Kan Chen, Alexander G. Schwing, Raquel Urtasun.
International Conference on Computer Vision (ICCV), Sydney, Australia, December 2013.
[PDF] [Project Page]

Non-iterative Normalized Feature Extraction in Large Viewpoint Variances Based on PCA of Gradient
J. Zhang, C. Song, D. Wen.
IS&T/SPIE Electronic Imaging (EI), Burlingame, California, USA, February 2013.


Machine Learning
Stanford University, Fall 2018 / Summer 2019, CS 229

Digital Circuits
ETH Zurich, Spring 2015, 252-0014-00L

Linear Algebra
ETH Zurich, Fall 2014, 401-0131-00L

Digital Circuits
ETH Zurich, Spring 2014, 252-0014-00L

Computer Science I
ETH Zurich, Fall 2013, 252-0847-00L