I'm an associate professor in the Stanford AI Lab (SAIL
) affiliated with DAWN
and the Statistical Machine Learning Group
). Our lab works on the foundations of the next generation of machine-learned systems.
- On the machine learning side, I am fascinated by how we can learn from increasingly weak forms of supervision and by the mathematical foundations of such techniques.
- On the systems side, I am broadly interested in how machine learning is changing how we build software and hardware. I'm particularly excited when we can blend ML and systems, e.g,. Snorkel, Overton (YouTube), or SambaNova.
Our work is inspired by the observation that data is central to these systems (crudely, "AI is driven by data—not code"
), and so data management principles (re-imagined) play a starring role in our work. Maybe this sounds like Silicon valley crazy talk, but crazily enough, you've probably used a system that uses these ideas from our lab in the last few hours due to amazing students and collaborations with Google ads
, and more.
While we're very proud of our research ideas and their impact, the lab's real goal is to help students become professors, entrepreneurs, and researchers. To that end, over a dozen members of our group have started their own professorships. With students and collaborators, I've been fortunate enough to cofound projects including SambaNova
, and Factory
along with two companies that are now part of Apple, Lattice (DeepDive) and Inductiv (HoloClean). I am fortunate to participate as an advisor or investor in some amazing companies including Agita Labs
, Pixie Labs
(now part of New Relic), Ramp
(now part of Discord), Thistle
, Ursa Computing
, and others.
- In ICML21, we describe some results on hyperbolic geometry, model evaluation, and stability.
- We continue our exploration of hyperbolic geometry showing that Busemann functions and ideal points allow for projections with higher fidelity (lower distortion) in hyperbolic space.
- In Mandoline, we think about how a user can combat distribution shift by allowing them to finely slice their data.
- In Catformer, we make transformers a bit more stable using concatenation.
- In NAACL21, a pair of papers about robustness gym a demonstration and an industry track paper comparing modern named entity linking systems.
- In ICLR2021, model patching, and self-supervision on medical images:
- Beidi Chen and friends talk about Mongoose, a learnable LSH framework for Efficient Training. Oral.
- Karan Goel et al talk about how to patch your models to handle differences among subgroups.
- Sarah Hooper et al describe techniques to (very) weakly supervise medical image segmentation tasks.
- Recent Talks
- In NeurIPS 2020, memory units, hidden strat, and non-euclidean geometry.
- Albert, Tri, Stefano, and Atri describe our work understanding recurrent models and memory from first principles using orthogonal polynomials in Hippo (code). Spotlight
- Nimit, Jared, Geoff, and Albert describe how to prevent some forms of hidden stratification (blog) and an overview video
- Ines, Albert, and Vaggos describe how to solve hierarchical clustering problems with hyperbolic geometry—with provable guarantees! code
A messy, incomplete log of old updates is here.