Danfei Xu

I am a Ph.D. student in CS at Stanford University. My advisors are Fei-Fei Li and Silvio Savarese who co-lead the Stanford Vision and Learning Lab. My research focuses on compositional and generalizable structures in robotics and vision.

Prior to joining Stanford, I received my B.S. from Columbia University (2015). I've worked/interned at DeepMind UK (2019), ZOOX (2017), Autodesk Research (2016), CMU RI (2014), and Columbia Robotics Lab (2013-2015).

Email  /  Google Scholar  /  CV (Jan 2019)  /  Github  /  Twitter

News
  • [July 2020] We are organizing the Advances and Challenges in Imitation Learning for Robotics workshop at RSS 2020!
  • [April 2020] I will be co-instructing the Stanford CS231n course in Spring 2020.
  • [March 2020] Released our latest paper on generalizable long-horizon imitation.
  • [Dec 2019] Presented Regression Planning Networks at NeurIPS 2019.
  • [Dec 2019] Presented Positive-Unlabeled Reward Learning at the NeurIPS 2019 DeepRL Workshop.
  • [Sep 2019] Regression Planning Networks accepted at NeurIPS 2019. Paper and code available.
Demos

Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations (2020)

6-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypoints (2020)
Preprints
Positive-Unlabeled Reward Learning
Danfei Xu, Misha Denil
(Short version) Late-Breaking Paper, NeurIPS Deep Reinforcement Learning Workshop 2019
(Long version) In Submission

An algorithm framework that simultaneously addresses the reward delusion problem in supervised reward learning and the overfitting discriminator problem in adversarial imitation learning.

Selected Publications
Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations
Ajay Mandlekar*, Danfei Xu*, Roberto Martin-Martin, Silvio Savarese, Li Fei-Fei
RSS, 2020

[website] [video]

Learning visuomotor policies that can generalize across long-horizon tasks by modeling latent compositional structures.

6-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypoints
Chen Wang, Roberto Martin-Martin, Danfei Xu, Jun Lv, Cewu Lu, Li Fei-Fei, Silvio Savarese, Yuke Zhu
ICRA, 2020

[website] [video] [code]

Real-time category-level 6D object tracking from RGB-D data.

Regression Planning Networks
Danfei Xu, Roberto Martin-Martin, De-An Huang, Yuke Zhu, Silvio Savarese, Li Fei-Fei
NeurIPS, 2019

[code] [poster]

A flexible neural network architecture for learning to plan from video demonstrations.

Continuous Relaxation of Symbolic Planner for One-Shot Imitation Learning
De-An Huang, Danfei Xu, Yuke Zhu, Silvio Savarese, Li Fei-Fei, Juan Carlos Niebles
IROS , 2019

One-shot imitation learning via hybrid neural-symbolic planning.

Situational Fusion of Visual Representation for Visual Navigation
William B. Shen, Danfei Xu, Yuke Zhu, Leonidas Guibas, Li Fei-Fei, Silvio Savarese
ICCV , 2019

Learning generalizable navigation policy from mid-level visual representations.

DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion
Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martin-Martin, Cewu Lu, Li Fei-Fei, Silvio Savarese
CVPR, 2019

[website] [video] [code]

Dense RGB-depth sensor fusion for 6D object pose estimation.

Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration
De-An Huang*, Suraj Nair*, Danfei Xu*, Yuke Zhu, Animesh Garg, Li Fei-Fei, Silvio Savarese, Juan Carlos Niebles
CVPR, 2019 (Oral)

Generate executable task graphs from video demonstrations.

Neural Task Programming: Learning to Generalize Across Hierarchical Tasks
Danfei Xu*, Suraj Nair*, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei, Silvio Savarese
ICRA, 2018

[website] [video] [Two Minute Papers]

Neural Task Programming (NTP) is a meta-learning framework that learns to generate robot-executable neural programs from task demonstration video.

PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
Danfei Xu, Ashesh Jain, Dragomir Anguelov
CVPR, 2018

End-to-end 3D Bounding Box Estimation via sensor fusion.

Scene Graph Generation by Iterative Message Passing
Danfei Xu, Yuke Zhu, Christopher B. Choy, Li Fei-Fei
CVPR, 2017

[website] [code]

We propose an end-to-end model that jointly infers object category, location, and relationships. The model learns to iteratively improve its prediction by passing messages on a scene graph.

3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
Christopher B. Choy, Danfei Xu*, JunYoung Gwak*, Silvio Savarese
ECCV, 2016

[website] [code]

We propose an end-to-end 3D reconstruction model that unifies single- and multi-view reconstruction.

Model-Driven Feed-Forward Prediction for Manipulation of Deformable Objects
Yinxiao Li , Yan Wang , Yonghao Yue , Danfei Xu, Michael Case , Shih-Fu Chang , Eitan Grinspun , Peter K. Allen
IEEE TASE, 2016

[website]

Deformable object manipulation with an application of personal assitive robot.

This is the journal paper of our "laundry robot" series:
ICRA 2015
IROS 2015
ICRA 2016

Topometric localization on a road network
Danfei Xu, Hernan Badino, Daniel Huber
IROS, 2015

Vision-based localization on a probabilistic road network.

Tactile identification of objects using Bayesian exploration
Danfei Xu, Gerald E. Loeb, Jeremy Fishel
ICRA, 2013

Object classification using multi-modal tactile sensing.

Teaching
  • [2020] Stanford CS 231n instructor
  • [2019] Stanford CS 231n teaching assistant & lecturer
  • [2018] Stanford CS 231n teaching assistant
  • [2018] Stanford CS 231a teaching assistant
Other Services
  • Reviewer: CVPR, ICCV, ECCV, IROS, ICRA, AAAI, IJRR, TPAMI, RA-L, NeurIPS
  • Program Committee: Scene Graph Representation and Learning Workshop (ICCV2019)
  • Tutorial Student Chair: Deep Representation and Estimation of State for Robotics (IROS2020)

Template source