Research
My research focus is human-centered artificial intelligence: Understanding human intelligence for developing biologically-inspired AI algorithms, as well as making AIs more compatible with humans. Please see my research summary for more details.
|
|
Understanding Human Intelligence in the Era of Artificial Intelligence.
Ruohan Zhang
Research Summary, 2020
paper
|
|
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models
Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, Li Fei-Fei
Conference on Robot Learning (CoRL), 2023   (Oral)
paper |
website
|
|
Mimicplay: Long-horizon Imitation Learning by Watching Human Play
Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, Anima Anandkumar
Conference on Robot Learning (CoRL), 2023   (Oral)
paper |
website
|
|
Quantifying the Effect of Visual Impairments on Daily Activities in Virtual, Interactive Environments
Wensi Ai, Sharon Lee, Li Fei-Fei, Jiajun Wu, Ruohan Zhang
Proceedings of the Annual Meeting of the Cognitive Science Society (CogSci), 2023
paper |
website
|
|
Primitive Skill-based Robot Learning from Human Evaluative Feedback
Ayano Hiranaka*, Minjune Hwang*, Sharon Lee, Chen Wang, Li Fei-Fei, Jiajun Wu, Ruohan Zhang (*equally contributed)
International Conference on Intelligent Robots and Systems (IROS), 2023
paper |
website
|
|
Partial-View Object View Synthesis via Filtering Inversion
Fan-Yun Sun, Jonathan Tremblay, Valts Blukis, Kevin Lin, Danfei Xu, Boris Ivanovic, Peter Karkus, Stan Birchfield, Dieter Fox, Ruohan Zhang, Yunzhu Li, Jiajun Wu, Marco Pavone, Nick Haber
arxiv, 2023
paper |
website |
code
|
|
Modeling Dynamic Environments with Scene Graph Memory
Andrey Kurenkov, Michael Lingelbach, Tanmay Agarwal, Chengshu Li, Emily Jin, Ruohan Zhang, Li Fei-Fei, Jiajun Wu, Silvio Savarese, Roberto MartĂn-MartĂn
International Conference on Machine Learning (ICML), 2023
paper |
code
|
|
Task-Driven Graph Attention for Hierarchical Relational Object Navigation
Michael Lingelbach, Chengshu Li, Minjune Hwang, Andrey Kurenkov, Alan Lou, Roberto MartĂn-MartĂn, Ruohan Zhang, Li Fei-Fei, Jiajun Wu
International Conference on Robotics and Automation (ICRA), 2023
paper |
code
|
|
BEHAVIOR-1K: A Benchmark for Embodied AI with 1,000 Everyday Activities and Realistic Simulation
Chengshu Li*, Ruohan Zhang*, Josiah Wong*, Cem Gokmen*, Sanjana Srivastava*, Roberto MartĂn-MartĂn*, Chen Wang*, Gabrael Levine*, Michael Lingelbach, Jiankai Sun, Mona Anvari, Minjune Hwang, Manasi Sharma, Arman Aydin, Dhruva Bansal, Samuel Hunter, Kyu-Young Kim, Alan Lou, Caleb R Matthews, Ivan Villa-Renteria, Jerry Huayang Tang, Claire Tang, Fei Xia, Silvio Savarese, Hyowon Gweon, Karen Liu, Jiajun Wu, Li Fei-Fei
(*equally contributed)
Conference on Robot Learning (CoRL), 2022   (Best Paper Nominee)
paper |
website
|
|
A Dual Representation Framework for Robot Learning with Human Guidance
Ruohan Zhang*, Dhruva Bansal*, Yilun Hao*, Ayano Hiranaka, Jialu Gao, Chen Wang, Roberto MartĂn-MartĂn, Li Fei-Fei, Jiajun Wu (*equally contributed)
Aligning Robot Representations with Humans Workshop at CoRL, 2022   (Spotlight)
Conference on Robot Learning (CoRL), 2022
paper |
appendix |
website
|
|
Interaction Modeling with Multiplex Attention
Fan-Yun Sun, Isaac Kauvar, Ruohan Zhang, Jiachen Li, Mykel Kochenderfer, Jiajun Wu, Nick Haber
Advances in Neural Information Processing Systems (NeurIPS), 2022
link |
code
|
|
How to Train your Decision-Making AIs?
Ruohan Zhang, Dhruva Bansal
The Gradient, 2022
link |
paper
|
|
Selective Visual Attention during Public Speaking in an Immersive Context
Mikael Rubin, Sihang Guo, Karl Muller, Ruohan Zhang, Michael Telch, Mary Hayhoe
Attention, Perception, & Psychophysics, 2022
link |
paper |
VSS2021 abstract
|
|
Machine versus Human Attention in Deep Reinforcement Learning Tasks
Sihang Guo, Ruohan Zhang, Bo Liu, Yifeng Zhu, Mary Hayhoe, Dana Ballard, Peter Stone
Advances in Neural Information Processing Systems (NeurIPS), 2021
link |
paper |
arxiv |
VSS2021 abstract
|
|
Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation
Lin Guan, Mudit Verma, Sihang Guo, Ruohan Zhang, Subbarao Kambhampati
Advances in Neural Information Processing Systems (NeurIPS), 2021   (Spotlight)
link |
paper |
arxiv |
talk
|
|
Recent Advances in Leveraging Human Guidance for Sequential Decision-Making tasks
Ruohan Zhang*, Faraz Torabi*, Garrett Warnell, Peter Stone. (*equally contributed)
Autonomous Agents and Multi-Agent Systems (JAAMAS), 2021
link |
paper |
arxiv
|
|
A Modular Attention Hypothesis for Modeling Visuomotor Behaviors
Ruohan Zhang
The University of Texas at Austin Ph.D. Dissertation, 2021
paper |
slides
|
|
Efficiently Guiding Imitation Learning Algorithms with Human Gaze
Akanksha Saran, Ruohan Zhang, Elaine Schaertl Short, Scott Niekum
Autonomous Agents and Multi-Agent Systems (AAMAS), 2021
paper |
arxiv |
bibtex |
code |
slides |
media
|
|
The Hierarchical Evolution in Human Vision Modeling
Dana H. Ballard, Ruohan Zhang
Topics in Cognitive Sciences, 2021
link |
paper
|
|
Human Gaze Assisted Artificial Intelligence: A Review
Ruohan Zhang, Akanksha Saran, Bo Liu, Yifeng Zhu, Sihang Guo, Scott Niekum, Dana Ballard, Mary Hayhoe
International Joint Conference on Artificial Intelligence (IJCAI) Survey Track, 2020
paper
|
|
Parallel Neural Processing with Gamma Frequency Latencies
Ruohan Zhang, Dana H. Ballard
Neural Computation, 2020
link |
paper
|
|
Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset
Ruohan Zhang, Calen Walshe, Zhuode Liu, Lin Guan, Karl S. Muller, Jake A. Whritner, Luxin Zhang, Mary M Hayhoe, Dana Ballard
AAAI Conference on Artificial Intelligence (AAAI), 2020
link |
paper |
arxiv |
dataset |
code |
poster |
AAAI2020 RLG Workshop talk
|
|
An Initial Attempt of Combining Visual Selective Attention with Deep Reinforcement Learning
Liu Yuezhang, Ruohan Zhang, Dana Ballard
arxiv, 2020
arxiv
|
|
Leveraging Human Guidance for Deep Reinforcement Learning Tasks
Ruohan Zhang, Faraz Torabi, Lin Guan, Dana Ballard, Peter Stone
International Joint Conference on Artificial Intelligence (IJCAI) Survey Track, 2019
paper |
arxiv
|
|
AGIL: Learning Attention from Human for Visuomotor Tasks
Ruohan Zhang, Zhuode Liu, Luxin Zhang, Jake Whritner, Karl Muller, Mary Hayhoe, Dana Ballard
European Conference on Computer Vision (ECCV), 2018
link |
paper |
arxiv |
dataset |
code |
VSS2018 abstract |
VSS2018 talk |
CCN2017 version
|
|
Modeling Sensory-Motor Decisions in Natural Behavior
Ruohan Zhang, Shun Zhang, Matthew Tong, Yuchen Cui, Constatin Rothkopf, Dana Ballard, Mary Hayhoe
PLoS Computational Biology, 2018
link |
paper |
VSS2017 abstract |
VSS2017 talk |
NETI2016 poster |
|
|
Model Checking For Safe Navigation Among Humans
Sebastian Junges, Nils Jansen, Joost-Pieter Katoen, Ufuk Topcu, Ruohan Zhang, Mary Hayhoe
International Conference on Quantitative Evaluation of SysTem (QEST), 2018
link |
paper
|
|
Fast and Precise Black and White Ball Detection for RoboCup Soccer
Jacob Menashe, Josh Kelle, Katie Genter, Josiah Hanna, Elad Liebman, Sanmit Narvekar, Ruohan Zhang, Peter Stone
RoboCup Symposium, 2017
paper |
code
|
|
Greedy Direction Method of Multiplier for MAP Inference of Large Output Domain
Xiangru Huang, Ian E.H. Yen, Ruohan Zhang, Qixing Huang, Pradeep Ravikumar, Inderjit S. Dhillon
Artificial Intelligence and Statistics (AISTATS), 2017
paper |
code
|
|
Participatory Art Museum: Collecting and Modeling Crowd Opinions
Xiaoyu Zeng, Ruohan Zhang
AAAI Conference on Artificial Intelligence (AAAI) Student Abstract, 2017
link |
paper
|
|
UT Austin Villa: Project-Driven Research in AI and Robotics
Katie Genter, Patrick MacAlpine, Jacob Menashe, Josiah Hanna, Elad Liebman, Sanmit Narvekar, Ruohan Zhang, Peter Stone
IEEE Intelligent Systems 31(2), 2016
link |
code
|
|
Dual Decomposed Learning with Factorwise Oracle for Structural SVM of Large Output Domain
Ian E.H. Yen, Xiangru Huang, Kai Zhong, Ruohan Zhang, Pradeep Ravikumar, Inderjit S. Dhillon
Advances in Neural Information Processing Systems (NIPS), 2016
link |
paper
|
|
Decision-Making Policies for Heterogeneous Autonomous Multi-Agent Systems with Safety Constraints
Ruohan Zhang, Yue Yu, Mahmoud El Chamie, Behçet Açikmese, and Dana H. Ballard
International Joint Conference on Artificial Intelligence (IJCAI), 2016
paper
|
|
Maximum Sustainable Yield Problem for Robot Foraging and Construction System
Ruohan Zhang, Zhao Song
International Joint Conference on Artificial Intelligence (IJCAI), 2016
paper
|
|
Cortical Spikes use Analog Sparse Coding
Dana Ballard, Ruohan Zhang, Luc Gentet
bioRxiv, 2020
paper
|
|
Attention Guided Imitation Learning and Reinforcement Learning
Ruohan Zhang
AAAI Conference on Artificial Intelligence (AAAI) Doctoral Consortium, 2019
paper |
NETI2019 poster
|
|
Learning Attention Model From Human for Visuomotor Tasks
Luxin Zhang, Ruohan Zhang, Zhuode Liu, Mary Hayhoe, Dana Ballard
AAAI Conference on Artificial Intelligence (AAAI) Student Abstract, 2018
link
|
|
Visual Attention Guided Deep Imitation Learning
Ruohan Zhang*, Zhuode Liu*, Luxin Zhang, Karl Muller, Mary Hayhoe, Dana Ballard. (* equally contributed)
NIPS Cognitively Informed Artificial Intelligence Workshop, 2017
paper
|
|
UT Austin Villa 2017 Team Description Paper for the Standard Platform League
Katie Genter, Josiah Hanna, Josh Kelle, Elad Liebman, Jacob Menashe, Sanmit Narvekar, Ruohan Zhang, Peter Stone
RoboCup Symposium, 2017
paper
|
|
UT Austin Villa 2016 Team Description Paper for the Standard Platform League
Katie Genter, Josiah Hanna, Josh Kelle, Elad Liebman, Jacob Menashe, Sanmit Narvekar, Rishi Shah, Ruohan Zhang, Peter Stone
RoboCup Symposium, 2016
paper
|
|
UT Austin Villa 2015 Team Description Paper for the Standard Platform League
Katie Genter, Josiah Hanna, Elad Liebman, Jacob Menashe, Sanmit Narvekar, Jivko Sinapov, Ruohan Zhang, Peter Stone
RoboCup Symposium, 2015
paper
|
|
Global Policy Construction in Modular Reinforcement Learning
Ruohan Zhang, Zhao Song, Dana Ballard
AAAI Conference on Artificial Intelligence (AAAI) Student Abstract, 2015
paper
|
|