Publications

(Email me at ebrun AT cs dot stanford dot edu for any papers listed without links)

Preprints

• The GPT Surprise: Offering Large Language Model Chat in a Massive Coding Class Reduced Engagement but Increased Adopters Exam Performances [link]
  Allen Nie, Yash Chandak, Miroslav Suzara, Malika Ali, Juliette Woodrow, Matt Peng, Mehran Sahami, Emma Brunskill, and Chris Piech
• Automated Reminders Reduce Incarceration for Missed Court Dates: Evidence from a Text Message Experiment [link]
  Alex Chohlas-Wood, Madison Coots, Joe Nudell, Julian Nyarko, Emma Brunskill, Todd Rogers, Sharad Goel

2024

• Evaluating Treatment Prioritization Rules via Rank-Weighted Average Treatment Effects [arxiv]
  Steve Yadlowsky, Scott Fleming, Nigam Shah, Emma Brunskill, Stefan Wager
  Journal of the American Statistical Association 2024 (accepted)
• Minimax-Regret Sample Selection in Randomized Experiments [arxiv]
  Yuchen Hu, Henry Zhu, Emma Brunskill, Stefan Wager
  Conference on Economics and Computation (EC) 2024
• Adaptive Instrument Design for Indirect Experiments [arxiv]
  Yash Chandak, Shiv Shankar, Vasilis Syrgkanis, Emma Brunskill
  International Conference on Learning Representations (ICLR) 2024
• Learning to Be Fair: A Consequentialist Approach to Equitable Decision-Making [draft]
  Alex Chohlas-Wood, Madison Coots, Henry Zhu, Emma Brunskill and Sharad Goel
  Management Science (accepted)
• Estimating Optimal Policy Value in Linear Contextual Bandits Beyond Gaussianity [link]
  Jonathan Lee, Weihao Kong, Aldo Pacchiano, Vidya Muthukumar, and Emma Brunskill
  Transactions on Machine Learning Research (TMLR) 2024
• Adaptive Interventions with User-Defined Goals for Health Behavior Change
  Aishwarya Mandyam*, Matthew Joerke*, Barbara Engelhardt, Emma Brunskill (*= co-first authors)
  Conference on Health, Inference, and Learning (CHIL) 2024
• Evaluating and Optimizing Educational Content with Large Language Model Judgments [arxiv]
  Joy He-Yueya, Noah D. Goodman, Emma Brunskill
  Education Data Mining Conference (EDM) 2024
• Estimating the Causal Treatment Effect of Unproductive Persistence [link] (Best paper nominee)
  Amelia Leon, Allen Nie, Yash Chandak, Emma Brunskill
  International Conference on Learning Analytics and Knowledge (LAK24)
• Improving Student Learning with Hybrid Human-AI Tutoring: A Three-Study Quasi-Experimental Investigation [link]
  Danielle R Thomas, Jionghao Lin, Erin Gatz, Ashish Gurung, Shivang Gupta, Kole Norberg, Stephen Fancsali, Vincent Aleven, Lee Branstetter, Emma Brunskill and Kenneth R Koedinger
  International Conference on Learning Analytics and Knowledge (LAK24)
• Brief, Just-in-Time Teaching Tips to Support Computer Science Tutors [link]
  Alan Y. Cheng, Ellie Tanimura, Joseph Tey, Andrew C. Wu and Emma Brunskill
  Proceedings of the ACM Technical Symposium on Computer Science Education (SIGCSE) 2024
• A Fast and Accurate Machine Learning Autograder for the Breakout Assignment [link]
  Evan Liu, David Yuan, Ahmed Ahmed, Elyse Cornwall, Juliette Woodrow, Kaylee Burns, Allen Nie,Emma Brunskill, Chris Piech and Chelsea Finn
  Proceedings of the ACM Technical Symposium on Computer Science Education (SIGCSE) 2024
• MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records [arxiv]
  Scott L. Fleming, Alejandro Lozano, William J. Haberkorn, Jenelle A. Jindal, Eduardo P. Reis, Rahul Thapa, Louis Blankemeier, Julian Z. Genkins, Ethan Steinberg, Ashwin Nayak, Birju S. Patel, Chia-Chun Chiang, Alison Callahan, Zepeng Huo, Sergios Gatidis, Scott J. Adams, Oluseyi Fayanju, Shreya J. Shah, Thomas Savage, Ethan Goh, Akshay S. Chaudhari, Nima Aghaeepour, Christopher Sharp, Michael A. Pfeffer, Percy Liang, Jonathan H. Chen, Keith E. Morse, Emma P. Brunskill, Jason A. Fries, Nigam H. Shah
  Association for the Advancement of AI Conference (AAAI 2024)
• Reinforcement Learning Tutor Better Supported Lower Performers in a Math Task [link]
  Sherry Ruan*, Allen Nie*, William Steenbergen, Jiayu He, JQ Zhang, Meng Guo, Yao Liu, Kyle Dang Nguyen, Catherine Y Wang, Rui Ying, James A Landay, Emma Brunskill (*= co first authors)
  Machine Learning Journal

2023

• In-Context Decision-Making from Supervised Pretraining [link][code]
  Jonathan Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2023 (spotlight)
• Waypoint Transformer: Reinforcement Learning via Supervised Learning with Intermediate Targets [link]
  Anirudhan Badrinath, Yannis Flet-Berliac, Allen Nie, Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2023
• Proportional Response: Contextual Bandits for Simple and Cumulative Regret Minimization [link]
  Sanath Kumar Krishnamurthy, Ruohan Zhan, Susan Athey, Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2023
• Experiment Planning with Function Approximation [link]
  Aldo Pacchiano, Jonathan Lee, Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2023
• Model-based Offline Reinforcement Learning with Local Misspecification. [link]
  Kefan Dong, Yannis Flet-Berliac, Allen Nie, Emma Brunskill
  Association for the Advancement of Artificial Intelligence (AAAI) 2023

• Adaptive Interventions with User-Defined Goals for Health Behavior Change
  Aishwarya Mandyam*, Matthew Joerke*, Barbara E. Engelhardt, Emma Brunskill
  Machine Learning for Health (ML4H)
• MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records
  Alejandro Lozano, William Haberkorn, Scott Fleming, Jenelle Jindal, Eduardo Reis, Rahul Thapa, Louis Blankemeier, Julian Genkins, Ethan Steinberg, Ashwin Nayak, Birju Patel, Chia-Chun Chiang, Alison Callahan, Zepeng Huo, Sergios Gatidis, Scott Adams, Oluseyi Fayanju, Shreya Shah, Thomas Savage, Ethan Goh, Akshay Chaudhari, Nima Aghaeepour, Christopher Sharp, Michael Pfeffer, Percy Liang, Jonathan Chen, Keith Morse, Emma Brunskill, Jason Fries, Nigam Shah
  Machine Learning for Health (ML4H)

• Texting and tutoring: Effects on K-3 reading during the pandemic [link]
  Rebecca D Silverman, Kristin Keane, Hsiaolin Hsieh, Emily Southerton, Renee C Scott, Emma Brunskil
  Journal of Educational Research
• Understanding the Impact of Reinforcement Learning Personalization on Subgroups of Students in Math Tutoring [link]
  Allen Nie, Anka Reuel, Emma Brunskill
  International Conference on Artificial Intelligence in Education

2022

• Data-Efficient Pipeline for Offline Reinforcement Learning with Limited Data
  Allen Nie, Yannis Flet-Berliac, Deon Richmond Jordan, William Steenbergen, Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2022
• Factored DRO: Factored Distributionally Robust Policies for Contextual Bandits
  Tong Mu, Yash Chandak, Tatsunori Hashimoto, Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2022
• Oracle Inequalities for Model Selection in Offline Reinforcement Learning
  Jonathan Lee, George Tucker, Ofir Nachum, Bo Dai, Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2022
• Giving Feedback on Interactive Student Programs with Meta-Exploration
  Evan Zheran Liu, Moritz Pascal Stephan, Allen Nie, Christopher J Piech, Emma Brunskill, Chelsea Finn
  Neural Information Processing Systems (NeurIPS) 2022
• Off-Policy Evaluation for Action-Dependent Non-stationary Environments
  Yash Chandak, Shiv Shankar, Nathaniel D. Bastian, Bruno Castro da Silva, Emma Brunskill, Philip S. Thomas
  Neural Information Processing Systems (NeurIPS) 2022
• Constrained Multi-objective Optimization with Contextual Multi-Armed Bandits
  Henry Zhu, Alex Chohlas-Wood, Madison Coots, Sharad Goel and Emma Brunskill
  ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) 2022 (non-archival)
• Learning to Be Fair: A Consequentialist Approach to Equitable Decision-Making [draft, in submission to journal]
  Alex Chohlas-Wood, Madison Coots, Henry Zhu, Emma Brunskill and Sharad Goel
  ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) 2022 (non-archival)
• Offline Policy Optimization with Eligible Actions [pdf]
  Yao Liu, Yannis Flet-Berliac, and Emma Brunskill
  Conference on Uncertainty in AI (UAI) 2022
• Constraint Sampling Reinforcement Learning: Incorporating Expertise For Faster Learning [arxiv] [code]
  Tong Mu, Georgios Theocharous, David Arbour, and Emma Brunskill
  Association for the Advancement of Artificial Intelligence (AAAI) 2022
• Identification of Subgroups With Similar Benefits in Off-Policy Policy Evaluation [arxiv]
  Ramtin Keramati, Omer Gottesman, Finale Doshi-Velez and Emma Brunskill
  AHLI Conference on Health, Inference, and Learning (CHIL) 2022
  Best paper at "Bridging the Gap: From Machine Learning Research to Clinical Practice" NeurIPS 2021 workshop
• Assessing Dataset Quality using Optimal Experimental Design for Linear Contextual Bandits
  Matthew Jorke, Jonathan Lee, Tong Mu, and Emma Brunskill
  Reinforcement Learning and Decision Making Symposium (RLDM) 2022
• Learning to be Process-Fair: Equitable Decision-Making using Contextual Multi-Armed Bandits
  Arpita Singhal, Henry Zhu and Emma Brunskill
  Reinforcement Learning and Decision Making Symposium (RLDM) 2022

2021

• Design of Experiments for Stochastic Contextual Linear Bandits [arxiv]
  Andrea Zanette*, Kefan Dong*, Jonathan Lee* and Emma Brunskill (* = co-first-authors)
  Neural Information Processing Systems (NeurIPS) 2021
• Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learning [arxiv]
  Andrea Zanette, Martin J. Wainwright and Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2021
• Universal Off-Policy Evaluation
  Yash Chandak, Scott Niekum, Bruno Castro da Silva, Erik Learned-Miller, Emma Brunskill, and Philip Thomas [arxiv]
  Neural Information Processing Systems (NeurIPS) 2021
  Best paper award at RLDM 2022
  
• Reinforcement Learning with State Observation Costs in Action-Contingent Noiselessly Observable Markov Decision Processes
  Alex Nam*, Scott Fleming* and Emma Brunskill (* = co-first-authors)
  Neural Information Processing Systems (NeurIPS) 2021
• Play to Grade: Testing Coding Games as Classifying Markov Decision Process
  Allen Nie, Emma Brunskill, and Chris Piech
  Neural Information Processing Systems (NeurIPS) 2021
• Online Model Selection for Reinforcement Learning with Function Approximation [arxiv]
  Jonathan Lee, Aldo Pacchiano, Vidya Muthukumar, Weihao Kong, and Emma Brunskill
  International Conference on Artificial Intelligence and Statistics (AISTATS) 2021
• EnglishBot: An AI-Powered Conversational System for Second Language Learning
  Sherry Ruan*, Liwei Jiang*, Qianyao Xu*, Glenn Davis, Zhiyuan Liu, Emma Brunskill, and James A. Landay
  International Conference on Intelligent User Interfaces (IUI) 2021
• Automatic Adaptive Sequencing in a Webgame [pdf]
  Tong Mu, Shuhan Wang, Erik Andersen, and Emma Brunskill
  Best short paper
  Conference on Intelligent Tutoring Systems (ITS) 2021
• Power Constrained Bandits [arxiv]
  Jiayu Yao, Emma Brunskill, Weiwei Pan, Susan Murphy, and Finale Doshi-Velez
  Machine Learning for Healthcare Conference 2021

2020

• Provably Good Batch Off-Policy Reinforcement Learning Without Great Exploration [arxiv]
  Yao Liu, Alekh Agarwal, Adith Swaminathan, and Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2020
• Off-policy Policy Evaluation For Sequential Decisions Under Unobserved Confounding [arxiv]
  Hongseok Namkoong, Ramtin Keramti, Steve Yadlowsky and Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2020
• Provably Efficient Reward-Agnostic Navigation with Linear Value Iteration [arxiv]
  Andrea Zanette, Alessandro Lazaric, Mykel Kochenderfer, and Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2020
• Learning When-to-Treat Policies [link]
  Xinkun Nie, Emma Brunskill, and Stefan Wager
  Journal of the American Statistical Association (JASA), accepted
• Learning Near Optimal Policies with Low Inherent Bellman Error [arxiv]
  Andrea Zanette, Alessandro Lazaric, Mykel Kochenderfer, and Emma Brunskill
  International Conference on Machine Learning (ICML) 2020
• Understanding the Curse of Horizon in Off-Policy Evaluation via Conditional Importance Sampling [arxiv]
  Yao Liu, Pierre-Luc Bacon and Emma Brunskill
  International Conference on Machine Learning (ICML) 2020
• Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitions [arxiv]
  Omer Gottesman, Joseph Futoma, Yao Liu, Soanli Parbhoo, Leo Anthony Celi, Emma Brunskill, Finale Doshi-Velez
  International Conference on Machine Learning (ICML) 2020
• Sublinear Optimal Policy Value Estimation in Contextual Bandits [link]
  Weihao Kong, Gregory Valiant and Emma Brunskill
  International Conference on Artificial Intelligence and Statistics (AISTATS) 2020
• Frequentist Regret Bounds for Randomized Least-Squares Value Iteration [link] [*Note: thanks to Taehyun Hwang for pointing out a small error in the proof in Appendix F, which we have corrected in the updated arxiv]
  Andrea Zanette, David Brandfonbrener, Emma Brunskill, Matteo Pirotta, Alessandro Lazaric
  International Conference on Artificial Intelligence and Statistics (AISTATS) 2020
• Being Optimistic to Be Conservative: Quickly Learning a CVaR Policy [link]
  Ramtin Keramati, Christoph Dann, Alex Tamkin, and Emma Brunskill
  Association for the Advancement of Artificial Intelligence Conference (AAAI) 2020
• Scaling Up Behavioral Science Interventions in Online Education [link]
  Rene Kizilcec, Justin Reich, Mike Yeomans, Christoph Dann, Emma Brunskill, Glenn Lopez, Selen Turkay, Joseph Williams, and Dustin Tingley.
  Proceedings of the National Academy of Sciences (PNAS)
• Towards Suggesting Actionable Interventions for Wheel Spinning Students [pdf]
  Tong Mu, Andrea Jetten and Emma Brunskill.
  Educational Data Mining (EDM) 2020
• Supporting Children's Math Learning with Feedback-Augmented Narrative [link]
  Sherry Ruan, Jiayu He, Rui Ying, Jonathan Burkle, Dunia Hakim, Anna Wang, Yufeng Yin, Lily Zhou, Qianyao Xu, Abdallah AbuHashem, Griffin Dietz, Elizabeth Murnane, Emma Brunskill, and James Landay
  Interaction Design and Children (IDC) 2020

2019

• Preventing undesirable behavior of intelligent machines [link to free access to our paper]
  Phil Thomas, Bruno Castro da Silva, Andrew G. Barto, Stephen Giguere, Yuriy Brun, and Emma Brunskill
  Science 2019
• Limiting Extrapolation in Linear Approximate Value Iteration [link]
  Andrea Zanette, Alessandro Lazaric, Mykel Kochenderfer and Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2019
• Almost Horizon-Free Structure-Aware Best Policy Identification with a Generative Model [link]
  Andrea Zanette, Mykel Kochenderfer and Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2019
• Offline Contextual Bandits with High Probability Fairness Guarantees [link]
  Blossom Metevier, Stephen Giguere, Sarah Brockman, Ari Kobren, Yuriy Brun, Emma Brunskill and Philip Thomas
  Neural Information Processing Systems (NeurIPS) 2019
• Off-Policy Policy Gradient with Stationary Distribution Correction [arxiv full]
  Yao Liu, Alekh Agarwal, Adith Swaminathan, and Emma Brunskill
  Conference on Uncertainty in AI (UAI) 2019 (oral presentation)
• Policy Certificates: Towards Accountable Reinforcement Learning [arxiv]
  Christoph Dann, Wei Wei, Lihong Li, and Emma Brunskill
  International Conference on Machine Learning (ICML) 2019
• Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Bounds [arxiv]
  Andrea Zanette and Emma Brunskill
  International Conference on Machine Learning (ICML) 2019
• Combining Parametric and Nonparametric Models for Off-policy Evaluation [arxiv}
  Omer Gottesman, Yao Liu, Emma Brunskill and Finale Doshi-Velez
  International Conference on Machine Learning (ICML) 2019
• Separable Value Functions Across Time-Scales [arxiv]
  Joshua Romoff, Peter Henderson, Ahmed Touati, Emma Brunskill, Joelle Pineau and Yann Ollivier
  International Conference on Machine Learning (ICML) 2019
• PLOTS: Procedure Learning from Observations using subTask Structure [pdf]
  Tong Mu, Karan Goel and Emma Brunskill
  Autonomous Agents and Multi-Agent Systems (AAMAS) 2019
• Learning Procedural Abstractions and Evaluating Discrete Latent Temporal Structure [pdf], [code], [code for metrics]
  Karan Goel and Emma Brunskill
  International Conference on Learning Representations (ICLR) 2019
  
  
• Where's the Reward? A Review of Reinforcement Learning for Instructional Sequencing [pdf soon]
  Shayan Doroudi, Vincent Aleven, and Emma Brunskill
  International Journal of Artificial Intelligence in Education (accepted)
• Not Everyone Can Write Great Examples But Great Examples Can Come From Anywhere [pdf soon]
  Shayan Doroudi, Ece Kamar and Emma Brunskill
  to appear in AAAI Conference on Human Computation and Crowdsourcing (HCOMP) 2019
• Fake It Till You Make It: Learning-Compatible Performance Support [pdf]
  Jonathan Bragg and Emma Brunskilll
  Conference on Uncertainty in AI (UAI) 2019
• Key Phrase Extraction for Generating Educational Question-Answer Pairs [pdf]
  Angelica Willis, Glenn Davis, Lakshmi Manoharan, Sherry Ruan, James Landay, and Emma Brunskill
  Learning at Scale (L@S) 2019,
• Value Driven Representation for Human-in-the-Loop Reinforcement Learning
  Ramtin Keramati and Emma Brunskill
  User Modelling, Adaptation and Personalization (UMAP) 2019, (23% acceptance rate)
• QuizBot: A Dialogue-based Adaptive Learning System for Factual Knowledge [pdf]
  Sherry Ruan, Liwei Jiang, Justin Xu, Bryce Tham, Zhengneng Qiu, Yeshuang Zhu, Elizabeth Murnane, Emma Brunskill, and James Landay
  Computer Human Interaction (CHI) 2019
• Fairer but Not Fair Enough: On the Equitability of Knowledge Tracing [pdf]
  Shayan Doroudi and Emma Brunskill
  Conference on Learning Analytics and Knowledge (LAK) 2019

2018

• Policy Certificates: Towards Accountable Reinforcement Learning [arxiv draft]
  Christoph Dann, Lihong Li, Wei Wei and Emma Brunskill.
  Neural Information Processing Systems (NeurIPS) Workshop on Ethical, Social and Governance Issues in AI 2018
• Distilling Information from a Flood: A Possibility for the Use of Meta-Analysis and Systematic Review in Machine Learning Research
  Peter Henderson and Emma Brunskill.
  Neural Information Processing Systems (NeurIPS) Workshop on Critiquing and Correcting Trends in Machine Learning 2018
• Representation Balancing MDPs for Off-Policy Policy Evaluation. [arxiv draft pdf]
  Yao Liu, Omer Gottesman, Aniruddh Raghu, Matthieu Komorowski, Aldo Faisal, Finale Doshi-Velez, Emma Brunskill.
  Neural Information Processing Systems (NeurIPS) 2018
• Learning Procedural Abstractions
  Karan Goel and Emma Brunskill
  Neural Information Processing Systems (NeurIPS) Infer2Control Workshop 2018
• When Simple Exploration is Sample Efficient: Identifying Sufficient Conditions for Random Exploration to Yield PAC RL Algorithms [arxiv]
  Yao Liu, Emma Brunskill.
  14th European Workshop on Reinforcement Learning (EWRL), 2018
• Sample Efficient Learning with Feature Selection for Factored MDPs. [pdf]
  Zhaohan Guo and Emma Brunskill.
  14th European Workshop on Reinforcement Learning (EWRL), 2018
• Problem Dependent RL Bounds Which Can Identify Bandit Structure in MDPs [pdf]
  Andrea Zanette and Emma Brunskill
  ICML 2018
• Decoupling Gradient-Like Learning Rules from Representations [pdf]
  Philip Thomas, Christoph Dann, and Emma Brunskill
  ICML 2018
• Shared Autonomy for Interactive Systems
  Sharon Zou, Tong Mu, Karan Goel, Michael Bernstein and Emma Brunskill
  UIST 2018 poster
• Exploring the Impact of the Default Option on Student Engagement and Performance in a Statistics MOOC [pdf]
  Emma Brunskill, Dawn Zimmaro and Candace Thille.
  WIP Learning at Scale (LAS) 2018
• Combining Adaptivity with Progression Ordering for Intelligent Tutoring Systems [pdf]
  Tong Mu, Shuhan Wang, Erik Andersen and Emma Brunskill
  WIP Learning at Scale (LAS) 2018
• Adaptive Natural-Language Targeting for Student Feedback [pdf]
  Sherry Ruan, Alex Kolchinski, Dan Schwartz and Emma Brunskill
  WIP Learning at Scale (LAS) 2018

2017

• Using Options and Covariance Testing for Long Horizon Off-Policy Policy Evaluation [pdf]
  Zhaohan Guo, Phil Thomas and Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2017
• Unifying PAC and Regret: Uniform PAC Bounds for Episodic RL (spotlight) [pdf], [code]
  Christoph Dann, Tor Lattimore and Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2017
• Regret Minimization in MDPs with Options without Prior Knowledge (spotlight) [link]
  Ronan Fruit, Matteo Pirotta, Emma Brunskill and Alessandro Lazaric
  Neural Information Processing Systems (NeurIPS) 2017
• Importance Sampling for Fair Policy Selection [pdf]
  Shayan Doroudi, Philip Thomas and Emma Brunskill
  Best paper
  the Conference on Uncertainty in AI (UAI) 2017
• Sample Efficient Policy Search for Optimal Stopping Domains [pdf]
  Karan Goel, Christoph Dann and Emma Brunskill.
  the International Joint Conference on AI (IJCAI) 2017
• The Misidentified Identifiability Problem in Bayesian Knowledge Tracing [pdf]
  Shayan Doroudi and Emma Brunskill.
  Nominated for best paper
  Educational Data Mining (EDM) 2017
• Trading off rewards and errors in multi-armed bandits [pdf]
  Akram Erraqabi, Alessandro Lazaric, Michal Valko, Emma Brunskill, and Yun-En Liu.
  International Conference on Artificial Intelligence and Statistics (AISTATS) 2017.
• The Robust Evaluation Matrix: Towards a More Principled Offline Exploration of Instructional Policies [pdf[slides]
  Shayan Doroudi, Vincent Aleven and Emma Brunskill
  Learning at Scale (LAS) 2017
• Importance Sampling with Unequal Support. [pdf, body only, supplemental only, ArXiv preprint (pdf)]
  P. S. Thomas and E. Brunskill.
  AAAI Conference on Artificial Intelligence (AAAI), 2017.
• Where to Add Actions in Human-in-the-Loop Reinforcement Learning. [pdf]
  Travis Mandel, Yun-En Liu, Emma Brunskill, Zoran Popović.
  AAAI Conference on Artificial Intelligence (AAAI) 2017.
• Predictive Off-Policy Policy Evaluation for Nonstationary Decision Problems, with Applications to Digital Marketing. [pdf]
  P. S. Thomas., G. Theocharous, M. Ghavamzadeh, I. Durugkar, and E. Brunskill.
  Conference on Innovative Applications of Artificial Intelligence (IAAI), 2017.
  • Related paper with same authors presented at the Workshop on Computational Frameworks for Personalization at ICML 2016.

2016

• Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning [link, reviews]
  P.Thomas and E.Brunskill
  International Conference on Machine Learning (ICML) 2016
• Energetic Natural Gradient Descent [link, reviews]
  P.Thomas, C.Dann, B. Castro da Silva and E.Brunskill
  International Conference on Machine Learning (ICML) 2016
• Efficient Bayesian Clustering for Reinforcement Learning [link]
  Travis Mandel, Yun-En Liu, Emma Brunskill and Zoran Popovic
  International Joint Conference on AI (IJCAI) 2016
• Latent Contextual Bandits and their Application to Personalized Recommendations for New Users [pdf]
  Li Zhou and Emma Brunskill
  International Joint Conference on AI (IJCAI) 2016
• PAC Continuous State Online Multitask Reinforcement Learning with Identification [link, supplemental]
  Yao Liu, Zhaohan Guo, and Emma Brunskill
  AAMAS 2016
• A PAC RL Algorithm for Episodic POMDPs [link]
  Zhaohan Guo, Shayan Doroudi and Emma Brunskill
  AISTATS 2016
• Offline Evaluation of Online Reinforcement Learning Algorithms [link]
  Travis Mandel, Yun-En Liu, Emma Brunskill and Zoran Popovic
  AAAI 2016
• Sequence Matters, But How Exactly? A Methodology for Evaluating Activity Sequences from Data
  Shayan Doroudi, Kenneth Holstein, Vincent Aleven and Emma Brunskill
  Educational Data Mining (EDM) 2016
• Questimator: Generating Knowledge Assessments for Arbitrary Topics [pdf]
  Qi Guo, Chinmay Kulkarni, Aniket Kittur, Jeffrey Bigham and Emma Brunskill
  International Joint Conference on AI (IJCAI) 2016
• Interface Design Optimization as a Multi-Armed Bandit Problem [pdf]
  James Lomas, Jodi Forlizzi, Nikhil Poonwala, Nirmal Patel, Sharan Shodhan, Kishan Patel, Ken Koedinger, and Emma Brunskill
  Computer Human Interaction (CHI) 2016.
• Towards a Learning Science for Complex Crowdsourcing Tasks [pdf]
  Shayan Doroudi, Ece Kamar, Emma Brunskill and Eric Horvitz
  Computer Human Interaction (CHI) 2016.
• Automatically Learning to Teach to the Learning Objective [pdf]
  Rika Antonova, Joe Runde, Dexter Lee, and Emma Brunskill
  work in progress in Learning at Scale 2016

2015

• Sample Complexity of Episodic Fixed-Horizon Reinforcement Learning [arxiv pdf]
  Christoph Dann and Emma Brunskill
  Neural Information Processing Systems (NeurIPS) 2015
• Faster Teaching via POMDP Planning [pdf]
  Anna Rafferty, Emma Brunskill, Tom Griffiths and Pat Shafto
  Cognitive Science (journal) 2015
• Concurrent PAC RL [pdf]
  Zhaohan (Daniel) Guo and Emma Brunskill.
  AAAI 2015
• The Queue Method: Handling Delay, Heuristics, Prior Data, and Evaluation in Bandits[link]
  Travis Mandel, Yun-En Liu, Emma Brunskill, Zoran Popovic.
  AAAI 2015
• From Predictive Models to Instructional Policies [pdf]
  Joseph Rollinson and Emma Brunskill
  Educational Data Mining (EDM) 2015.
• Towards Understanding How to Leverage Sense-making, Induction/Refinement and Fluency to Improve Robust Learning [pdf]
  Shayan Doroudi, Kenneth Holstein, Vincent Aleven and Emma Brunskill.
  Educational Data Mining (EDM) 2015.
• Learning the Features Used to Decide How to Teach [link]
  Min Hyung Lee, Joe Runde, Warfa Jabril, Zhouying Wang, and Emma Brunskill.
  Learning at Scale 2015. Work in Progess paper.

2014

• PAC-inspired Option Discovery in Lifelong Reinforcement Learning [pdf]
  Note:Lazaric and Fruit pointed out that PAC-SMDP and PAC-MDP are not always directly comparable, hence our paper's analysis of the benefit of options over primitive actions does not always hold.
  E.Brunskill and L.Li
  ICML 2014
• Resource-Efficient Stochastic Optimization of a Locally Smooth Function under Correlated Bandit Feedback [pdf]
  M.Azar, A.Lazaric and E.Brunskill
  ICML 2014
• Trading Off Scientific Knowledge and User Learning with Multi-Armed Bandits [pdf]
  Yun-En Liu, Travis Mandel, Emma Brunskill, and Zoran Popovic.
  Educational Data Mining (EDM) 2014.
• Towards Automatic Experimentation of Educational Knowledge [pdf]
  Honorable mention
  Yun-En Liu, Travis Mandel, Emma Brunskill, and Zoran Popovic.
  Computer Human Interaction (CHI) 2014.
• Offline Policy Evaluation Across Representations with Applications to Educational Games [pdf]
  Travis Mandel, Yun-En Liu, Sergey Levine, Emma Brunskill, Zoran Popovic.
  Autonomous Agents and Multi-Agent Systems (AAMAS) 2014.

2013

• Sequential Transfer in Multi-armed Bandit with Finite Set of Models [link]
  M.Azar, A.Lazaric and E.Brunskill
  Neural Information Processing Systems (NeurIPS) 2013
• Regret Bounds for Reinforcement Learning with Policy Advice [link]
  M.Azar, A.Lazaric and E. Brunskill
  European Conference on Machine Learning (ECML) 2013
• Sample Complexity of Multi-task Reinforcement Learning. [pdf]
  E.Brunskill and L.Li
  Conference on Uncertainty in Artificial Intelligence (UAI) 2013
• Understanding Sequential Decisions via Inverse Reinforcement Learning.
  Siyuan Liu, Miguel Araujo, Emma Brunskill, Rosaldo Rossetti, Joao Barros, Ramayya Krishnan
  MDM 2013
• Predicting Player Moves in an Educational Game: A Hybrid Approach* [pdf]
  Nominated for best paper
  Yun-En Liu, Travis Mandel, Eric Butler, Erik Andersen, Eleanor O'Rourke, Emma Brunskill, Zoran Popovic.
  Educational Data Mining (EDM) 2013.
• Estimating Student Knowledge from Paired Interaction Data. [pdf]
  A.Rafferty, J.Davenport, and E.Brunskill
  Educational Data Mining (EDM) 2013
• New Potentials for Data-driven Intelligent Tutoring System Development and Optimization.
  Koedinger, K.R., Brunskill, E., Baker, R.S.J.d., McLaughlin, E.A., Stamper, J.
  AI Magazine
• Towards Operationalizing Outlier Detection in Community Health Programs
  T.McCarthy, B.DeRenzi, J.Blumenstock and E.Brunskill
  Note in Information Communications Technology for International Development (ICTD) 2013
• Analysis of the Impact of Errors Made During Health Data Collection Using Mobile Phones: Exploring Error Modeling and Automatic Diagnosis.
  S.Palkar and E.Brunskill
  ACM DEV 2013 (Poster)

2012

• Incentive Decision Processes [pdf]
  S.Reddi and E.Brunskill
  Conference on Uncertainty in Artificial Intelligence (UAI) 2012
• The Impact on Individualizing Student Models on Necessary Practice Opportunities [pdf]
  Nominated for best paper
  J.I.Lee and E.Brunskill
  International Conference on Educational Data Mining (EDM) 2012
• Policy Building -- An Extension To User Modeling [pdf]
  M.Yudelson and E.Brunskill
  International Conference on Educational Data Mining (EDM) 2012
• Bayes-optimal reinforcement learning for discrete uncertainty domains.
  E.Brunskill
  AAMAS 2012 (Extended Abstract)
• Global and regional hearing impairment prevalence: an analysis of 42 studies in 29 countries [link]
  G.Stevens, S.Flaxman, E.Brunskill, M. Mascarenhas, C.Mathers, M.Finucane
  The European Journal of Public Health

2011 and prior

• Estimating prerequisite structure from noisy data [pdf]
  E.Brunskill
  International Conference on Educational Data Mining (EDM) 2011
• Partially observable sequential decision making for problem selection in an intelligent tutoring system [pdf]
  E.Brunskill and S.Russell
  poster in International Conference on Educational Data Mining (EDM) 2011
• Faster teaching by POMDP planning [pdf]
  A.Rafferty, E.Brunskill, T.Griffiths, and P.Shafto
  International Conference on Artificial Intelligence in Education (AIED) 2011
• Designing mobile interfaces for novice and low-literacy users [pdf]
  I. Medhi, S. Patnaik, E.Brunskill, S. N. Nagasena Gautama, W. Thies, and K. Toyama
  ACM Transactions on Computer-Human Interaction 2011
• Efficient planning under uncertainty with macro-actions [pdf]
  R.He, E. Brunskill and N. Roy
  Journal of Artificial Intelligence Research 2011
• Evaluating an adaptive multi-user educational tool for low-resource regions [pdf]
  E.Brunskill, S.Garg, C.Tseng, J.Pal and L.Findlater
  International Conference on Information and Communication Technologies and Development (ICTD) 2010
• RAPID: A reachable anytime planner for imprecisely-sensed domains [pdf]
  E.Brunskill and S.Russell
  Uncertainty in Artificial Intelligence (UAI) 2010
• PUMA: planning under uncertainty with macro-actions [pdf]
  R.He, E.Brunskill and N.Roy
  AAAI Conference on Artificial Intelligence (AAAI) 2010
• Planning in partially-observable switching-mode continuous domains [pdf]
  E.Brunskill, L.Kaelbling, T.Lozano-Perez, and N.Roy
  Annals of Mathematics and Artificial Intelligence 2010
• When policies can be trusted: analyzing a criteria to identify optimal policies in MDPs with unknown model parameters [pdf]
  (This corrected version makes a 2 sentence clarification that only the final DeltaQ represents an estimate of g)
  E.Brunskill
  International Conference on Automated Planning and Scheduling (ICAPS) 2010
• Provably efficient learning with typed parametric models [pdf]
  E.Brunskill, B. Leffler, L. Li, M. Littman, and N. Roy.
  Journal of Machine Learning Research 2009
• Evaluating the accuracy of data collection on mobile phones: a study of forms, SMS, and voice [pdf]
  S.Patnaik, E.Brunskill, and W.Thies.
  International Conference on Information and Communication Technologies and Development (ICTD) 2009
• Where to go: interpreting natural directions using global inference [pdf]
  Y.Wei, E.Brunskill, T.Kollar and N.Roy.
  International Conference on Robotics and Automation (ICRA) 2009
• CORL: a continuous-state offset-dynamics reinforcement learner [pdf]
  E.Brunskill, B.Leffler, L.Li, M.Littman and N.Roy.
  Uncertainty in Artificial Intelligence (UAI) 2008
  
• Perceptual switch rates with ambiguous structure-from-motion figures in bipolar disorder [pdf].
  K.Krug, E.Brunskill, A.Scarna, G.Goodwin and A.Parker.
  Proceedings of the Royal Society B 2008.
• A supervised learning approach for collision detection in legged locomotion. [pdf]
  F.Doshi, E.Brunskill, A.Shkolnik, T.Kollar, K.Rohanimanesh, R.Tedrake, and N.Roy
  International Conference on Intelligent Robots and Systems (IROS) 2007
• Topological mapping using spectral clustering and classification [pdf]
  E.Brunskill, T.Kollar and N.Roy
  International Conference on Intelligent Robots and Systems (IROS) 2007
• Adaptive state space construction with reinforcement learning for robots.
  E.Brunskill, E.Uchibe, and K.Doya
  poster International Conference on Robotics and Automation (ICRA) 2006
• SLAM using incremental probabilistic PCA and dimensionality reduction. [pdf]
  E.Brunskill and N.Roy.
  International Conference on Robotics and Automation (ICRA) 2005

Workshop and Symposium Papers

• Routing for rural health: optimizing community health worker visit schedules [pdf]
  E.Brunskill and N.Lesh
  position paper AAAI Spring Symposium on Artificial Intelligence for Development 2010
• Learning to identify locally actionable health anomalies [pdf]
  K.Chen, E.Brunskill, J.Dick and P.Dhadialla
  position paper AAAI Spring Symposium on Artificial Intelligence for Development 2010
• How close is close enough? Finding optimal policies in PAC-style reinforcement learning.
  E. Brunskill
  Abstract Neural Information Processing Systems (NeurIPS) 2008 Workshop on Model Uncertainty and Risk in Reinforcement Learning
• Continuous-state POMDPs with hybrid dynamics. [pdf]
  E.Brunskill, L.Kaelbling, T.Lozano-Perez, and N.Roy.
  Proceedings of the International Symposium on Artificial Intelligence and Mathematics (ISAIM) 2008
• Continuous state POMDPs for object manipulation tasks.
  E.Brunskill
  Association for the Advancement of Artificial Intelligence (AAAI) 2007 Doctoral Consortium
• Lessons from prototyping a microfinance distance learning tool. [pdf]
  E.Brunskill and T.Parikh
  CHI workshop on User Centered Design for International Development 2007
• Building peer-to-peer systems with Chord, a distributed lookup service.
  F.Dabek, E.Brunskill, F.Kaashoek, D.Karger, R.Morris, I.Stoica, and H.Balakrishnan.
  in Proceedings of the 8th Workshop on Hot Topics in Operating Systems (HotOS-VIII) 2001

Ph.D. Thesis

• Compact Parametric Models for Efficient Sequential Decision Making in High-dimensional, Uncertain Domains MIT June 2009

Other Publications

• LittleDog learning locomotion project
  CSAIL research abstract 2006
• Impact of Bipolar Disorder on Ambiguous Structure-from-Motion Percepts
  E.Brunskill, et al.
  Society for Neuroscience Abstract (SFN) 2003