photo of two users' hands interacting with a touch-sensitive table using the Access Overlays tool to gain understanding of a large map accessible without sight

project areas

touchscreen accessibility

friendsourcing Q&A for the blind

image captioning

social tech for parents of children with special needs

social tech for autism

health info seeking

crowdworkers with disabilities

improving communication via AAC

accessible AR and VR

accessible photography

dyslexia and web search

accessibility & social technologies


My research explores the relationship between accessibility and social technologies. In particular, I am interested in how interaction design can have a positive impact on people with disabilities by making it easier for them to connect with others in social and professional contexts. I am also interested in how interaction traces from social technologies (e.g., "big data") can provide insight into the activities, challenges, and successes surrounding technology use by people with disabilities, and/or provide early warning signs of health concerns. I manage the MSR Ability team at Microsoft Research, and am an affiliate member of MSR NExT's Enable group.

teaching & service

Together with Richard Ladner, I co-organized a joint University of Washington + Microsoft Research seminar series on the topic of "Computers & Disabilities" (CS590W) during the Fall 2010 academic quarter, and we also co-taught the same course at UW in Winter 2017. I served as a faculty panelist for the 2012 and 2017 ACM ASSETS doctoral symposium, and I was the local arrangements chair for the 2013 ASSETS conference. Richard Ladner and I co-organized the 2016 UW-MSR Summer Institute on the topic of Expanding Accessibility Research. I was the papers co-chair for the 2017 ASSETS conference, and the papers co-chair for the "Health, Accessibility, and Aging" subcommittee for CHI 2018.

research projects

Touchscreen Accessibility for the blind

Friendsourcing Q&A for the blind

  • survey of blind people's use of social networking sites, and experiment comparing their use of friendsourcing vs. crowdsourcing for Q&A [ CSCW 2013 paper ]
  • analysis of visual challenges facing blind people based on questions posted to a friend/crowdsourcing service [ CHI 2013 paper ]
  • proposing and studying the concept of social microvolunteering as a mechanism for friendsourcing on behalf of altruistic causes [ CHI 2015 paper ]

Image Captioning

  • study of blind Twitter users' needs regarding image captioning, and recommendations on how to make visual media in social media more accessible to blind users [ CHI 2016 paper ]
  • A study of how automated image captions in social media are interpreted by blind users and how to better support appropriate user skepticism of incorrect captions. [ CHI 2017 paper ]
  • A study of how human-in-the-loop AI can improve the utility of automatically-generated image captions for users with visual impairment. [ HCOMP 2017 paper and appendix ]
  • introducing proxemic audio interfaces as an interaction technique for immersive and varied rich description of artwork for people with visual impairment. [ IMWUT 2017 paper & video ]
  • Caption Crawler: a system for automatically providing alt text for online images using reverse image search [ CHI 2018 paper ]
  • Rich Representations of Visual Content for Screen Reader Users: rich, interactive updates to traditional "alt text" representations [ CHI 2018 paper ]

Social technologies for parents of children with special needs

  • study of Facebook and Twitter use by mothers of children aged 3 and under, including those with special needs/developmental delays [ CSCW 2014 paper ]
  • study of ways in which parents of children with special needs use social media, including suggestions for changes to social platforms to better support this demographic [ ICWSM 2014 paper ]

Social technologies for people on the autism spectrum

  • SIDES: a tabletop game for social skills development aimed at youth with Asperger's syndrome [ CSCW 2006 paper ]
  • SWARM: a wearable device to facilitate awareness of one's own or others' emotional states [ TEI 2015 paper ]
  • Supporting Neurodiverse Software Engineers in Technical Workplace Settings [ ASSETS 2015 paper ]
  • A study of how search engines and Twitter are used to seek and share health information, including disability issues [ CHI 2014 paper ]

Crowdsourcing employment and disability

  • A study indicating that people with disabilties view online crowdsourcing work as a potentially important source of employment, and identifying the technical obstacles to full participation in crowdwork by people with disabilities. [ CSCW 2015 paper ]
  • A study exploring senior citizens' understanding of and interest in crowd work, and reflecting on how crowd work systems might evolve to engage this audience. [ CHI 2016 paper ] [ HCOMP 2015 poster abstract ]
  • A framework for subcontracting microtasks that may benefit workers with disabilities. [ CHI 2017 paper ]

Improving communication via AAC devices

  • Reframing AAC as groupware and supplementing AAC with a mobile phone app (AACrobat) to facilitate communication partner awareness and engagement [ CSCW 2017 paper | Video Figure ]
  • Interview study identifying limitations of current AAC devices for personality expression and presenting design guidelines and evaluation metrics for more expressive AAC [ CSCW 2017 paper ]
  • Expanding the expressiveness of AAC via supplementary "awareness displays" for conveying emotion and/or communication logistics. [ CHI 2017 paper ]
  • Automating e-Tran board style input via a smartphone app for interpreting simple gaze gestures and translating them into text. [ CHI 2017 paper ]
  • Analyzing the limitations of eye gaze sensing for interactive applications and considering how applications can be re-designed to mitigate these challenges. [ CHI 2017 paper ]
  • Introducing the dynamic, cascading dwell technique for error-reduction and improved throughput in eye typing. [ CHI 2017 paper ]
  • Using computer vision to provide word and phrase suggestions to AAC systems that faciltiate discussing co-located objects. [ DIS 2017 paper ]
  • Voicesetting interfaces for improved expressivity in AAC via non-speech audio backchannel reactions, the use of emoji to trigger automated SSML markup for common emotional intonations, and a detailed eye-gaze operated prosody editor. [ CHI 2018 paper ]

Accessible Augmented Reality and Virtual Reality

  • Canetroller: A novel haptic and auditory interface that enables blind users to physically navigate virtual reality environments [ CHI 2018 paper ]

Accessible Photography

  • Understanding the accessibility needs of users with motor impairments. [ CHI 2018 paper ]
  • An investigation of the use of photo-capture and photo-sharing apps by low-vision teens, including Instagram and Snapchat. [ CHI 2018 paper ]
  • Understanding how people use web search as a linguistic support tool [ WWW 2017 paper ]
  • Interview, survey, and online study demonstrating the differential needs of dyslexic and non-dyslexic web searchers. [ CHI 2018 paper ]
  • Analysis of how web page features impact readability for users wtih dyslexia. [ SIGIR 2018 paper ]