Tao Dong
Dr. Tao Dong is a User Experience Researcher at Google, located in Mountain View, California. His current research is focused on cross-device user interfaces and multi-device eco-systems. He holds a PhD in the field of Human-Computer Interaction from the School of Information at the University of Michigan.
Research Areas
Authored Publications
Google Publications
Other Publications
Sort By
Blending Accessibility in UI Framework Documentation to Build Awareness
The 25th International ACM SIGACCESS Conference on Computers and Accessibility, ACM (2023)
Preview abstract
The lack of accessibility awareness among industry professionals is one of the reasons for rampant inaccessible websites and applications. This problem is exacerbated by the industry norm of having a single place dedicated to accessibility in the documentation of UI frameworks, which makes accessibility difficult for developers to discover and implement as part of their workflows. This paper presents the Blended Approach (BA), a novel approach and framework for improving accessibility awareness through documentation. Unlike the conventional practice, it recommends sprinkling and repeating short snippets on accessibility throughout the documentation while linking developers to detailed explanations on the dedicated accessibility page. Thus, BA places the topic of accessibility on an equal footing as other common programming concerns such as performance, security, and UX. As a case study, we applied BA to the onboarding tutorial of Flutter, a popular UI toolkit. The positive feedback we received in our evaluation with 11 professional developers suggests BA can be a viable and effective approach.
View details
Synthesis-Assisted Video Prototyping From a Document
Brian R. Colonna
Christian Frueh
UIST 2022: ACM Symposium on User Interface Software and Technology (2022)
Preview abstract
Video productions commonly start with a script, especially for talking head videos that feature a speaker narrating to the camera. When the source materials come from a written document -- such as a web tutorial, it takes iterations to refine content from a text article to a spoken dialogue, while considering visual compositions in each scene. We propose Doc2Video, a video prototyping approach that converts a document to interactive scripting with a preview of synthetic talking head videos. Our pipeline decomposes a source document into a series of scenes, each automatically creating a synthesized video of a virtual instructor. Designed for a specific domain -- programming cookbooks, we apply visual elements from the source document, such as a keyword, a code snippet or a screenshot, in suitable layouts. Users edit narration sentences, break or combine sections, and modify visuals to prototype a video in our Editing UI. We evaluated our pipeline with public programming cookbooks. Feedback from professional creators shows that our method provided a reasonable starting point to engage them in interactive scripting for a narrated instructional video.
View details
An Early Rico Retrospective: Three Years Of Uses For A Mobile App Dataset
Biplab Deka
Forrest Huang
Jeffrey Nichols
Artificial Intelligence for Human Computer Interaction: A Modern Approach (2021)
Preview abstract
The Rico dataset, containing design data from more than9.7k Android apps spanning 27categories, was released in 2017. It exposes visual, textual, structural, and interactive design properties of more than72k unique UI screens. Over the years since its release, the original paper has been cited nearly 100 times according to Google Scholar and the dataset has been used as the basis for numerous research projects. In this chapter, we describe the creation of Rico using a system that combined crowdsourcing and automation to scalably mine design and interaction data from Android apps at runtime. We then describe two projects that we conducted using the dataset: the training of an autoencoder to identify similarity between UI designs, and an exploration of the use of Google’s Material Design within the dataset using machine learned models. We conclude with an overview of other work that has used Rico to understand our mobile UI world and build data-driven models that assist users, designers, and developers.
View details
The Impact of “Cosmetic Changes” on the Usability of Error Messages
Kandarp Khandwala
Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, ACM (2019), LBW0273
Preview abstract
Programmatic errors are often difficult to resolve due to poor usability of error messages. Applying theories of visual perception and techniques in visual design, we created three visual variants of a representative error message in a modern UI framework. In an online experiment, we found that the visual variants led to substantial improvements over the original error message in both error comprehension and resolution. Our results demonstrate that seemingly cosmetic changes to the presentation of an error message can have an oversized impact on its usability.
View details
The Moving Context Kit: Designing for Context Shifts in Multi-Device Experiences
Jeffrey Nichols
Julia Haines
Michael Gilbert
Proceedings of the 2017 Conference on Designing Interactive Systems, ACM, New York, NY, USA, pp. 309-320
Preview abstract
Multi-device product designers need tools to better address ecologically valid constraints in naturalistic settings early in their design process. To address this need, we created a reusable design kit of scenarios, “hint” cards, and a framework that codifies insights from prior work and our own field study. We named the kit the Moving Context Kit, or McKit for short, because it helps designers focus on context shifts that we found to be highly influential in everyday multi-device use. Specifically, we distilled the following findings from our field study in the McKit: (1) devices are typically specialized into one of six roles during parallel use—notifier, broadcaster, collector, gamer, remote, and hub, and (2) device roles are influenced by context shifts between private and shared situations. Through a workshop, we validated that the McKit enables designers to engage with complex user needs, situations, and relationships when incorporating novel multi-device techniques into the products they envision.
View details
Understanding the Challenges of Designing and Developing Multi-Device Experiences
Jeffrey Nichols
DIS '16 Proceedings of the 2016 Conference on Designing Interactive Systems
Preview abstract
As the number of computing devices available to users continues to grow, personal computing increasingly involves using multiple devices together. However, support for multi-device interactions has fallen behind users' desire to leverage the diverse capabilities of the devices that surround them. In this paper, we report on an interview study of 29 designers and developers in which we investigate the barriers to creating useful, usable, and delightful multi-device experiences. We uncovered three key challenges: 1) the difficulty in designing the interactions between devices, 2) the complexity of adapting interfaces to different platform UI standards, and 3) the lack of tools and methods for testing multi-device user experiences. We discuss the technological and business factors behind these challenges and potential ways to lower the barriers they impose.
View details
No Results Found
