Show Me Your Face: Towards an Automated Method to Provide Timely Guidance in Visual Analytics

Teaser Image
Author
Abstract
Providing guidance during a Visual Analytics session can support analysts in pursuing their goals more efficiently. However, the effectiveness of guidance depends on many factors: Determining the right timing to provide it is one of them. Although in complex analysis scenarios choosing the right timing could make the difference between a dependable and a superfluous guidance, an analysis of the literature suggests that this problem did not receive enough attention. In this paper, we describe a methodology to determine moments in which guidance is needed. Our assumption is that the need of guidance would influence the user state-of-mind, as in distress situations during the analytical process, and we hypothesize that such moments could be identified by analyzing the user's facial expressions. We propose a framework composed by a facial recognition software and a machine learning model trained to detect when to provide guidance according to changes of the user facial expressions. We trained the model by interviewing eight analysts during their work and ranked multiple facial features based on their relative importance in determining the need of guidance. Finally, we show that by applying only minor modifications to its architecture, our prototype was able to detect a need of guidance on the fly and made our methodology well suited also for real-time analysis sessions. The results of our evaluations show that our methodology is indeed effective in determining when a need of guidance is present, which constitutes a prerequisite to providing timely and effective guidance in VA.
Keywords
Journal Article
Year of Publication
2022
Journal
IEEE Transactions on Visualization and Computer Graphics
Volume
28
Start Page
4570-4581
Issue
12
Number of Pages
12
Date Published
07/2021
Type of Article
Journal
ISSN Number
1077-2626
DOI
reposiTUm Handle
Funding projects
Paper
Download citation