EvalBench: A Software Library for Visualization Evaluation

Aim: 

It is generally acknowledged in the information visualization research field that it is necessary to evaluate visualization techniques, but the difficulties of conducting such evaluations still remain a topic. There is a need for a solid evaluation infrastructure to encourage information visualization researchers to carry out an evaluation of their tools and ideas. To stimulate the effort on this issue, the researchers need solutions how to integrate different methods for evaluation into their prototypes and how to collect and measure the data produced by the users participating a study.

Team: 

To pave the way for future experimenters in the HCI field, the Java software library EvalBench was developed that focused on flexibility and reusability and should be easy to integrate into existing Java visualization applications. It provides an extendable architecture for experimenters to perform task-based controlled experiments to evaluate user performance of visualization or interaction techniques.

The visualization tool that needs to be evaluated has to implement the EvaluationDelegate interface in order to react on the different states of the experiment (e.g. use a distinct dataset for the visualization for a certain session or task). The library is capable of importing the design-time attributes of tasks defined in an external file (e.g. XML) for an experiment session. During the execution of an experiment, the run-time attributes of the tasks and the interaction log are stored separately for each experiment session.

The particular features are:

  • It can be easily integrated into existing visualization tools.
  • It allows perform task-based evaluations with automated data collection methods.
  • It provides an user interface for the test persons to complete the tasks.

Papers:

Aigner, W.; Hoffmann, S. & Rind, A.: EvalBench: A Software Library for Visualization Evaluation, Computer Graphics Forum, Special Issue "EuroVis 2013", Vol. 32, No. 3, forthcoming, 2013.
Hoffmann, S.: Empirical evaluation of a visualization technique with semantic zoom, Master Thesis, Vienna University of Technology, 2011.

More information can be found at www.evalbench.org and the source code of the library can be forked at GitHub.

Funding: 

This work was supported by the Austrian Federal Ministry of Economy, Family and Youth via CVAST, a Laura Bassi Centre of Excellence (#822746), by the Austrian Science Fund (FWF) via the HypoVis project (#P22883), and by the European Commission via the MobiGuide project (#FP7-287811).

Contact: 
Status: 
finished