University of Twente Student Theses


Quantitative and Efficient Usability Testing in High Risk System Development Under Diversity of User Groups

Vos, Wendy Marijke (2011) Quantitative and Efficient Usability Testing in High Risk System Development Under Diversity of User Groups.

[img] PDF
Abstract:Infusion pumps are involved in 30% of reported (irreversible) incidents on the ICU and OR, as being dynamic and complex environments characterized by high activity, cognitive strain, extensive use of technology, and time stress (Bogner, 1994). Most designated ‘causes’ involve a variety of user errors, materializing from poorly designed user interfaces. Through evaluation studies, it is widely recognized that poorly designed user interfaces do induce latent errors (Lin, 1998), and operating inefficiencies, even when operated by well-trained, competent users. In the current study, a prototype infusion pump was submitted to a quantitative and efficient usability evaluation test in which our goal was twofold. First, we wanted to gain reliable quantified insights into its safety level, shaped through usability testing and triage heuristics in data analysis. Second, we focused on the quality of the current design with regard to the origin of problems found and if designing for both user groups was possible, respectively using a list of ergonomical and cognitive design principles and problem distribution analysis. With respect to the first research goal, we established that, for reliable quantitative estimates (implementing confidence intervals and variance of defect visibility) of numbers of problems expected, one needs large sample sizes (n). With the use of the LNBzt model we established that even 34 participants did not render an 85% standard for D, as proposed by Nielsen. We only reached an 80% level. Only when eliminating possible false positives (making our data set more efficient) that we reached our goal of 90% for D. Further, by using heuristics, we showed that, contrary to current belief, ‘false positives’ occur when using Retrospective Task Analysis in testing, jeopardizing the high face validity of this method in usability testing. Removing false positives as not being usability problems after all, helped to make progress more favorable (efficient). As to the second research objective, based on Human Factors Engineering principles, we concluded the current modular, split screen design to be a very good basis for further optimizing through (re-)design, confirmed by very positive user ratings. Concerning the design for both user groups, problem distribution slightly differed between both groups, suggesting that it would be better to design with user-profiles, to be loaded when starting the pump. The advantage of such an approach would be that profiles of additional future user groups can, in the near future, be programmed and that, per user group expertise, the interface design can be as intuitive as possible, rendering a highly supportive device.
Item Type:Essay (Master)
Faculty:BMS: Behavioural, Management and Social Sciences
Subject:77 psychology
Programme:Psychology MSc (66604)
Link to this item:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page