University of Twente Student Theses




[img] PDF
Abstract:Objectives: This study evaluates the reliability of the mystery shopping method by testing the accuracy of the mystery shopper when reporting facts and investigating the possible presence of halo effects in mystery shopping reports. Furthermore, this study evaluates the influence of time delay between observation and reporting on the accuracy of mystery shopping reports and the possible relationship between time delay and halo effects. Method: A 2*3 experimental design was set up (employee with sufficient expertise vs. employee without sufficient expertise and no time delay vs. 1 hour time delay vs. 24 hours time delay). 94 mystery shoppers visited a service desk thinking they were investigating the service quality of that service desk. If fact, the behavior of the mystery shopper was the subject of the study and the participants did not know the situation was set up. To test the accuracy of mystery shoppers, the mystery shoppers observed six factual environmental factors which they could report either correctly or incorrectly afterwards. To test possible halo effects, the behavior of the employee was negatively manipulated. When a mystery shopper encountered an employee without sufficient expertise, it was tested if other constructs (physical environment, policies & proficiencies, overall evaluation) were also evaluated more negatively, which indicates a halo effect. To test the influence of time delay, the mystery shoppers had to fill in the questionnaire corresponding to one of the three time delay conditions. Results: The current study indicates that mystery shoppers are for 71% accurate when they do not work under time pressure. When mystery shoppers do experience time pressure, they are only for 48% accurate. Having previous mystery shopping experience also influences the accuracy of mystery shoppers positively. At least nine mystery shopping visits per service outlet are necessary to obtain accurate mystery shopping results. Halo effects were found within the employee construct and on two policy & proficiencies items. No halo effects on the physical environment construct and on the four other policy & proficiencies items were found. Besides, time delay between observation and reporting (until 24 hours) does neither influence the accuracy of mystery shoppers, nor does it increase halo effects in mystery shopping reports. Discussion: The current study shows that mystery shoppers do not always provide accurate data. To increase the reliability of mystery shopping, this study suggests that mystery shoppers should not work under time pressure, experienced mystery shoppers should be hired and at least 9 mystery shopping visits per outlet should be executed. Furthermore, halo effects could be present in mystery shopping reports, especially within the employee construct, though they do not seem very threatening. No halo effects were found on the physical environment, so mystery shopping data on this subject is reliable. Time delay between observation and reporting (until 24 hours) does not threaten the reliability of mystery shopping reports, since no differences were found within the three time delay conditions regarding accuracy and halo effects.
Item Type:Essay (Master)
Faculty:BMS: Behavioural, Management and Social Sciences
Subject:05 communication studies
Programme:Communication Studies MSc (60713)
Link to this item:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page