Automated observation of competency-related behavior in serious gaming

This exploratory study uses a cooperative digital game to explore possibilities for automatic observation of competency-related behavior in listening, verbal communication, taking initiative and decision making. During game play, audio and video of participants is recorded. Audio and video is analyzed using vocal and facial expression analysis. Demonstration of competency-related behavior is verified using a self-report and peer survey, completed by participants after game play. Self-report and peer surveys are correlated with vocal, facial expression and game data to determine to what extent competency-related behaviors can be predicted using computerized observations. Results show that, although behavior can to a large extent be predicted using game data, vocal and facial expression analysis, the predictors of these behaviors do not logically explain the behavior predicted. Several propositions are developed to help guide future research.