Effects of Visualizing Lip Movements on Learning Chinese Orthography

This study examined what the influence was of adding the visualization of lip movements to the translation, orthography and audio of pronunciation on reading skills of Chinese characters. The second question was how the visualization of lip movements affected learning processes and how learning processes affected pronunciation and translation. 51 participants without knowledge of Chinese learned 40 Chinese characters. We compared learning in two conditions, in a within-subjects design. First condition: participants learned 20 characters with a person pronouncing the character next to the translation and written Chinese character. Second condition: participants learned another 20 characters with only the audio of pronunciations of characters and next to the translation and the written character. All participants were tested on performance of pronunciation and translation. This study showed that visualizing pronunciations influenced recalling pronunciations and translations, but only when participants familiarized with Chinese first with only the audio of pronunciations. Visualizing pronunciations did not influence learning processes, but learning translations had a negative relation with the amount of times listened to the audio clips. Results suggested that prior knowledge is needed to reduce cognitive load to improve recall and that passively learning by just repetition of audio clips harms learning translations.