舞出言语 : 语音与肢体动作之间的跨感官对应关系 = Dancing to tongues : an investigation of crossmodal correspondence between speech sounds and body movements

Abstract

Can we transfer our experience in one sensory modality to another sensory modality, for instance, from sights to sounds? Researchers of crossmodal correspondence have discovered many such curious perceptual congruencies in their explorations. This research examines the perceived matching of speech sounds to physical movements as an investigation of sound symbolic crossmodal correspondences. Chearles, Farm, & Styles (2014) showed that participants with dance backgrounds could accurately match syllabic speech sounds with its corresponding dance movements produced by an experienced dancer. As an extension of this novel paradigm in psycholinguistics, we compared the accuracy of Mandarin Chinese speakers and non-Mandarin speakers in matching Mandarin-occuring speech sounds with its corresponding dance movements. An experienced dancer was invited to perform dance movements to a selected list of four phonemes, carefully controlled for three conditions of duration and rhythm of articulation, based on her own interpretation. These motions were recorded on video and presented to participants online, who chose the sound they feel is best suited to the movement. Results show that participants in both language groups were able to reliably match the correct movement to its corresponding sound, even when looking at rhythm or phoneme trials alone, thereby indicating a high degree of intersensory mappings across visual, auditory and kinaesthetic domains.