A pilot study on Mandarin Cued Speech

Overview & Synopsis

Cued Speech (CS) is a communication system developed for deaf people, which exploits hand cuesto complement speechreading at the phonetic level. Currently, it is estimated that CS has beenadapted to over 60 languages; however, no official CS system is available for Mandarin Chinese. Thisarticle proposes a novel and efficient Mandarin Chinese CS system, satisfying the main criterion thatthe hand coding constitutes a complement to the lips’ movements. We propose to code vowels [i, u,y] as semiconsonants when they are followed by other Mandarin finals, which reduces the numberof Mandarin finals to be coded from 36 to 16. We establish a coherent similarity between MandarinChinese and French vowels for the remaining 16 finals/vowels, which allows us to take advantageof the French CS system. Furthermore, by investigating the lips viseme distribution based on a newcorpus, an optimal allocation of the 16 Mandarin vowels to different hand positions is obtained. AGaussian classifier was used to evaluate the average separability of different allocated vowel groups,which gives 92.08%, 92.33%, and 92.73% for the three speakers, respectively. The consonants aremainly designed according to their similarities with the French CS system, as well as some considerationson the special Mandarin consonants. In our system, the tones of Mandarin are coded withhead movements.