Professor of Psychology, University of California, Santa Cruz
Developing and Evaluating Embodied Agents
Speech perception and communication are usually successful becauseperceivers optimally integrate several sources of information,particularly information from the face as well as the voice. Ourresearch agenda aims to create computer-animated agents that produceaccurate auditory and visible speech, as well as realistic facialexpressions, emotions, and gestures. The invention of such agents hasawesome potential to benefit virtually all individuals, but especiallythose with hearing, speech, and communication problems. Ourcomputer-animated talking head, Baldi, speaks in real time on aninexpensive PC platform and involves phoneme synthesis controlled bycoarticulation constraints. Recent modifications include additionaland modified control parameters, texture mapping, three generations ofa tongue, hard palate, controls for paralinguistic information andaffect in the face, text-to-visible speech synthesis, alignment withnatural speech, and auditory speech to visible speech synthesis. Thetalk will describe this technology, the evaluation of itseffectiveness, how it is used in psychological experimentation to testtheories of pattern recognition, and its applications in languagetraining and education.