Facial Animation
Geometry Manipulation Based on a Pseudo Muscle Model
Lucia an Italian Talking Head
Piero Cosi
ISTITUTO DI SCIENZE E TECNOLOGIE DELLA COGNIZIONE
SEZIONE DI PADOVA - “FONETICA E DIALETTOLOGIA”
Via Martiri della libertà, 2 – 35137 adova (Italy)
e-mail: [email protected] - www: http://www.pd.istc.cnr.it
Incontro con Frank H. Guenther
Director of CNS Speech Lab, Boston University, Boston MA
"Neurolinguistica: produzione e percezione del parlato e controllo senso
motorio"
Martedì 22 settembre 2009 - ore 10,30
Sala delle Statue, Palazzo Rospigliosi
Via XXIV Maggio, 43 - Roma
Copyright, 2009 © ISTC-SPFD-CNR
ISTC Applications
SONIC
SPHINX
CLSU Toolkit
ILT – Italian Literacy Tutor
Copyright, 2009 © ISTC-SPFD-CNR
Outline
Motivation
Who is working on similar … stuff?
 Movie Companies & Studios
 Animation Software Companies
 Research Institutes and Universities
ISTC-SPFD – LUCIA & FESTIVAL
 Emotions (Audio & Video)
and….what about the future!?
Motivation
human-like conversational agents
(Virtual Characters, Computer Characters, Virtual Agents,
Conversational Agents, Interface Agents…)
human-like characters
they interact naturally, using facial gestures, eye contact,
speech patterns, posture and gesture, just like we might
use them when conversing with other people
one might see them in computer games or in the media
users can interact and cooperate with them to complete a
shared task
Soggetti Coinvolti
Movie Companies & Studios
Games
ecc.
animazione
ecc.
BALDI
LUCIA
GRETA
ecc.
ecc.
PC, web, telefonia, ...
RICERCA
(modelli fisici)
LUCIA a new emotive/expressive Italian
talking head
http://www2.pd.istc.cnr.it/LUCIA
ELITE System
A/D
(16KHz)
Acoustic signals
A / D (100Hz)
Articulatory signals
ELITE
Video Processor
Track Animation example
Facial Animation
Facial
Modelling / Animation
Geometry
manipulations
Parametrization
Pseudo
muscle
model
Pure vector
based
model
Spline
model
Free form
deformation
Wrinkle
generation
Facial Animation & TTS
FESTIVAL
Italian TTS
segmental/lexical/semantic
information
WAV
LUCIA
Input Text
AVengine
FAP
(TXT2Animation)
http://www2.pd.istc.cnr.it/FESTIVAL
http://www2.pd.istc.cnr.it/LUCIA
emotions and expressions
universal expressions (fear)
Audiovisual Emotive/Expressive Speech
neutral
sadness
fear
Lucia
disgust
joy
Emotion Player
Copyright, 2007 © ISTC-SPFD-CNR
“ILT” – Italian Literacy Tutor
“Interactive Books
MANY THANKS!
and
WELCOME
to
INTERSPEECH
2011
Firenze
27-31 August 2011