GDC is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Use the GDC 2021 Session Viewer to browse and sort the ever-growing list of sessions by pass type, topic, and format. You will be able to build your personal schedule directly within the event platform, once live early July 2021.

Machine Learning Summit: Full-Body Animation Generation for Expressive NPCs

Yu Ding  (Animation researcher, Netease)

Pass Type: All Access Pass, Summits Pass

Topic: Programming

Format: Session

Vault Recording: TBD

Animations are extremely important to augment the believability and effectiveness of non-player characters (NPCs) for game users. Our session describes a novel approach, relying on deep learning technologies, to automatically synthesize high-quality and life-like full-body animations for talking NPCs. The animations involve lip and chin, upper facial expression (eyebrows, upper and bottom eyelids, and eyeballs), head rotation, torso and hand gestures, and legs and feet. The synthesized animations can reflect speech prosody, express the emotional state expressed by speech and correspond to specific personalities and professions of NPCs. Without any manual intervention, it takes only less than 500ms to compute full-body animation trajectories for an utterance which lasts about 5 to 20 seconds. The automatic animation generator allows animators or artists to be released from manually creating animations and processing motion capture data.


Attendees will learn how to build an animation generator with a state-of-the-art approach to automatically computing animation trajectories. They will learn about the technical details, potential benefits, and practical challenges of state-of-the-art probabilistic models and computational algorithms for animation synthesis technologies. Attendees who are more interested in understanding how to adopt deep learning into production will also benefit from the presentation.

Intended Audience

This presentation is appropriate for anyone with an interest in facial expression and body language generation, multimodal data processing, or applying deep learning methods into the game industry as well as producers, technical directors, or artists who are curious about how these technologies shifts are affecting the industry. The presentation will provide some basic concepts on algorithms and will not be targeted toward only those working with machine/deep learning.