GDC is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

View, browse and sort the ever-growing list of sessions by pass type, topic, and format. With this Session Scheduler, you can build your schedule in advance and access it during the show via export or with the GDC Mobile App, once live. Sessions do fill up so please arrive early to sessions that you would like to attend. Adding a session to your schedule does not guarantee you a seat.

Machine Learning Summit: Full-Body Animation Generation for Expressive NPCs

Yu Ding (Animation Researcher, NetEase, Inc., China)

Pass Type: All Access Pass, Core+Summits Pass, Summits Pass - Get your pass now!

Topic: Programming

Format: Session

Vault Recording: TBD

Audience Level: All

Animations are extremely important to augment the believability and effectiveness of non-player characters (NPCs) for game users. Our session describes a novel approach, relying on deep learning technologies, to automatically synthesize high-quality and life-like full-body animations for talking NPCs. The animations involve lip and chin, upper facial expression (eyebrows, upper and bottom eyelids, and eyeballs), head rotation, torso and hand gestures, and legs and feet. The synthesized animations can reflect speech prosody, express the emotional state expressed by speech and correspond to specific personalities and professions of NPCs. Without any manual intervention, it takes only less than 500ms to compute full-body animation trajectories for an utterance which lasts about 5 to 20 seconds. The automatic animation generator allows animators or artists to be released from manually creating animations and processing motion capture data.


Attendees will learn how to build an animation generator with a state-of-the-art approach to automatically computing animation trajectories. They will learn about the technical details, potential benefits, and practical challenges of state-of-the-art probabilistic models and computational algorithms for animation synthesis technologies. Attendees who are more interested in understanding how to adopt deep learning into production will also benefit from the presentation.

Intended Audience

This presentation is appropriate for anyone with an interest in facial expression and body language generation, multimodal data processing, or applying deep learning methods into the game industry as well as producers, technical directors, or artists who are curious about how these technologies shifts are affecting the industry. The presentation will provide some basic concepts on algorithms and will not be targeted toward only those working with machine/deep learning.