GDC is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

View, browse and sort the ever-growing list of sessions by pass type, track, and format. With this Session Viewer, you can view GDC 2023 session details, speakers and share your favorites via social media. You will be able to build your schedule and access it during the show via export or with the Mobile App, once live. Sessions do fill up and seating is first come, first serve, so arrive early to sessions that you would like to attend.

Machine Learning Summit: GPT-3 Powered Text to Lifelike Speech and Animation for NPCs

Dao Si  (AI Team Leader, JNG Studio of NUVERSE, NUVERSE)

Pass Type: All Access Pass, Summits Pass

Topic: Programming

Format: Lecture

Vault Recording: TBD

Audience Level: All

Performance-driven narrative video games needed NPCs' performance to be realistic and depict a wide range of believable emotions. Accurate sentiment analysis and semantic understanding of the text can better help games' audio and animation content generation.

This session describes a novel system in 'Earth Revival', using GPT-3 to measure sentiment and extract semantic features, to automatically synthesize emotional voices and high-quality emotional, expressive full-body animations for talking NPCs. In this system, the speech synthesis system introduces paralinguistic elements to achieve realistic emotional expression, which can produce natural-sounding voices for final game releases or content updates. What's more, the automatic full-body animation generation model uses the multi-modal context of speech text, audio, and speaker identity to produce the arbitrary beat and semantic full-body animation together.

This system of GPT-3 powered text to lifelike speech and animation can significantly improve the narrative process and minimize time and cost.

Takeaway

Attendees will see how a system of GPT-3 powered text to lifelike speech and animation can significantly improve the narrative process and minimize time and cost. They can acquire the implementation detail of each component and improve the development efficiency.

Intended Audience

This is for those who are interested in auto character animation generation, such as animators, technical animators, animation programmers, game designers, and more. Basic knowledge of machine learning techniques is preferred, but not required.