Agenda
Neural Spatial Audio in 'Roco Kingdom'
Maode Shi (Game Engine Specialist, Tencent Games)
YITING LI (Technical Audio Designer, Tencent Games)
Pass Type:
Academic Pass, Festival Pass, Game Changer Pass, Indie Pass, Start-Up Pass -
Get your pass now!
Track:
Programming, Audio
Audience Level:
Intermediate
Format:
Lecture
Vault Recording: TBD
Audience Level: Intermediate
This talk presents a neural spatial audio system designed for real-time game deployment. Unlike existing geometric acoustics approaches that impose ray-tracing costs or demand extensive manual authoring of rooms, portals, and propagation nodes, this system uses lightweight GPU neural networks to infer direct-sound and indirect-sound parameters from the listener's position. A bus-based decomposition makes rendering cost independent of active source count. Manual volume placement and region tagging are eliminated entirely: a one-click GPU cloud bake converts offline ray-traced simulation into compact neural weights loaded directly at runtime. Performance measurements show under 100 microseconds per listener update on desktop GPUs and under 200 microseconds on iPhone 14. Perceptual evaluation confirms near-just-noticeable-difference error across key room-acoustic metrics. The session covers the core architecture, production deployment challenges, and validation data from a shipped implementation.
Takeaway
Attendees will learn how to model and encode direct/indirect acoustic parameters into lightweight neural networks, how to properly train the models and fit them into a basis-bus architecture to contain GPU costs, and how to fit the pipeline into production.
Intended Audience
This session targets audio programmers, technical artists, and engine engineers. Attendees benefit from basic familiarity with acoustic concepts such as reverberation and occlusion, and a general understanding of neural network inference. No prior experience with machine learning training or spatial audio APIs is required.