GDC 2025 Session Viewer

In For Honor, AI bots are essential for maintaining consistent gameplay in both multiplayer and single-player modes. Traditionally, creating these bots is a labor-intensive process, taking around four weeks per hero to define unique behaviors and difficulty levels. Our challenge was to automate this process to keep up with the game's frequent updates. We utilized reinforcement learning (RL) and self-play techniques, allowing bots to train by playing against existing scripted bots and themselves. This method proved to be efficient in getting bots that are good enough to hold their own against players. To tailor difficulty, we developed an action masking system, giving designers control over the RL bots which is normally a frequent issue with this kind of tech. Additionally, we combined RL with traditional scripted logic, like behavior trees, to create hybrid bots that integrate seamlessly into all game scenarios. These bots have gone through extensive testing to make sure they are up to par with their scripted counterpart and have now shipped in the game. This presentation will share our automated bot creation process, which drastically reduced development time required to make bots. We will also cover how the For Honor production is switching over to this new system for the creation of bots for future heroes. We also aim to provide practical insights into how ML and traditional AI can be synergized to streamline game development.

Takeaway

This is a case study of applying reinforcement learning to automate parts of a game production. The main takeaways are ML/RL can help streamline some aspect of game productions. ML/RL can work hand in hand with a designer. ML/RL can synergize with classic AI. How a live production is integrating ML in their workflow.

Intended Audience

AI programmer, gameplay programmer, game designers, producer, directors, anyone interested in ml applied to game development.