Data science and artificial intelligence are everywhere. So are video games. It’s no surprise that it was only a matter of time until people started getting creative with combining the two in unique ways. And no, I’m not talking about improving in-game AI (because clearly, Skyrim doesn’t care), and I’m not talking about analyzing game sales either. Today, I want to look at some fun ways that people have used reinforcement learning in the world of gaming, ranging from creating new AI to beating the game using bots.
First of all, what is reinforcement learning?
Short answer (aka the answer for non-data scientists): it’s a data science thing. Long answer (for the data scientists): Reinforcement learning is a paradigm of learning, where the AI agent learns through experience. It interacts with the environment and gets rewards from the environment as feedback. In gamer terms, when you’re fighting an NPC or boss and it keeps learning how you play and gets harder as time goes on. Or if you’re lucky, you get a companion AI that gets better, not worse, by working with you.
Now, to connect that logic to data science, some smart folks have created their own AI/bots that learn how the game works, and keep getting better through experience. You can also think of it as learning by experience, or through trial and error. Because of this experimental nature, agents (aka the AI), can learn a set of rules even in an environment or situation it’s never been in before.
Starting simple with tic-tac-toe
Gaming isn’t just PS5 and OP PCs – it can be as simple as good ‘ol Tic-Tac-Toe. With their online portal, KNIME developed a tic-tac-toe bot that learned as it played. By creating two agents that played against each other a set number of times, and by being given a very vague general idea of the rules of tic-tac-toe, these agents were able to learn the game and be quite formidable. In an experiment where the two agents were allowed to play more games to better learn the game, they were given the “hard mode” title.
Moving on to Chess
Chess is a lot more complicated than the 3×3 square in tic-tac-toe. It has a broader grid to move across, more rules in general, and more pieces with their own rules. It’s hard enough for a human to learn Chess, let alone an AI agent. A while back, Google’s subsidiary DeepMind repurposed AlphaGo, a game-playing AI known for beating world-famous Go players, to play Chess instead, now calling it AlphaZero. In less than 4 hours of learning Chess from start to finish, AlphaZero was able to beat the world’s best Chess program, Stockfish 8, in a 100-game match-up.
Going virtual with Atari
Atari had more than just Pong and that terrible E.T. game, so it was a fun place for some researchers to start. DeepMind also used RL to play Atari a bit, training an agent to play Pong, Space Invaders, and more. In their research paper, they created a single agent that could play seven different games (which is still probably more than your out-of-touch parents can play), reliably, and able to surpass the skill of a human player. See it in action here.
Researchers from NVIDIA also explored the design and implementation of a CUDA port of the Atari Learning Environment (ALE), a system for developing and evaluating DRL algorithms using Atari games.
Doting with Dota 2
A few years ago, OpenAI created the OpenAI Five, a team of five AI agents designed to play the MOBA game, Dota 2 (Defense of the Ancients), using reinforcement learning, to play against human players. In fact, this project really supported the idea of learning over time; in the span of a few months, the agents when from just competing with high-ranking players to demolishing even the best players in the competition. I’m going to use that as my excuse for why I lost really, really badly when I (once) attempted Dota 2. Yep. AI.
Actually designing games with reinforcement learning
Reinforcement learning isn’t just used for creating OP AI – but it can be used to design games and game levels too. RL has been used for procedural content generation, which means randomly generated levels/scenarios/bosses/etc. For non-gamers, think of it as playing a level of Mario, and each new level is completely unique each time you play it. For those really out of the loop, it’s like driving to work but your office is different every day you go in. It keeps things exciting.
Hope for better AI
Here’s where I get really curious – Microsoft and Ninja Theory working together to make better AI. Project Paidia is a collaboration project between the two to use reinforcement learning to develop better gaming AI, such as for team-based games that require excellent collaboration. They give an example of the Ninja Theory game Bleeding Edge that requires players to work with AI to score points and achieve goals. It’s still early, but I’m hopeful that this will lead to better AI companions in gaming.
Taking over the galaxy in StarCraft II
Possibly the example that got the most news attention in recent memory, another AI from Google’s DeepMind – AlphaStar – was able to place in the top 0.15% of all game players in StarCraft II, an incredibly popular online game with thousands of players – all known for being quite devoted and passionate about becoming an expert at the game.
Hope for the future
When you combine everything above, there are a few common themes among the possibilities that reinforcement learning can bring to gaming. We’re on the cusp of better random level generation, final boss-tier AI in competition, and most importantly, better AI companions. Not everything will be improved by tomorrow, but with so many tech giants now involved in gaming, the future is bright for AI and gaming.
Learn more about reinforcement learning
There’s quite a lot that you can do with reinforcement learning – whether it’s related to video games or not. The core skills can be used across a variety of purposes, from stock trading and finance to cybersecurity and art. Regardless of your application, there’s always a use for reinforcement learning. If you register for our upcoming Ai+ Training session, “Reinforcement Learning for Game Playing and More,” you’ll gain the core skills you need to apply RL however you want. Session highlights include:
- Gain knowledge of the latest algorithms used in reinforcement learning.
- Understand OpenAI Gym environment
- Build your custom environment in Gym
- Using TensorFlow build an RL agent to play the Game of Atari
- Learn to apply RL in tasks other than games
Plus, as more and more organizations learn about the benefits of reinforcement learning, it’s a great way for you to stand out and find your niche instead of being just a machine learning engineer – gotta stand out in the job market somehow!
