This talk will introduce the formal Imitation Learning problem and discuss two main categories of agent training in the Imitation Learning paradigm: behavioral cloning and interactive experts. The talk will also include examples of where it could be used to solve real-world problems and a demonstration of the DAgger algorithm.
Reinforcement Learning has seen an explosion of work in the last few years with some high-profile results such as DeepMind’s success at Go with AlphaZero. Most of the success has been demonstrated on games which, while impressive, have certain properties that don’t translate to real-world challenges faced by practitioners. First, these games can be simulated efficiently at the massive scales needed to train the RL algorithms. Second, many of the games, like Chess, Go, and Breakout, are fully observable, in that everything about the state of the world is available to the learning agent at each iteration. For many sequential decision processes, however, there may be no simulator and the state of the world is only partially observable at any given time.
[Related Article: Reinforcement Learning vs. Differentiable Programming]
Imitation Learning is a related approach to Reinforcement Learning, but instead of having the AI agent learn from scratch through its own exploration, Imitation Learning is about learning decision policies from expert demonstrations. This then becomes a supervised learning problem that tries to learn the policy of the expert rather than a policy that maximizes long-term reward. While Imitation Learning may not produce superhuman-level performance on competitive tasks like games, it can achieve human-level performance on other tasks, such as controlling a tracking camera during a sports broadcast.
