Evolving neural network agents in the NERO video game.pdf

In Proceedings of the IEEE 2005 Symposium on Computational Intelligence and Games (CIG’05). Piscataway, NJ: IEEE Winner of the Best Paper Award at CIG’05 Evolving Neural Network Agents in the NERO Video Game Kenneth O. Stanley Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 USA kstanley@ Bobby D. Bryant Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 USA bdbryant@ Risto Miikkulainen Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 USA risto@ Abstract- In most modern video games, character be- havior is scripted; no matter how many times the player exploits a weakness, that weakness is never repaired. Yet if game characters could learn through interacting with the player, behavior could improve during game- play, keeping it interesting. This paper introduces the real-time NeuroEvolution of Augmenting Topologies (rt- NEAT) method for evolving increasingly complex arti- ficial neural networks in real time, as a game is being played. The rtNEAT method allows agents to change and improve during the game. In fact, rtNEAT makes possible a new genre of video games in which the player teaches a team of agents through a series of customized training exercises. In order to demonstrate this concept in the NeuroEvolving Robotic Operatives (NERO) game, the player trains a team of robots for combat. This paper describes results from this novel application of machine learning, and also demonstrates how multiple agents can evolve and adapt in video games like NERO in real time using rtNEAT. In the future, rtNEAT may allow new kinds of educational and training applications that adapt online as the user gains new skills. 1 Introduction The world video game market in 2002 was between $15 billion and $20 billion, larger than even that of Hollywood (Thurrott 2002). Video games have become a facet of many people’s lives and the market continues to expand. Because there are millio