Neural reinforcement learning for behaviour synthesis

Claude Touzet (1)

1 LNIA - Laboratoire de Neurosciences intégratives et adaptatives

We present the results of a research aimed at improving the Q-learning method through the use of artificial neural networks. Neural implementations are interesting due to their generalisation ability. Two implementations are proposed: one with a competitive multilayer perceptron and the other with a self-organising map. Results obtained on a task of learning an obstacle avoidance behaviour for the mobile miniature robot Khepera show that this last implementation is very effective, learning more than 40 times faster than the basic Q-learning implementation. These neural implementations are also compared with several Q-learning enhancements, like the Q-learning with Hamming distance, Q-learning with statistical clustering and Dyna-Q.

Keywords: Neural Q-learning; reinforcement learning; obstacle avoidance behaviour; self-organising map; autonomous robotics

Domains: Artificial Intelligence [cs.AI]; Machine Learning [cs.LG]; Neural and Evolutionary Computing [cs.NE]; Robotics [cs.RO]

Claude Touzet. Neural reinforcement learning for behaviour synthesis. Robotics and Autonomous Systems, 1997, ⟨10.1016/S0921-8890(97)00042-0⟩. ⟨hal-01337989⟩