Reinforcement Learning and Optimal Adaptive Control: An Overview and Implementation examples

SG Khan, G Herrmann, FL Lewis, AG Pipe, CR Melhuish

Research output: Contribution to journalArticle (Academic Journal)peer-review

110 Citations (Scopus)

Abstract

This paper provides an overview of the reinforcementlearning and optimaladaptive control literature and its application to robotics. Reinforcementlearning is bridging the gap between traditional optimal control, adaptive control and bio-inspired learning techniques borrowed from animals. This work is highlighting some of the key techniques presented by well known researchers from the combined areas of reinforcementlearning and optimal control theory. At the end, an example of an implementation of a novel model-free Q-learning based discrete optimaladaptive controller for a humanoid robot arm is presented. The controller uses a novel adaptive dynamic programming (ADP) reinforcementlearning (RL) approach to develop an optimal policy on-line. The RL joint space tracking controller was implemented for two links (shoulder flexion and elbow flexion joints) of the arm of the humanoid Bristol-Elumotion-Robotic-Torso II (BERT II) torso. The constrained case (joint limits) of the RL scheme was tested for a single link (elbow flexion) of the BERT II arm by modifying the cost function to deal with the extra nonlinearity due to the joint constraints.
Translated title of the contributionReinforcement Learning and Optimal Adaptive Control: An Overview and Implementation examples
Original languageEnglish
Pages (from-to)42 - 59
Number of pages18
JournalAnnual Reviews on Control
Volume36(1)
DOIs
Publication statusPublished - Apr 2012

Fingerprint Dive into the research topics of 'Reinforcement Learning and Optimal Adaptive Control: An Overview and Implementation examples'. Together they form a unique fingerprint.

  • University of Texas at Arlington

    Guido Herrmann (Visitor)

    Dec 2011

    Activity: Visiting an external institution typesVisiting an external academic institution

Cite this