Adaptive Dynamic Programming Using Lyapunov Function Constraints

Thomas Gohrt, Pavel Osinenko, Stefan Streif

Research output: Contribution to journalArticlepeer-review

6 Citations (Scopus)


This letter is concerned with a stabilizing adaptive dynamic programming (ADP) approach to approximate solution of a given infinite-horizon optimal control problem. Since the latter problem cannot, in general, be solved exactly, a parametrized function approximator for the infinite-horizon cost function is introduced in ADP (so called 'critic'). This critic is used to adapt the parameters of the function approximator. The so called 'actor' in turn derives the optimal input of the system. It is a notoriously hard problem to guarantee closed-loop stability of ADP due to the use of approximation structures in the control scheme. Since at least stabilizability is always assumed in the analyzes of ADP, it is justified to invoke a respective Lyapunov function. The proposed ADP scheme explicitly uses the said Lyapunov function to simultaneously optimize the critic and guarantee closed-loop stability. A Hessian-free optimization routine is utilized for the actor and critic optimization problems. Convergence to prescribed vicinities of the optima is shown. A computational study showed significant performance improvement for the critic-based approach compared a nominal stabilizing controller for a range of initial conditions.

Original languageEnglish
Article number8725559
Pages (from-to)901-906
Number of pages6
JournalIEEE Control Systems Letters
Issue number4
Publication statusPublished - Oct 2019
Externally publishedYes


  • Adaptive control
  • optimal control
  • optimization algorithms


Dive into the research topics of 'Adaptive Dynamic Programming Using Lyapunov Function Constraints'. Together they form a unique fingerprint.

Cite this