A Q-learning predictive control scheme with guaranteed stability

Lukas Beckenbach, Pavel Osinenko, Stefan Streif

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)

Abstract

Model-based predictive controllers are used to tackle control tasks in which constraints on state, input or both need to be satisfied. These controllers commonly optimize a fixed finite-horizon cost, which relates to an infinite-horizon (IH) cost profile, while the resulting closed-loop under the predictive controller yields an in general suboptimal IH cost. To capture the optimal IH cost and the associated control policy, reinforcement learning methods, such as Q-learning, that approximate said cost via a parametric architecture can be employed. Conversely to predictive controllers, however, closed-loop stability has rarely been investigated under the approximation associated controller in explicit dependence of these parameters. It is the aim of this work to incorporate model-based Q-learning into a predictive control setup as to provide closed-loop stability in online learning, while eventually improving the performance of finite-horizon controllers. The proposed scheme provides nominal asymptotic stability and the observation was made that the suggested learning approach could in fact improve the performance against a baseline predictive controller.

Original languageEnglish
Pages (from-to)167-178
Number of pages12
JournalEuropean Journal of Control
Volume56
DOIs
Publication statusPublished - Nov 2020
Externally publishedYes

Keywords

  • Cost shaping
  • Nominal stability
  • Predictive control
  • Q-Learning

Fingerprint

Dive into the research topics of 'A Q-learning predictive control scheme with guaranteed stability'. Together they form a unique fingerprint.

Cite this