Abstract:

Robotic setups often need finetuned controller parameters both at low and tasklevels. Finding
an appropriate set of parameters through simplistic protocols, such as manual tuning or
grid search, can be highly timeconsuming. This thesis proposes an automatic controller tuning
framework based on linear optimal control combined with Bayesian optimization. With
this framework, an initial set of controller gains is automatically improved according to the
performance observed in experiments on the physical plant.
In the tuning scenario that we propose, we assume we can measure the performance of the
control system in experiments through an appropriate cost. However, we only allow a limited
number of experimental evaluations (e.g. due to their intrinsic monetary cost or effort). The
goal is to globally explore a given range of controller parameters in an efficient way, and return
the best known controller at the end of this exploration.
At each iteration, a new controller is generated and tested on a closedloop experiment in
the real plant. Then, the recorded data is used to evaluate the system performance using a
quadratic cost. We reiterate in order to solve a global optimization problem, whose goal is to
learn most about the location of the global minimum from the limited number of experiments.
We use the Linear Quadratic Regulator (LQR) formulation as a standard way to compute
optimalmultivariate controllers given a linear plant model and quadratic cost. We parametrize
the LQR weights in order to obtain controllers with appropriate robustness guarantees.
The underlying Bayesian optimization algorithm is Entropy Search (ES), which represents
the latent objective as a Gaussian process and constructs an explicit belief over the location of
the objective minimum. This method maximizes the information gain from each experimental
evaluation. Thus, this framework shall yield improved controllers with fewer evaluations
compared to alternative approaches.
A sevendegreeoffreedomrobot arm balancing an inverted pole is used as the experimental
demonstrator. Results of a twodimensional tuning problem are shown in two different
contexts: in the first setting, a wrong linear model is used to compute a nominal controller,
which destabilizes the actual plant. The automatic tuning framework is still able to find a stabilizing
controller after a few iterations. In the second setting, a fairly good linearmodel is used
to compute a nominal controller. Even then, the framework can still improve the initial performance
by about 30%. In addition, successful results on a fourdimensional tuning problem
indicate that the method can scale to higher dimensions.
Themain and novel contribution of this thesis is the development of an automatic controller
tuning framework combining ES with LQR tuning. Albeit ES has been tested on simulated
numerical optimization problems before, this work is the first to employ the algorithm for
controller tuning and apply it on a physical robot platform. Bayesian optimization has recently
gained a lot of interest in the research community as a principled way for global optimization
of noisy, blackbox functions using probabilistic methods. Thus, this work is an important
contribution towardmaking thesemethods available for automatic controller tuning for robots.
In conclusion, we demonstrate in experiments on a complex robotic platform that Bayesian
optimization is useful for automatic controller tuning. Applying Gaussian process optimization
for controller tuning is an emerging novel area. The promising results of this work open
up many interesting research directions for the future.
In future work, we aim at scaling this framework further than this problem to higher dimensional
systems such as balancing of a full humanoid robot for which a rough linear model
can be obtained. In addition, comparing ES with other global optimizers from the literature
may be of interest. Investigating safety considerations such as avoiding unstable controllers
during the search is also part of future research. 