The geometry of the solution set of nonlinear optimal control problems

In an optimal control problem one seeks a time-varying input to a dynamical systems in order to stabilize a given target trajectory, such that a particular cost function is minimized. That is, for any initial condition, one tries to find a control that drives the point to this target trajectory in the cheapest way. We consider the inverted pendulum on a moving cart as an ideal example to investigate the solution structure of a nonlinear optimal control problem. Since the dimension of the pendulum system is small, it is possible to use illustrations that enhance the understanding of the geometry of the solution set. We are interested in the value function, that is, the optimal cost associated with each initial condition, as well as the control input that achieves this optimum. We consider different representations of the value function by including both globally and locally optimal solutions. Via Pontryagin's maximum principle, we can relate the optimal control inputs to trajectories on the smooth stable manifold of a Hamiltonian system. By combining the results we can make some firm statements regarding the existence and smoothness of the solution set.