Mountain Car

Mountain Car
State dimension:	1
Differential states:	2
Discrete control functions:	1

The Mountain Car problem proposes a vehicle stuck in a “well.” It lacks the power to directly climb out of the well, but instead must accelerate repeatedly forwards and backwards until it has achieved the energy necessary to exit the well.

The problem is a popular machine learning test case. It first appeared in the PhD thesis of Andrew Moore in 1990. [2]. The implementation here is taken from [1] and based on that given by Melnikov, Makmal, and Briegel [3]. Its dynamics are given by a two-dimensional ODE model.

The optimal integer control functions exhibits a bang bang structure.

Mathematical formulation

$\begin{array}{lll}\underset{u}{\min }& & t_f\\ \text{subject to}\\ \dot{x}(t)& =& v(t),\\ \dot{v}(t)& =& 0.001\cdot u(t)-0.0025\cdot \cos (3\cdot x(t)),\\ x(0)& =& -0.5,\\ v(0)& =& 0,\\ x(t_f)& =& 0.5,\\ v(t_f)& \ge & 0,\\ u(t)& \in & [-1,1]\mathrm{\forall }t\in [0,t_f]\end{array}$

Reference Solutions

Here is one local solution to the above control problem.

• Reference solution plots
• 
  States and discretized control for a local optimum.

Miscellaneous and Further Reading

This formulation detailed description can be found in [1].

References

[1] Multidisciplinary Optimal Control Library: https://openmdao.org/dymos/docs/latest/examples/mountain_car/mountain_car.html
[2] Andrew William Moore. Efficient memory-based learning for robot control. Technical Report UCAM-CL-TR-209, University of Cambridge, Computer Laboratory, November 1990. URL: https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-209.pdf, doi:10.48456/tr-209.
[3] Alexey A Melnikov, Adi Makmal, and Hans J Briegel. Projective simulation applied to the grid-world and the mountain-car problem. arXiv preprint arXiv:1405.5459, 2014.