Mountain Car: Difference between revisions
RobertLampel (talk | contribs) |
RobertLampel (talk | contribs) No edit summary |
||
| (12 intermediate revisions by the same user not shown) | |||
| Line 5: | Line 5: | ||
}} | }} | ||
The '''Mountain Car problem''' proposes a vehicle stuck in a | The '''Mountain Car problem''' proposes a vehicle stuck in a "well". It lacks the power to directly climb out of the well, but instead must accelerate repeatedly forwards and backwards until it has achieved the energy necessary to exit the well. | ||
The problem is a popular machine learning test case. It first appeared in the PhD thesis of Andrew Moore in 1990 | The problem is a popular machine learning test case. It first appeared in the PhD thesis of Andrew Moore in 1990 [[#Moo90 | [2]]]. The implementation here is taken from [[#openmdao | [1]]] and based on that given by Melnikov, Makmal, and Briegel [[#MMB14 | [3]]]. | ||
Its dynamics are given by a two-dimensional [[:Category:ODE model|ODE model]]. | Its dynamics are given by a two-dimensional [[:Category:ODE model|ODE model]]. | ||
== Mathematical formulation == | == Mathematical formulation == | ||
| Line 22: | Line 20: | ||
\quad x(0) &=& -0.5, \\ | \quad x(0) &=& -0.5, \\ | ||
\quad v(0) &=& 0, \\ | \quad v(0) &=& 0, \\ | ||
\quad x(t_f) &=& | \quad x(t_f) &=& 0.5, \\ | ||
\quad v(t_f) & \geq & 0, \\ | \quad v(t_f) & \geq & 0, \\ | ||
\quad u(t) & \in & [-1, 1] \ \quad \forall t \in [0,t_f] | \quad u(t) & \in & [-1, 1] \ \quad \forall t \in [0,t_f] | ||
| Line 33: | Line 31: | ||
Here is one local solution to the above control problem. | Here is one local solution to the above control problem. | ||
<gallery caption="Reference solution plots" widths=" | <gallery caption="Reference solution plots" widths="500px" heights="300px" perrow="1"> | ||
Image:Mountain_Car.png| States and discretized control for a local optimum. | Image:Mountain_Car.png| States and discretized control for a local optimum. | ||
</gallery> | </gallery> | ||
== Miscellaneous and Further Reading == | == Miscellaneous and Further Reading == | ||
This formulation detailed description can be found in [[# | This formulation and a detailed description can be found in [[#openmdao|[1]]]. | ||
== References == | == References == | ||
<span id=" | <span id="openmdao">[1]</span> Multidisciplinary Optimal Control Library: https://openmdao.org/dymos/docs/latest/examples/mountain_car/mountain_car.html<br> | ||
<span id="Moo90">[2]</span> Andrew William Moore. Efficient memory-based learning for robot control. Technical Report UCAM-CL-TR-209, University of Cambridge, Computer Laboratory, November 1990. URL: https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-209.pdf, doi:10.48456/tr-209.<br> | |||
<span id="MMB14">[3]</span> Alexey A Melnikov, Adi Makmal, and Hans J Briegel. Projective simulation applied to the grid-world and the mountain-car problem. arXiv preprint arXiv:1405.5459, 2014.<br> | |||
[[Category:MIOCP]] | [[Category:MIOCP]] | ||
[[Category: | [[Category:Bang bang]] | ||
Latest revision as of 08:27, 27 January 2026
| Mountain Car | |
|---|---|
| State dimension: | 1 |
| Differential states: | 2 |
| Discrete control functions: | 1 |
The Mountain Car problem proposes a vehicle stuck in a "well". It lacks the power to directly climb out of the well, but instead must accelerate repeatedly forwards and backwards until it has achieved the energy necessary to exit the well.
The problem is a popular machine learning test case. It first appeared in the PhD thesis of Andrew Moore in 1990 [2]. The implementation here is taken from [1] and based on that given by Melnikov, Makmal, and Briegel [3]. Its dynamics are given by a two-dimensional ODE model.
Mathematical formulation
Reference Solutions
Here is one local solution to the above control problem.
- Reference solution plots
-
States and discretized control for a local optimum.
Miscellaneous and Further Reading
This formulation and a detailed description can be found in [1].
References
[1] Multidisciplinary Optimal Control Library: https://openmdao.org/dymos/docs/latest/examples/mountain_car/mountain_car.html
[2] Andrew William Moore. Efficient memory-based learning for robot control. Technical Report UCAM-CL-TR-209, University of Cambridge, Computer Laboratory, November 1990. URL: https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-209.pdf, doi:10.48456/tr-209.
[3] Alexey A Melnikov, Adi Makmal, and Hans J Briegel. Projective simulation applied to the grid-world and the mountain-car problem. arXiv preprint arXiv:1405.5459, 2014.