Landing Throttleable Hybrid Rockets with Hierarchical Reinforcement Learning in a Simulated Environment

Author

Francesco Alessandro Stefano Mikulis-Borsoi, University of New HampshireFollow

Date of Award

Spring 2020

Project Type

Senior Honors Thesis

College or School

CEPS

Department

Computer Science

Program or Major

Computer Science

Degree Name

Bachelor of Science

First Advisor

Marek Petrik

Abstract

In this paper, I develop a hierarchical Markov Decision Process (MDP) structure for completing the task of vertical rocket landing. I start by covering the background of this problem, and formally defining its constraints. In order to reduce mistakes while formulating different MDPs, I define and develop the criteria for a standardized MDP definition format. I then decompose the problem into several sub-problems of vertical landing, namely velocity control and vertical stability control. By exploiting MDP coupling and symmetrical properties, I am able to significantly reduce the size of the state space compared to a unified MDP formulation. This paper contains two major contributions: 1) the development of a standardized MDP definition framework and 2) a hierarchical MDP structure that is able to successfully land the rocket within the goal bounds more than 95% of the time. I validate this approach by comparing its performance to a baseline RRT search, underlining the advantages of rapid-decision making compared to online planning in the field of Artificial Intelligence (AI).

Recommended Citation

Mikulis-Borsoi, Francesco Alessandro Stefano, "Landing Throttleable Hybrid Rockets with Hierarchical Reinforcement Learning in a Simulated Environment" (2020). Honors Theses and Capstones. 521.
https://scholars.unh.edu/honors/521