Oliver Jia-Richards

Doctoral Candidate - MIT Space Propulsion Laboratory

Contents

  1. Dyna-style Learning and Planning for Maneuvering a Spacecraft around an Asteroid
  2. Optimization of a Low-Thrust Trajectory from Geostationary Orbit to Landing on an Asteroid
  3. Direct Model Reference Adaptive Control for a Quadcopter Altitude Controller
  4. Modeling of Electric Field and Ion Trajectories Inside a Micro-Satellite Electrospray Thruster

Dyna-style Learning and Planning for Maneuvering a Spacecraft around an Asteroid

Final project for the course "Planning Under Uncertainty" taken at MIT in Fall 2019. Explores the usage of the Dyna algorithm, which combines direct and indirect reinforcement learning, for maneuvering a spacecraft around an obstacle, in this case a very small asteroid. The Dyna algorithm has the agent learn a model of its dynamics and the reward function in order to simulate interactions with the environment in between real experiences. This allows the agent to more effectively update its estimated value function in the precense of model changes, such as if a spacecraft's thruster degrades. The plots below show an example of this property being used in order to safely maneuver the spacecraft around an asteroid when the spacecraft's thrusters degrade to 25% effectiveness. The goal is for the spacecraft to maneuver from its initial position (-2, 0) to the goal region (2, 0) while avoiding collisions with the asteroid. The dash-dot trajectory shows the optimal trajectory if the spacecraft's thrusters are fully functional and the grey trajectory shows the response of standard Q-learning to thruster degradation. Since standard Q-learning only uses actual experiences to update the value function, it is unable to propagate model changes to the value function and the spacecraft crashes into the asteroid. The black trajectory shows the Dyna algorithm where interactions with the environment are simulated 250 or 500 times between actual experiences. Initially, the spacecraft follows the same trajectory as standard Q-learning as it re-learns the model. However, after a few steps, it is able to use the re-learned model in order to correct the value function and safely avoid the asteroid.

Further details can be found in the final report.


Optimization of a Low-Thrust Trajectory from Geostationary Orbit to Landing on an Asteroid

Final project for the course "Optimal Control and Estimation" taken at MIT in Spring 2018.

Further details can be found in the final report.


Direct Model Reference Adaptive Control for a Quadcopter Altitude Controller

Final project for the course "Feedback Control Systems" taken at MIT in Fall 2017. Direct Model Reference Adaptive Control (DMRAC) was tested, both in simulation and in experiment, in order to adapt the gains for a quadcopter altitude controller. DMRAC uses a reference model of the desired system response and tunes the controller gains in order to have the actual system response match the reference. The plots below show an experimental test run where the quadcopter was commanded to move between two altitudes (1 m and 1.5 m). Initially, with incorrect controller gains, the actual system response (solid line) does not match the reference system response (dash-dot line). However, over time the controller adapts its gains such that by the end of the test, the actual system response is similar to the reference. In the right plot, the dashed line represents the expected controller gains for the actual system to match the reference model (both gains should settle to the same value). The actual controller gains settle on a value around twice the expected, indicating modelling errors.

Further details can be found in the final report.


Modeling of Electric Field and Ion Trajectories Inside a Micro-Satellite Electrospray Thruster

Final project for the course "Introduction to Numerical Simulation" taken at MIT in Fall 2017.

Further details can be found in the final project summary.