Efficient Physical Prediction and Inference

Making predictions in real-world situations is difficult, as we must be able to deal with scenes that have a large number and variety of objects in many different configurations. In addition, physics is a chaotic system, where imperceptible details can influence the outcome: a gust of wind can blow a frisbee off course, or a ball hitting a rough wall can take an unpredictable bounce. Despite these challenges, people are able to produce relatively accurate predictions and inferences quickly enough to act on the world. I study the approximations this system makes to be able to work so efficiently and so robustly.

In my research, I have found that the mind can handle these challenges by sampling from a small set of stochastic physical simulations. Adding noise to our simulations can help smooth over the imperceptible features that cause physical outcomes to diverge. Sampling a set of these simulations allows us to understand the range of possible outcomes the world might take. And we often can get away with using just a small number of these samples to efficiently make and update predictions over time.

Inference is a more challenging problem to solve than prediction: if we have a mental model of how the world works it is easy to transform the initial state of the world into a hypothesized future state; however, it is much more computationally difficult to reverse the process to infer the initial state from the final state. I have found that we make and update these inferences by keeping track of a small number of hypotheses that are updated based on how well observations make the physical behavior expected under each hypothesis. This inference algorithm can be approximated in a neurally plausible system, providing a testable hypothesis for how the brain performs this information processing.

Related papers:

Kevin A Smith
Research Scientist in Brain and Cognitive Sciences