People must often infer what might have transpired in the past to bring about the present state of the world, a task called retrodiction. We hypothesize that retrodiction relies on similar cognitive mechanisms to prediction – inferring possible futures based on the present state of the world. Here we investigate how people perform on physical reasoning tasks that differ only in that people are asked to do either prediction or retrodiction. We find that average behavior is similar between tasks across a range of difficulty, though there was greater variability in retrodiction responses. We propose two ways in which prediction and retrodiction might be related; however, neither sufficiently explains the similarities and differences across tasks. We suggest that both tasks rely on similar cognitive processes, but that further research is needed to determine the exact relation.