From building towers to picking an orange from a stack of fruit, assessing support is critical for successfully interacting with the physical world. But how do people determine whether one object supports another? In this paper we develop the Counterfactual Simulation Model (CSM) of physical support. The CSM predicts that people judge physical support by mentally simulating what would happen to a scene if the object of interest were removed. Three experiments test the model by asking one group of participants to judge what would happen to a tower if one of the blocks were removed, and another group of participants how responsible that block was for the tower’s stability. The CSM accurately captures participants’ predictions about what would happen by running noisy simulations that incorporate different sources of uncertainty. Participants’ responsibility judgments are closely related to counterfactual predictions: the more likely the tower would be predicted to fall if a block were removed, the more responsible this block was judged for the tower’s stability. By construing physical support as preventing from falling, the CSM provides a unified account across dynamic and static physical scenes of how causal judgments arise from the process of counterfactual simulation.