Improving commonsense reasoning in multimodal models

Humans navigate the world by filling in observational gaps through commonsense knowledge and reasoning. Although large-scale foundation models have achieved strong performance on commonsense benchmarks, they struggle to reason beyond surface-level patterns, particularly in novel and ambiguous settings. This thesis develops methods to evaluate and improve commonsense reasoning in foundation models across language, vision, and video modalities. We first introduce frameworks that generate and leverage temporal inferences, improving performance on downstream language tasks requiring event reasoning. We then extend this idea to the visual domain with an architecture that incorporates commonsense inferences, improving reasoning abilities for knowledge-intensive Visual Question Answering (VQA). Next, we turn to diagnosing the limits of current models, designing evaluations that reveal systematic weaknesses in their spatio-temporal and causal reasoning. Targeting these gaps, we introduce a human-inspired framework for detecting surprising events in video, improving performance across five video understanding tasks. Overall, our results highlight that explicitly modeling the implicit dimensions of human reasoning, such as commonsense, causes, effects, and beliefs, can significantly enhance the robustness and reasoning abilities of modern AI systems. Looking ahead, our findings motivate future work on agents that reason adaptively, maintain robust internal world models, and integrate richer spatio-temporal understanding for real-world deployment.