Video Generation & Reasoning
Every week we pick one paper and go deep — video generation, world models, physical reasoning, diffusion, flow matching, and everything in between.
Upcoming
Video Models Are Zero-Shot Learners and Reasoners
The remarkable zero-shot capabilities of Large Language Models (LLMs) have propelled natural language processing from task-specific models to unified, generalist foundation models. Could video models be on a trajectory towards general-purpose vision understanding, much like LLMs developed general-purpose language understanding? We demonstrated that Veo 3 can zero-shot solve a broad variety of tasks it wasn't explicitly trained for: segmenting objects, detecting edges, editing images, understanding physical properties, recognizing object affordances, simulating tool use, completing symmetric patterns, solving mazes, and much more. These emergent abilities indicate that video models are on a path to becoming unified, generalist vision foundation models.
VChain: Chain-of-Visual-Thought for Reasoning in Video Generation
Recent video generation models can produce smooth and visually appealing clips, but they often struggle to synthesize complex dynamics with a coherent chain of consequences. Accurately modeling visual outcomes and state transitions over time remains a core challenge. In contrast, large language and multimodal models exhibit strong visual state reasoning and future prediction capabilities. To bridge these strengths, we introduce VChain, a novel inference-time chain-of-visual-thought framework that injects visual reasoning signals from multimodal models into video generation. Specifically, VChain contains a dedicated pipeline that leverages large multimodal models to generate a sparse set of critical keyframes as snapshots, which are then used to guide the sparse inference-time visual-state adaptation of a pre-trained video generator only at these key moments. Our approach is tuning-efficient, introduces minimal overhead and avoids dense supervision. Extensive experiments on complex, multi-step scenarios show that VChain significantly enhances the quality of generated videos.
Do Joint Audio-Video Generation Models Understand Physics?
TBD
Demystifying Video Reasoning
Recent advances in video generation have revealed an unexpected phenomenon: diffusion-based video models exhibit non-trivial reasoning capabilities. Prior work attributes this to a Chain-of-Frames (CoF) mechanism, where reasoning is assumed to unfold sequentially across video frames. In this work, we challenge this assumption and uncover a fundamentally different mechanism. We show that reasoning in video models instead primarily emerges along the diffusion denoising steps. Through qualitative analysis and targeted probing experiments, we find that models explore multiple candidate solutions in early denoising steps and progressively converge to a final answer, a process we term Chain-of-Steps (CoS). Beyond this core mechanism, we identify several emergent reasoning behaviors critical to model performance: (1) working memory, enabling persistent reference; (2) self-correction and enhancement, allowing recovery from incorrect intermediate solutions; and (3) perception before action, where early steps establish semantic grounding and later steps perform structured manipulation.
Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm
The "Thinking with Text" and "Thinking with Images" paradigms significantly improve the reasoning abilities of LLMs and VLMs — but images capture only single moments, and the separation of text and vision hinders unified multimodal understanding. "Thinking with Video" leverages video generation models such as Sora-2 to use video frames as a unified medium for multimodal reasoning. The accompanying Video Thinking Benchmark (VideoThinkBench) covers vision-centric tasks (e.g., eyeballing puzzles) and text-centric tasks (e.g., GSM8K, MMMU). Sora-2 emerges as a capable reasoner: comparable to SOTA VLMs on vision-centric tasks — surpassing GPT-5 by 10% on eyeballing puzzles — with 92% accuracy on MATH and 69.2% on MMMU.
Video Generation Models are General-Purpose Vision Learners
Driven by next-token prediction, NLP shifted from task-specific models into powerful generalist foundation models. What, then, is the equivalent catalyst needed to achieve a general-purpose model in computer vision? This talk contends that large-scale text-to-video generation serves as a pivotal pre-training paradigm for computer vision, providing the necessary spatiotemporal priors, vision-language alignment, and scalability required for general visual intelligence. GenCeption leverages a pre-trained video generative diffusion backbone to define a feed-forward perception model, capable of performing various vision tasks steered by text instructions — achieving state-of-the-art performance across depth, surface normal, and camera pose estimation, expression-referring segmentation, and 3D keypoint prediction, often matching or surpassing specialized models (e.g., DepthAnything3, SAM3, D4RT, VGGT-Omega, Sapiens, David, Genmo, and Lotus-2). The video generative pretrained backbone outperforms alternative pretraining paradigms, exhibits preliminary data and model scaling properties along with exceptional data efficiency, and triggers intriguing emergent behaviors — suggesting that video generation is not merely a synthesis tool, but a foundational path toward generalist vision intelligence for the physical world.
Past Sessions
Embodied Reasoning with World Models
I'll present a couple methods showing how we can effectively reason with video models. I'll first talk about how we can directly reason with video models. I'll then talk about how video models can be combined with VLMs for hybrid reasoning and with policies for low level reasoning.
Video Models Can Reason with Verifiable Rewards
TBD
Think Visually, Reason Textually: Vision-Language Synergy in ARC
Abstract reasoning from minimal examples remains a core unsolved problem for frontier foundation models. The Abstraction and Reasoning Corpus (ARC-AGI) provides a rigorous testbed for this capability, demanding conceptual rule induction and transfer to novel tasks. We introduce two synergistic strategies: Vision-Language Synergy Reasoning (VLSR), which decomposes ARC-AGI into modality-aligned subtasks, and Modality-Switch Self-Correction (MSSC), which leverages vision to verify text-based reasoning for intrinsic error correction. Our approach yields up to 4.33% improvement over text-only baselines across diverse flagship models.
Can Models Think Without Language? Video as the Next Substrate of Intelligence
Reasoning is often viewed as inseparable from language, driven by the remarkable success of large language models. But is language the only medium through which intelligence can emerge? In this talk, I will present A Very Big Video Reasoning Suite (VBVR), an inaugural effort toward a new paradigm of model reasoning that uses video as the substrate of thought. Unlike language, video naturally captures rich spatial and temporal structure, opening new possibilities for how models perceive, reason about, and make predictions. I will also share recent findings on how reasoning emerges through video generation, and how these developments may reshape the future landscape of large multimodal models.
