pipette
ESEspañol

EPIC: Epipolar-Consistent 360{\deg} Immersive Stereo Video Generation

Debabrata Mandal, Dongdong Fu, Jonathon Miller, William Villareal, Xi Peng, Praneeth Chakravarthula

PreprintBold claims, read criticallyReal-world use

In the authors' words

Immersive displays can enable rich and diverse virtual experiences. Manually authoring every possible experience to realize this potential, however, is prohibitively expensive, difficult to scale, and impractical. Generative AI models could remove this bottleneck, but today's models are built for conventional displays and cannot generate the high-resolution, stereoscopic content required for immersive viewing. Further, temporal and stereo inconsistencies that may be tolerable on conventional displays can become highly disruptive when viewed through an immersive headset. Here, we address this gap with a zero-shot generative pipeline that extends existing video diffusion models into 4K stereoscopic videos. Inspired from binocular vision and depth perception, we develop an epipolar-aware image matching metric that captures the temporal and stereo geometric inconsistencies across views. We then use this metric as a preference signal for direct preference optimization with limited training data. Our work enables stereo video generation and provides a scalable path for bringing generative content to immersive displays, allowing diverse mixed reality experiences on demand.

Main resultLimitation the authors admit

Appeared: Thursday, October 1. arXiv. Preprint, not yet peer-reviewed.