Stability AI Taps into GenAI Video Generation with Stable Video 4D

Stability AI has debuted Stable Video 4D, adding a new dimension to its array of generative AI models. As the market starts to saturate with generative AI tools for video generation, Stable Video 4D claims to be different. Built on the foundation of Stability AI’s Stable Video Diffusion model, the new model takes the concept further by accepting video input and generating multiple novel-view videos from 8 different perspectives. 

“We see Stable Video 4D being used in movie production, gaming, AR/VR, and other use cases where there is a need to view dynamically moving 3D objects from arbitrary camera angles,” Varun Jampani, Team Lead, 3D Research at Stability AI, told VentureBeat.

Jampani explained that the four dimensions include width (x), height (y), depth (z), and time (t). That means Stable Video 4D can view a moving 3D object from various camera angles as well as at different timestamps. 

“The key aspects that enabled Stable Video 4D are that we combined the strengths of our previously-released Stable Video Diffusion and Stable Video 3D models and fine-tuned it with a carefully curated dynamic 3D object dataset,” he explained.

Further, Stable Video 4D is a first-of-its-kind network where a single network does both novel view synthesis and video generation. The AI model can process single-object videos of several seconds with a plain background, and the company plans to generalize it to longer videos and complex scenes.

Stable Video 4D is now available for research and evaluation on Hugging Face, and Stability AI is yet to reveal the availability of its commercial options for the future.