In a significant leap for artificial intelligence research, Stability AI introduced a cutting-edge tool on Tuesday known as Stable Video Diffusion. This free AI research tool has the ability to transform any static image into a brief video, showcasing the continuous evolution of AI capabilities. While it marks a pioneering step in AI video synthesis, Stability AI acknowledges that the technology is still in its early stages.
Stability AI made waves last year with the launch of Stable Diffusion, an open-weights image synthesis model that triggered a surge of interest and experimentation within the AI community. Now, with Stable Video Diffusion, the company aims to replicate that success in the realm of AI video synthesis. The tool currently offers two models, “SVD” and “SVD-XT,” capable of producing image-to-video synthesis at different frame lengths.
Stable Video Diffusion operates locally on machines equipped with Nvidia GPUs, enabling users to experiment with image-to-video synthesis. The generated video clips are typically 2-4 seconds long, with resolutions of 576×1024. During local testing, a 14-frame generation took approximately 30 minutes on an Nvidia RTX 3060 graphics card. However, users have the option to expedite the process by utilizing cloud services like Hugging Face and Replicate, though some services may involve additional costs.
In experiments, the generated animations exhibit scenes with static elements while incorporating dynamic effects like panning, zooming, or animated elements such as smoke and fire. Notably, individuals depicted in photos may not exhibit movement, highlighting the tool’s current limitations.
Stability AI emphasizes that Stable Video Diffusion is still in its early stages and is primarily intended for research purposes. The company underlines that the tool is not yet suitable for real-world or commercial applications. As Stability AI continues to receive feedback and updates the models with the latest advancements, they stress the importance of user insights in refining the tool for potential future releases.
