Adobe rivals OpenAI’s Sora with new Firefly Video model: What is it and how to use?

Adobe has rolled out its text-to-video generation feature on Firefly AI, positioning it against OpenAI’s Sora and Google Veo AI models. The new AI model is available via the redesigned Firefly web app and is being positioned as the first “commercially safe AI video generation model”.

The new Generative video model allows users to both generate text to video and image to video. Using text-to-video feature users can generate a video using just text description while image-to-video allows users to reference an image along with the text prompt to create customized footage. 

“Generate Video (beta), powered by the Firefly Video Model, empowers creative professionals with tools to generate video clips from a text prompt or image, use camera angles to control shots, create professional-quality images from 3D sketches, craft atmospheric elements and develop custom motion design elements.” Adobe explained in a release about the new feature.

Currently, the video generation model is capable of generating videos of up to 1080p for 5 seconds at 24 frames per second and take around 90 seconds or longer to create. Adobe says it is working on bringing a lower resolution model for high-speed iteration and a 4K model for production work. 

Unlike many other AI companies that are being sued for using copyrighted data, Adobe says it only trains Firefly on the content it has permission to use which includes Adobe Stock and public domain data. The company also noted that it does not use customer content for training its AI.

Adobe says that Firefly is integrated with other Adobe Creative Cloud applications, including Photoshop, Premier Pro and Adobe Express in order to improve the seamless editing process. 

The San Jose-California based company also announced two other features for its paid users. First, Scene to Image allows users to create ‘professional quality images’ from 3D sketches, which Adobe says can be used to seamlessly render production-ready assets from 3D shapes. 

Second, Translate Audio and Translate Video allows users to (drumroll) translate spoken dialog in over 20 languages. Adobe says its new tool allows creators to quickly and easily translate videos and audio while matching the voice, tone, cadence and acoustic of the original content. 

Adobe Firefly subscription plan pricing: 

Adobe has also announced a new Standard and Pro plan. The Standard plan allows users up 2,000 video/audio credits per month  and 20 vidoe generations of 5 second 1080p. It is priced at ₹797.68/month or 7,986.24 for an year, inclusive of GST. 

Meanwhile, the Pro plan which allows for 7,000 video/audio credits per month and 70 5 second 1080p video generations is priced at ₹2,394.22/month or ₹23,944.56 for the whole year.