Adobe has expanded its Firefly family of creative generative AI models to video, in addition to new breakthroughs in its Image, Vector and Design models. The Firefly Video Model, now in limited public beta, is the first publicly available video model designed to be “commercially safe,” Adobe said.
Perhaps the biggest news that came out of Monday’s Adobe Max conference for broadcasters was the addition of generative AI to Premiere Pro, one of the most popular video-production systems in the Media & Entertainment industry. However, the “Generative Extend” AI beta is not full-on generative AI, but rather a feature that allows creators to extend clips to cover gaps in footage, smooth out transitions or hold onto shots longer for perfectly timed edits.
The company said it trains its Firefly generative AI models on licensed content, such as Adobe Stock or public-domain content, and its features are developed under its AI Ethics principles of accountability, responsibility and transparency—aka what it calls “Content Credentials.”
“Since founding the Content Authenticity Initiative in 2019, Adobe has championed the widespread adoption of Content Credentials as the industry standard for transparency in digital content, now supported by over 3,700 members,” the company said. “Content Credentials, which act like a ‘nutrition label’ for digital content to show how it was created and edited, are applied to select Firefly-powered features across Creative Cloud to indicate the use of generative AI.”
Since Firefly’s first beta release in March 2023, Adobe said it has been used to generate more than 13 billion images—an increase of more than 6 billion over the past six months.
“The usage of Firefly within our creative applications has seen massive adoption, and it’s been inspiring to see how the creative community has used it to push the boundaries of what’s possible,” Adobe Chief Technology Officer, Digital Media Ely Greenfield said. “We’re thrilled to bring creative professionals even more tools for ideation and creation, all designed to be commercially safe.”
The beta for Adobe’s Firefly Video Model (beta) extends Adobe’s family of generative AI models, which already includes an Image Model, Vector Model and Design Model, “making Firefly the most comprehensive model offering for creative teams,” it said. It is available today through a limited public beta to garner initial feedback from a small group of creative professionals, which will be used to continue to refine and improve the model, according to Adobe.
Within one year of being launched, Firefly was brought into Photoshop, Express, Illustrator, Substance 3D and more, while supporting various workflows in Creative Cloud applications. Firefly also supports text prompts in over 100 languages.
Other new features include:
- Text to Video and Image to Video (beta), for user controls and stunning video clips: Powered by the Firefly Video Model and now rolling out in limited public beta in the Firefly web app, creators can access new Text to Video and Image to Video capabilities. With Text to Video, video editors can generate footage from text prompts, access various camera controls, such as angle, motion and zoom, to fine-tune videos, and reference images for b-roll generation to seamlessly fill gaps in a video timeline. Image to Video capabilities allow creators to bring still shots or illustrations to life by transforming them into stunning live-action clips.
- Firefly Image 3 enhancements for faster generations: With the latest evolution of Firefly Image 3 Model, creators of all levels can ideate by generating images in seconds with results that are up to 4 times faster than previous models—now available on the Firefly web app.
- Generative Workspace (beta) in Photoshop: Adobe Firefly-powered Generative Workspace in Photoshop allows designers to simultaneously ideate, brainstorm and iterate concepts.
- Firefly Vector Model (beta) advancements in Illustrator: Adobe Illustrator brought Generative Shape Fill (beta), Generative Recolor and Text to Pattern, all powered by the latest Firefly Vector Model (beta) earlier this year, allowing designers to quickly ideate or add detailed vectors in their own unique style to existing artwork and designs. The latest version of Firefly Vector Model allows creators to further control the density of elements in a single pattern to change how tightly those elements are packed together.
Adobe at the conference also previewed “Project Concept,” a new capability for multiplayer, collaborative, creative concept development that lets creative professionals remix images in real time, so they can conceptualize live within a single canvas.
In Firefly Services, a collection of creative and generative APIs for enterprises, Adobe unveiled new offerings to scale production workflows. This includes Dubbing and Lip Sync, now in beta, which uses generative AI for video content to translate spoken dialogue into different languages while maintaining the sound of the original voice with matching lip sync. “Bulk Create, Powered by Firefly Services” is also now in beta and will let professioonals edit large volumes of images more efficiently, streamlining tasks such as resizing or background removal.
The Firefly Video Model is in limited public beta via waitlist at firefly.adobe.com. During this limited public beta, generations are free. Adobe will share more information about Firefly video generation offers and pricing when the Firefly Video Model moves out of limited public beta.
TV Tech sister brand TechRadar Pro has more.
