Hollywood and tech executives embrace AI to cut costs and engage audiences

Samira Panah Bakhtiar, head of media, entertainment, gaming and sports at AWS, said AI presents promising new ways to generate more revenue from the catalog of movies and shows that, for many studios, remain largely unused.

“I think the most incredible opportunity right now for our customers who have decades and decades of archival content is their ability to search and discover what they're sitting on,” Bakhtiar said.

AWS is working with other AI companies like Anthropic to find ways for media companies to classify stock footage by characteristics like lead actor and sentiment analysis so media owners can better understand what has untapped value for consumers.

Understanding where content can and should be distributed, then creating and translating subtitles, audio descriptions, or dubbing audio files into a new language using AI are other applications that interest entertainment executives. But for many creative content companies, building tools in-house isn’t an option. That’s where startups come in.

“We recognized very early on, especially in this wave of generative AI, that large foundation-based technology companies were going to be very important to our future, and that we weren’t going to be able to invest at a level that would allow us to do things on our own,” said Phil Wiser, Paramount’s CTO.

Not only can AI potentially help companies like Paramount distribute their content more effectively to audiences, it can also reduce the cost of reaching audiences who may not be able to consume media as it was created.

Full audio descriptions of scenes, for example, are expensive to do properly. “I hope that using AI will now make these accessibility capabilities more widespread in media in general,” Wiser said.

Companies like Microsoft, AWS, and Nvidia are all eager to see the next big startup emerge, and content developers like Paramount are prime customers for companies looking to revolutionize how media is created and distributed. Here are some of the startups AI and entertainment leaders have their eyes on.

Track

In its last funding round, Runway was valued at $1.5 billion, and its text-to-video capabilities have attracted partnerships from Paramount, AWS, Nvidia and Google.

The point of Monday's discussion wasn't whether Runway could make movies in a complete way, but rather to speed up the work of marketing traditionally made films.

“Marketing is one of the areas where we’re seeing a lot of adoption because you have to create spin-offs of content very quickly, and you have to do it within the current workflow that you’re using,” Wiser said of how Paramount is using Runway’s technology.

This technology was also used in the Oscar-winning film “Everything everywhere at once.”

Krikey.ai

Krikey also lets users turn text into images, but it specializes in animation, allowing players to create three-dimensional avatars of themselves, for example. This type of technology paves the way for some of AI's most important applications in marketing and advertising.

“We’re seeing a growing interest in marketing with this human equivalent or digital character that allows fans to interact with the characters,” Wiser said. Krikey is part of AWS’s startup accelerator cohort, according to Pitchbook.

Cuebrique AI

Cuebric is able to transform still images into multidimensional landscapes suitable for cinema. Partners with Nvidia and AWS for its inference computing. Amazon’s Bakhtar cited Cuebric among several other startups using AI to create visuals. The company’s co-founders said they wanted democratize cinema.

Advanced AI

Luma AI, another AI image generation startup named by Bakhtar, has raised over $68 million, according to Bain, with recent backing from Andreessen Horowitz. Luma’s Genie text-to-3D image generation model is the company’s core offering. Co-founder and CEO Amit Jain worked on Apple Vision Pro and its machine learning applications before co-founding Luma.

xSpirits

Eliot Sakhartov, Microsoft's business and technology strategist for media and entertainment, described xMentium as a “completely unsexy” AI tool his team uses.

xMentium helps speed up the drafting and referencing of legal documents and contracts. The company has raised about $7 million in funding, according to Pitchbook, and is specifically targeting the entertainment industry.

Speechmatics

Michael Kaplan, Nvidia's head of global business development for generative AI in media and entertainment, pointed to Speechmatics, a UK-based startup that creates AI transcription and translation tools that can be used for subtitling.

Speechmatics is a member of Nvidia's Inception program, a network of over 20,000 startups.

“You can actually unlock a whole bunch of content in a geographic area that would never normally be able to get it,” Sakhartov said of AI transcription and translation of subtitles.

Twelve laboratories

Twelve Labs, also mentioned by Nvidia’s Kaplan, uses AI not only to listen to spoken language in content, but also to understand what’s happening more broadly. That added context can help seamlessly divide content into chapters or insert ads where they make sense. It also offers more capabilities for searching within videos using more ephemeral search terms like tone and actions.

Sakhartov presented a hypothetical scenario in which such a tool could take your favorite show and the length of your commute, and custom-edit an episode with just your favorite parts tailored to that time window.

ElevenLabs

Voice-cloning startup ElevenLabs could expand the possibilities of distributing films and TV shows in new languages ​​through AI dubbing. The London and New York-based startup reached unicorn status earlier this year. Its digital voice replicas have also been used for marketing purposes.

A digital version of Jensen Huang, CEO of Nvidia debuted at Computex 2024 in Taiwan in June, and the tech titan's voice was generated in English and Mandarin by ElevenLabs.

Hour 1

Speaking of Huang, Israeli startup Hour One's technology helped animate Huang”digital human.“The rendering was so convincing,” Kaplan admitted, “that I didn't know. I thought it was him.”

Founded in 2019, Hour One has been pushing the potential of digital humans and lifelike avatars for longer than most. The company is also a member of Nvidia’s Inception program. Among entertainment applications, the company sees an opportunity to generate moving avatars of well-known broadcasters for real-time updates in news and sports.