Connect with us

Tech

Google's video-to-audio converter creates music from pixels

Google's video-to-audio converter creates music from pixels

 


If you've ever seen video clips from AI video generators like OpenAI's Sora, you'll notice that they don't have any audio attached to them, but Google's DeepMind lab may have come up with a solution.

Google unveiled its video-to-audio (V2A) technology, saying it has developed a tool that can use pixels from a video to generate synchronized audio. Editors can also insert text prompts in the language they need.

Google released a series of example videos using its AI video generator Veo, then used its V2A tool to “match character and tone.”

But V2A technology isn't just useful for AI video: Google DeepMind researchers say the technology can also be used for traditional footage, “including archival material and silent films.”

V2A can apparently generate “an unlimited number of soundtracks for any video input,” and text prompts can be used to guide the audio output with positive or negative prompts, the latter directing it away from a particular tone or style.

“This flexibility gives users more control over V2A's audio output, enabling them to quickly try out different audio outputs and choose the one that works best,” DeepMind wrote in a blog post.

Rather than an autoregressive architecture, the Google researchers took a “diffusion-based approach” to building the model. The V2A system encodes the video input into a compressed representation, and the diffusion model constructs audio from random noise based on the video footage. The audio output is then decoded, converted into an audio waveform, and combined with the video data.

Google says it trained the model using video, audio, and additional annotations that help the model understand the connections between visual events and audio sounds.

The researchers believe their model is novel because, while V2A technology can understand raw pixels, text prompts are optional.

“The system also eliminates the need for manual coordination of generated sound and video, eliminating the need for tedious coordination of various sound, visual and timing elements,” he adds.

However, the audio output is limited because it depends on the quality of the video input: any distortion or other unwanted artifacts in the video will affect the quality of the audio.

Lip sync, which could be extremely beneficial for AI video creators, doesn’t appear to have been mastered by V2A, as the model can’t “adjust based on the transcript.”

This sounds like an appealing tool for video editors, but for now the tool is only available to DeepMind researchers, who want to subject it to “rigorous safety evaluation and testing” before releasing it to the public.

DeepMind didn't say how its V2A tool was trained, but as Tech Radar points out, Google has a potential advantage because it owns YouTube, the world's largest video-sharing platform.

Sources

1/ https://Google.com/

2/ https://petapixel.com/2024/06/21/googles-video-to-audio-tool-generates-music-from-pixels/

The mention sources can contact us to remove/changing this article

What Are The Main Benefits Of Comparing Car Insurance Quotes Online

LOS ANGELES, CA / ACCESSWIRE / June 24, 2020, / Compare-autoinsurance.Org has launched a new blog post that presents the main benefits of comparing multiple car insurance quotes. For more info and free online quotes, please visit https://compare-autoinsurance.Org/the-advantages-of-comparing-prices-with-car-insurance-quotes-online/ The modern society has numerous technological advantages. One important advantage is the speed at which information is sent and received. With the help of the internet, the shopping habits of many persons have drastically changed. The car insurance industry hasn't remained untouched by these changes. On the internet, drivers can compare insurance prices and find out which sellers have the best offers. View photos The advantages of comparing online car insurance quotes are the following: Online quotes can be obtained from anywhere and at any time. Unlike physical insurance agencies, websites don't have a specific schedule and they are available at any time. Drivers that have busy working schedules, can compare quotes from anywhere and at any time, even at midnight. Multiple choices. Almost all insurance providers, no matter if they are well-known brands or just local insurers, have an online presence. Online quotes will allow policyholders the chance to discover multiple insurance companies and check their prices. Drivers are no longer required to get quotes from just a few known insurance companies. Also, local and regional insurers can provide lower insurance rates for the same services. Accurate insurance estimates. Online quotes can only be accurate if the customers provide accurate and real info about their car models and driving history. Lying about past driving incidents can make the price estimates to be lower, but when dealing with an insurance company lying to them is useless. Usually, insurance companies will do research about a potential customer before granting him coverage. Online quotes can be sorted easily. Although drivers are recommended to not choose a policy just based on its price, drivers can easily sort quotes by insurance price. Using brokerage websites will allow drivers to get quotes from multiple insurers, thus making the comparison faster and easier. For additional info, money-saving tips, and free car insurance quotes, visit https://compare-autoinsurance.Org/ Compare-autoinsurance.Org is an online provider of life, home, health, and auto insurance quotes. This website is unique because it does not simply stick to one kind of insurance provider, but brings the clients the best deals from many different online insurance carriers. In this way, clients have access to offers from multiple carriers all in one place: this website. On this site, customers have access to quotes for insurance plans from various agencies, such as local or nationwide agencies, brand names insurance companies, etc. "Online quotes can easily help drivers obtain better car insurance deals. All they have to do is to complete an online form with accurate and real info, then compare prices", said Russell Rabichev, Marketing Director of Internet Marketing Company. CONTACT: Company Name: Internet Marketing CompanyPerson for contact Name: Gurgu CPhone Number: (818) 359-3898Email: [email protected]: https://compare-autoinsurance.Org/ SOURCE: Compare-autoinsurance.Org View source version on accesswire.Com:https://www.Accesswire.Com/595055/What-Are-The-Main-Benefits-Of-Comparing-Car-Insurance-Quotes-Online View photos

ExBUlletin

to request, modification Contact us at Here or [email protected]