Tech

I gave Google Gemini 1.5 a video of a total solar eclipse and asked it to write a song – here it is

Google Gemini Pro 1.5 is a transformative moment for multimodal artificial intelligence. You can feed video, audio, or image files and ask questions about their content.

To see how well it works, I fed the Gemini Pro 1.5 a complete silent video of the moment of a recent total solar eclipse that was visible across North America.

Using the Google Cloud platform VertexAI, we were also able to give Gemini some instructions along with video clips. I had her AI music generator write lyrics and prompts to create a song inspired by the content of the video.

I then entered the prompts and lyrics into Udio to create the song, sent the complete track back to Gemini Pro 1.5, and asked it to listen to the track and plot a music video.

What is Google Gemini Pro 1.5?

Google released the Gemini family of models in November, starting with the smaller Nano, which is available on some Android smartphones. He then released Pro, which powers the Gemini chatbot, and finally he released Gemini Ultra, his GPT-4 level of power, in January.

Last month, the search giant released its first update to the Gemini family, launching Gemini Pro with a massive 1 million token context window, a mix of expert architectures for improved responsiveness and accuracy, and true multimodal capabilities. 1.5 has been released.

Currently available to developers only through API calls or the VertexAI cloud platform, this advanced functionality will soon be introduced to Gemini chatbots.

Some of the features include the ability to upload audio files such as songs or speeches, video files of exercise or solar eclipses, and ask Gemini questions about the files.

create songs from videos

(Image credit: Gemini Pro 1.5)

Although you can't use Gemini Pro 1.5 to directly generate music (Google has other AI models for creating both music and videos), you can create prompts and lyrics.

I fed the AI model a short 25-second clip showing the moment of total solar eclipse from a recent total solar eclipse visible from the United States, and fed it into an AI music generator to create lyrics and an inspired song. I asked them to provide both possible prompts. In video.

New @Google developers released today: – Gemini 1.5 Pro now available in over 180 countries via Gemini API in public preview – Speech (speech) understanding features and new features for easier file processing Supports File API – new embedding model! https://t.co/wJk1e1BG1E April 9, 2024

see next

Gave this to me as a hint to Udio. “A grand orchestral piece in three different movements, the first building is full of tension and anticipation as it approaches, the second slows down during totality, becoming dreamy and mysterious. , the third building begins a triumphant crescendo as the sun again emerges from behind the moon.”

These were the lyrics for the chorus. “The solar eclipse has arrived, a celestial show. In the moment of darkness before light grows, the stars appear to play in the midday sky. A once-in-a-lifetime spectacle, as the world passes by.”

I don't think Gemini Pro 1.5 is as creative as Claude 3, ChatGPT, or Gemini Ultra. The lyrics on these platforms tend to be more original, but their ability to analyze videos is very good. We were able to evaluate different moments in the video and capture the changes that were reflected in the lyrics.

Create music videos from songs

(Image courtesy of Pika Labs)

One of the latest updates to Google Gemini 1.5 is the ability to grab a song or piece of audio and analyze its content. This is especially helpful when planning music video ideas for the song, especially if you're working in a hurry.

We took a Udio-generated song using Gemini Pro 1.5 prompts and lyrics and asked the AI model to plot a shot-by-shot music video based on the audio files.

We got a series of 5-second shots of each part of the song, including the intro and chorus. Each section presented a prompt such as “Generate a beautiful landscape with a sunrise or birds flying.'' I then entered that prompt into Pika Labs.

This could allow robots to act completely independently using only a microphone and camera.

This is an unusual example of how Gemini Pro 1.5 can be used, but it shows what is possible and what can even be built by third-party developers. For example, you can use Gemini Pro 1.5 as a middle layer between an AI music generator and an AI video platform like his LTX Studio or Runway to create songs and music videos in one click.

The real benefits of larger context windows will emerge as AI applications such as Meta's smart glasses and Rabbit R1 begin to hit the market. If Google can tackle the latency issue, AI will be able to analyze large amounts of real-world data and provide live feedback and information.

This could help visually impaired people understand audio instructions, create the first step towards truly self-driving cars, or give robots the ability to act completely independently using just a microphone and camera. It can be used to give or give.

Tom's Guide Details

Sources

1/ https://Google.com/

2/ https://www.tomsguide.com/ai/google-gemini/i-gave-google-gemini-15-a-video-of-the-total-eclipse-and-asked-it-to-write-a-song-heres-what-it-sounds-like

The mention sources can contact us to remove/changing this article

What Are The Main Benefits Of Comparing Car Insurance Quotes Online

LOS ANGELES, CA / ACCESSWIRE / June 24, 2020, / Compare-autoinsurance.Org has launched a new blog post that presents the main benefits of comparing multiple car insurance quotes. For more info and free online quotes, please visit https://compare-autoinsurance.Org/the-advantages-of-comparing-prices-with-car-insurance-quotes-online/ The modern society has numerous technological advantages. One important advantage is the speed at which information is sent and received. With the help of the internet, the shopping habits of many persons have drastically changed. The car insurance industry hasn't remained untouched by these changes. On the internet, drivers can compare insurance prices and find out which sellers have the best offers. View photos The advantages of comparing online car insurance quotes are the following: Online quotes can be obtained from anywhere and at any time. Unlike physical insurance agencies, websites don't have a specific schedule and they are available at any time. Drivers that have busy working schedules, can compare quotes from anywhere and at any time, even at midnight. Multiple choices. Almost all insurance providers, no matter if they are well-known brands or just local insurers, have an online presence. Online quotes will allow policyholders the chance to discover multiple insurance companies and check their prices. Drivers are no longer required to get quotes from just a few known insurance companies. Also, local and regional insurers can provide lower insurance rates for the same services. Accurate insurance estimates. Online quotes can only be accurate if the customers provide accurate and real info about their car models and driving history. Lying about past driving incidents can make the price estimates to be lower, but when dealing with an insurance company lying to them is useless. Usually, insurance companies will do research about a potential customer before granting him coverage. Online quotes can be sorted easily. Although drivers are recommended to not choose a policy just based on its price, drivers can easily sort quotes by insurance price. Using brokerage websites will allow drivers to get quotes from multiple insurers, thus making the comparison faster and easier. For additional info, money-saving tips, and free car insurance quotes, visit https://compare-autoinsurance.Org/ Compare-autoinsurance.Org is an online provider of life, home, health, and auto insurance quotes. This website is unique because it does not simply stick to one kind of insurance provider, but brings the clients the best deals from many different online insurance carriers. In this way, clients have access to offers from multiple carriers all in one place: this website. On this site, customers have access to quotes for insurance plans from various agencies, such as local or nationwide agencies, brand names insurance companies, etc. "Online quotes can easily help drivers obtain better car insurance deals. All they have to do is to complete an online form with accurate and real info, then compare prices", said Russell Rabichev, Marketing Director of Internet Marketing Company. CONTACT: Company Name: Internet Marketing CompanyPerson for contact Name: Gurgu CPhone Number: (818) 359-3898Email: [email protected]: https://compare-autoinsurance.Org/ SOURCE: Compare-autoinsurance.Org View source version on accesswire.Com:https://www.Accesswire.Com/595055/What-Are-The-Main-Benefits-Of-Comparing-Car-Insurance-Quotes-Online View photos