Optimize traffic for AI apps generated by network functions

Many companies are exploring how to incorporate the benefits of generative AI (gen AI) into their business. According to the 2023 Gartner report, “We Shape AI, AI Shapes Us: 2023 IT Symposium/Xpo Keynote Insights” (October 16, 2023), most organizations use or plan to use AI on a daily basis to improve productivity. In the 2024 Gartner CIO and Technology Executive Survey, 80% of respondents said they plan to adopt generative AI within three years.1

Enterprises looking to deploy large-scale language models (LLMs) face a unique set of network challenges compared to serving traditional web applications, because generative AI applications behave very differently than most other web applications.

For example, web applications typically exhibit predictable traffic patterns, with requests and responses being processed in a relatively short time (usually in milliseconds). In contrast, due to their multimodal nature, gen AI inference applications can have fluctuating request/response times, which can present unique challenges. At the same time, LLM queries often consume 100% of a GPU or TPU's compute time, compared to more typical request processing that runs in parallel. Due to the computational cost, inference latencies can range from seconds to minutes.

Typical Web Traffic

Generation AI Traffic

As a result, traditional round-robin or utilization-based traffic management techniques are generally not suitable for gen AI applications. To deliver the best end-user experience for gen AI applications and efficiently use limited and costly GPU and TPU resources, we recently announced several new networking capabilities to optimize traffic for AI applications.

Many of these innovations are built into Vertex AI and are now available with cloud networking so you can use them with whichever LLM platform you choose.

Let's take a closer look.

1. Accelerating AI training and inference with cross-cloud networks

According to an IDC report, 66% of enterprises cite generative AI and AI/ML workloads as one of the primary use cases for multi-cloud networking.2 This is because the data required for model training/fine-tuning, search augmentation generation (RAG), or grounding resides in many different environments. This data must be accessed or copied remotely so that the LLM model can access it.

Last year, Google introduced the Cross-Cloud Network, which provides service-centric any-to-any connectivity built on Google's global network, making it easier to build and assemble distributed applications across clouds.

Cross-Cloud Network includes products that provide reliable, secure, SLA-backed cross-cloud connectivity for high-speed data transfer between clouds, helping move the vast amounts of data required to train AI models. Products included in the solution include Cross-Cloud Interconnect, which provides managed interconnection with 10 Gbps or 100 Gbps bandwidth with a 99.99% SLA and end-to-end encryption.

In addition to providing secure and reliable data transfer for AI training, cross-cloud networking also enables you to run AI model inference applications across hybrid environments, for example, by accessing models running in Google Cloud from application services running in another cloud environment.

Sources 1/ https://Google.com/ 2/ https://cloud.google.com/blog/products/networking/networking-capabilities-optimize-traffic-for-generative-ai-apps The mention sources can contact us to remove/changing this article

What Are The Main Benefits Of Comparing Car Insurance Quotes Online

LOS ANGELES, CA / ACCESSWIRE / June 24, 2020, / Compare-autoinsurance.Org has launched a new blog post that presents the main benefits of comparing multiple car insurance quotes. For more info and free online quotes, please visit https://compare-autoinsurance.Org/the-advantages-of-comparing-prices-with-car-insurance-quotes-online/ The modern society has numerous technological advantages. One important advantage is the speed at which information is sent and received. With the help of the internet, the shopping habits of many persons have drastically changed. The car insurance industry hasn't remained untouched by these changes. On the internet, drivers can compare insurance prices and find out which sellers have the best offers. View photos The advantages of comparing online car insurance quotes are the following: Online quotes can be obtained from anywhere and at any time. Unlike physical insurance agencies, websites don't have a specific schedule and they are available at any time. Drivers that have busy working schedules, can compare quotes from anywhere and at any time, even at midnight. Multiple choices. Almost all insurance providers, no matter if they are well-known brands or just local insurers, have an online presence. Online quotes will allow policyholders the chance to discover multiple insurance companies and check their prices. Drivers are no longer required to get quotes from just a few known insurance companies. Also, local and regional insurers can provide lower insurance rates for the same services. Accurate insurance estimates. Online quotes can only be accurate if the customers provide accurate and real info about their car models and driving history. Lying about past driving incidents can make the price estimates to be lower, but when dealing with an insurance company lying to them is useless. Usually, insurance companies will do research about a potential customer before granting him coverage. Online quotes can be sorted easily. Although drivers are recommended to not choose a policy just based on its price, drivers can easily sort quotes by insurance price. Using brokerage websites will allow drivers to get quotes from multiple insurers, thus making the comparison faster and easier. For additional info, money-saving tips, and free car insurance quotes, visit https://compare-autoinsurance.Org/ Compare-autoinsurance.Org is an online provider of life, home, health, and auto insurance quotes. This website is unique because it does not simply stick to one kind of insurance provider, but brings the clients the best deals from many different online insurance carriers. In this way, clients have access to offers from multiple carriers all in one place: this website. On this site, customers have access to quotes for insurance plans from various agencies, such as local or nationwide agencies, brand names insurance companies, etc. "Online quotes can easily help drivers obtain better car insurance deals. All they have to do is to complete an online form with accurate and real info, then compare prices", said Russell Rabichev, Marketing Director of Internet Marketing Company. CONTACT: Company Name: Internet Marketing CompanyPerson for contact Name: Gurgu CPhone Number: (818) 359-3898Email: [email protected]: https://compare-autoinsurance.Org/ SOURCE: Compare-autoinsurance.Org View source version on accesswire.Com:https://www.Accesswire.Com/595055/What-Are-The-Main-Benefits-Of-Comparing-Car-Insurance-Quotes-Online View photos