Amazon investigating embarrassment over scraping abuse allegations

Amazon's cloud division has launched an investigation into Perplexity AI, asking whether the AI ​​search startup is violating Amazon Web Services rules by scraping websites that have tried to prevent such scraping, WIRED reports.

An AWS spokesperson, who spoke to WIRED on the condition of anonymity, confirmed that the company is investigating Perplexity. WIRED has previously found that the startup, backed by the Jeff Bezos family fund and Nvidia and recently valued at $3 billion, appears to be relying on scraped website content that is blocked by the Robots Exclusion Protocol, a common web standard. While the Robots Exclusion Protocol is not legally binding, terms of use generally are.

The Robots Exclusion Protocol is a decades-old web standard that lets you place a plain text file (such as wired.com/robots.txt) on a domain to specify pages that automated bots and crawlers shouldn't visit. Companies that use scrapers can choose to ignore the protocol, but most companies have traditionally respected it. An Amazon spokesperson told WIRED that AWS customers must follow the robots.txt standard when crawling websites.

AWS' terms of service prohibit customers from using its services for any illegal activity, and customers are responsible for complying with its terms and all applicable laws, a spokesperson said in a statement.

The investigation into Perplexitys' activities follows a June 11 Forbes report that the startup had allegedly stolen at least one article. WIRED's investigation confirmed the activity and found further evidence of scraping abuse and theft by systems linked to Perplexitys' AI-powered search chatbot. Engineers at WIRED's parent company, Cond Nast, block Perplexitys' crawlers on all of its websites using robots.txt files. However, WIRED found that the company had accessed its servers using an undisclosed IP address, 44.221.181.252, to access Cond Nast properties and scrape Cond Nast websites at least hundreds of times in the past three months.

Machines associated with Perplexity appear to be conducting extensive crawls of news websites that ban bots from accessing their content, and spokespeople for The Guardian, Forbes and The New York Times said they have also found the IP address on their servers multiple times.

WIRED traced the IP address to a virtual machine, known as an Elastic Compute Cloud (EC2) instance, hosted on AWS. The investigation was launched after WIRED asked whether using AWS infrastructure to scrape prohibited websites violated the company's terms of use.

Last week, Perplexity CEO Aravind Srinivas responded to WIRED's inquiry by first saying that the questions posed to the company reflected a fundamental, deep-seated misunderstanding of Perplexity and how the Internet works. Srinivas then told Fast Company that the covert IP addresses WIRED saw scraping the Cond Nast website and a test site we created were operated by a third-party company that provides web crawling and indexing services. He declined to name the company, citing non-disclosure agreements. When asked if he would tell the third-party company to stop crawling WIRED, Srinivas said, “It's complicated.”

Sources 1/ https://Google.com/ 2/ https://www.wired.com/story/aws-perplexity-bot-scraping-investigation/ The mention sources can contact us to remove/changing this article

What Are The Main Benefits Of Comparing Car Insurance Quotes Online

LOS ANGELES, CA / ACCESSWIRE / June 24, 2020, / Compare-autoinsurance.Org has launched a new blog post that presents the main benefits of comparing multiple car insurance quotes. For more info and free online quotes, please visit https://compare-autoinsurance.Org/the-advantages-of-comparing-prices-with-car-insurance-quotes-online/ The modern society has numerous technological advantages. One important advantage is the speed at which information is sent and received. With the help of the internet, the shopping habits of many persons have drastically changed. The car insurance industry hasn't remained untouched by these changes. On the internet, drivers can compare insurance prices and find out which sellers have the best offers. View photos The advantages of comparing online car insurance quotes are the following: Online quotes can be obtained from anywhere and at any time. Unlike physical insurance agencies, websites don't have a specific schedule and they are available at any time. Drivers that have busy working schedules, can compare quotes from anywhere and at any time, even at midnight. Multiple choices. Almost all insurance providers, no matter if they are well-known brands or just local insurers, have an online presence. Online quotes will allow policyholders the chance to discover multiple insurance companies and check their prices. Drivers are no longer required to get quotes from just a few known insurance companies. Also, local and regional insurers can provide lower insurance rates for the same services. Accurate insurance estimates. Online quotes can only be accurate if the customers provide accurate and real info about their car models and driving history. Lying about past driving incidents can make the price estimates to be lower, but when dealing with an insurance company lying to them is useless. Usually, insurance companies will do research about a potential customer before granting him coverage. Online quotes can be sorted easily. Although drivers are recommended to not choose a policy just based on its price, drivers can easily sort quotes by insurance price. Using brokerage websites will allow drivers to get quotes from multiple insurers, thus making the comparison faster and easier. For additional info, money-saving tips, and free car insurance quotes, visit https://compare-autoinsurance.Org/ Compare-autoinsurance.Org is an online provider of life, home, health, and auto insurance quotes. This website is unique because it does not simply stick to one kind of insurance provider, but brings the clients the best deals from many different online insurance carriers. In this way, clients have access to offers from multiple carriers all in one place: this website. On this site, customers have access to quotes for insurance plans from various agencies, such as local or nationwide agencies, brand names insurance companies, etc. "Online quotes can easily help drivers obtain better car insurance deals. All they have to do is to complete an online form with accurate and real info, then compare prices", said Russell Rabichev, Marketing Director of Internet Marketing Company. CONTACT: Company Name: Internet Marketing CompanyPerson for contact Name: Gurgu CPhone Number: (818) 359-3898Email: [email protected]: https://compare-autoinsurance.Org/ SOURCE: Compare-autoinsurance.Org View source version on accesswire.Com:https://www.Accesswire.Com/595055/What-Are-The-Main-Benefits-Of-Comparing-Car-Insurance-Quotes-Online View photos