Is ChatGPT the key to stopping deepfakes? – UBNow: News and Opinions for UB Faculty

When most people think about artificial intelligence, they probably think and worry about ChatGPT and deepfakes.

AI-generated text and images often appear in our social media feeds and other websites we visit, sometimes without our knowledge, and are often used to spread unreliable and misleading information.

But what if a text generation model like ChatGPT could actually detect deepfake images?

A UB-led research team applied large-scale language models (LLMs) such as OpenAI's ChatGPT and Google's Gemini to detecting deepfakes of human faces. Research recently presented at the IEEE/CVF conference on Computer Vision and Pattern Recognition found that while LLMs' performance lags behind state-of-the-art deepfake detection algorithms, their natural language processing could make them a more practical detection tool in the future.

“LLM differs from existing detection methods in that it can explain its detection results in a way that humans can understand, such as identifying a false shadow or mismatched earrings,” said Shiwei Liu, lead author of the study and Innovation Professor in the Department of Computer Science and Engineering at SUNY Empire College of Engineering and Applied Science. “Although LLM was not designed or trained for deepfake detection, its semantic knowledge makes it well suited for this purpose, and we expect to see further work on this application.”

The research was conducted in collaboration with the University at Albany and the Chinese University of Hong Kong in Shenzhen. It was supported by the National Science Foundation.

ChatGPT is trained on much of the text available on the internet (around 300 billion words) to find statistical patterns and relationships between words to generate responses.

More recent versions of ChatGPT and other LLMs can also analyze images: these multimodal LLMs use large databases of captioned photos to find relationships between words and images.

“Humans do the same thing: We're constantly assigning semantic descriptions to images, whether it's a stop sign or a viral meme,” said Shan Jai, lead author of the study and assistant lab director of the UB Media Forensics Lab. “In this way, images become a language of their own.”

The Media Forensic Lab team decided to test whether GPT-4 with Vision (GPT-4V) and Gemini 1.0 could distinguish between real and AI-generated faces. They gave them thousands of images of real and deepfake faces and asked them to identify signs of manipulation or possible synthetic artifacts.

ChatGPT showed 79.5% accuracy in detecting synthetic artifacts in images generated by latent diffusion and 77.2% accuracy in images generated by StyleGAN.

“This is comparable to previous deepfake detection methods, so with proper rapid guidance, ChatGPT can detect AI-generated images quite well,” said Liu, who is also co-director of UB's Center for Information Integrity.

More importantly, ChatGPT was able to explain its decisions in easy-to-understand terms: when presented with an AI-generated photo of a man wearing glasses, the model correctly pointed out that “the hair on the left side of the image is a bit blurry” and that “the transition between the person and the background is a bit abrupt and lacking depth.”

“Existing deepfake detection models can tell us the probability that an image is real or fake, but few can tell us why they reached that conclusion. And even if we looked into the underlying mechanisms of the models, we would come across features that we don't understand,” Liu says. “In contrast, everything ChatGPT outputs is human-understandable.”

This is because ChatGPT bases its analysis solely on semantic knowledge: whereas traditional deepfake detection algorithms distinguish real from fake by training on large datasets of images that are labeled as real or fake, LLM's natural language capabilities provide a commonsense understanding of reality, such as the typical symmetry of human faces and how real photographs look, at least when not hallucinated.

“Once ChatGPT's vision component understands an image as a human face, the language component can infer that a face would typically have two eyes, and so on,” Lyu says. “The language component provides a deeper connection between visual and language concepts.”

ChatGPT's semantic knowledge and natural language processing make it a more user-friendly deepfake tool for both users and developers, the study concludes.

“Typically, we translate our insights into deepfake detection into programming language. Now, all of this knowledge lives within a single model, and we just need to use natural language to extract that knowledge,” Liu said.

ChatGPT's performance was well below state-of-the-art deepfake detection algorithms, which boast accuracy rates in the mid-to-high 90s.

This is in part because LLM was unable to capture statistical differences in signal levels that are invisible to the human eye but are often used by detection algorithms to spot AI-generated images.

“ChatGPT only focused on semantic-level anomalies,” Lyu said. “Thus, the semantic intuitiveness of ChatGPT's results may actually be a double-edged sword for deepfake detection.”

And other LLMs may not be as effective at explaining their analysis: While it performed as well as ChatGPT at inferring the presence of synthetic artifacts, Gemini's supporting evidence was often nonsensical, such as pointing out a mole that didn't exist.

Another drawback is that LLM often refuses to analyze images: When asked directly if a photo was generated by AI, ChatGPT usually responds with “Sorry, we can't accommodate that request.”

“The model is programmed not to respond if it doesn't reach a certain level of confidence,” Liu says. “We know that ChatGPT has information relevant to deepfake detection, but we still need human operators to stimulate parts of its knowledge base. Prompt engineering is effective, but it's not very efficient. So our next step is to go down a level and really fine-tune the LLM specifically for this task.”

Sources 1/ https://Google.com/ 2/ https://www.buffalo.edu/ubnow/stories/2024/07/chat-gpt-deepfakes.html The mention sources can contact us to remove/changing this article

What Are The Main Benefits Of Comparing Car Insurance Quotes Online

LOS ANGELES, CA / ACCESSWIRE / June 24, 2020, / Compare-autoinsurance.Org has launched a new blog post that presents the main benefits of comparing multiple car insurance quotes. For more info and free online quotes, please visit https://compare-autoinsurance.Org/the-advantages-of-comparing-prices-with-car-insurance-quotes-online/ The modern society has numerous technological advantages. One important advantage is the speed at which information is sent and received. With the help of the internet, the shopping habits of many persons have drastically changed. The car insurance industry hasn't remained untouched by these changes. On the internet, drivers can compare insurance prices and find out which sellers have the best offers. View photos The advantages of comparing online car insurance quotes are the following: Online quotes can be obtained from anywhere and at any time. Unlike physical insurance agencies, websites don't have a specific schedule and they are available at any time. Drivers that have busy working schedules, can compare quotes from anywhere and at any time, even at midnight. Multiple choices. Almost all insurance providers, no matter if they are well-known brands or just local insurers, have an online presence. Online quotes will allow policyholders the chance to discover multiple insurance companies and check their prices. Drivers are no longer required to get quotes from just a few known insurance companies. Also, local and regional insurers can provide lower insurance rates for the same services. Accurate insurance estimates. Online quotes can only be accurate if the customers provide accurate and real info about their car models and driving history. Lying about past driving incidents can make the price estimates to be lower, but when dealing with an insurance company lying to them is useless. Usually, insurance companies will do research about a potential customer before granting him coverage. Online quotes can be sorted easily. Although drivers are recommended to not choose a policy just based on its price, drivers can easily sort quotes by insurance price. Using brokerage websites will allow drivers to get quotes from multiple insurers, thus making the comparison faster and easier. For additional info, money-saving tips, and free car insurance quotes, visit https://compare-autoinsurance.Org/ Compare-autoinsurance.Org is an online provider of life, home, health, and auto insurance quotes. This website is unique because it does not simply stick to one kind of insurance provider, but brings the clients the best deals from many different online insurance carriers. In this way, clients have access to offers from multiple carriers all in one place: this website. On this site, customers have access to quotes for insurance plans from various agencies, such as local or nationwide agencies, brand names insurance companies, etc. "Online quotes can easily help drivers obtain better car insurance deals. All they have to do is to complete an online form with accurate and real info, then compare prices", said Russell Rabichev, Marketing Director of Internet Marketing Company. CONTACT: Company Name: Internet Marketing CompanyPerson for contact Name: Gurgu CPhone Number: (818) 359-3898Email: [email protected]: https://compare-autoinsurance.Org/ SOURCE: Compare-autoinsurance.Org View source version on accesswire.Com:https://www.Accesswire.Com/595055/What-Are-The-Main-Benefits-Of-Comparing-Car-Insurance-Quotes-Online View photos