AI-powered sonar on smart glasses tracks gaze and facial expressions

Researchers at Cornell University have developed two technologies that track people's gaze and facial expressions through sonar-like sensing.

The technology is small enough to fit into off-the-shelf smart glasses and virtual reality (VR) or augmented reality (AR) headsets, while consuming significantly less power than similar tools that use cameras.

Both use speakers and microphones attached to the frames of the glasses to bounce inaudible sound waves off your face and pick up reflected signals caused by facial and eye movements. One of his devices, GazeTrak, is the first eye-tracking system to rely on acoustic signals. His second, EyeEcho, is the first glasses-based system to continuously and accurately detect facial expressions and reproduce them in real-time through an avatar.

Ke Li is the principal investigator for the GazeTrak smart eyewear tracking technology.

The device can last several hours on battery in smart glasses and more than a day on VR headset.

Cheng Zhang, assistant professor of information science in the Cornell Ann S. Bowers College of Computing and Information Sciences He said no. Zhang, who created the new device, directs the Smart Computer Interfaces for Future Interactions (SciFi) Lab.

Ke Li, a doctoral student in information science who led the development of GazeTrak and EyeEcho, said, “In a VR environment, detailed facial expressions and eye movements can be used to better interact with other users. We need to recreate it.”

Using audio signals instead of video also reduces privacy concerns, Lee said. There are a number of camera-based systems for tracking facial expressions and eye movements in this research field, such as Vision Pro and Oculus, and there are also commercial products, he said. But not everyone wants a wearable's camera to be able to capture you and your surroundings all the time.

Li will present GazeTrak: Exploring Acoustic-based Eye Tracking on a Glass Frame at the Annual International Conference on Mobile Computing and Networking (MobiCom24) in September. I will make an announcement. From the 30th to October 4th.

“As VR/AR headsets become significantly smaller and eventually become similar to today's smart glasses, privacy concerns related to systems that use video will become increasingly important,” co-authors said. said Franois Gimbletire, a Cornell Bowers CIS professor of information science. Multicollege Department of Design Technology Both technologies are extremely small and power efficient, making them lightweight and sleek, making them ideal for AR glasses.

For GazeTrak, the researchers placed one speaker and four microphones inside the frame of each eye of the glasses, reflecting and picking up sound waves from the eyeball and the area around the eye. The resulting audio signal is fed into a customized deep learning pipeline that uses artificial intelligence to continuously infer the direction of a person's gaze.

Although GazeTrak still doesn't work as well as state-of-the-art eye-tracking technology that relies on cameras, the new device proves the concept that audio signals can also be useful. The researchers believe that with further optimization, they can achieve the same accuracy and reduce the number of speakers and microphones needed.

In EyeEcho's case, one speaker and one microphone are placed next to the hinge of the glasses, pointing downwards to capture the movement of the skin as facial expressions change. The reflected signals are also interpreted using AI.

This technology allows users to make hands-free video calls through their avatars, even in noisy cafes or on the street. While some smart glasses have the ability to recognize faces and distinguish between some specific facial expressions, there are currently none that continuously track facial expressions like EyeEcho.

Lee will be presenting this work, “EyeEcho: Continuous and low We present “Output Facial Expression Tracking”.

These two advances have uses beyond enhancing a person's VR experience. GazeTrak can be used with Screen Reader to read portions of text aloud when people with low vision browse websites.

GazeTrak and EyeEcho may also be useful in diagnosing and monitoring neurodegenerative diseases such as Alzheimer's disease and Parkinson's disease. In these conditions, patients often have abnormal eye movements and dull facial expressions, and this type of technology could potentially track the progress of the disease from the comfort of a patient's home.

Several Cornell researchers also contributed to the study, including information science doctoral students Ruidong Zhang, Mose Saketa, and Saif Mahmud. James Chen 24 years old, Sean Chen 24 years old, Kenny Liang 24 years old. and Sicheng ying, master's student at the University of Edinburgh.

This research was supported by the National Science Foundation and the IGNITE Innovation Acceleration Program.

Patricia Waldron is a writer in the Cornell Ann S. Bowers College of Computing and Information Sciences.

Sources 1/ https://Google.com/ 2/ https://news.cornell.edu/stories/2024/04/ai-powered-sonar-smartglasses-tracks-gaze-and-facial-expressions The mention sources can contact us to remove/changing this article

What Are The Main Benefits Of Comparing Car Insurance Quotes Online

LOS ANGELES, CA / ACCESSWIRE / June 24, 2020, / Compare-autoinsurance.Org has launched a new blog post that presents the main benefits of comparing multiple car insurance quotes. For more info and free online quotes, please visit https://compare-autoinsurance.Org/the-advantages-of-comparing-prices-with-car-insurance-quotes-online/ The modern society has numerous technological advantages. One important advantage is the speed at which information is sent and received. With the help of the internet, the shopping habits of many persons have drastically changed. The car insurance industry hasn't remained untouched by these changes. On the internet, drivers can compare insurance prices and find out which sellers have the best offers. View photos The advantages of comparing online car insurance quotes are the following: Online quotes can be obtained from anywhere and at any time. Unlike physical insurance agencies, websites don't have a specific schedule and they are available at any time. Drivers that have busy working schedules, can compare quotes from anywhere and at any time, even at midnight. Multiple choices. Almost all insurance providers, no matter if they are well-known brands or just local insurers, have an online presence. Online quotes will allow policyholders the chance to discover multiple insurance companies and check their prices. Drivers are no longer required to get quotes from just a few known insurance companies. Also, local and regional insurers can provide lower insurance rates for the same services. Accurate insurance estimates. Online quotes can only be accurate if the customers provide accurate and real info about their car models and driving history. Lying about past driving incidents can make the price estimates to be lower, but when dealing with an insurance company lying to them is useless. Usually, insurance companies will do research about a potential customer before granting him coverage. Online quotes can be sorted easily. Although drivers are recommended to not choose a policy just based on its price, drivers can easily sort quotes by insurance price. Using brokerage websites will allow drivers to get quotes from multiple insurers, thus making the comparison faster and easier. For additional info, money-saving tips, and free car insurance quotes, visit https://compare-autoinsurance.Org/ Compare-autoinsurance.Org is an online provider of life, home, health, and auto insurance quotes. This website is unique because it does not simply stick to one kind of insurance provider, but brings the clients the best deals from many different online insurance carriers. In this way, clients have access to offers from multiple carriers all in one place: this website. On this site, customers have access to quotes for insurance plans from various agencies, such as local or nationwide agencies, brand names insurance companies, etc. "Online quotes can easily help drivers obtain better car insurance deals. All they have to do is to complete an online form with accurate and real info, then compare prices", said Russell Rabichev, Marketing Director of Internet Marketing Company. CONTACT: Company Name: Internet Marketing CompanyPerson for contact Name: Gurgu CPhone Number: (818) 359-3898Email: [email protected]: https://compare-autoinsurance.Org/ SOURCE: Compare-autoinsurance.Org View source version on accesswire.Com:https://www.Accesswire.Com/595055/What-Are-The-Main-Benefits-Of-Comparing-Car-Insurance-Quotes-Online View photos