Connect with us

Tech

Real-time iris tracking and depth estimation

 



Posted by Andrey Vakunov, Dmitry Lagun, Research Engineer, Google Research

A wide range of real-world applications, including computational photography (eg portrait mode and grind reflections) and augmented reality effects (eg virtual avatars) rely on tracking the iris to estimate eye position. Once accurate iris tracking is available, you will find that you can measure the distance from the camera to the user without using a dedicated depth sensor. This allows you to improve a variety of use cases, from computational photos to virtual fitting of appropriately sized glasses and hats, to the ease of use of adopting font sizes depending on the viewer’s distance.

Iris tracking is a difficult task to solve on mobile devices due to limited computing resources, different light conditions, and the presence of occlusions such as hair and people squinting. Often, sophisticated dedicated hardware is used, limiting the range of devices to which the solution can be applied.

FaceMesh can be used to drive a virtual avatar (center). By further adopting iris tracking (right), the vitality of the avatar will be greatly improved. Examples of eye color changes made possible by MediaPipe Iris.

Today we are announcing the release of MediaPipe Iris, a new machine learning model for accurate iris estimation. Built from work on the MediaPipe face mesh, this model tracks landmarks, including irises, pupils, and eye contours, in real time using a single RGB camera, without the need for dedicated hardware. I can do it. By using the iris landmark, the model can also measure the distance between the subject and the camera with a relative error of less than 10% without the use of a depth sensor. Please note that iris tracking does not guess where people are looking and does not provide any kind of ID recognition. Most modern phones, desktops, thanks to the fact that the system is implemented in MediaPipe, an open source cross-platform framework for researchers and developers to build world-class ML solutions and applications. It can be run on a laptop, or even the web.

Visionary personal usability prototype: Observed font size remains constant regardless of the distance of the device from the user.

ML Pipeline for Iris Tracking The first step in the pipeline leverages our previous work with 3D face meshes, using precision facial landmarks to generate a mesh of approximate facial geometry. I will. From this mesh, separate the eye regions of the original image for use in the iris tracking model. Then the problem is divided into two parts. It is the estimation of the outline of the eye and the position of the iris. We designed a multitasking model that consisted of an integrated encoder with separate components for each task and made available task-specific training data.

Example of tracking iris (blue) and eyelids (red).

To train the model from the cropped eye area, we manually annotated ~50k images representing different lighting conditions and head poses from geographically diverse areas, as shown below. ..

An eye area annotated with eyelid (red) and iris (blue) contours. The cropped eye region forms the input to the model and predicts landmarks via separate components.

Depth from the iris: Depth estimation from a single image Our iris tracking model can determine the distance from the object to the camera with less than 10% error without requiring any special hardware. This is done by relying on the fact that the horizontal diameter of the iris of the human eye is approximately constant at 11.7 ± 0.5 mm over a wide population. [1, 2, 3, 4], With some simple geometric arguments. For illustration purposes, consider a pinhole camera model that projects onto a sensor of square pixels. The distance to the subject can be estimated from the facial landmarks using the focal length of the camera. The focal length of the camera can be obtained using the camera capture API or along with other camera specific parameters directly from the EXIF ​​metadata of the captured image. Considering the focal length, the distance from the subject to the camera is directly proportional to the physical size of the subject’s eyes, as shown below.

The subject distance (d) can be calculated from the focal length (f) and iris size using a similar triangle. Left: MediaPipe’s iris that predicts the distance in cm in Pixel 2 only from iris tracking. Depth sensor. Right: Ground truth depth.

To quantify the accuracy of this method, we compared it to the iPhone 11 depth sensor by collecting front-facing synchronized video and depth images of over 200 participants. Using a laser ranging device, we ly verified that the error of the iPhone 11 depth sensor is less than 2% at distances up to 2 meters. Our evaluation shows that the depth estimation approach using iris size has a mean relative error of 4.3% and a standard deviation of 2.4%. Testing the approach on participants with and without eyeglasses (without considering their contact lenses) shows that eyeglasses increase the average relative error by only 4.8% (standard deviation 3.1%). I understand. We have not tested this approach for participants with eye disorders (such as senile arc or pannus). Given that MediaPipe Iris does not require any special hardware, these results suggest that devices with a wide range of cost points may be able to obtain metric depth from a single image.

Histogram of estimation error (left) and comparison of estimated and actual distances by the iris (right).

MediaPipe Iris Released We are releasing iris and depth estimation models as a cross-platform MediaPipe pipeline that can run on desktop, mobile and web. As I mentioned in a recent Google Developer Blog post about MediaPipe on the web, I leverage WebAssembly and XNNPACK to run the Iris ML pipeline locally in the browser without sending any data to the cloud.

MediaPipe’s WASM stack allows you to run your model locally in your browser. Left: Iris tracking. Right: Depth from the iris calculated from the photo including EXIF ​​data. You can try iris tracking here and iris depth measurement here.

Future Directions We plan to extend the MediaPipe Iris model to more stable tracking, reduce errors, and expand into accessibility use cases. We strongly believe in sharing code that enables reproducible research in different disciplines, rapid experiments, and the development of new ideas. The documentation and accompanying model cards detail the intended use, restrictions and fairness of the models to ensure that their use is consistent with Google’s AI principles. Please note that any form of monitoring or identification is clearly out of scope and will not be valid with this technology. By providing this iris perception capability to the wider R&D community, we hope that creative use cases emerge and stimulate responsible new applications and new research tools.

For more MediaPipe ML solutions, see the latest updates on our solutions page and the Google Developer blog.

Acknowledgments Thanks to Artsiom Ablavatski, Andrei Tkachenka, Buck Bourdon, Ivan Grishchenko, and Gregory Karpiak for their support in model evaluation and data collection. Yury Kartynnik, Valentin Bazarevsky, and Artsiom Ablavatski who developed the mesh technology. Aliaksandr Shyrokau and the annotation team were diligent in data preparation. Vidhya Navalpakkam, Tomer Shekel, Kai Kohlhoff (domain expertise), Fan Zhang, Esha Uboweja, Tyler Mullen, Michael Hays, Chuo-Ling Chang (help to integrate model into MediaPipe) Continuing to build this technology Matthias Grundmann, Florian Schroff, and Ming Guang Yong for their support.

What Are The Main Benefits Of Comparing Car Insurance Quotes Online

LOS ANGELES, CA / ACCESSWIRE / June 24, 2020, / Compare-autoinsurance.Org has launched a new blog post that presents the main benefits of comparing multiple car insurance quotes. For more info and free online quotes, please visit https://compare-autoinsurance.Org/the-advantages-of-comparing-prices-with-car-insurance-quotes-online/ The modern society has numerous technological advantages. One important advantage is the speed at which information is sent and received. With the help of the internet, the shopping habits of many persons have drastically changed. The car insurance industry hasn't remained untouched by these changes. On the internet, drivers can compare insurance prices and find out which sellers have the best offers. View photos The advantages of comparing online car insurance quotes are the following: Online quotes can be obtained from anywhere and at any time. Unlike physical insurance agencies, websites don't have a specific schedule and they are available at any time. Drivers that have busy working schedules, can compare quotes from anywhere and at any time, even at midnight. Multiple choices. Almost all insurance providers, no matter if they are well-known brands or just local insurers, have an online presence. Online quotes will allow policyholders the chance to discover multiple insurance companies and check their prices. Drivers are no longer required to get quotes from just a few known insurance companies. Also, local and regional insurers can provide lower insurance rates for the same services. Accurate insurance estimates. Online quotes can only be accurate if the customers provide accurate and real info about their car models and driving history. Lying about past driving incidents can make the price estimates to be lower, but when dealing with an insurance company lying to them is useless. Usually, insurance companies will do research about a potential customer before granting him coverage. Online quotes can be sorted easily. Although drivers are recommended to not choose a policy just based on its price, drivers can easily sort quotes by insurance price. Using brokerage websites will allow drivers to get quotes from multiple insurers, thus making the comparison faster and easier. For additional info, money-saving tips, and free car insurance quotes, visit https://compare-autoinsurance.Org/ Compare-autoinsurance.Org is an online provider of life, home, health, and auto insurance quotes. This website is unique because it does not simply stick to one kind of insurance provider, but brings the clients the best deals from many different online insurance carriers. In this way, clients have access to offers from multiple carriers all in one place: this website. On this site, customers have access to quotes for insurance plans from various agencies, such as local or nationwide agencies, brand names insurance companies, etc. "Online quotes can easily help drivers obtain better car insurance deals. All they have to do is to complete an online form with accurate and real info, then compare prices", said Russell Rabichev, Marketing Director of Internet Marketing Company. CONTACT: Company Name: Internet Marketing CompanyPerson for contact Name: Gurgu CPhone Number: (818) 359-3898Email: [email protected]: https://compare-autoinsurance.Org/ SOURCE: Compare-autoinsurance.Org View source version on accesswire.Com:https://www.Accesswire.Com/595055/What-Are-The-Main-Benefits-Of-Comparing-Car-Insurance-Quotes-Online View photos



picture credit

ExBUlletin

to request, modification Contact us at Here or [email protected]