AI-Doctor’s accuracy in retinal disease detection redefines medical diagnosis

In a recent study published in Cell Reports Medicine, researchers developed AI-Doctor, an advanced artificial Intelligence (AI) system designed to interpret fundus fluorescein angiography (FFA) images precisely.

It aids in the accurate diagnosis and treatment suggestions for various Ischemic Retinal Diseases (IRDs).   

Study: An artificial intelligence system for the whole process from diagnosis to treatment suggestion of ischemic retinal diseases Xinyu Zhao,1,2,6 Zhenzhe Lin,1,6 Shanshan Yu,1,6 Jun Xiao,1 Liqiong Xie,1 Yue Xu,1 Ching-K. Image Credit: Yurchanka Siarhei/Shutterstock.com

Background 

In ophthalmology, AI has been extensively explored for disease screening or diagnosis. Still, a comprehensive AI model simulating physicians’ extensive thought processes from diagnosis to treatment suggestions remains a pivotal need.

This is crucial for conditions like IRDs reliant on intricate FFA for evaluation, where interpretation challenges exist, particularly in underserved areas.

While technologies like optical coherence tomography angiography (OCTA) have surfaced, their application is confined, emphasizing the need for AI to bridge the significant gap in healthcare services by promoting accessibility and equity in eye care.

Current AI models, primarily focused on specific diseases, lack extensive practical implementation, including treatment suggestions. Therefore, further research is imperative for developing versatile AI models to ensure equitable and efficient eye care solutions where specialized knowledge is scarce and interpreting diagnostic images is complex.

About the study

In the study, experienced technicians utilized Spectralis Heidelberg Retina Angiograph + Optical Coherence Tomography (HRA+OCT) to capture FFA images from various hospitals, adhering to a unified image capture protocol. The images were categorized into different phases for both clinical and research applications.

A team of authorized ophthalmologists, well-versed with standardized training, executed the classification tasks, ensuring the precision of the image b phase identification and IRD diagnosis.

The segmentation task involved the development of a model focusing on Branch Retinal Vein Occlusion (BRVO) and Diabetic Retinopathy (DR) with non-perfusion areas (NPAs), utilizing FFA images and relying on specialists’ annotations and refinements for achieving accurate results.

An additional check on model generalizability was conducted with images of other IRDs, which included Central Retinal Vein Occlusion (CRVO) and retinal vasculitis.

The developed Ai-Doctor consisted of two models, one for classification and another for segmentation, employing state-of-the-art convolutional neural networks, Unet and ResNet-152.

For improved clinical applications, a Clinically Applicable Ischemia Index (CAII) was proposed, aiming to facilitate the identification of patients with higher risk, using FFA images captured from different orientations and applying detailed principles for image selection and CAII calculations.

Specialists initially divided the eyes into groups based on their need for laser therapy, utilizing the characteristics of FFA images, medical records, and clinical experience.

The CAII values for each eye in the groups were quantified under the segmentation model, setting the CAII values as the threshold for laser therapy suggestions and deriving sensitivity and specificity under the corresponding threshold to explore the optimal threshold, subsequently validating them with an independent external dataset.

The evaluation of classification and segmentation models was rigorously conducted using statistical tests and measures such as accuracy, recall, and precision, along with the analysis of Receiver Operating Characteristic (ROC) and areas under the ROC curve.

The Dice similarity coefficient (DSC), Intersection over Union (IoU) value, and F1 score were utilized for segmentation assessment. Empirical ROC curves were examined to establish optimal CAII thresholds for recommending laser therapy.

These meticulous analyzes and subsequent statistical assessments ensured the thorough evaluation of the developed models and methodologies.

Study results  

In the present research, a novel AI-Doctor system was developed and meticulously validated, leveraging 24,316 images. The design was systematically structured, utilizing distinct datasets for classification tasks in training and internal validation at Zhongshan Ophthalmic Center (ZOC), and external tests at Shenzhen Eye Hospital (SEH) and Foshan Second People’s Hospital (FSPH).

An additional 1,295 images were dedicated to refining the segmentation model, focusing on NPAs and branch retinal vein occlusion areas.

Ai-Doctor has showcased promising results, displaying high precision, recall, and accuracy in multiple datasets for identifying image phases and diagnosing common IRDs, aligning its diagnostic competency with experts.

It effectively utilized heatmap visualization to facilitate precise identification of diverse conditions like diabetic retinopathy, revealing a focus on more pronounced features in the images, especially where misclassification occurred.

For segmentation, the research compared Unet-VGG16 and Unet-Swin Transformer models, with the former exhibiting superior performance and robustness in segmenting NPAs and BRVO areas, both in internal validation and external test sets.

When trained with a diversified set of images, including those of diabetic retinopathy and BRVO, Ai-Doctor’s segmentation model demonstrated enhanced versatility and applicability, showing high Dice similarity coefficients even in previously unencountered conditions.

The study introduced a clinically applicable ischemia index (CAII), calculated based on segmentation results of 55°-viewing field FFA images.

The established optimal CAII thresholds showcased high sensitivity and specificity in determining the need for laser therapy, with subsequent validations reinforcing the reliability of these thresholds in classifying diverse conditions such as BRVO and diabetic retinopathy.

AI-Doctor enhances clinical utility by automatically generating comprehensive AI reports for FFA image interpretations and maintains high efficiency, completing the entire process from image phase identification to ischemic area segmentation in approximately eight seconds. This efficiency emphasizes AI-Doctor’s potential for rapid application in clinical environments.

This research underscores AI-Doctor’s transformative potential in ophthalmological diagnostics, substantiating its reliability, precision, and versatility in medical image analysis.

The AI system’s capability to provide detailed insights and its quick processing time render it a pivotal advancement, potentially reshaping diagnostic approaches in ophthalmology and substantiating its significance in medical image interpretation. 

The AI-Doctor’s detailed, rapid, and precise interpretations mark a transformative progression in medical diagnostics, especially in ophthalmology, by combining efficiency, reliability, and extensive applicability.

The research’s implications are profound, highlighting the potential of advanced AI in revolutionizing medical diagnostics through detailed, efficient, and precise interpretations, paving the way for groundbreaking advancements in ophthalmology.