Multi-center deep learning for placental segmentation in obstetric ultrasound with multi-observers and cross-country generalization

image annotation

A total of 7,500 ultrasound images, randomly sampled from obstetric ultrasound exams recorded in Copenhagen, Denmark between 1 January and 31 December 2018, were available. Sample size was based on feasibility. The images were taken as part of a Danish screening program that includes early pregnancy aneuploidy screening, late pregnancy fetal malformation screening, and other indication-based tests. All ultrasound images were regularly stored locally in the Astraia database (Astraia Software GmbH, Munich, Germany) and were accessible for all these examinations. Finally, the images for annotation were anonymized and stored on a secure server.

The presence or absence of placenta in the images was recorded. Then, for images containing the placenta, the annotation tool Labelme²⁷ was used to outline the placenta and create a ground truth dataset. This manual annotation was performed by the lead author (LA) on all his 7500 ultrasound images.

In addition, in 500 randomly selected images, placental locations were labeled as belonging to either the anterior or posterior wall of the uterus.

To compare the performance of the model to that of the clinical staff, 5 fetal medicine specialists, 10 obstetricians and 10 midwives were randomly selected from 24 randomized samples representing all trimesters. The ultrasound images were annotated (8/9/7 images from the 1st/2nd/3rd trimester, respectively). ). The participant provided her written informed consent and participation was voluntary. A fetal medicine specialist was a specialist in fetal medicine, an obstetrics and gynecology resident employed by a department of obstetrics and gynecology, but without a professional degree, and a midwife with no ultrasound experience licensed midwifery I was a teacher. There was no subject overlap between this additional test and training sets.

ethics

This study was approved by the Danish Data Protection Agency (protocol number P-2019-310) and the Danish Patient Safety Authority (protocol number 3-3031-2915/1). Under Danish law, the Danish Patient Safety Authority can grant permission to use patient chart data for research purposes without the individual patient’s consent. This was done prior to data acquisition.

This study was conducted in accordance with relevant guidelines and regulations.

Model architecture and training

Mask R-CNN model proposed by He et al.²⁸ It was used in this study because it is widely used for instance segmentation in many different domains.The model was initialized with a backbone pretrained on the ImageNet database and trained on the data using PyTorch²⁹During training, the model was provided with images and corresponding binary masks, which encoded the position of each region of placental tissue in the image pixel-by-pixel. Masks were extracted from polygons created during the annotation process. During inference, the model identified all regions predicted to contain placental tissue and output a mask for each region. Additionally, the model predicted each prediction confidence score between 0 and 1 for the presence of placental tissue.

image and partition

Of the 7500 images screened, 2130 images were found to contain the placental region during the annotation process. These were randomly split into a training set, a validation set and a test set containing 1704, 208 and 218 images. Additionally, 208 and 218 images without placental regions were added to the validation and test sets to estimate the robustness of the model against false negatives. The training/validation/test partitions were created to prevent subject images from appearing in multiple partitions.

Of the 500 images screened and annotated for anterior or posterior placental position, 164 images showed placental tissue. These were divided into training, validation, and test partitions containing 116, 24, and 24 images, respectively. Validation and test sets were created with equal numbers of anterior and posterior placental regions. No images without placental regions were added in this experiment.

A validation set was used to find the optimal hyperparameters. This includes the acceptance threshold, training period, and amount of Random Augmentation applied.

Additional tests were performed on 100 annotated images from the Barcelona public dataset. These images were acquired using Voluson E6, Voluson S8, Voluson S10 (GE Medical Systems, Zipf, Austria), and Aloka (Aloka CO., LTD); and Voluson S10 (GE Medical Systems, Zipf, Austria).

Random Augmentation of Training Data

The model was trained both with and without the addition of Random Augmentation. Random Augmentation was added to the training data to make the model more robust. Geometry transform enhancements such as shear and rotation were added to both the image and the corresponding mask, whereas per-pixel enhancements such as brightness and color inversion were applied only to the image. The amount of dilation applied constitutes a hyperparameter that is tuned by adjusting the number of random dilations applied to each image.

For each image in the training partition, a number of randomly augmented images were created and added to the set. A non-extended version was also kept in the training set.

training

The model was trained on a training set of 100 epochs using the Adam optimization algorithm. Model performance was computed on the validation set after each epoch and the best performing model was saved as the final output model. The model was trained on an Nvidia Quadro RTX 6000 24 GB GPU with a batch size of 5 and a training time of ~6 minutes per epoch.

Model prediction processing

During inference, the model output was processed in two steps.

confidence

All predicted placental regions with confidence scores below the set threshold (acceptance threshold = 0.5) were discarded.

Prediction overlap

If the model predicted multiple placental regions in the image, and one of the regions was completely within another region, the region with the lowest confidence was discarded.

evaluation

The model was evaluated on validation and test sets using two different approaches:

image classification

The model was evaluated for its ability to detect the presence of placental tissue in images without considering correspondence between predicted placental regions and ground truth regions. This task was approached as a binary image classification problem where the segmentation map generated by a model or annotator for a given image is mapped as follows: Sky again not empty.

In addition to quantifying the model’s performance on ground truth annotation, we compared the model’s accuracy with human performance.

image segmentation

We next evaluated the model for its ability to pinpoint placental regions. Overlap between annotations and predictions was quantified as the Intersection Over Union fraction (IoU).

We then evaluated the performance of the model on 24 annotated test images in three groups of clinicians with varying levels of experience with ultrasound: 5 fetal medicine specialists, 10 obstetricians and 10 gynecologists. midwives) performance. I compared her IoU and ground truth of the model to his IoU with the ground truth of the annotator. For IoU comparisons, only images containing placenta were included in the analysis.

All analyzes were further divided into three semesters.

Generalization of cross-country

We used open access datasets to determine how well the final model performed on data from different populations^Five From Barcelona, ​​Spain, as a final cross-validation test. We selected 100 mid-gestation ultrasound images and manually annotated these images using the same annotation procedure described above. Next, we compared model performance across datasets.

report

Results will be reported according to TRIPOD guidelines³⁰.