Deep learning based uterine fibroid detection in ultrasound images

Abstract

Uterine fibroids are common benign tumors originating from the uterus’s smooth muscle layer, often leading to symptoms such as pelvic pain, and reproductive issues. Early detection is crucial to prevent complications such as infertility or the need for invasive treatments like hysterectomy. One of the main challenges in diagnosing uterine fibroids is the lack of specific symptoms, which can mimic other gynecological conditions. This often leads to under-diagnosis or misdiagnosis, delaying appropriate management. In this research, an attention based fine-tuned EfficientNetB0 model is proposed for the classification of uterine fibroids from ultrasound images. Attention mechanisms, permit the model to focus on particular parts of an image and move forward the model’s execution by empowering it to specifically go to imperative highlights whereas overlooking irrelevant ones. The proposed approach has used a total of 1990 images divided into two classes: Non-uterine fibroid and uterine fibroid. The data augmentation methods have been connected to improve generalization and strength by exposing it to a wider range of varieties within the training data. The proposed model has obtained the value of accuracy as 0.99. Future research should focus on improving the accuracy and efficiency of diagnostic techniques, as well as evaluating their effectiveness in diverse populations with higher sensitivity and specificity for the detection of uterine fibroids, as well as biomarkers to aid in diagnosis.

Introduction

Fibroids are non-cancerous/benign growths that occur in the muscle wall of the uterus of the woman. These Uterine Fibroids (UF) are commonly found in middle-aged and elderly women with an occurrence rate of 20–25% in women over 30 years old. Pelvic pain, infertility and heavy menstrual bleeding are the common symptoms of UF [1]. However, hormonal imbalances and genetics may be one of the reasons but the accurate cause of UF is unknown. UF can have a substantial impact on women’s reproductive health and quality of life as UFs are a major cause of hysterectomies worldwide, losing their uterus each year due to fibroids [2]. Ultrasound (US) imaging is a non-invasive and commonly used to diagnose and monitor UF. Initial diagnosis of UF typically involves ultrasound images but treatment depend on the type, size, and location, as well as the symptoms and the reproductive goals. Generally, medications are suggested to reduce fibroid size and to control the symptoms [3, 4]. However, manual identification of UFs in the US can be challenging for small or obscured lesions. Therefore, deep learning offers a promising method for automatic classification of UFs in ultrasound images. As per the literature review, it has been observed that deep learning models can achieve high accuracy in classifying non-fibroids and fibroids by surpassing human performance in certain cases. Moreover, it helped in removing human bias and subjectivity from the classification process leading to more consistent results. Automated classification can potentially lead to earlier detection of UFs, allowing for timely intervention and improved patient outcomes by saving radiologists and sonographers valuable time. Therefore, in this proposed work, attention mechanism is combined with EfficientNetB0 model for the uterine fibroids classification from ultrasound images. The chief offerings of the study are as follows:

• 1. An attention based fine-tuned EfficientNetB0 model is proposed for the classification of uterine fibroids from ultrasound images. Attention mechanisms, permit the model to focus on detailed parts of an image and improve the model’s performance by enabling it to selectively attend to important features while ignoring irrelevant ones.

• 2. The data augmentation techniques have been applied to improve model generalization and robustness by revealing it to a wider range of disparities in the training data.

The rest of the research is shown as: Sect. 2 shows the literature review, followed by dataset description in Sect. 3, methodology in Sect. 4 followed by results in Sect. 5, conclusion and future scope in Sect. 6.

Literature review

The researchers had performed work on the classification of UF. They had worked using 3D CNN using a dataset of 3D ultrasound images and had obtained the value of accuracy as 91.3% [5]. The researchers commended the 3D CNN points of interest over conventional 2D in identifying UF due to its upgraded capacity to seizure spatial information. Authors [6] attained 98.8% accuracy by employing a pre-trained ResNet50 CNN calibrated on their dataset of 2D ultrasound images. A study [7] accomplished 96.4% accuracy utilizing the VGG16 model, which was prepared on their ultrasound images dataset. The proposed model extracted highlights from US images employing a grouping of convolutional layers. The proposed model had an accuracy of 97.5%, illustrating the adequacy of DCNNs inside the assurance of UF [8]. In a study [9], examiners proposed a cross breed DL illustrated to recognize UF. The ultrasound images were fed into the show, and highlights were removed utilizing a gathering of CNN and dreary neural frameworks. The revelations of the think about show up that the hybrid DL show has the potential for utilize in helpful picture dealing with since the proposed demonstrate accomplished an accuracy of 96.8%. For the location of UF from ultrasound images, another thinks about [10] proposed a DCNN plan. The proposed shows extricated characteristics from ultrasound images utilizing a combination of convolutional and pooling layers. The proposed demonstration showed up that DCNN can be profitable for recognizing UF with an accuracy of 96.7%. A DL-based system was proposed for programmed UF location from ultrasound pictures in think about [11] to extricate qualities from the ultrasounds.

Dataset description

Input dataset

The dataset comprises 1990 images divided into two classes: Non-uterine fibroid (NUF) and uterine fibroid (UF) as shown in Fig. 1 [12]. The data is split into an 80 − 20 ratio for training and testing, respectively. for the test set, the total NUF is 223 and UF is 173, whereas for the train set, the total NUF is 892 and the total UF is 702. Each image is resized to a uniform size of 224 × 224 pixels. This dataset is crucial for the development and evaluation of deep learning models aimed at the automated detection and classification of uterine fibroids. The class imbalance between the two classes presents a challenge that must be addressed to ensure the model’s robustness and effectiveness. The utilization of such a dataset enables the exploration and implementation of various deep learning architectures and algorithms for improved diagnosis and treatment planning in the context of uterine fibroids.

Fig. 1

(a) Non-Uterine Fibroid (NUF): The red arrow points to the fibroid mass located outside the uterus. (b) Uterine Fibroid (UF): The blue arrow points to the fibroid mass within the uterus. The dataset

Data augmentation

Information increase could be a vital strategy in machine learning for misleadingly growing a dataset by making altered adaptations of images [13]. This process makes a difference move forward demonstrate generalization and robustness by uncovering it to a wider range of varieties within the training data, such as rotations, translations, flips, and changes in brightness or contrast. By augmenting the dataset, the model learns to recognize objects in various positions, orientations, and lighting conditions, making it more effective when applied to real-world data.

• Random Rotation: Randomly rotates the image by a factor of up to 0.15, introducing variations to the orientation of the images.

• Random Translation: Randomly translates the image horizontally and vertically by up to 10% of the image height and width, respectively, adding positional variance.

• Random Flip: Randomly flips the image horizontally or vertically, augmenting the dataset with mirror images.

• Random Contrast: Randomly adjusts the contrast of the image by a factor of up to 0.1, modifying the intensity of pixel values.

These augmentation methods offer assistance to avoid overfitting and move forward the model’s generalization by exposing it to a more extensive extend of varieties inside the dataset, eventually upgrading the model’s execution on unseen information. After the application of information augmentation procedures presently the image count expanded to 10,000. Out of which 8000 images are taken for training and 2000 images are taken for testing reasons.

Methodology

The Fig. 2 has outlined the process of developing a deep learning model for ultrasound image classification. The specific task here is to distinguish between uterine fibroids (UF) and non-uterine fibroids (NUF) in ultrasound images. The process started with a data source of ultrasound images. Subsequently, data augmentation has been applied to enhance the robustness of the model by including some operations like random rotations, translations, flips and contrast adjustments. Following the augmentation, the data has been fed into a pre-trained EfficientNetB0 model, which is a kind of convolutional neural network (CNN) architecture that has been specifically a kind of artificial neural network suggested to analyse visual images. EfficientNetB0 model has acted as a feature extractor by learning patterns from the data. Furthermore, an attention mechanism has been incorporated to the EfficientNetB0 model which has allowed the proposed model to focus on relevant aspects of the input data that are most relevant to the task. Afterward, the global average pooling layer was employed to sum up the features that have been extracted by the EfficientNetB0 model [14], which were afterward followed by a dropout layer that randomly dropped out a specified percentage of neurons during the training to mitigate the overfitting. Henceforth, the data has been served to a dense layer with a softmax activation function. The softmax function has generated a probability distribution over two set of classes that is Uterine Fibroid (UF) or Non-Uterine Fibroid (NUF).

Fig. 2

Proposed methodology

Lastly, the hyperparameters such as learning rate, optimizer, and batch size of the model have been tuned to manage the learning process of the model. Tuning of these hyper-parameters has involved setting of their values to accomplish optimal model performance. This optimization process is normally concerned the evaluation of the model’s performance on a validation set with diverse combinations of hyperparameter values. The objective is to determine the set of hyperparameters that has obtained the best balance between efficiency and accuracy.

EfficientNetB0 model

EfficientNetB0 is a CNN architecture that has been proposed to obtain the balance between the state-of-the-art performance and computational efficiency. Scaling approach has been employed that allowed the EfficientNet model to get better performance as compared to existing models while being more efficient in terms of computational resources in terms of width, depth and resolution. EfficientNetB0 has achieved better performance by scaling these dimensions in a organized way than the other models that only scale one or two dimensions while claiming computational efficiency. EfficientNetB0 has appeared to realize state-of-the-art execution on different picture classification assignments while being more proficient in terms of show measure and computational assets compared to other models. EfficientNetB0 has scaled the width by increasing the number of channels in each layer, the depth of the network by increasing the number of layers and the resolution by increasing the input image size. This multi-dimensional scaling approach has contributed to the model’s superior performance. It is efficient on resource-constrained gadgets such as portable phones or implanted frameworks, where computational assets are constrained [15]. In this work, the EfficientNetB0 model has been optimized through fine-tuning which entailed freezing the weights of the pre-trained base model (base_model) while adding new layers (GlobalAveragePooling2D, Dropout, Dense) on top of it for a new classification task. The frozen base model has acted as a feature extractor by capturing general features from the input images. The new added layers are then trained to learn task-specific features from these extracted features to enhance the model’s performance.

Fine-tuning is performed because pre-trained models like EfficientNetB0 have already learned rich representations from a large dataset (e.g., ImageNet) and can generalize well to new tasks with less data. Instead of training a model entirely from scratch, the power of fine-tuning has been employed. This process has taken a pre-trained model, that is EfficientNetB0, which has already learned valuable features from a massive dataset. These learned features have been adapted to the specific task by fine-tuning for the classification of uterine fibroids. This scheme has offered several benefits. Fine-tuning has led to faster progress as the model does not need to re-learn the elementary image recognition skills which permitted it to focus on the unique characteristics of fibroids in ultrasound images resulting in quicker training times. While dealing with the limited data, fine-tuning has improved the performance. The pre-trained model has acted as a strong basis and fine-tuning has aided it in adapting to the specific patterns in the fibroid classification dataset. This is very helpful when the dataset is fairly small as there is less information for the model to learn from scratch. Finally, fine-tuning has aided in avoiding the overfitting. The pre-trained weights were performed as a method of control which prevented the model from learning specific details in the training data that might not generalize well to unseen images. Moreover, fine-tuning has permitted the model to transfer knowledge acquired from the pre-trained model’s substantial dataset, even if the unfamiliar fibroid dataset is lesser.

Proposed attention mechanism based EfficientNetB0 method

In this research work, a new model named as Attention based EfficientNetB0 has been developed where EfficientB0 is a CNN architecture known for its performance and efficiency which has been improved by the addition of attention mechanisms. The attention mechanism has allowed the model to prioritize the precise areas of an image that are most appropriate for the desired task. This has improved the performance of the model by allowing it to select the significant features while ignoring the inappropriate ones. In the context of EfficientNetB0, attention mechanisms have been typically applied in the form of attention layers, which are added at the several phases of the network. These layers have enhanced the capability of model to capture fine-grained details and long-range dependencies within an image which lead to better generalization and improve feature representation.

Therefore, attention mechanisms have been incorporated to further enhance the capabilities of the model. The integration of attention mechanisms has significantly enhanced the performance of EfficientNetB0 by capturing crucial dependencies and contextual information from images by mimic the human visual focus that allowed the model to highlight specific image regions which are most relevant to the task. There are two main types of attention mechanisms that can be beneficial: Self-Attention has enabled the EfficientNetB0 to capture long-range dependencies and contextual information within the image. It has essentially allowed different regions of the image to communicate with each other that lead to more comprehensive understanding of the content. By employing self-attention, EfficientNetB0 has effectively learned the global dependencies within an image. This attention mechanism has aided in identifying relationships between distant image elements for improving performance in various tasks. Another Spatial Attention mechanism has focused on specific locations within the image. It has focused on specific image regions highlighting the areas of interest while suppressing non-relevant ones in tasks where spatial information is essential, such as object detection and segmentation. Integrating attention mechanisms into EfficientNetB0’s architecture has involved the addition of attention layers. A prevalent approach is to integrate the self-attention through the transformer mechanism which has enhanced the model’s ability to extract relevant information from images by boosting its performance in computer vision tasks.

Results

Hyperparameter tuning

In this work, proposed model has been trained using four distinct hyperparameters named as batch size, optimizer, learning rate and epochs.

Batch size

It determined the number of training examples used in a single update of the model’s internal parameters during gradient descent. It could be a vital hyperparameter in deep learning models that influences both the training speed and the quality of the model. Choosing a suitable batch estimate depends on the particular dataset size, model complexity, and available resources. Batch size is regularly tuned along with other hyperparameters to optimize the execution of the demonstrate. Batch size is related to the training set size (N) and the number of iterations per epoch (M) by the given formula.

$${\rm{N = batch size \times M}}$$

Where N is the total number of training examples. M is the number of iterations per epoch.

Optimizer

Nadam Optimizer is an optimization algorithm in place of the Adam optimizer that has combined the benefits of Nesterov accelerated gradient (NAG) descent and Adam. By integration of the NAG technique, Nadam is the modification over Adam by enabling more precise and stable convergence. It has integrated the NAG technique to adjust the update direction based on momentum with adaptive learning rates for each parameter. This integration has permitted Nadam to offer steadier and more effective optimization in comparison to the other optimizers. It is helpful in providing training to deep neural networks where fast convergence and robustness to noisy data are crucial. Nadam’s adaptive learning rate method has assisted in navigating complex loss settings which makes it a widespread selection for various deep learning tasks.

Epochs

An epoch is represented as a single training cycle where the entire dataset is fed through the model once. During each epoch, the model has renewed its internal parameters (weights and biases) based on the errors (loss) it has faced in the training. Training for more epochs has permitted the model to learn from the data multiple times to improve its performance. However, it may lead to overfitting. Overfitting occurs when a model has memorized the training data too well, losing its ability to generalize to unseen data. It’s a balancing act: train for too few epochs and the model might underfit (fail to learn the patterns in the data), train for too many and you risk overfitting. Epochs are related to the batch size (B), the total number of training examples (N), and the number of iterations per epoch (M) by the formula:

$$M = N/B$$

Learning rate

The learning rate is a hyperparameter in deep learning that controls the step size during optimization. The learning rate is typically set before training and can be fixed or adjusted dynamically during training using techniques like learning rate schedules or adaptive learning rate methods. In this work, the learning rate is set to 0.00005. Lower learning rates often result in a more stable optimization process, as the updates to the parameters are smaller and less likely to lead to divergence. With a lower learning rate, the optimization algorithm takes smaller steps toward the minimum point, potentially allowing it to find a more precise solution.

Accuracy and loss analysis

The Fig. 3 represents the accuracy metrics of a model trained over multiple epochs. Each row corresponds to an epoch number, and the columns indicate the accuracy achieved on the training data (Accuracy) and the validation data (Val_Accuracy) at that epoch. The accuracy values show a clear trend of improvement over epochs for both the training and validation sets. Initially, at epoch 1, the model started with a relatively low training accuracy of 56.11% on the training data and an even lower 34.85% validation accuracy, indicating that the model is not performing well and likely underfitting. However, as the training progresses, the model’s performance has been improved significantly. By epoch 4, the model has achieved a high training accuracy of 96.70% on the training data and 86.11% validation accuracy on the validation data, indicating that the model is learning the underlying patterns in the data well. Towards the later epochs, the model’s performance has continued to improve, with accuracy values nearing 100% on both the training and validation sets. From these experiments, it has been observed that the model has learned the dataset’s features effectively and is performing very well, likely indicating that it has reached a point of overfitting, especially as the validation accuracy started to plateau.

Fig. 3

Proposed Attention based fine-tuned EfficientNetB0 model’s accuracy

Figure 4 displays the loss metrics of a proposed model trained over ten epochs, showing both training loss (Loss) and the validation loss (Val_Loss) at each epoch. At initial state, during epoch 1, both the training and validation losses are relatively high, which indicated that the model has not performed well and likely has high error rates.

Fig. 4

Loss of Attention based fine-tuned EfficientNetB0 model

However, as the training progressed, the losses steadily has decreased which indicated that the model has improved its performance by learning the dataset’s patterns. By epoch 4, the losses have reduced significantly, indicating that the model has become more accurate by making fewer errors. This trend continued with the losses decreasing further in each subsequent epoch. Towards the later epochs, the model’s performance has continued to improve, with the losses approaching very low values. This suggested that the model has learned the dataset well and has performed well at a high level of accuracy, especially as the validation loss closely tracked the training loss, indicating that the model did not have any overfitting issue.

State-of-art comparative analysis

The table has listed a comparative analysis of various techniques and their corresponding accuracy rates. Table 1, it has shown that Dilna et al. [11] have achieved an accuracy of 95.1% in classifying ultrasound scanned uterus images indicating the effectiveness of their classification method. Furthermore, Behboodi et al. [10] have utilized UNet-based networks for US diagnostic imaging and have achieved an accuracy of 86.2% with the consideration of the utility of this architecture for medical image analysis. Besides, Li et al. [16] have used deep learning on the ChEMBL dataset by achieving an accuracy of 85% which shows the potential of deep learning for pharmaceutical research. Moreover, Tang et al. [17] has introduced the AR-Unet Network and has achieved an impressive accuracy of 94.56% on the AR-Unet dataset which indicated the robustness of approach. Then, Yang et al. [18] utilized neural networks on an ultrasound image and achieved an accuracy of 88.5% which demonstrated the effectiveness of deep learning for medical image analysis. Girija et al. [19] has employed various data mining techniques on 450 patients with an accuracy of 89.54% which has illustrated the importance of data mining in healthcare research. Additionally, Huo et al. [20] have used a Deep learning-based method on a dataset of 3870 ultrasound images with an accuracy of 87.45%. Overall, the accuracies attained by these studies have displayed the efficiency of different methods and procedures in healthcare with each approach showcasing its strengths in diverse perspectives.

Table 1 State-of-art comparison

Conclusion and future scope

Fibroids in the uterine are a gynaecological disorder that can substantially influence the health of women and their quality of life. Fibroids can impede the implantation of a fertilized egg or interrupt the blood flow to the uterus which may lead to recurrent miscarriages or infertility. Thus, early detection of uterine fibroids (UF) is essential for maintaining the fertility. By early treatment of fibroids, these risks can be reduced, and women can have a better chance of conceiving and carrying a pregnancy. Though, all fibroids don’t cause any symptoms or require any form of treatment, therefore, detection of those that are likely to cause problems can help in providing treatment plans to individual needs. This can lessen the use of unnecessary treatments and minimize the impact of fibroids. Another challenge is the lack of reliable diagnostic tests for uterine fibroids. Therefore, in this work, for the early detection of fibroids in the uterine, an attention mechanism based fine-tuned EfficientNetB0 model has been proposed for the classification of uterine fibroids and non-fibroids from ultrasound images.

Future research directions include the use of biomarkers that can specify the occurrence of fibroids at an early stage or predict their growth and progression at diverse stages. Biomarkers such as specific proteins or genetic markers could help to improve the accuracy of diagnosis and guide in treatment decisions. This will help to ensure that new diagnostic techniques are applicable and accessible to a wide range of women, regardless of their age, ethnicity, or socioeconomic status.