Application of improved Unet network in the recognition and segmentation of lung CT images in patients with pneumoconiosis

Abstract

Background

Pneumoconiosis has a significant impact on the quality of patient survival. This study aims to evaluate the performance and application value of improved Unet network technology in the recognition and segmentation of lesion areas of lung CT images in patients with pneumoconiosis.

Methods

A total of 1212 lung CT images of patients with pneumoconiosis were retrospectively included. The improved Unet network was used to identify and segment the CT image regions of the patients’ lungs, and the image data of the granular regions of the lungs were processed by the watershed and region growing algorithms. After random sorting, 848 data were selected into the training set and 364 data into the validation set. The experimental dataset underwent data augmentation and were used for model training and validation to evaluate segmentation performance. The segmentation results were compared with FCN-8s, Unet network (Base), Unet (Squeeze-and-Excitation, SE + Rectified Linear Unit, ReLU), and Unet + + networks.

Results

In the segmentation of lung CT granular region with the improved Unet network, the four evaluation indexes of Dice similarity coefficient, positive prediction value (PPV), sensitivity coefficient (SC) and mean intersection over union (MIoU) reached 0.848, 0.884, 0.895 and 0.885, respectively, increasing by 7.6%, 13.3%, 3.9% and 6.4%, respectively, compared with those of Unet network (Base), and increasing by 187.5%, 249.4%, 131.9% and 51.0%, respectively, compared with those of FCN-8s, and increasing by 14.0%, 31.2%, 4.7% and 9.7%, respectively, compared with those of Unet network (SE + ReLU), while the segmentation performance was also not inferior to that of the Unet + + network.

Conclusions

The improved Unet network proposed shows good performance in the recognition and segmentation of abnormal regions in lung CT images in patients with pneumoconiosis, showing potential application value for assisting clinical decision-making.

Introduction

Pneumoconiosis is a common occupational lung disease, which is mainly caused by long-term work in environments containing dust particles. It is widespread in industrialized countries, especially among workers in the mining, construction and metallurgical industries. The incidence of pneumoconiosis varies from region to region, but there is a general trend of high incidence. According to statistics, pneumoconiosis is one of the main causes of occupational disease death in many countries [1]. Therefore, the prevention and treatment of pneumoconiosis are crucial, including strengthening dust control in the working environment, regular physical examination and rational treatment [2].

The pathological changes of pneumoconiosis include fibrosis and scarring of lung tissue and damage to the alveolar walls. These changes lead to decreased lung function, dyspnea and other associated symptoms [3]. The diagnosis of pneumoconiosis usually relies on a comprehensive assessment of clinical symptoms, occupational history, X-ray examination and lung function tests. Chest X-rays and computed tomography (CT) scans are commonly used imaging techniques in the detection and diagnosis of pneumoconiosis.

Currently, image analysis of pneumoconiosis mainly relies on physicians’ visual judgment and manual marking segmentation [4], which is subjective, time-consuming and error-prone [5]. In order to improve accuracy and efficiency, researchers began to explore automatic identification and segmentation methods based on computer algorithms [6]. The segmentation methods are mainly based on feature extraction and pattern recognition of chest CT images. Computer algorithms are able to automatically detect and locate abnormal areas of the lungs, such as nodules, plaques and fibrotic lesions, and perform quantitative analysis [7]. These methods reduce human error and improve the consistency and reliability of diagnosis [8].

In recent years, deep learning techniques have also made important breakthroughs in image recognition, localization and segmentation [9,10,11,12,13]. Fully Convolutional Network (FCN) [14] is a widely used method. By replacing the fully connected layers of traditional Convolutional Neural Network (CNN) with convolutional layers, it is able to accomplish the accurate segmentation of contents and objects. For example, in the field of medical imaging, FCN network can be used to accurately segment the tumor regions in medical images such as chest CT or chest MRI through training the network. There are three structures of FCN networks, FCN-32s, FCN-16s, and FCN-8s, among which FCN-8s shows the best segmentation effect [15]. Ronneberger [16] et al. proposes Unet after improving and expanding on the basis of FCN, which mainly contains the encoding-decoding structure and skip connection, and it has been widely used in the field of image segmentation. However, due to the simple structure of the Unet network(Base), the feature extraction ability of the lesion region is insufficient, resulting in its model training is prone to overfitting phenomenon and relatively poor segmentation results. The Unet + + network proposed by Zhou et al. [17] in 2018 is partially adapting the direct hopping connection between the Unet encoder and decoder to a nested dense hopping connection, but it still does not allow for exploring enough information from multiscale features.

Therefore, in response to the current “difficult diagnosis” of pneumoconiosis, we proposed an improved Unet network, which aimed to fully automate the identification and segmentation of suspected abnormal regions of different sizes and locations of lung CT images in patients with pneumoconiosis, to provide a powerful support for clinical diagnosis and treatment decisions and to improve the prognostic outcomes of patients. In practical application, this model can be used in rural areas to solve the problem of diagnosis and treatment, promote the development of primary medical care, thus alleviating the problem of “difficult to diagnose” pneumoconiosis in rural areas, promoting the development of “Internet + Intelligent Medical Care”, and promoting the implementation of the strategy of Healthy China.

Materials and methods

Patients

The study retrospectively selected patients with pneumoconiosis who were treated at the West China Fourth Hospital of Sichuan University. The inclusion criteria covered the following requirements: (1) individuals with a history of dust exposure; (2) patients whose lung radiographs met or exceed the acceptable quality criteria set out in the GBZ70-2015 guidelines for the diagnosis of occupational pneumoconiosis; and (3) positive cases who had been formally diagnosed with pneumoconiosis and who had obtained diagnostic certificates from qualified units. On the other hand, the exclusion criteria included subjects with pre-existing pulmonary or pleural diseases that would interfere with the diagnosis or grading of pneumoconiosis. These may include but are not limited to, pneumothorax, pleural effusion, or incomplete resection of lung tissue on one side.

A total of 20 patients were included in the study, of which 19 were male and 1 was female. The age distribution was mainly concentrated between 30 and 60 years old, with only 4 patients younger than 30 years old. 13 patients were between 30 and 60 years old, and there were also 3 patients older than 60 years old. The mean age of the study population was 45 years and the median age was 38 years. The age and gender distribution was shown in Fig. 1.

Fig. 1

Population pyramid in frequency

The study was approved by the Medical Ethics Review Committee of West China Fourth Hospital of Sichuan University, with the review committee approval number HXSY-EC-2,023,042.

Data source and processing

This study used a set of CT image dataset provided by West China Fourth Hospital of Sichuan University and developed a system Based on an innovative algorithm for marker extraction. The algorithm integrated a variety of advanced digital image processing techniques, such as watershed algorithm, region growing algorithm, flood processing, morphology and edge detection [18], which were improved and combined according to the image features in order to achieve segmentation and extraction of CT images for pneumoconiosis. The algorithm had high accuracy, strong reliability and completeness, and had been verified by expert doctors with an accuracy rate of 93.6%. It can help doctors diagnose and treat pneumoconiosis more quickly, shorten the treatment time and improve the treatment effect.

Network structure

8 S network

FCN-8s replaces the traditional fully connected layers with fully convolutional layers, allowing the network to accept input images of arbitrary size and output pixel-level predictions of the same size. The FCN model can use FCN-32s, FCN-16s, and FCN-8s structures depending on the segmentation granularity, and 8s is the feature map recovered from 8-fold downsampling to the input size. The FCN-8s network structure was shown in Fig. 2.

Fig. 2

Schematic diagram of FCN-8s network

Unet network

The layer structure of the encoding area of the Unet network mainly consists two 3 × 3 convolutional layers represented by light blue, and a max pooling layer of size 2 × 2 represented by light gray, and the layer structure of the decoding area mainly includes 2 × 2 up-sampling represented by light gray and 2 × 2 convolutional layers represented by light green in transposed convolutional layer, and two 3 × 3 convolutional layers represented by light blue, and all of the convolutions use the Rectified Linear Unit (ReLU) [19] as the activation function. The Unet network structure was shown in Fig. 3.

Fig. 3

Schematic diagram of Unet network

Unet + + network

Unet + + splits the encoder and decoder parts of the original Unet model into submodules, each of which contains an encoder and a decoder. In the encoder, each submodule downsamples the input image twice consecutively and extracts features using a convolutional layer; in the decoder, each submodule up-samples and fuses the output of the previous level with the encoder features, and then performs the decoding operation at the next level. Ultimately, the output of Unet + + is formed by combining the outputs of all the submodules.The structure of Unet + + network was shown in Fig. 4.

Fig. 4

Schematic diagram of Unet + + network

Related work

In this study, Gaussian Error Linear Unit (GeLU) [20] was introduced as an activation function in improved Unet network. Although ReLU and GeLU activation functions are widely used in the field of deep learning, the choice of using GeLU in some specific cases may lead to better performance [21].The new activation function GeLU solved the problem that ReLU can lead to the disappearance of non-positive gradient and uneven data distribution. GeLU introduced the Sigmoid function on the basis of ReLU, which made its output smoother, and thus it could reduce the problem of oscillations occurring in the neural network [22], and made the model have better generalization ability and accuracy to improve the performance of lung particle segmentation. The formula and function image (Fig. 5) were expressed as follows:

$$GeLU(x)=0.5 \times x(1+\tanh \left[ {\sqrt {\frac{2}{\pi }} (x+0.0447{x^3})} \right])$$

(1)

Fig. 5

GeLU activation function

This study incorporated the Squeeze-and-Excitation(SE) attention mechanism into the Unet network, which mainly increased the model’s attention to important features by adaptively adjusting the weights of each channel in the input feature map. Specifically, the SE attention mechanism first compressed the input feature map into a vector using a global average pooling operation, and then learned a weight vector through two fully connected layers, which contained the weight information of each channel. Finally, this weight vector was multiplied with the original input feature map to obtain the enhanced feature map. In the Unet model, the SE attention mechanism can be used to enhance the attention during feature extraction and reconstruction. The structure of the SE module was shown in Fig. 6.

Fig. 6

Structure of SE module

In this study, the residual mechanism was introduced to improve the doubleconv part of the Unet network, and each convolutional layer was followed by a Gelu activation function and a normalization layer (BatchNorm2d) to improve the feature learning ability of the model [23], and SE attention mechanism was added to Unet network The structure of improved Unet network was shown in Fig. 7.

Fig. 7

Schematic diagram of improved Unet network

At the same time, in order to compare the role of the GeLU activation function, we introduced Unet(SE + ReLU) to compare with the improved Unet(SE + GeLU).

Furthermore, in order to demonstrate the impact of data augmentation techniques on model performance, we conducted ablation experiments.

Moreover, in order to explore whether data distribution and random grouping affect the performance and generalization ability of the model, we conducted a five-fold cross-validation experiment.

Evaluation indicators and methods

In this study, Dice similarity coefficient, positive prediction value (PPV), sensitivity coefficient (SC) and mean intersection over union (MIoU) were used to measure the accuracy of the improved Unet network for segmentation of abnormal regions of lung CT images [24], and to verify the superiority of the segmentation performance of the improved Unet network by comparing it with the metrics of the mainstream network models.

Dice similarity coefficient is the most common measure in medical image segmentation tasks, and it is a measure of the overlap region between labeled and modeled segmentation results. The formula was expressed as follows:

$$\begin{array} {l}{\textcircled{1}Dice(A,B)=2\frac{\vert A \cap B\vert}{ \vert A\vert +\vert B \vert }}\\{\textcircled{2}Dice=\frac{2TP}{ 2TP+FN+FP }}\end{array}$$

(2)

In the formula (2), ① and ② are the two representations of Dice indexes, A and B in Equation ① represent the labeling map and segmentation map contour area, respectively. TP, FP, and FN in Equation ② represent the number of true positive, false positive, and false negative, respectively.

Generally, it is considered that Dice > 0.7 indicates that the model segmentation and the doctor’s manual segmentation of the region have a high degree of repetition, and the segmentation effect is excellent [25].

The PPV evaluation index indicates the ratio of the number of true positive samples to the number of predicted positive samples. The formula was expressed as follows:

$$PPV=\frac{{TP}}{{TP+FP}}$$

(3)

The SC evaluation index indicates the ratio of the number of true positive samples to the number of actual positive samples. The formula was expressed as follows.

$$SC=\frac{{TP}}{{TP+FN}}$$

(4)

The MIoU evaluation index indicates the average of the ratios of the intersection and concatenation of the true label and the predicted value.The formula was expressed as follows.

$$MIoU=\frac{1}{{k+1}}\sum\limits_{{i=0}}^{k} {\frac{{TP}}{{FN+FP+TP}}}$$

(5)

Giga Floating-point Operations Per Second (GFLOPS), Params, Frame Per Second(FPS) were used to evaluate the inference speed.

Statistical methods

R statistical software (version 4.3.3) was used for data analysis. The Shapiro-Wilk method was used to test whether the data satisfied normal distribution. Based on the results of normality and variance chi-square tests for each evaluation metric, Wilcoxon signed rank sum test was used to perform the statistical test of the performance evaluation metrics obtained from the improved Unet network and other networks, and the statistical test was corrected by the Benjamini and Hochberg FDR (BH) method, α = 0.05 [26].

Results

Results of pre-processing and enhancement of CT image data

The CT image data were further preprocessed before applying the improved Unet network for the experiments. Watershed algorithm was introduced for dividing the labeled image pixels into foreground and background, and three-channel processing was employed to address the possible errors caused by binarization processing. Meanwhile, the accuracy of the segmented images was further improved by combining the region growing algorithm to accurately control and adjusted the segmented images by removing the misdetected objects and adding the missed objects.

Next, a flood processing algorithm was applied and the segmented image was regionally grown using different features according to a pre-determined regional guide to obtain more complete and detailed information about the target boundaries, which facilitated the extraction of particles and substance one by one. All image segments underwent uniform processing, and their resolution were adjusted to a size of 512 × 512. Finally, the data were cleaned to exclude the effects caused by incomplete extraction of dust particles and other reasons. After selecting 1,212 patient image samples that met the requirements, 848 of them were used as the training set and the other 364 samples as the validation set by randomized sorting, and all Dicom format samples were converted to PNG format images.

Afterward, the original images underwent lung parenchyma extraction, and the corresponding gold standard images were masked. Subsequently, the processed data were resized to a resolution of 512 × 512. The experimental data volume were enlarged by data cleaning and supervised single-sample data enhancement operations [27], and geometric transformations and DropBlock regularization methods [28] were used to effectively avoid overfitting problems. A fixed-size rectangular region was cropped from the center of the image using the center crop operation to resize the image to a uniform size. And the CT maps without foreground targets are removed. The data after enhancement were used for model training and validation. The image processing results were shown in Fig. 8.

Fig. 8

Schematic diagram of processing results. A1: Original image; B1: Ground truth image; A2: Ground truth image after mask processing; B2: Original image after extracting lung parenchyma

The results of the ablation study were shown in Table 1.

Table 1 Summary of ablation study for the improved unet model and data augmentation

Parameter setting and segmentation effects of the improved Unet network

The experiment used the Pytorch deep learning framework to build the experimental environment, which ran on Pycharm software. The specific experimental network code is available through the corresponding author.

In this study, the input image shapes for all networks were all 512 × 512 × 1, with a depth of 8. Additionally, all networks utilized the Adam optimizer for optimization. According to the GPU memory and image data size, in order to improve the network training efficiency, as well as for the model to be more stable in the later stages of training, the following excellent empirical values of network parameters were obtained by comparing a large number of parameter adjustment experiments: the number of training samples batch_size for all the networks was set to 2, and the training times of network training as steps _ per _ epoch was 606 (i.e., the number of data set samples / batch_size = 1212/ 2). The number of training rounds for each round of network training num_classes was set to 2 according to the correlation between the parameters, and the total number of training rounds epochs was set to 100 (the total number of iterations of the network was 60,600), and the learning rate was initialized to 0.00005. The learning rate update strategy and mixed accuracy training were used to optimize the neural network model to improve the training speed and reduce memory consumption.

The variation of the Loss curve of the improved Unet network was shown in Fig. 9, where it can be observed that the Loss value was higher at the beginning of the training, but with the increase of the number of iterations, the Loss gradually decreased and stabilized. This indicated that the improved Unet network was able to gradually learn a better feature representation during the training process and improve the segmentation performance. The variation of Accuracy curves for the improved Unet network and other networks was shown in Fig. 10.

Fig. 9

Loss curve variation diagram

Fig. 10

Accuracy comparison diagram

Comparison of the segmentation effect of the improved Unet network and the base network

Figure 11 illustrated the segmentation effect of the improved Unet network, FCN-8s network, Unet(Base) network, Unet(SE + ReLU) network and Unet + + network.

Fig. 11

Schematic diagram of FCN-8s, Unet(Base), improved Unet(Ours), Unet(SE + ReLU) and U-net + + network segmentation result

Results of five-fold cross-validation

Figure 12 illustrated the Accuracy curves of five-fold cross-validation.

Fig. 12

Results of five-fold cross-validation

Results of multiple comparative experiments

The experimental networks used in this study included the FCN-8s network, the Unet network (Base), the Unet(SE + ReLU) network, the improved Unet(SE + GeLU) network and the Unet + + network. And the training parameters of all networks were consistent.

Figure 13 demonstrated the segmentation performance of the improved Unet network and Unet (Base) network on the CT image dataset.

Fig. 13

Comparison of segmentation metrics of improved Unet network and Unet (base) network

Table 2 demonstrated the segmentation performance of each network on a dataset of lung CT images of pneumoconiosis patients. The median values of Dice, PPV, SC and MIoU for the improved Unet network in this study were 0.848, 0.884, 0.895 and 0.885, respectively, increasing by 7.6%, 13.3%, 3.9% and 6.4%, respectively, compared with those of Unet network (Base), and increasing by 187.5%, 249.4%, 131.9% and 51.0%, respectively, compared with those of FCN-8s, and increasing by 14.0%, 31.2%, 4.7% and 9.7%, respectively, compared with those of Unet network (SE + ReLU), while the segmentation performance was also not inferior to that of the Unet + + network.

It could be seen that the performance of the improved Unet network was significantly better than that of the Unet network (Base) and FCN-8s. As can be seen from the P-values in Table 1, all of the above performance enhancements were statistically significant except for the Unet + + network, suggesting that the improved Unet network was effective in segmenting the particulate region of the lung.

Table 2 Data comparison in the experimental network

Table 3 demonstrated the inference speed of different experimental networks.

Table 3 Inference speed in the experimental network

Discussion

When working with lung CT images of the patients with pneumoconiosis, accurate identification and segmentation of the abnormal areas is crucial for making therapeutic decisions. It helps to prevent the early spread of pulmonary particles, improve patient prognosis, and reduce mortality and disability.

The hospital where the patients were diagnosed and treated is the only Tertiary Specialized Hospital for occupational diseases among the 44 hospitals under the auspices of the National Health Commission of China, and it diagnoses and treats patients with pneumoconiosis from all over the southwest region of China. The academic level of pneumoconiosis diagnosis is in the leading position in China. The epidemiologic characteristics (sex, age) of the patients in this study were similar to those of pneumoconiotics from other regions [29,30,31].

We could find that data augmentation significantly enhanced the segmentation performance of the model, especially in our model. We hypothesize that this may be due to the fact that removing redundant information and noise from the edges of the image and retaining the main feature regions of the image improves the performance of the model. By cropping different central regions, multiple different samples can be generated, thus expanding the dataset and improving the generalization ability of the model.

Each of the several networks mentioned in this paper has its own characteristics in processing medical images, and they differ in capturing specific structures and regions: the FCN-8s is a fully convolutional network mainly used for semantic segmentation tasks, and it has certain advantages for segmenting organs and structures in large regions. In medical image segmentation, Unet can effectively capture fine structures and edges in medical images, and has better results for segmenting cells, organs, and lesion regions, etc. Unet + + introduces a deeper network structure and a more complex feature fusion mechanism, which is advantageous for processing complex medical images. However, these methods have a common problem that manual labeling of data has a great influence on the segmentation results and is time-consuming.

To address this issue, this study applied the improved Unet network to the recognition and segmentation of granular regions in lung CT images. By comparing with the other networks, the improved Unet network showed a better segmentation performance. Compared to the FCN-8s network, the improved Unet network had better performance in capturing detail and edge parts, and segmented the target object more accurately. Compared to the Unet(Base) network, the improved Unet network had higher segmentation accuracy and better segmentation results for more accurate localization and segmentation of the target region. Compared with the Unet (SE + ReLU) network, the segmentation results of the improved Unet network were better, showing the superiority of the GeLU activation function. Compared with the Unet + + network, the improved Unet network performed equally in terms of stability and clarity of segmentation results. Therefore, the experimental results showed that the improved Unet network had great advantages in image segmentation tasks and achieved good segmentation results in different scenarios, showing potential application value for clinical decision aid.

This research method improved the doubleconv structure of the Unet network and introduces the residual mechanism, which can effectively avoid the problem of degradation of model performance due to training overfitting caused by the disparity of lung region sizes and the disappearance of lung region detail information after multi-layer convolution. The introduction of the SE attention mechanism can make the neural network pay more attention to the useful feature information, so as to improve the model’s accuracy, ppv and sc value and other indicators, to better distinguish the effective features and reduce their impact on the model.

The method in this study improved the activation function of the Unet network, and the GeLU activation function enhances the nonlinear representation and generalization ability of the network, allowing the model to maintain accuracy while being lightweight, and reduces the computational bottleneck and memory cost for model training. The adoption can better accommodate the unbalanced scenario of background and lung particle regions. The method adopted mixed-accuracy training to speed up computation and reduce memory consumption, thus making the training of large-scale models more efficient. Based on this, the use of appropriate learning rate updating strategy can better control the parameters such as gradient size and direction in the optimization process.The joint use of mixed-accuracy training and learning rate updating strategies can better optimize the neural network, thus improving the convergence speed and stability of the whole network.

The proposed method achieved a processing speed of 69.70 FPS, indicating its potential for real-time applications. A higher FPS value signified faster model execution, which was crucial for applications that require quick response times. At the same time, it should be pointed out that the improved Unet network proposed in this study further improves the segmentation performance of the Unet model on the suspected particulate region of the patient’s lungs, but the segmentation accuracy still has a certain amount of room for improvement. The lung parenchyma extraction operation was performed in the experimental data processing of this study, and the generalization ability of the model might be further improved if CT image data without extracted lung parenchyma were used to train the network. Currently, the model has been applied to pneumoconiosis and pulmonary nodule. However, to extend this study to cancer, it is essential first to explore the datasets used for such applications. The primary difference between tumors and pneumoconiosis nodules lies in their morphology: tumors tend to be larger in size, whereas pneumoconiosis nodules are smaller, typically measuring only 0.3–0.5 cm. Tumors are generally more isolated, often appearing as single entities, whereas pneumoconiosis nodules are more numerous and diffusely distributed. Given that the model has successfully identified the smaller and more numerous pneumoconiosis nodules, it is reasonable to expect that it could also identify the larger and less numerous tumors. Thus, this model has potential applications for lung cancer, brain tumors, and other cancers. These are elements that need to be studied and addressed in future research.