KR20190090986A - System and method for assisting chest medical images reading - Google Patents

Abstract

The present invention relates to a method and a system for assisting chest medical image reading. The method for assisting chest medical image reading comprises the following steps: obtaining a chest medical image of a patient by a medical image obtaining unit; adjusting the brightness of the chest medical image obtained by a histogram matching unit; segmenting only a lung part in the chest medical image of which the brightness is adjusted from a segmentation unit; standardizing the segmented lung part by a space standardization unit; and reading the standardized medical image in the space standardization unit from a reading unit.

Description

Chest medical image reading support system and method {SYSTEM AND METHOD FOR ASSISTING CHEST MEDICAL IMAGES READING}

The present disclosure relates to a medical image reading support system as a whole, and more particularly, to a chest medical image reading support system and method using artificial intelligence.

This section provides background information related to the present disclosure which is not necessarily prior art. In addition, in the present specification, direction indications such as up / down, up / down, etc. are based on the drawings.

In the medical field, a doctor displays a medical image of a patient on a monitor, reads out a medical image displayed on the monitor, and observes the state of the lesion and changes over time. In image diagnosis using a medical image, a doctor diagnoses anomalous shadows or the like from a medical image to be diagnosed. The reading made by the doctor at the time of image diagnosis includes the judgment result and the reading opinion on which the judgment is based on what is found in the abnormal shadow. At this time, the reading opinion and the estimated disease name recorded by the doctor are not necessarily correct. For example, if a doctor cannot spend a lot of time reading a single medical image, or if a doctor with low reading experience observes abnormal shadows or difficult shadows that are difficult to find in a medical image, the abnormal shadows There was a possibility to omit or misjudge the judgment of presumptive disease. In order to solve the problems that may occur in the reading of the medical image, a system for automatically detecting a diseased part when analyzing a medical image by digitizing the medical image and supporting a medical image reading by a computer has been developed. The computer aided system is CAD (Computer Aided Diagnosis). CAD automatically detects abnormal shadow candidates as disease sites. In the detection process of abnormal shadows, medical image data is subjected to computer processing.

1 is a view showing an example of a medical image reading support system described in Japanese Patent No. 599514. For convenience of description, some symbols and terms have been changed.

The medical image reading support system 10 includes a medical image obtaining unit 11, an abnormal shadow detection unit 12, a CAD finding generation unit 13, and the like. In particular, a support vector machine, a neural network, a Bayesian network, or the like may be used as the inference device for estimating the disease name in the CAD finding generation unit 13.

2 is a diagram illustrating an example of a medical image reading support system described in Korean Laid-Open Patent Publication No. 2017-0046105. For convenience of description, some symbols and terms have been changed.

The medical image reading support system uses a deep learning algorithm to determine whether the tuberculosis is infected by reading a lung image of the patient.

Although the conventional medical image reading support system disclosed in FIGS. 1 and 2 improves the accuracy of the medical image reading support system by utilizing artificial intelligence (AI), the quality of the medical image determined by the artificial intelligence is low. Since artificial intelligence cannot properly determine medical images, it is difficult to improve the accuracy of the medical image reading support system.

The present disclosure is to provide a medical image reading support system and method for improving the accuracy of the medical image reading support system using artificial intelligence by improving the quality of medical images judged by artificial intelligence.

This will be described later in the section titled 'Details of the Invention.'

SUMMARY OF THE INVENTION Herein, a general summary of the present disclosure is provided, which should not be construed as limiting the scope of the present disclosure. of its features).

According to one aspect of the present disclosure (According to one aspect of the present disclosure), a method of supporting chest medical image reading, the method comprising: obtaining, by a medical image acquisition unit, a chest medical image of a patient; Adjusting brightness of a chest medical image obtained by a histogram matching unit; Segmenting the lungs in the chest medical image in which the segmentation unit is adjusted in brightness; Standardizing the segmented lung part by the spatial standardization unit; And a reading unit reading the normalized medical image by the spatial standardizing unit.

According to another aspect of the present disclosure (According to another aspect of the present disclosure), a system for supporting a chest medical image reading, comprising: a medical image acquisition unit for acquiring a chest medical image of a patient; A histogram matching unit matching the brightness of the chest medical image of the patient acquired by the medical image obtaining unit; Segmentation unit for segmenting the lung portion that needs to be read in the chest medical image of the patient with matching brightness; A spatial standardization unit for standardizing segmented lung parts; In addition, a readout unit for reading a medical image of a standardized patient is provided.

This will be described later in the section titled 'Details of the Invention.'

1 is a view showing an example of a medical image reading support system described in Japanese Patent No. 599514,
2 is a view showing an example of a medical image reading support system described in Korean Patent Laid-Open No. 2017-0046105;
3 is a view showing an example of a medical image reading support system according to the present disclosure;
4 is a diagram illustrating a histogram matching unit according to the present disclosure;
5 is a diagram illustrating a segmentation unit according to the present disclosure;
6 is a view for explaining a spatial standardization unit according to the present disclosure;
7 is a flowchart illustrating an example of a method using a medical image reading support system according to the present disclosure;
8 is a flowchart illustrating still another example of a method using a medical image reading support system according to the present disclosure;
9 is a view showing an example of a read statement made by a reading unit.

The present disclosure will now be described in detail with reference to the accompanying drawing (s).

3 is a diagram illustrating an example of a medical image reading support system according to the present disclosure.

The medical image reading support system 100 includes a medical image obtaining unit 110, a histogram matching unit 120, a segmentation unit 130, a spatial standardization unit 140, and a reading unit 150. The medical image acquisition unit 110 acquires a medical image of a patient that needs reading. The medical image acquisition unit 110 may directly obtain a medical image from a digital medical imaging apparatus such as an X-ray apparatus, a computed tomography (CT), or magnetic resonance imaging (MRI). Or by a medical image transmission system (PACS) from a server storing digital medical images taken by a digital medical imaging apparatus, or connected to an external storage device (eg FDD, HDD, CD drive, etc.) To obtain a medical image of the patient, but is not limited thereto. The histogram matching unit 120 matches intensity of the medical image of the patient obtained by the medical image acquisition unit 110. That is, the artificial intelligence can better read the medical image of the patient by matching the brightness of the patient's medical image with the standard brightness as a reference, which will be described in detail with reference to FIG. 4. The segmentation unit 130 segments the portion of the medical image whose brightness is standardized by the histogram matching unit 120 for reading. The method is described in detail with reference to FIG. 5. Spatial Nomalizaton 140 normalizes the size of the medical image segmented by the segmentation unit 130 to a predetermined size. A detailed method will be described with reference to FIG. 6. The reading unit 150 reads the medical image normalized in size by the spatial standardizing unit 140 by AI, and then prepares a reading and provides it to the doctor. In particular, by pre-processing the medical image by the histogram matching unit 120 and the spatial standardization unit 140, when the AI reads the medical image, the reading efficiency may be improved to increase the reading accuracy. The AI used for the reading unit 150 is preferably a deep neural network. Artificial neural deep neural networks also typically require learning to read medical images of patients. The data used to train deep neural networks is called training data. In the present disclosure, since the medical image of the patient in need of reading is subjected to a preprocessing process characterized by the histogram matching unit 120 and the spatial standardization unit 140, the deep neural used in the medical image reading support system 100 of the present disclosure. It is preferable that a plurality of medical images, which are learning data of a network, also undergo the same preprocessing process. For example, a plurality of medical images used as learning data are obtained by the medical image acquisition unit 110 and then the medical images are passed through the histogram matching unit 120, the segmentation unit 130, and the spatial standardization unit 140. After the preprocessing, the deep neural network of the reading unit 150 is used as training data. Furthermore, the medical image of the patient used for reading may also be used as learning data of the deep neural network used in the medical image reading support system 100 of the present disclosure, so that the reading ability of the deep neural network may be continuously improved. The artificial intelligence learning method used in the reading unit 150 will be described again with reference to FIG. 8. The medical image reading support system 100 is also substantially the same as the conventional CAD based system described in FIGS. 1 and 2 except as described in this disclosure. For example, the medical image reading support system 100 may be a computer system, a micro processing unit (MPU), a central processing unit (CPU), a graphic processing unit (GPU) or a tensor processing unit (TPU), a cache memory, It may include hardware such as a data bus and a software configuration for performing a specific purpose. Although the present disclosure has been described without limitation on the type of medical image, the medical image assisted reading support system described in the present disclosure has been found to be particularly effective when reading chest medical images.

4 is a diagram illustrating a histogram matching unit according to the present disclosure.

The histogram matching unit 120 modifies the medical image brightness of the patient obtained by the medical image acquisition unit 110 to improve readability by artificial intelligence (AI). To this end, the histogram matching unit 120 first selects a medical image having a uniform distribution of brightness as a standard medical image for histogram matching. The data for selecting a standard medical image may use training data for training a deep neural network. The method of selecting a standard medical image for histogram matching is a method of dicoming a plurality of chest X-rays for use as training data of a deep neural network, for example, when a medical image of a patient to be read is a chest X-ray image. The image header is used to analyze the brightness distribution of each chest X-ray image. The plurality of chest X-ray medical images is referred to as a first reference medical image. The first reference medical image may be a medical image of a patient in need of reading, or may be a medical image of another patient. Afterwards, chest X-ray images having a brightness range of 10 to 1,000 are selected from the first reference medical image and are referred to as second reference medical images. For example, it is preferable that the second reference medical image has a brightness distribution of 10 to 1,000 sections of 80% or more. A brightness average is calculated for each of the second reference medical images. Thereafter, the brightness averages of the second reference medical images are summed, and then the average of the brightness averages of the second reference medical images are calculated. Thereafter, among the second reference medical images, the medical image having the brightness average closest to the average of the second reference medical image brightness averages is selected as the standard medical image for histogram matching. In the brightness histogram 200 of the standard medical image for matching the selected histogram, it can be seen that the bright and dark areas are evenly spread as shown in FIG. The X axis of the histogram 200 represents the brightness, and the Y axis represents the number of pixels having the corresponding brightness. After selecting the standard medical image for histogram matching, the brightness histogram of the medical image of the patient to be read is obtained. 4 (b) shows an example of a brightness histogram 220 of a chest X-ray medical image 210 of a patient who needs reading and a chest X-ray medical image 210 of a patient who needs reading. The brightness histogram 220 of the chest X-ray medical image 210 of the patient in need of reading indicates that the light and dark areas are uneven. As such, the brightness histogram 220 of the chest X-ray medical image 210 of the patient requiring uneven light and dark areas is matched with the brightness histogram 200 of the standard medical image for histogram matching. The brightness histogram 220 of the chest X-ray medical image 210 is modified to be similar to the brightness histogram 200 of the standard medical image so that the light and dark areas are spread evenly. According to the modified histogram, the brightness of the chest X-ray medical image 210 of the patient that needs to be read may be modified to improve the readability of artificial intelligence (AI). A chest histogram 220 of a patient's chest X-ray medical image 210 that needs to be read is matched with a brightness histogram 200 of a standard medical image for histogram matching, and thus a chest X-ray medical image of a patient who needs to read a A method of modifying the brightness histogram 220 for 210 in a form similar to the brightness histogram 200 of a standard medical image may use a known histogram matching method. For example, a brightness histogram for a chest X-ray medical image 210 of a patient who needs to calculate a brightness histogram 200 distribution of a standard medical image and read based on the calculated brightness histogram 200 distribution of a standard medical image. The brightness histogram 220 of the chest X-ray medical image 210 of the patient in need of reading may be modified to calculate the distribution of the data in a form similar to the brightness histogram 200 of the standard medical image.

5 is a diagram illustrating a segmentation unit according to the present disclosure.

Segmentation unit 130 divides the portion that needs to be read. For example, only the lungs are segmented from the chest X-ray image, especially when the chest X-ray image is a medical image of a patient who needs to read. In the present disclosure, for example, a lung segmentation image 320 obtained by dividing only the lung portion 310 from a chest X-ray image 300, which is a medical image of a patient, using U-net AI learned using artificial intelligence. )

6 is a diagram illustrating a spatial standardization unit according to the present disclosure.

The spatial standardizer 140 expands the image obtained by the segmentation unit 130. For example, in FIG. 6 (a), which is a chest X-ray medical image before segmentation, the actual lung region often includes not only the dark portion 400 but also the bright portion 410 below. However, in FIG. 6B, which is a medical image divided by the segmentation unit 130, it can be seen that only the dark portion 400 is divided into lung regions. Spatial normalization unit 140 is, for example, 1.5 times from the up, down, left, and right coordinates calculated by segmenting the lungs by the segmentation unit 130 and up, down, left, and right coordinates of the lung image divided by the segmentation unit 130 using the Contour function. The image is expanded to obtain a medical image as shown in FIG. Compared to FIGS. 6 (c) and 6 (a), unnecessary portions of the top, bottom, left, and right sides have been deleted, and compared to FIG. 6 (b), the lung area is expanded, so that the actual lung area is larger than the medical image obtained by the segmentation unit 130. It will include a lot. Preferably, the spatial standardization unit 140 then makes the size of the image obtained by expanding the image obtained by the segmentation unit 130 to a constant size, for example, in a square shape. Optimized size for the deep neural network, which is the artificial intelligence of the reading unit 150, and the width and height to be the same size. It is desirable that the constant size be as close as possible to the actual X-ray image size (Fig. 6 (a)). Converting to a smaller size than the actual X-ray image can result in some missing information and less accurate lesions.

7 is a flowchart illustrating an example of a method using a medical image reading support system according to the present disclosure.

In the method of using the chest medical image reading support system according to the present disclosure, first, a medical image of a patient is obtained by a medical image obtaining unit (S1). The acquired medical image is adjusted for contrast by the histogram matching unit (S2). Subsequently, the segment to be read is divided from the medical image processed by the contrast ratio by segmentation (S3). For example, as shown in FIG. 5, a lung image may be segmented from a chest X-ray medical image. The application of the present disclosure has been shown to be particularly effective when applied, particularly when reading lung disease from chest medical images. Thereafter, the area of the medical image divided by the spatial standardization unit is expanded, and then the size of the medical image is adjusted (S4). Thereafter, the reading unit reads the medical image preprocessed by the histogram matching unit and the spatial standardization unit and creates a reading (S5). In particular, the reading unit reads the medical image by the deep neural network which is artificial intelligence. The doctor then reads the reading made by the reading unit to determine whether the disease is determined from the medical image of the patient.

8 is a flow chart showing another example of a method using the medical image reading support system according to the present disclosure, which has been described in terms of learning.

First, a plurality of medical images are prepared (S10). The plurality of medical images may be two-dimensional chest X-ray images or three-dimensional medical images such as CT images. Medical imaging is not limited to humans, but can of course be applied to animals such as dogs and pigs. In the case of 3D medical imaging, a 3D segmentation technique is required, and the segmented lung needs to be expanded and sized in the 3D direction.

Next, histogram matching is performed on the plurality of medical images (S20). Preferably, histogram matching is performed on the plurality of medical images. More preferably, the histogram matching technique of the method shown in Fig. 4 is applied. By applying this technique (selecting a standard medical image), even if another medical image is added later, the image is based on the already selected standard medical image. It will have the advantage of handling.

Next, segmentation is performed on each medical image (S30). For this segmentation, known deep neural networks such as Fully Convolutional Network (FCN) and U-Net can be used. The network may be learned using the already learned or using the plurality of medical images as learning data. Preferably, the histogram matching shown in FIG. 4 and the medical image to which the technique shown in FIGS. 5 and 6 are applied may be supervised and trained, and transfer learning may be used.

Next, the image is cut out from the original medical image in consideration of the bright part 410 shown in FIG. 6 (S40). Each of these cropped images is not constant in size (because the size of the lungs varies from patient to patient), so it is advisable to resize to a certain size. It is possible to adjust the size by averaging the sizes of the plurality of cropped images, but it is desirable to adjust the image by setting one standard since the image may be continuously added. The segmentation network itself may be learned in the form of directly dividing the bright portion 410 shown in FIG. 6, but is not desirable in terms of accuracy.

Next, the reading unit is trained based on the plurality of pre-processed medical images (S45). The reader can function as Classification (existence of malicious nodule), Detection (determination of position and size of nodule), Segmentation (determination of specific position, size, shape of nodule), and depending on the function, AlexNet, GoolgleNet, ResNet, FCN, U-Net, GAN may be formed of one Deep Neural Network of the same type or a combination thereof. The rear part of the reading unit is provided with a part for presenting the result of the Deep Neural Network (s) in the form of a reading as shown in FIG. 9, which is already known in the technique shown in FIG. 1. That is, the reading unit or the read reading of the form shown in FIG. 9 (b) is created by the reading unit for the lung disease part 500 shown in FIG. 9 (a).

The read support system thus learned is used for the preparation of the read or temporary read document (S50). If a medical image of a particular patient is provided, preferably through all of the steps (S20, S30, S40) and then provided to the reading unit, the reading unit or the read edition shown in FIG. 9 is prepared by the reading unit. In summary, as a preferred embodiment, the present disclosure provides (1) learning of a preprocessing network (2) learning of a reader network using a plurality of medical images that have passed through the learned preprocessing network (3) medical images of specific patients The medical image of the specific patient who has undergone the preprocessing (4) through the learned preprocessing network includes a readout or a readout reading through the learned readout.

Various embodiments of the present disclosure will be described below.

(1) a method for supporting chest medical image reading, the medical image obtaining unit acquiring a chest medical image of a patient; Adjusting brightness of a chest medical image obtained by a histogram matching unit; Segmenting the lungs in the chest medical image in which the segmentation unit is adjusted in brightness; Standardizing the segmented lung part by the spatial standardization unit; And a reading unit reading the normalized medical image by the spatial standardizing unit.

(2) adjusting the brightness of the chest medical image obtained by the histogram matching unit

Selecting a chest medical image having a uniform brightness distribution as a standard medical image for histogram matching; Obtaining a brightness histogram of the acquired chest medical images of the patient; Matching the brightness histogram of the acquired chest medical image with the brightness histogram of the standard medical image; And correcting the brightness histogram of the acquired chest medical image of the patient based on the brightness histogram of the standard medical image.

(3) selecting a medical image having a uniform distribution of brightness as a standard medical image for histogram matching may include obtaining brightness distributions for a plurality of first reference medical images that are a plurality of chest medical images; Selecting, as the plurality of second reference medical images, a brightness distribution of 10 to 1,000 sections of the plurality of first reference medical images being 80% or more of the total; Calculating respective brightness averages for the plurality of second reference medical images; Calculating an average of brightness averages of each of the plurality of second reference medical images after summing brightness averages of each of the plurality of second reference medical images; And selecting a standard medical image for histogram matching having the brightness average closest to the average of the brightness averages of each of the second reference medical images, from among the plurality of second reference medical images. Way

(4) the step of standardizing the segmented lung part by the spatial normalization unit may include obtaining an expanded medical image including the segmented lung part; And increasing or decreasing the size of the expanded medical image, including the segmented lung portion, to a predetermined size.

(5) Obtaining an extended medical image including the segmented lung part is performed by performing chest medical examination to obtain an extended medical image including the segmented lung part using the up-down, left-right coordinates and the Contour function calculated when segmenting the lung part. How to support image reading

(6) how the readout supports chest medical image readings operated by artificial intelligence (AI)

(7) How artificial intelligence (AI) supports chest medical image readings powered by deep neural networks

(8) learning data for training the deep neural network of the reading unit includes: acquiring, by the medical image acquisition unit, a plurality of chest medical images; Adjusting brightness of a plurality of chest medical images obtained by a histogram matching unit; Segmenting only the lungs in the plurality of chest medical images of which the segmentation unit is adjusted in brightness; And standardizing the segmented lung part by the spatial standardization unit.

(9) a system for supporting a chest medical image reading, the system comprising: a medical image obtaining unit obtaining a chest medical image of a patient; A histogram matching unit matching the brightness of the chest medical image of the patient acquired by the medical image obtaining unit; Segmentation unit for segmenting the lung portion that needs to be read in the chest medical image of the patient with matching brightness; A spatial standardization unit for standardizing segmented lung parts; And a readout unit configured to read standardized medical images of the patient.

(10) One of the segmentation unit and the reading unit is a chest medical image reading support system operated by artificial intelligence (AI).

According to the present disclosure, it is possible to obtain a chest medical image reading support system that improves the reading accuracy of an artificial deep neural network.

Medical image reading support system: 10. 100

Claims (10)

A method of supporting chest medical image reading,
Obtaining, by the medical image acquisition unit, a chest medical image of the patient;
Adjusting brightness of a chest medical image obtained by a histogram matching unit;
Segmenting the lungs in the chest medical image in which the segmentation unit is adjusted in brightness;
Standardizing the segmented lung part by the spatial standardization unit; And
And a reading unit reading the normalized medical image by the spatial standardizing unit. The method according to claim 1,
Adjusting the brightness of the chest medical image obtained by the histogram matching unit
Selecting a chest medical image having a uniform brightness distribution as a standard medical image for histogram matching;
Obtaining a brightness histogram of the acquired chest medical images of the patient;
Matching the brightness histogram of the acquired chest medical image with the brightness histogram of the standard medical image; And
And correcting the brightness histogram of the acquired chest medical image based on the brightness histogram of the standard medical image. The method according to claim 2,
Selecting a chest medical image with a uniform brightness distribution as a standard medical image for histogram matching
Obtaining brightness distributions for a plurality of first reference medical images that are a plurality of chest medical images;
Selecting, as the plurality of second reference medical images, a brightness distribution of 10 to 1,000 sections of the plurality of first reference medical images being 80% or more of the total;
Calculating respective brightness averages for the plurality of second reference medical images;
Calculating an average of brightness averages of each of the plurality of second reference medical images after summing brightness averages of each of the plurality of second reference medical images; And
Selecting a standard medical image for histogram matching that has the brightness average closest to the average of the brightness averages of each of the second reference medical images among the plurality of second reference medical images; . The method according to claim 1,
The spatial standardization standardizes the segmented lung part
Obtaining an expanded medical image including a segmented lung portion; And
Increasing or decreasing the size of the expanded medical image, including the segmented lung portion, to a predetermined size. The method according to claim 4,
Obtaining extended medical images, including segmented lungs,
A method of supporting chest medical image reading that obtains an expanded medical image, including segmented lung portions, using calculated top, bottom, left and right coordinates and the Contour function when segmenting the lung. The method according to claim 1,
The reading unit supports a chest medical image reading operated by artificial intelligence (AI). The method of claim 6,
Artificial intelligence (AI) in the reader to support chest medical image readings operated by deep neural networks. The method according to claim 7,
The training data for training the deep neural network of the reader
Obtaining, by the medical image obtaining unit, a plurality of chest medical images;
Adjusting brightness of a plurality of chest medical images obtained by a histogram matching unit;
Segmenting only the lungs in the plurality of chest medical images of which the segmentation unit is adjusted in brightness; And
And standardizing the segmented lung part by the spatial normalization unit. 3. A system for supporting chest medical image reading,
A medical image acquisition unit obtaining a chest medical image of the patient;
A histogram matching unit matching the brightness of the chest medical image of the patient acquired by the medical image obtaining unit;
Segmentation unit for segmenting the lung portion that needs to be read in the chest medical image of the patient with matching brightness;
A spatial standardization unit for standardizing segmented lung parts; And,
And a readout for reading medical images of the standardized patient. The method according to claim 9,
One of the segmentation unit and the reading unit is a chest medical image reading support system operated by artificial intelligence (AI).

Images