CN112084901A - Automatic detection method and system of airport runway area in high resolution SAR image based on GCAM - Google Patents

Abstract

The invention discloses a GCAM-based high-resolution SAR image airport runway area automatic detection method and a system thereof, wherein the GCAM-based high-resolution SAR image airport runway area automatic detection method comprises the steps of carrying out downsampling on a high-resolution SAR image to generate a medium-resolution image; inputting the medium-resolution image into a geographic space context attention mechanism network GCAM to extract a runway area; and carrying out coordinate mapping on the extracted runway area to obtain the detection result of the runway area of the final high-resolution SAR image. Experiments show that compared with DeeplaLV 3+, RefineNet and MDDA networks, the method is high in precision and short in time consumption, can fully learn the geospatial information of the SAR image airport, and can realize high-precision, rapid and automatic extraction of the runway area of the high-resolution SAR image airport.

Description

Automatic detection method of airport runway area in high-resolution SAR image based on GCAM system

technical field

The invention relates to an airport runway area automatic detection technology, in particular to a GCAM-based high-resolution SAR image airport runway area automatic detection method and system.

Background technique

Airports are important transportation hubs and military facilities, and the detection of airport targets from Synthetic Aperture Radar (SAR) images has become an important application. SAR has the advantages of all-day and all-weather imaging, penetrating clouds and fog, etc., but SAR images are less readable and more complicated to interpret than optical images, so most airport detection is usually based on optical remote sensing images. With the increasing resolution of SAR images and more and more data, the research on using SAR images to extract airports has gradually increased in recent years, and related research has also begun to deepen. Traditional airport extraction methods are time-consuming and labor-intensive, and most of them are only effective for airport extraction from optical images, but they are less effective for airport extraction from SAR images. Therefore, it has far-reaching and urgent practical significance to realize the automatic and rapid extraction of airport runway areas on high-resolution SAR images. In addition, using the airport runway area to mask the aircraft detection can also greatly reduce the false alarms in the aircraft detection and improve the aircraft detection accuracy.

Airport detection has a wide range of applications in terminal navigation, accident search and rescue, and aircraft positioning. The runway area is one of the main components of the airport. In the research on airport detection, most of the researches on the detection of airports are based on optical remote sensing images. At present, airport detection has been carried out by the traditional method of extracting airport edge line segments in the existing technology, but the method of extracting line segments requires the airport to have obvious linear characteristics, which is not suitable for large-scale civil airports with many terminals and weak runway linear characteristics. Not applicable; some schemes use sparse reconstruction saliency model (SRS) and target-aware active contour model (TAACM) to complete airport detection, which enhances the detail extraction of airports; and some schemes combine vision The saliency analysis model, the bidirectional complementary saliency analysis module and the saliency active contour model (SOACM) are used for airport contour extraction. This method is applicable to most optical remote sensing images; SAR images have strong penetration ability and can work without interference. , and can obtain rich ground object information, this advantage makes SAR images gradually become the experimental object of airport detection. There are also some schemes that combine traditional line segment grouping methods and saliency analysis models to detect airports on small SAR images, but this method is not suitable for detecting airports on large SAR images; some schemes propose a combination of The PolSAR airport runway detection algorithm based on polarization features and random forest is optimized, but this method can only effectively extract parallel runway features in the airport.

In recent years, deep learning has achieved very good results in the direction of semantic segmentation. Semantic segmentation is a deep learning method that performs feature learning based on image pixels to achieve different categories of images. Airport detection needs to extract all airport features. This principle is consistent with the idea of semantic segmentation, so the method of combining deep learning with airport detection begins to appear. For example: a prior art proposes an airport detection method combining deep learning YOLO model and saliency analysis model; a prior art combines deep learning Goole-LF network and support vector machine SVM method to detect airports; a prior art combines Deep learning Faster-CNN network and spatial analysis method for airport extraction; an existing technology builds an end-to-end deep transferable convolutional deep learning network to detect airports; but the above methods all apply deep learning to optical remote sensing images For example, and because of the lack of airport sample data, deep learning models often overfit during training. For high-resolution SAR image airport extraction, an existing technology proposes a deep learning network MDDA (Mult-level and densely dual attention) for high-resolution SAR image runway area, which can achieve high-precision airport extraction, but requires more Large datasets and long training times. Therefore, it is of practical significance to seek a deep learning method suitable for small sample datasets and can efficiently extract airports.

The deep learning network is developing very rapidly, and the DeepLab series in deep learning has excellent performance in the field of semantic segmentation. In 2014, DeepLabv1 was proposed, and Atrous Conv was cited for the first time, which solved the problems of traditional CNN algorithm in the presence of signals in pixel labels and spatial invariance, and improved model capture through Conditional Random Field (CRF). The ability of fine details, DeepLabv1 won the second place in the PASCAL semantic segmentation challenge; DeepLabv2 was proposed in 2016, and DeepLabv2 further proposed the ASPP (Astrous spatialpyramid pooling) module on the basis of DeepLabv1, so as to capture contextual semantic information from a multi-scale direction, The backbone network VGG-16 is replaced by ResNet, which overcomes the problem of feature resolution reduction caused by pooling in traditional CNN; DeepLabv3 appeared in 2017, and DeepLabv3 improved ASPP on the basis of DeepLabv2, making the network performance better; DeepLabv3+ in 2018 Further improvement on the basis of DeepLabv3, DeepLabv3+ introduces an encoding-decoding structure, uses DeepLabv3 as the encoding part, designs a simple and effective decoding block, and adds a depthwise separable convolution to the backbone network. DeepLabv3+ makes the model maintain Under the premise of performance, the amount of calculation and parameters are effectively reduced.

Therefore, in view of the problems existing in SAR image airport extraction, how to achieve high-precision, fast and automatic extraction of high-resolution SAR image airport runway areas based on deep learning has become a key technical problem to be solved urgently.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention: aiming at the above-mentioned problems of the prior art, a GCAM-based high-resolution SAR image airport runway area automatic detection method and system are provided. The present invention can fully learn the geospatial information of the SAR image airport, and can Realize high-precision, fast and automatic extraction of airport runway area from high-resolution SAR images.

In order to solve the above-mentioned technical problems, the technical scheme adopted in the present invention is:

A GCAM-based high-resolution SAR image airport runway area automatic detection method, comprising:

1) Downsampling the high-resolution SAR image to generate a medium-resolution image;

2) Input the medium resolution image to the geospatial context attention mechanism network GCAM to extract the runway area;

3) Perform coordinate mapping on the extracted runway area to obtain the detection result of the final high-resolution SAR image.

Optionally, the down-sampling of the high-resolution SAR image in step 1) specifically refers to performing a 5-fold down-sampling process on the SAR image by using a pixel value extraction method.

Optionally, the geospatial context attention mechanism network GCAM includes an encoding block and a decoding block, and the encoding block includes a residual network ResNet, a multi-scale squeeze pyramid MSP, and an edge refinement module EDM, and the residual network ResNet is used for Perform feature extraction on the input data set to obtain preliminary features. The multi-scale extrusion pyramid MSP is used to obtain global context information from different pooling convolution layer operations at different resolutions for the preliminary features. The edge refinement module EDM is used for To strengthen the network edge extraction ability for the preliminary features, the output of the multi-scale extrusion pyramid MSP and the edge refinement module EDM are further fused to obtain multi-level features; the decoding block is used to combine the preliminary features and multi-level features to perform semantic segmentation of the airport runway area. Extract the runway area.

Optionally, the residual network ResNet is an improved residual network obtained by replacing ordinary two-dimensional convolution with atrous convolutions with atrous rates of 2, 4, 8, and 16 on the basis of the residual network ResNet_101.

Optionally, the multi-scale squeeze pyramid MSP includes a multi-receptive field parallel pooling working layer and an effective attention module eSE, and the multi-receptive field parallel pooling working layer is composed of a 1×1 convolution with a dilation ratio of 1. , three 3×3 convolutions with dilation rates of 6, 12, and 18, a global average pooling module GAP and a stripe pooling module SP are built in parallel; the stripe pooling module SP is designed for an input size of H ×W two-dimensional feature tensor, using the band-shaped pooling window H×1 to perform the pooling operation in the horizontal direction, and the band-shaped pooling window 1×W to perform the pooling operation in the vertical direction, respectively. The element values are averaged to obtain the output of stripe pooling in the horizontal direction and the output of stripe pooling in the vertical direction, and then two one-dimensional convolutions are used for the output of stripe pooling in the horizontal direction and the output of stripe pooling in the vertical direction respectively. The output is expanded in the left and right and up and down directions, and the size of the two feature maps after the expansion is the same, and then the two feature maps after the expansion are fused, and finally the original data and the fused data after Sigmoid processing are multiplied as The final obtained H×W two-dimensional feature tensor output; the effective attention module eSE first learns the feature F _avg through global average pooling for the input feature map Xi, and processes the feature F _avg through the fully connected layer to obtain the weight Matrix W _C , the weight matrix W _C is re-adjusted by the Sigmoid function to obtain the channel attention feature A _eSE , and then the channel attention feature A _eSE is applied to the input feature map Xi to obtain the refined feature map X _refine , and finally the The refined feature map X _refine performs feature re-screening to obtain global context information.

Optionally, the edge refinement module EDM includes a global convolution module GCB that enhances the close connection between the feature map and the pixel classification layer and the ability to process feature maps of different resolutions to obtain global information. The edge refinement module BR for the edge extraction capability of the coding block; the global convolution module GCB includes a k×k large convolution kernel and a feature combination module, and the k×k large convolution kernel includes two routes, one route The convolution of k×1×c×c is composed of the convolution of 1×k×c×c, and the other route is composed of the convolution of 1×k×c×c and the convolution of k×1×c×c, where c is the number of channels, and the output results of the two channels are input into the feature combination module together to obtain the feature Sum _{W×H × C} ; the edge refinement module BR passes through the small convolution kernel, the activation function, the small convolution kernel, the activation function, the small The convolution kernel is processed, and then the processing results are superimposed on the original feature Sum _W×H×C , and finally the feature map after refining the edge of the runway area is obtained.

Optionally, the decoding block performs edge information decoding on the feature map obtained by reducing the dimension of the output feature of the coding block through 1×1 convolution and using the edge refinement module EDM to refine the edge of the runway area, and performing bilinear decoding. 4 times upsampling, then connect the preliminary features output by the residual network ResNet after 1×1 convolutional dimension reduction and the results obtained by bilinear 4 times upsampling, and then apply the connected features to a 3×3 volume The product is used to refine the features, and finally a simple bilinear 4x upsampling is performed to obtain the final segmentation result.

In addition, the present invention also provides a GCAM-based high-resolution SAR image airport runway area automatic detection system, including:

A down-sampling program unit for down-sampling high-resolution SAR images to generate medium-resolution images;

The runway area extraction program unit is used to input the medium resolution image to the geospatial context attention mechanism network GCAM to extract the runway area;

The coordinate mapping program unit is used to perform coordinate mapping for the extracted runway area to obtain the final detection result.

In addition, the present invention also provides a GCAM-based high-resolution SAR image airport runway area automatic detection system, comprising a computer device including an interconnected microprocessor and a memory, the microprocessor being programmed or configured to execute Steps of the GCAM-based high-resolution SAR image airport runway area automatic detection method, or a computer program programmed or configured to execute the GCAM-based high-resolution SAR image airport runway area automatic detection method stored in the memory .

In addition, the present invention also provides a computer-readable storage medium storing a computer program programmed or configured to execute the GCAM-based high-resolution SAR image airport runway area automatic detection method.

Compared with the prior art, the present invention has the following advantages: the present invention includes down-sampling a high-resolution SAR image to generate a medium-resolution image; inputting the medium-resolution image into the geospatial context attention mechanism network GCAM to extract the runway area; ; Carry out coordinate mapping for the extracted runway area to obtain the detection result of the final high-resolution SAR image. By combining deep learning and SAR image extraction of airport runway area, the geospatial information of the SAR image airport can be fully learned, and the high-resolution SAR image can be fully learned. High-precision, fast, and automatic extraction of airport runway areas from high-resolution SAR images.

Description of drawings

FIG. 1 is a schematic diagram of a basic principle of a method according to an embodiment of the present invention.

FIG. 2 is a schematic structural diagram of an improved residual network in an embodiment of the present invention.

FIG. 3 is a schematic structural diagram of a stripe pooling module SP in an embodiment of the present invention.

FIG. 4 is a schematic structural diagram of an effective attention module eSE in an embodiment of the present invention.

FIG. 5 is a schematic structural diagram of a global convolution module GCB and an edge refinement module BR in an embodiment of the present invention.

FIG. 6 is a SAR image, a label, and an optical remote sensing image of a sample label of an airport in an embodiment of the present invention.

FIG. 7 is a runway extraction result for Airport I in an embodiment of the present invention.

FIG. 8 is a runway extraction result for Airport II in an embodiment of the present invention.

FIG. 9 is a runway extraction result for Airport III in an embodiment of the present invention.

Detailed ways

As shown in Figure 1, the GCAM-based high-resolution SAR image airport runway area automatic detection method in this embodiment includes:

1) Downsampling the high-resolution SAR image to generate a medium-resolution image;

2) Input the medium resolution image to the geospatial context attention mechanism network GCAM to extract the runway area;

3) Perform coordinate mapping on the extracted runway area to obtain the detection result of the final high-resolution SAR image.

In this embodiment, the down-sampling of the high-resolution SAR image in step 1) specifically refers to performing a 5-fold down-sampling process on the SAR image by using a pixel value extraction method. It mainly includes two parts of downsampling: one is the downsampling of the sample images of the dataset, and the other is the downsampling of three high-resolution test SAR images, which are then sampled as medium-resolution SAR images.

In order to quickly extract the airport runway area of the SAR image, this embodiment proposes a geospatial contextual attention mechanism network GCAM (Geospatial Contextual Attention Mechanism). As shown in Figure 2, the geospatial contextual attention mechanism network GCAM includes encoding blocks and decoding. Blocks, coding blocks include residual network ResNet, Multi-scale Squeeze Pyramid (Multi-scale Squeeze Pyramid) and edge refinement module EDM (Edge Detection Module). Residual network ResNet is used to perform feature extraction on the input dataset to obtain preliminary features , the multi-scale extrusion pyramid MSP is used to obtain global context information from different pooling convolution layer operations at different resolutions for preliminary features, and the edge refinement module EDM is used to strengthen the network edge extraction ability for preliminary features. The output of the pyramid pressing MSP and the edge refinement module EDM is further fused to obtain multi-level features; the decoding block is used to combine the preliminary features and multi-level features to perform semantic segmentation of the airport runway area to extract the runway area. The coding block first uses the residual network ResNet to perform preliminary feature extraction on the input data set; the multi-scale extruded pyramid MSP and the edge refinement module EDM re-extract and fuse these preliminary features respectively. In terms of efficiency, different pooling convolution layer operations are used to obtain global context information. The edge refinement module EDM strengthens the network edge extraction capability and further integrates multi-level features; the decoding block adopts edge thinning decoding, and a part of the decoding block receives multiple data from the encoding block. Hierarchical high-level features, part of which receive preliminary features from the residual network ResNet, enable semantic segmentation of airport runway areas.

The residual network ResNet is the backbone network of the geospatial context attention mechanism network GCAM. The residual network ResNet has the characteristics of skip connections and optimized residuals. Its structure can speed up training and improve the accuracy of the model. It is very suitable for building a semantic segmentation network. In order to solve the problem that the network pooling operation is easy to lose detailed features, as shown in Figure 2, the residual network ResNet used in this embodiment is based on the residual network ResNet_101, and the hole rate is 2, 4, 8 and 16. The improved residual network obtained after replacing the ordinary two-dimensional convolution with the atrous convolution of . The hole convolution will solve the problem that the network pooling operation is easy to lose detailed features, and the addition of the hole convolution will not increase the number of parameters of the residual network ResNet, but it can keep the feature map size larger in the later convolution layer. This is beneficial to the detection of target pixels and improves the overall performance of the model. Considering the addition of atrous convolution, for any position j in the picture, apply the filter ω(k) on the input feature x[j+r.k], then the output y(j) is:

The rate r introduces r-1 0 values between sampling points, effectively extending the receptive field from k×k to k+(k-1)(r-1) without increasing the amount of parameters and computation. Figure 2 shows the improved structural part of the improved residual network. We copied the last block (block) of the residual network ResNet_101 4 times and built it in parallel, but the simple parallel work of the block does not use the network to obtain deep semantic information, which will cause the features to be concentrated in the smaller feature maps of the last few layers. And continuous strided convolutions do not utilize semantic segmentation. Therefore, in this embodiment, atrous convolutions with atrous rates of 2, 4, 8, and 16 are used to replace ordinary two-dimensional convolutions, thereby improving the final output stride. The addition of atrous convolution changes the resolution of some feature maps, which makes the final output features of the residual network ResNet_101 not only have high-dimensional and low-resolution feature maps, but also include some low-dimensional and high-resolution features, realizing multi-dimensional features. fully extracted.

Referring to Figure 1, the Multi-Scale Squeeze Pyramid MSP includes multi-receptive field parallel pooling working layers and an effective attention module eSE.

Referring to Figure 1, the multi-receptive field parallel pooling working layer consists of a 1×1 convolution with a dilation rate of 1, three 3×3 convolutions with a dilation rate of 6, 12, and 18, and a global average pooling module GAP. It is constructed in parallel with a stripe pooling module SP; the stripe pooling module SP uses a band-shaped pooling window H×1 to perform pooling operations in the horizontal direction for a two-dimensional feature tensor with an input size of H×W. The band-shaped pooling window 1×W performs the pooling operation in the vertical direction. The element values in the pooling kernel are averaged to obtain the output of stripe pooling in the horizontal direction and the output of stripe pooling in the vertical direction. The output of stripe pooling in the direction and the output of stripe pooling in the vertical direction use two one-dimensional convolutions to expand the output in the left and right and up and down directions, and the size of the two feature maps after expansion is the same. The two feature maps are fused, and finally the original data and the data after sigmoid processing are multiplied to obtain the final H×W two-dimensional feature tensor output; The feature map contains 256 channels and rich semantic information. It is first input to the multi-receptive field parallel pooling working layer. 3×3 convolution, a global average pooling module GAP and a stripe pooling module SP are built in parallel. Four dilated convolutions with different dilation rates can effectively capture multi-scale information from different receptive fields; the addition of global average pooling downsamples features to prevent network overfitting; stripe pooling captures local information of features; The receptive field parallel pooling working layer achieves multi-scale feature fusion.

The stripe pooling module SP (Stripe Pooling) can overcome the shortcomings of general pooling that are prone to false alarms. As shown in Fig. 3, when a two-dimensional feature tensor x∈R ^H×W is input, the stripe pooling module SP utilizes band-like pooling windows H×1 and 1×W to pool in the horizontal and vertical directions, respectively operation, and average the element values in the pooling kernel, and use this value as the pooling output value. The output y ^h =R ^H of stripe pooling in the horizontal direction is:

为是水平方向上条纹池化对任意矩阵元素的输出，x_i,j为池化核内所有的矩阵元素。In the above formula,

is the output of stripe pooling for any matrix element in the horizontal direction, and x _i,j are all matrix elements in the pooling kernel.

The output y ^v = R ^W of stripe pooling in the vertical direction is:

为是垂直方向上条纹池化对任意矩阵元素的输出，x_i,j为池化核内所有的矩阵元素。In the above formula,

is the output of stripe pooling for any matrix element in the vertical direction, and x _i,j are all matrix elements in the pooling kernel.

After H×1 and 1×W kernelization, two one-dimensional convolutions are used to dilate the output in the left-right and up-down directions. After the expansion, the two feature maps are of the same size, and then fused. Finally, the original data and the data processed by the Sigmoid function are multiplied, and the result is output. In the stripe pooling layers in the horizontal and vertical directions, the discretely distributed pixel regions and the banded pixel regions can easily depend on each other. Since the convolution kernel is long and narrow, and the convolution kernel shape is narrow along the opposite dimension, it is easy to capture the local information of the feature. All of these features make stripe pooling superior to square kernel-based average pooling.

Effective Squeeze-and-Excitation Module (eSE) is used to screen features from channel information after receiving multi-scale features. Referring to Figure 4, the effective attention module eSE firstly learns the feature F _avg through global average pooling for the input feature map Xi, and processes the feature F _avg through the fully connected layer (Fullyconnect, FC) to obtain the weight matrix W _C , and the weight matrix W _C readjusts the extracted channel attention feature A _eSE through the Sigmoid function, and then multiplies the channel attention feature A _eSE with the input feature map Xi to obtain the refined feature map X _refine , so as to give each input Xi pixel by pixel Perform weight assignment to realize feature re-screening. Among them, a fully connected layer (Fullyconnect, FC) and a sigmoid function are used to readjust the input feature map to extract useful channel information.

When the size of the input feature map is X _i ∈ R ^C×W×H , the effective channel attention map A _eSE (X _i )∈R ^C×1×1 is calculated as follows:

A _eSE (X _i )=σ(W _C (F _gap (X _i )))

In the above formula, A _eSE (X _i ) represents the channel attention feature A _eSE extracted from the input feature map Xi, σ is the Sigmoid function, W _C is the weight matrix, and F _gap (X _i ) is the input feature map. The feature F _avg obtained by the global average pooling of Xi, and the function expression of F _gap (X _i ) is:

In the above formula, Xi,j represent all the elements in the matrix of the feature map Xi.

The channel attention feature A _eSE is applied to the input feature map Xi to obtain the refined feature map X _refine . The expression is as follows:

表示异或。输入的特征图Xi是来自多尺度挤压金字塔MSP输出的多尺度特征图。将A_eSE(X_i)作为通道特征注意力应用到多尺度特征图中，会使得多尺度特征更具信息性。最后将输出的特征图逐元素的输入到精细化特征图X_refine进行特征再筛选。In the above formula,

Represents an exclusive or. The input feature map Xi is the multi-scale feature map from the output of the multi-scale squeeze pyramid MSP. Applying A _eSE (X _i ) as channel feature attention to multi-scale feature maps makes multi-scale features more informative. Finally, the output feature map is input element by element to the refined feature map X _refine for feature re-screening.

Referring to Figure 1, it can be seen that the multi-scale squeeze pyramid MSP and the edge refinement module EDM work in parallel, and simultaneously receive the output feature map from the improved residual network. As shown in Figure 1, the edge refinement module EDM includes a global convolution module GCB (Global Convolutional Block), which is used to enhance the close connection between the feature map and the pixel classification layer and the ability to process feature maps of different resolutions to obtain global information. The edge refinement module BR (Boundary Refinement) improves the edge extraction ability of the coding block from the global information. The edge refinement module EDM can effectively solve the problem of pixel classification and localization in semantic segmentation. The global convolution module GCB increases the size of the convolution kernel to the spatial size of the feature map, so that the feature map and the pixel classification layer are closely related. , so as to enhance the ability to process different features and obtain global information; and then introduce the edge refinement module BR to further improve the network edge extraction ability.

As shown in FIG. 5 , in this embodiment, the global convolution module GCB includes a large convolution kernel of k×k and a feature combination module. The large convolution kernel of k×k includes two paths, one path is k×1×c The convolution of ×c is composed of the convolution of 1×k×c×c, and the other route is composed of the convolution of 1×k×c×c and the convolution of k×1×c×c, where c is the number of channels, The output results of the two channels are input into the feature combination module together to obtain the feature Sum _W×H×C ; the edge refinement module BR processes the feature Sum _W×H×C through a small convolution kernel, an activation function, and a small convolution kernel in turn, and then the The processing results are superimposed on the original feature Sum _W×H×C , and finally the feature map after refining the edge of the runway area is obtained.

Referring to Figure 5, the global convolution module GCB adopts the convolution construction mode to make full use of the multi-channel information of the features. In the problem of pixel classification, the global convolution module GCB adopts a large convolution kernel, so that the semantic information corresponding to each pixel will not be changed due to image transformation (translation, flip, etc.), and the relationship between pixels is closer; On the problem of pixel point positioning, the global convolution module GCB uses full convolution, and uses the principle of matrix decomposition to replace the k×k large kernel convolution with 1×k and k×1, k×1 and 1×k convolutions. The amount of parameters is reduced and the amount of calculation is reduced, and each pixel type can be matched with the corresponding correct type to achieve accurate pixel segmentation. Since the global convolution module GCB does not have a BN layer (Batch Normalization) and an activation function, an edge refinement module BR with a small convolution kernel is introduced to prevent the misclassification of object boundary pixels and achieve accurate classification and positioning.

As shown in Figure 1, the decoding block performs edge information decoding on the feature map obtained by reducing the dimension of the output feature of the coding block through 1×1 convolution and using the edge refinement module EDM to refine the edge of the runway area, and performing bilinear decoding. 4 times upsampling, then connect the preliminary features output by the residual network ResNet after 1×1 convolutional dimension reduction and the results obtained by bilinear 4 times upsampling, and then apply the connected features to a 3×3 volume The product is used to refine the features, and finally a simple bilinear 4x upsampling is performed to obtain the final segmentation result. The input of the decoding block consists of two parts: the output features of the encoding block and the preliminary features output by the residual network ResNet. The output features of the coding block are first reduced by 1×1 convolution, then edge information is decoded by EDM, and then upsampled by bilinear 4 times. This operation is to fully decode the edge information while reducing the number of feature channels; The corresponding feature connections of the backbone network with the same spatial resolution, because the features from the backbone network contain some low-level features, and the low-level features usually contain a large number of channels, so we also use a 1 × 1 convolution to reduce the number of channels and reduce unnecessary network. channel calculation.

In this embodiment, step 3) is used to perform coordinate mapping on the extracted runway area to obtain the final detection result. Since the coordinate mapping is the same as the existing method, it will not be repeated here. After the geospatial context attention mechanism network GCAM realizes the segmentation of the airport runway area of the medium-resolution SAR image, the coordinate mapping method is used to process the result map, and then obtain The resulting plot of the high-resolution SAR original image. Finally, the result image and the original image are visualized to realize the extraction of the runway area from the high-resolution SAR image.

The following will perform experimental verification of the GCAM-based high-resolution SAR image airport runway area automatic detection method in this embodiment. The experimental environment is as follows: CPU Inter Xeon Gold 5120; GPU (single) NVIDIA RTX 2080Ti; the data set uses the SAR images of the Gaofen-3 system. First, the 10 airport sample images are reduced by 5 times by using the method of pixel extraction. Sampling; then use LaberImage software for pixel labeling, which is divided into two categories: runway area and background. We arbitrarily cut 10 downsampled medium-resolution SAR images into images larger than 480×480 to make a small-sample dataset, generating a total of 466 images. The ratio of training set and validation set is 4:1. Subfigures (a) to (c) in Figure 6 are the SAR image, label and optical remote sensing image of an airport sample label respectively; the connected area where the label a is located is the runway area, which includes the runway, taxiing Roads, parking pads, and aircraft; the remaining individual connected areas are the background.

The parameters are set as follows: the learning rate is set to 0.00001 during network training, and the weight decay coefficient is set to 0.995. The batch size of the input image is 1, and the network is trained for 100 iterations, saving an epoch every 5 times. During the training process, the input image is randomly cropped, and the window size of random cropping is 480×480.

In this embodiment, PA (Pixel accuracy, Pixel accuracy) and IOU (Intersectionover Union, Intersectionover Union) are used as parameters for verifying the accuracy of track extraction. PA represents the proportion of correctly labeled pixels to total pixels; IOU represents the ratio of intersection and union between segmentation results and labels; MPA (Mean pixel accuracy) represents the proportion of correctly classified pixels for each category; MIOU (Mean intersection over union) represents the average of IOUs on each category. Specifically:

Assuming that there are k+1 classes (including a class of backgrounds), in the above formula, P _ij represents the number of pixels that belong to class i but is predicted to be class j, which is a false positive sample, and P _ji represents that it belongs to class j but is predicted is the number of pixels in class i, which is a false negative sample, and P _ii represents the real number of pixels in class i.

In order to verify the efficiency of the method of this embodiment in extracting the airport runway area of the SAR image, a total of three sets of comparative experiments were performed. The method of this embodiment is compared with DeepLabV3+, RefineNet and MDDA. There are three experimental airports, namely Airport I with a size of 12000×15000, Airport II with a size of 9600×9600, and Airport III with a size of 15000×17500. None of these airports have been used in the dataset. MDDA is a deep learning network that we proposed before for extracting airport runway areas in SAR images. DeepLabV3+ and RefineNet are both mainstream networks for semantic segmentation. The dataset used in the experiment is our manually annotated 466 small-sample dataset. Because the network outputs a down-sampled medium-resolution image, the sizes of airports I, II, and III after down-sampling are 2400×3000, 2000×2000, and 3000×3500, respectively. Finally, we perform coordinate mapping on the result graph to directly obtain the result graph before downsampling. We analyzed the network training time, the image testing time, and the extraction accuracy of the runway area before and after sampling.

Figures 7 to 9 show the extraction results of airport runway areas for airports I, II, and III, respectively. Among them, (a) is the original high-resolution SAR image, (b) is the medium-resolution SAR image after downsampling by 5 times, (c) is the category label of the runway area, red is the runway area, and black is the non-runway area, that is, the background; (d) is the extraction result of RefineNet for medium resolution SAR images, (e) is the extraction result of MDDA for medium resolution SAR images, (f) is the extraction result of DeepLabV3+ for medium resolution SAR images, (g) is this Extraction results of the example method (GCAM) for the intermediate resolution SAR image; (h) is the fusion map of the RefineNet result (d) and the intermediate resolution SAR image (b), (i) is the MDDA result (e) and the intermediate resolution SAR image. The fusion image of the rate SAR image (b), (j) is the fusion image of the DeepLabV3+ result (f) and the medium resolution SAR image (b), (k) is the result (g) of the method in this embodiment (GCAM) and the medium resolution SAR image (b). The fusion map of the rate SAR image (b); (l)-(o) is the fusion map of the result (d)-(g) after coordinate mapping processing and the high-resolution original image (a); where the label is 1 The area marked with 2 is the runway area; the area frame marked 2 is the false detection frame, that is, the background false detection is the part of the runway area; the area frame marked 3 marks the missed detection frame, that is, the part of the runway area that has not been detected.

1. Experiment results and analysis of Airport I.

As shown in sub-figure (a) in Figure 7, Airport I is mainly composed of a large area of long runway area and aircraft landing pads. There are a large number of aircraft in the airport, and there are obvious aircraft target highlights; traffic lines.

We test the medium-resolution SAR image of Airport I with a test image size of 2400×3000. As shown in sub-figures (d)-(g) in Figure 7, the extraction result of the method in this embodiment is the closest to the label, and MDDA's extraction of part of the edge of the runway area is incomplete; DeepLabV3+ misses a small part of the runway area Phenomenon; RefineNet has the worst extraction effect. According to the visual graphs (h)-(k), we mark the main missed boxes. There is no large missed detection area in the method of this embodiment, while MDDA has 2 main missed detection areas, and DeepLabV3+ has 4 relatively obvious missed detection areas. The inspection areas are all on the edge of the runway area. Comparing the network result (j) of the method of this embodiment with the extraction result (k) of DeepLabV3+, it can be seen that the addition of the edge refinement module EDM in the method of this embodiment strengthens the learning of network edge features.

2. Experiment results and analysis of Airport II.

Airport II has simpler features than Airport I. The runway area of Airport II is mainly composed of long and straight runways. Except for a small building complex near the edge of the airport, there is no large residential area, but there are many water areas around. The water area is imaged under the synthetic aperture radar and has the same dark black features as the runway, which is disturbing to the network distinguishing features.

We test a medium-resolution SAR image of Airport II with a size of 2000 × 2000. Figure 8 shows the runway area extraction of Airport II. Comparing the subgraphs (d), (e), (f), (g) and (c) in FIG. 8 , it can be seen that the extraction result of the method of this embodiment has no false alarm and the extraction effect is the best. It can be seen from the subgraphs (h)-(k) in Fig. 8 that the method of this embodiment has only one small missed frame; MDDA has one false frame and four obvious missed frames; DeepLabV3+ There are many false alarms and the most missed frames; for the left edge area of Airport II, RefineNet has several missed areas; their extraction effects need to be improved. It is illustrated that the edge extraction capability and the false alarm removal capability of the method in this embodiment are both the best, which also highlights the advantages of the multi-scale extrusion pyramid MSP in the method in this embodiment.

3. Experiment results and analysis of Airport Ⅲ.

The runway area structure and surrounding structures of Airport III are the most complex. There are many airstrips, taxiways, rest stops and parking pads. Airport III is a civil airport, and most of the runway areas of Airport III are short runways, and there is no large area of long straight runways. The SAR features of surrounding ground objects are mostly gray and bright spots, which have obvious feature contrast with the airport runway area, which reduces the probability of network misjudgment. However, the edge features of Airport III are more complex and contain the most edge information, which requires the network to have better ability to learn global semantic information and to effectively decode edge information.

We test a medium-resolution SAR image of Airport III with a size of 3000 × 3500. Comparing the subgraphs (d)-(l) in Fig. 9, the extraction effect of the method in this embodiment is also the best, and only some small areas are missed; MDDA detection has two obvious false alarms; DeepLabV3+ has a large number of missed detections, It shows that its ability to learn edge information is not strong; RefineNet has a large number of false alarms and the extraction effect is the worst. This also improves the effectiveness of the edge decoding of the method in this embodiment.

In order to more intuitively reflect the high efficiency of the method of this embodiment for extracting the airport runway area. Table 1 presents the extraction accuracy of the medium-resolution SAR images of the three airports under different algorithms. The average extraction accuracy of the method in this embodiment for the three airport runway areas reaches 0.9823, and the average IOU reaches 0.9665, which are all higher than MDDA, DeepLabV3+ and RefineNet. According to Table 1, the difference between the PA and IOU values of the method in this embodiment is very small for the runway area of the same airport, indicating that the method in this embodiment can almost completely extract the runway area without false alarms; DeepLabV3+ is prone to false alarms, so the same airport runway There is a certain difference between the values of PA and IOU in the area, and false alarms will reduce the IOU value of the runway area; although the overall extraction effect of MDDA is not bad, it is insufficient in learning the details of the small sample data set; the PA and IOU values of RefineNet are both is the lowest.

Table 1: Analysis of extraction accuracy for different networks.

Table 2 presents the training time of the different algorithms on the small sample dataset and the test time on the medium-resolution SAR images of the three airports. According to Table 2, from the point of view of the training time for the small sample data set, our network only needs about 2 hours of training time; the training time of MDDA is the longest, nearly 8 hours, and the effect of MDDA training small samples is obviously not great. The samples are good; the training time of DeepLabV3+ and RefineNet is similar to that of the method in this example, but the accuracy is quite different. From the test time of the medium-resolution SAR images of the three airports, the smaller the image size, the shorter the test time. The average test time of the method in this embodiment is only 16.95s, the average test time of RefineNet is 16.69s, and the average test time of DeepLabV3+ The average test time is 15.89s, and the test time of MDDA is about 2.5 times that of the method in this embodiment. The addition of MSP and EDM brings a certain amount of parameters to the network, which is also the reason why the training time and testing time of the method in this embodiment are slightly longer than those of DeepLabV3+. The shorter the network training time and the image testing time, the more efficient the actual project will be. . Therefore, on the whole, the method of this embodiment can achieve high-precision and rapid extraction in the processing of small sample data sets of SAR images, and the method of this embodiment has high efficiency.

Table 2: Dataset training time and test time for medium resolution airport images for different networks.

It can be seen from this that the method of this embodiment can realize the rapid and automatic extraction of the airport runway area of the high-resolution SAR image. Our network design is very lightweight, which greatly shortens the network layer iteration time, reduces network training time and image testing time. MSP enables the network to learn global features at multiple scales and encode effective features. The parallel working mode of EDM and MSP strengthens the learning between contextual semantic information, and EDM enables edge information to be fully decoded and extracted. At the same time, our network is more suitable for training small-sample datasets. Today, there is no public large dataset of SAR airports for semantic segmentation, which can only be manually annotated. The application of small samples is more conducive to saving labor time and cost. In general, our network is superior to the mainstream algorithm DeepLabV3+ in terms of extraction accuracy, dataset training time and image testing time, and the performance of GCAM surpasses the algorithm MDDA we proposed before, realizing efficient automatic .

To sum up, in order to realize the rapid and automatic extraction of high-resolution SAR images, a GCAM-based automatic detection method for airport runway areas of high-resolution SAR images is proposed. The method includes three parts: High-resolution SAR image downsampling processing, GCAM extraction of airport runway area, GCAM generated coordinate mapping of the resulting image. The downsampling process makes a single training sample contain more airport information, which is conducive to making a small sample data set; MSP adds stripe pooling and four parallel convolutions to work together to learn features at multiple scales, and the eSE module performs useful feature screening. ; EDM helps the network learn edge semantic information, and coordinate mapping processing can obtain the extraction results of the original high-resolution SAR image. In the detection test of three airport runway areas, our network is the best compared to DeepLabV3+, RefineNet and MDDA, with MPA up to 0.98 and MIOU up to 0.96. In addition, our network training data set time is only 2.25h, and the average image test time is only 16.94s. From the extraction results, GCAM has no false alarms and few missed detections, and can efficiently extract the airport runway area. In addition, GCAM can improve the detection efficiency in practical engineering; and after extracting the airport runway area, it can shorten the detection range of subsequent aircraft extraction and save time.

In addition, this embodiment also provides a GCAM-based high-resolution SAR image airport runway area automatic detection system, including:

A down-sampling program unit for down-sampling high-resolution SAR images to generate medium-resolution images;

The runway area extraction program unit is used to input the medium resolution image to the geospatial context attention mechanism network GCAM to extract the runway area;

The coordinate mapping program unit is used to perform coordinate mapping for the extracted runway area to obtain the detection result of the final high-resolution SAR image.

In addition, the present embodiment also provides a GCAM-based high-resolution SAR image airport runway area automatic detection system, including a computer device, the computer device including an interconnected microprocessor and a memory, the microprocessor being programmed or configured to Perform the steps of the aforementioned GCAM-based high-resolution SAR image airport runway area automatic detection method, or a computer program programmed or configured to execute the aforementioned GCAM-based high-resolution SAR image airport runway area automatic detection method is stored in the memory.

In addition, the present embodiment also provides a computer-readable storage medium storing a computer program programmed or configured to execute the aforementioned GCAM-based high-resolution SAR image airport automatic detection method.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein. The present application refers to flowcharts of methods, apparatus (systems), and computer program products according to embodiments of the present application and/or processor-executed instructions generated for implementing a process or processes and/or block diagrams in a flowchart. A means for the function specified in a block or blocks. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams. These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

The above are only the preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above-mentioned embodiments, and all technical solutions under the idea of the present invention belong to the protection scope of the present invention. It should be pointed out that for those skilled in the art, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims (10)

1. A GCAM-based high-resolution SAR image airport runway area automatic detection method is characterized by comprising the following steps:

1) down-sampling the high-resolution SAR image to generate a medium-resolution image;

2) inputting the medium-resolution image into a geographic space context attention mechanism network GCAM to extract a runway area;

3) and carrying out coordinate mapping on the extracted runway area to obtain a final detection result of the high-resolution SAR image.

2. The GCAM-based high-resolution SAR image airport runway area automatic detection method according to claim 1, characterized in that, the down-sampling of the high-resolution SAR image in step 1) is specifically a 5-fold down-sampling processing of the SAR image by adopting a pixel value extraction method.

3. The GCAM-based high resolution SAR image airport runway area automatic detection method of claim 1, wherein the geospatial context attention mechanism network GCAM comprises a coding block and a decoding block, the coding block comprises a residual network ResNet, a multi-scale extrusion pyramid MSP and an edge refinement module EDM, the residual network ResNet is used for performing feature extraction on an input data set to obtain preliminary features, the multi-scale extrusion pyramid MSP is used for obtaining global context information from different resolutions with different pooling convolutional layer operations for the preliminary features, the edge refinement module EDM is used for enhancing network edge extraction capability for the preliminary features, and outputs of the multi-scale extrusion pyramid MSP and the edge refinement module EDM are further fused to obtain multi-level features; the decoding block is used for carrying out semantic segmentation on the runway area of the airport by combining the preliminary features and the multi-level features to extract the runway area.

4. The GCAM-based high-resolution SAR image airport runway area automatic detection method according to claim 3, wherein the residual error network ResNet is an improved residual error network obtained by replacing ordinary two-dimensional convolution with hole convolutions with hole rates of 2, 4, 8 and 16 on the basis of the residual error network ResNet _ 101.

5. The GCAM-based high-resolution SAR image airport runway area automatic detection method of claim 3, wherein the multi-scale extrusion pyramid MSP comprises a multi-field parallel pooling working layer and an effective attention module eSE, wherein the multi-field parallel pooling working layer is built by a 1 x 1 convolution with a voidage of 1, a 3 x 3 convolution with three voidages of 6,12 and 18 respectively, a global average pooling module GAP and a stripe pooling module SP in parallel; the stripe pooling module SP performs pooling operation in the horizontal direction by utilizing a stripe pooling window H multiplied by 1 and pooling operation in the vertical direction by utilizing a stripe pooling window H multiplied by 1 in the vertical direction aiming at a two-dimensional feature tensor with the input size of H multiplied by W, averages element values in a pooling kernel respectively to obtain output of stripe pooling in the horizontal direction and output of stripe pooling in the vertical direction, then performs expansion in the left-right direction and the up-down direction on the output respectively by using two one-dimensional convolutions aiming at the output of stripe pooling in the horizontal direction and the output of stripe pooling in the vertical direction, the two expanded feature maps have the same size, then fuses the two expanded feature maps, and finally multiplies the original data and the data subjected to Sigmoid processing to obtain the output of the H multiplied by W two-dimensional feature tensor; the active attention Module eSE learns first by globally averaged pooling of features F for the input feature map Xi_avgWill feature F_avgObtaining a weight matrix W by full connection layer processing_CThe weight matrix W_CReadjusting the extracted channel attention feature A through Sigmoid function_eSEThen the channel attention feature A_eSEApplying the input feature map Xi to obtain a refined feature map X_refineFinally, the refined characteristic diagram X is obtained_refineAnd performing feature re-screening to obtain global context information.

6. The GCAM-based high resolution SAR image airport runway area automatic detection method of claim 3, characterized in that the edge refinement module EDM comprises a global convolution module GCB for enhancing the close relation of feature maps and pixel classification layers and the ability to process feature maps of different resolutions to obtain global information, an edge refinement module BR for enhancing the edge extraction ability of coded blocks from global information; the global convolution module GCB comprises a big convolution kernel of kxk and a characteristic combination module, wherein the big convolution kernel of kxk comprises two paths, one path consists of convolution of kx01 x1 cxc x 2c and convolution of 1 x3 kxc x c, the other path consists of convolution of 1 xkxxc x c and convolution of kx1 xc x c, wherein c is the number of channels, and output results of the two paths are input into the characteristic combination module together to obtain the characteristic Sum_W×H×C(ii) a The edge refinement module BR targets a feature Sum_W×H×CSequentially processing the data by a small convolution kernel, an activation function and a small convolution kernel, and then overlapping the processing result to the original characteristic Sum_W×H×CAnd finally obtaining a characteristic diagram after the edges of the refined runway area.

7. The GCAM-based high-resolution SAR image airport runway area automatic detection method of claim 3, wherein the decoding block performs 1 x 1 convolution dimensionality reduction on output features of a coding block, performs edge information decoding on a feature map obtained by an edge refinement module EDM after refining the runway area edge, performs bilinear 4-fold upsampling, then connects the result obtained by performing 1 x 1 convolution dimensionality reduction on preliminary features output by a residual error network ResNet and bilinear 4-fold upsampling, then applies a 3 x 3 convolution to the connected features to refine the features, and finally performs a simple bilinear 4-fold upsampling, thereby obtaining the final segmentation result.

8. A GCAM-based high-resolution SAR image airport runway area automatic detection system is characterized by comprising:

the down-sampling program unit is used for down-sampling the high-resolution SAR image to generate a medium-resolution image;

a runway area extraction program unit for inputting the medium resolution image into a geographic space context attention mechanism network GCAM to extract a runway area;

and the coordinate mapping program unit is used for carrying out coordinate mapping on the extracted runway area to obtain a detection result of the final high-resolution SAR image.

9. A GCAM-based high resolution SAR image airport runway area automatic detection system comprising a computer device comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to perform the steps of the GCAM-based high resolution SAR image airport runway area automatic detection method according to any one of claims 1 to 7, or the memory has stored therein a computer program programmed or configured to perform the GCAM-based high resolution SAR image airport runway area automatic detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program programmed or configured to perform the GCAM-based high resolution SAR image airport runway area automatic detection method of any of claims 1-7.

Images