Novel digital instrument identification method for power distribution room
Technical Field
The invention relates to the technical field of digital image recognition, in particular to a novel method for recognizing a digital instrument of a power distribution room
Technical Field
Digital instruments have been widely used in the field of substations. However, due to the limitations of cost privacy and working conditions, many digital meters do not have special communication interfaces, so that the readings cannot be automatically identified, and the relevant readings need to be entered manually. The manual entry of the meter reading usually consumes a large amount of manpower and time, makes mistakes easily in a long-time and high-frequency working environment, and cannot acquire and enter the meter reading under the unattended condition such as night. Therefore, the method for manually inputting the reading of the digital instrument of the power distribution station has the defects of low efficiency, poor reliability, incapability of meeting the real-time performance, low intelligent level and the like. In this case, it is necessary to automatically acquire numerical information by image processing and image recognition based on computer vision technology.
In recent years, with the common installation of cameras and the popularization and application of inspection robots, various works of transformer substations gradually develop towards automation, and the recognition and recording work of instrument data also continuously develops towards digitization and intellectualization. Many digital image processing methods have been applied to digital instrument recognition automatic recognition, one is a method of positioning and recognizing according to manually designed features, such as zhuangzhen positioning the digital instrument area using edge detection method, and then performing digital recognition according to the stroke and shape features of the number; the bei cheng jie uses an image thresholding method to locate the digital instrument area and then performs digital recognition based on the projection method and digital stroke features. The other type is a positioning and recognition method based on deep learning automatic feature extraction, for example, a YOLOv2 network is used by Rayson Laroca and the like to position a digital instrument area, and a comparison test of CR-NET, CRNN and other networks on digital recognition is carried out. Most conventional methods of meter area extraction and identification of individual numbers are based on ideal viewing angles and lighting environments. Once the premise is broken away, the identification accuracy and robustness of most traditional methods are greatly reduced, and the identification effect is difficult to guarantee. And the traditional method can only process single-angle and single-form numbers and cannot effectively process multi-angle and multi-form numbers in the transformer substation. The target detection based on the deep learning can automatically extract features, effectively improve the precision, and can obtain better identification accuracy under the condition of changeable illumination, angle and form, so that the target detection based on the deep learning is widely applied at present.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a novel digital instrument recognition method with high recognition accuracy and high operation efficiency, and aims to solve the problems of low recognition accuracy and poor algorithm robustness of the background of the prior art on multi-angle and multi-form instrument numbers.
In order to achieve the purpose, the invention adopts the following technical scheme:
a novel digital instrument identification method comprises the following steps:
step 1: extracting a digital instrument area, and independently dividing a plurality of digital instruments to be identified;
step 2: extracting vertex coordinates of the digital instrument, and carrying out perspective transformation on the digital instrument area obtained in the step (1) to obtain an inclination-corrected digital instrument area;
and step 3: preprocessing the digital instrument image after the inclination correction obtained in the step (2);
and 4, step 4: dividing a single number in the preprocessed digital instrument image obtained in the step (3) by using an outer contour detection method;
and 5: identifying the single divided digital area by using a traditional method, and summarizing and outputting an identification result if the identification of each single digital area is successful; otherwise, jumping to step 6;
step 6: and (3) recognizing the digital instrument image after the inclination correction obtained in the step (2) by using a pre-trained target recognition model, and summarizing and outputting a recognition result.
Further, the step 1 comprises the following steps:
step 1.1: carrying out binarization processing on the original image, and extracting the outline of a digital instrument area;
step 1.2: and (3) fitting the outline of the digital instrument area in the step 1.1 by using a polygon, and further independently dividing the plurality of digital instruments in the extracted digital instrument area.
Still further, the step 2 comprises the following steps:
step 2.1: in the digital instrument area which is independently segmented in the step 1.2, pixels corresponding to the gray scale of the digital instrument are searched in the outline, the coordinates of the corresponding pixels are processed and stored, and the coordinates of four vertexes of the digital instrument are found by utilizing a sorting method;
step 2.2: and (3) finding an optimal single mapping transformation matrix H, namely finding a conversion matrix from the source plane to the target plane by using the vertex coordinates of the digital instrument before the non-tilt correction obtained in the step (2.1) and the vertex coordinates of the digital instrument after the expected tilt correction.
Step 2.3: and (3) performing perspective transformation on the digital instrument area obtained in the step (1) by using the optimal single mapping transformation matrix H obtained in the step (2.2), namely projecting the digital instrument area to a new view plane to obtain the digital instrument area after inclination correction.
In step 2.2, an optimal single mapping transformation matrix H between four two-dimensional point pairs is calculated using an RANSAC method, and the used RANSAC method achieves the goal by repeatedly selecting a group of random subsets in data, and the steps are as follows:
2.2.1: randomly extracting 4 sample data from the data set, wherein the 4 samples are not collinear, and calculating a transformation matrix H and marking as a model M;
2.2.2: calculating projection errors of all data in the data set and the model M, and adding an inner point set I if the errors are smaller than a threshold value;
2.2.3: if the number of the elements of the current internal point set is greater than the number of the elements of the optimal internal point set I _ best, updating the I _ best to I, and updating the iteration times k;
2.2.4: if the iteration number is more than k, exiting: otherwise, adding 1 to the iteration number, and repeating the steps.
Still further, the step 3 includes the steps of:
step 3.1: converting the digital instrument area image after the inclination correction obtained in the step (2) into an HSV space, and performing binarization segmentation on the image by using an HSV threshold value;
step 3.2: morphological treatment: and performing closed operation on the image, expanding the image firstly and then corroding the image, eliminating the breakpoint in a single digit and removing small white spots in a non-digit area.
Further, the step 5 comprises the following steps:
step 5.1: scanning two upper horizontal lines, two lower horizontal lines and three vertical lines in a single digital region as shown in fig. 2, wherein if N and more than N pixels with non-0 gray levels are continuously scanned, the corresponding seven-segment code tube is bright;
step 5.2: finding corresponding numbers according to the lighting condition of the seven-segment codes obtained in the step 5.1, and if a matching relation cannot be found, failing to identify corresponding single numbers;
step 5.3: judging the result of the step 5.2, if the identification of each single number is successful, summarizing and outputting the identification result; otherwise, jump to step 6.
Further, the step 6 comprises the following steps:
step 6.1: making and marking a data set of the digital instrument panel to be detected;
step 6.2: modifying and optimizing training parameters in a Yolov4 configuration file;
step 6.3: training a digital instrument panel data set by using a Yolov4 network structure;
and 6.4, identifying the digital instrument image after the inclination correction obtained in the step 2 by utilizing the YOLOv4 model obtained by training, and summarizing and outputting the identification result.
In the related step 6.3, the specific structure of the model is as follows: the yoloV4 adopts cspdarknet53 on a main feature extraction network, uses a cspdnet structure on the basis of the original darknet53, uses SPP (spatial pyramid) and PAN (Path Aggregation network) structures in a feature pyramid part, and uses a cutmax and mosaic multi-picture fusion technology in the aspect of data expansion in a feature utilization part or a head using YoloV3 in a yoloV4 network structure; simulating image occlusion by using Dropblock; using table smoothing to alleviate overfitting; the activating function adopts a hash activating function; CIoU is introduced as regression loss, so that the regression of the target frame is more stable
In the invention, a digital instrument area is extracted by collecting an electric appliance cabinet image of a power distribution room, the digital instrument area is subjected to inclination correction, the digital area image subjected to inclination correction is preprocessed, the image subjected to inclination correction is segmented to obtain a single digital image, the single digital image is subjected to digital recognition by a threading method to obtain a matching result, whether the success rate of the single digital image in the threading method recognition is 100% or not is judged, if yes, the threading method digital recognition result is used as the recognition result of the single digital image, if not, the image subjected to inclination correction is re-recognized by a previously trained YOLOv4 model to obtain a recognition result, and finally, the recognition result of the single digital image is output.
The beneficial effects of the invention are as follows: the invention provides a novel digital instrument recognition method, which comprises the steps of extracting a digital instrument area to perform inclination correction, performing area segmentation on an individual number, performing primary recognition by using a threading method, and correcting a result by using a YOLOv4 algorithm model, thereby obtaining a correct recognition result of a digital instrument of a power distribution room. The proposed method for identifying the digital instrument of the power distribution room based on YOLOv4 has great reference value for detecting and identifying the digital instrument of the power distribution room.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a schematic diagram of a conventional identification method in step 5 of the present invention;
fig. 3 is a schematic diagram of the network structure of YOLOv4 in step 6 of the present invention.
Detailed Description
The following describes the embodiments of the present invention in detail with reference to the drawings and examples (taking an image of an electrical cabinet of a local distribution substation as an example).
Referring to fig. 1 to 3, a novel digital instrument recognition method includes the following steps:
step 1: independent partitioning of multiple digital meters.
Extracting a digital instrument area, and independently segmenting a plurality of digital instruments to be identified, wherein the process comprises the following steps:
step 1.1: carrying out binarization processing on the original image, and extracting the outline of a digital instrument area;
step 1.2: and (3) fitting the outline of the digital instrument area in the step 1.1 by using a polygon, and further independently dividing the plurality of digital instruments in the extracted digital instrument area.
Step 2: and performing inclination correction processing on the extracted digital instrument.
Extracting vertex coordinates of the digital instrument, calculating an optimal single mapping transformation matrix H by using an RANSAC method, and carrying out perspective transformation on the digital instrument area obtained in the step 1 to obtain an inclination-corrected digital instrument area, wherein the process comprises the following steps:
step 2.1: in the digital instrument area which is independently segmented in the step 1.2, pixels corresponding to the gray scale of the digital instrument are searched in the outline, the coordinates of the corresponding pixels are processed and stored, and the coordinates of four vertexes of the digital instrument are found by utilizing a sorting method;
step 2.2: calculating an optimal single mapping transformation matrix H between the four two-dimensional point pairs by using the vertex coordinates of the digital instrument before non-tilt correction and the vertex coordinates of the digital instrument after expected tilt correction, which are obtained in the step 2.1, by using an RANSAC method, namely finding a conversion matrix from a source plane to a target plane;
the RANSAC (random sample consensus) method used in this step achieves the goal by iteratively selecting a set of random subsets in the data. The selected subset is assumed to be an in-office point and verified by the following method:
1): a model is adapted to the assumed local interior point, that is, all unknown parameters can be calculated from the assumed local interior point; (solving for affine transformations, at least three points are required)
2): testing all other data with the model obtained in 1), and if a certain point is suitable for the estimated model, considering it to be an in-office point;
3): if enough points are classified as the assumed intra-office points, the estimated model is reasonable enough;
4): then, all the assumed intra-office points are used to re-estimate the model, since it is estimated only by the initial assumed intra-office points;
5): finally, the model is evaluated by estimating the error rate of the local interior point and the model.
The algorithm comprises the following specific steps:
2.2.1: randomly extracting 4 sample data (the four samples are not collinear) from the data set to calculate a transformation matrix H, and marking the transformation matrix H as a model M;
2.2.2: calculating projection errors of all data in the data set and the model M, and adding an inner point set I if the errors are smaller than a threshold value;
2.2.3: if the number of the elements of the current internal point set is greater than the number of the elements of the optimal internal point set I _ best, updating the I _ best to I, and updating the iteration times k;
2.2.4: if the iteration number is more than k, exiting: otherwise, adding 1 to the iteration number, and repeating the steps.
Step 2.3: and (3) performing perspective transformation on the digital instrument area obtained in the step (1) by using the optimal single mapping transformation matrix H obtained in the step (2.2), namely projecting the digital instrument area to a new view plane to obtain the digital instrument area after inclination correction.
And step 3: and (4) preprocessing the digital instrument image.
Preprocessing the digital instrument image after the inclination correction obtained in the step 2, wherein the process is as follows:
step 3.1: converting the digital instrument area image after the inclination correction obtained in the step (2) into an HSV space, and performing binarization segmentation on the image by using an HSV threshold value;
step 3.2: morphological treatment: and performing closed operation on the image, expanding the image firstly and then corroding the image, eliminating the breakpoint in a single digit and removing small white spots in a non-digit area.
And 4, step 4: independent segmentation of individual digits in a digital instrument image.
And (3) dividing a single number in the preprocessed digital instrument image obtained in the step (3) by using an outer contour detection method.
And 5: the "threading method" identifies individual numbers.
Identifying the single segmented digital area by using a threading method, and summarizing and outputting identification results if the identification of each single digital area is successful; otherwise, jumping to step 6, and the process is as follows:
step 5.1: scanning two upper horizontal lines, two lower horizontal lines and three vertical lines in a single digital region as shown in fig. 2, wherein if N and more than N pixels with non-0 gray levels are continuously scanned, the corresponding seven-segment code tube is bright;
step 5.2: finding corresponding numbers according to the lighting condition of the seven-segment codes obtained in the step 5.1, and if a matching relation cannot be found, failing to identify corresponding single numbers;
step 5.3: judging the result of the step 5.2, if the identification of each single number is successful, summarizing and outputting the identification result; otherwise, jump to step 6.
Step 6: the numerical identification was performed using the YOLOv4 model.
Recognizing the digital instrument image after the inclination correction obtained in the step 2 by using a pre-trained YOLOv4 model, and summarizing and outputting the recognition result, wherein the process is as follows:
step 6.1: making and marking a data set of the digital instrument panel to be detected;
step 6.2: modifying and optimizing training parameters in a Yolov4 configuration file;
step 6.3: training a digital instrument panel data set by using a Yolov4 network structure;
and 6.4, identifying the digital instrument image after the inclination correction obtained in the step 2 by utilizing the YOLOv4 model obtained by training, and summarizing and outputting the identification result.
The specific structure of the model in the involved step 6.3 is as follows: the YoloV4 adopts cspdarknet53 on a main feature extraction network, and uses a cspnet structure on the basis of the original darknet 53. In the feature pyramid part, SPP (spatial gradient porous) and PAN (Path Aggregation network) structures are used, and in the feature utilization part, the head of Yolov3 is still used. The technology of fusion of cutmix and mosaic multiple pictures is used in the aspect of data augmentation in the YoloV4 network structure; simulating image occlusion by using Dropblock; using table smoothing to alleviate overfitting; the activating function adopts a hash activating function; CIoU is introduced as regression loss, so that the regression of the target frame is more stable.
The main network flow chart of YoloV4 is shown in fig. 3, and the flow chart is as follows: inputting 416x416x3 pictures into a trunk feature extraction network cspdarknet53, performing Darknet convolution, performing a series of residual error network structures, compressing the pictures in height and width, increasing channels, continuously performing down-sampling to obtain higher semantic information, operating the last three feature layers with richer semantics in a Yolov4 network, performing maximum pooling on the result of performing three times of convolution on the feature layer at the bottom layer obtained in the cspdarknet53 by using pooling cores with different sizes by an SPP, stacking the result after the pooling, performing three times of convolution, performing feature extraction from bottom to top on the result and the second and third feature layers from the reciprocal of the trunk feature extraction network by using PANet, then performing feature extraction from top to bottom, and finally converting the extracted features into prediction results by using yolohead in the Yolov3 network.
The embodiments described in this specification are merely illustrative of implementations of the inventive concepts, which are intended for purposes of illustration only. The scope of the present invention should not be construed as being limited to the particular forms set forth in the examples, but rather as being defined by the claims and the equivalents thereof which can occur to those skilled in the art upon consideration of the present inventive concept.