CN118865426A - A method for extracting key information from airport luggage tags - Google Patents

A method for extracting key information from airport luggage tags Download PDF

Info

Publication number
CN118865426A
CN118865426A CN202410906373.9A CN202410906373A CN118865426A CN 118865426 A CN118865426 A CN 118865426A CN 202410906373 A CN202410906373 A CN 202410906373A CN 118865426 A CN118865426 A CN 118865426A
Authority
CN
China
Prior art keywords
image
text
label
key information
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410906373.9A
Other languages
Chinese (zh)
Inventor
邹驰誉
王红斌
陈霜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kunming University of Science and Technology
Original Assignee
Kunming University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kunming University of Science and Technology filed Critical Kunming University of Science and Technology
Priority to CN202410906373.9A priority Critical patent/CN118865426A/en
Publication of CN118865426A publication Critical patent/CN118865426A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/40—Document-oriented image-based pattern recognition
    • G06V30/41—Analysis of document content
    • G06V30/414—Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/146—Aligning or centring of the image pick-up or image-field
    • G06V30/1463—Orientation detection or correction, e.g. rotation of multiples of 90 degrees
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/146—Aligning or centring of the image pick-up or image-field
    • G06V30/147—Determination of region of interest
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/16—Image preprocessing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/16—Image preprocessing
    • G06V30/1607—Correcting image deformation, e.g. trapezoidal deformation caused by perspective
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/16—Image preprocessing
    • G06V30/162—Quantising the image signal
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/16—Image preprocessing
    • G06V30/164—Noise filtering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/16—Image preprocessing
    • G06V30/166—Normalisation of pattern dimensions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/18—Extraction of features or characteristics of the image
    • G06V30/1801—Detecting partial patterns, e.g. edges or contours, or configurations, e.g. loops, corners, strokes or intersections
    • G06V30/18019—Detecting partial patterns, e.g. edges or contours, or configurations, e.g. loops, corners, strokes or intersections by matching or filtering
    • G06V30/18038—Biologically-inspired filters, e.g. difference of Gaussians [DoG], Gabor filters
    • G06V30/18048—Biologically-inspired filters, e.g. difference of Gaussians [DoG], Gabor filters with interaction between the responses of different filters, e.g. cortical complex cells
    • G06V30/18057—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G06V30/19007—Matching; Proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
    • G06V30/19173—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/40—Document-oriented image-based pattern recognition
    • G06V30/41—Analysis of document content
    • G06V30/416—Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/40—Document-oriented image-based pattern recognition
    • G06V30/42—Document-oriented image-based pattern recognition based on the type of document

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a key information extraction method of airport luggage labels, and belongs to the technical field of image processing and aviation luggage management systems. The method comprises the steps of preprocessing an acquired original image of an airport luggage tag, adopting a convolutional neural network model to primarily identify the edge of the luggage tag, determining a text region image of a target tag, calculating inclination and distortion in the image through an automatic perspective correction algorithm, applying a perspective matrix to carry out geometric correction, secondarily identifying through the convolutional circular neural network, and extracting key information such as passenger names, flight numbers, luggage serial numbers and the like. The invention provides an efficient and accurate baggage tag image recognition and key information extraction method for an airport baggage handling system, thoroughly changes the traditional baggage handling flow and greatly improves the handling speed and accuracy.

Description

Method for extracting key information of airport luggage tag
Technical Field
The invention belongs to the technical field of image processing and aviation luggage management systems, and particularly relates to a key information extraction method of airport luggage labels.
Background
In modern airport baggage handling systems, ensuring quick, accurate identification of baggage tags is critical to improving efficiency and passenger satisfaction. The machine vision and deep learning technology set can remarkably reduce the luggage mishandling event caused by human errors by automatically identifying and extracting key information on the luggage label, such as passenger name, flight number and the like.
Currently, the more popular deep learning algorithms mainly include Convolutional Neural Network (CNN), recurrent Neural Network (RNN), long-short-term memory network (LSTM), and generation countermeasure network (GAN), etc. The Convolutional Neural Network (CNN) has weak time series data processing capability, cannot capture the time dependency relationship well, and requires a large amount of computing resources and time in the training process. The cyclic neural network (RNN) has the problems of gradient disappearance and gradient explosion, is difficult to train long-sequence data, has higher calculation complexity and has low training speed. Long and Short Term Memory (LSTM) has a complex structure, long training time, and high requirements for hardware resources, particularly high memory consumption. The generation of the countermeasure network (GAN) training process is unstable, and it is difficult to control the quality of the generated result, requiring a large amount of training data and computing resources.
The invention improves the accuracy of identifying the airport luggage labels by combining the Convolutional Neural Network (CNN) and the Convolutional Recurrent Neural Network (CRNN), also remarkably accelerates the processing speed and can meet the requirement of high-efficiency operation of airports.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a key information extraction method of airport luggage labels.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: a key information extraction method of airport luggage labels comprises the following steps:
(1) Acquiring an original image of an airport luggage tag, and performing size adjustment, normalization processing, gray level conversion, binarization processing and denoising processing on the original image to obtain a preprocessed image;
(2) Extracting text character features of the preprocessed image by using a multi-layer convolution network to obtain a text image, performing binarization processing on the text image to determine the boundary of a label text region in the image, and finally expanding and corroding the optimized boundary by a morphological processing method to obtain the label text region image;
(3) Extracting, classifying, deciding and correcting text direction characteristics of the tag text region image by using a multi-layer convolutional neural network, extracting target character characteristics of the corrected text region image to obtain probability distribution, and matching the target character characteristics with identifiers based on the probability distribution to obtain a target tag text region image;
(4) Transforming the target label text region image through a label text region to obtain label text region coordinates, and then dividing through a self-perspective correction algorithm to obtain each label image;
(5) And carrying out secondary identification on each label image by using a convolutional cyclic neural network, and then extracting key information.
As a preferred embodiment of the present invention, in the step (1), the original picture is resized using an interpolation method, so that the new picture size after the adjustment satisfies the formula (1):
Where I (x, y) is the pixel value ,I(x1,y1),I(x2,y1),I(x1,y2),I(x2,y2) of the new image at pixel (x, y) is the pixel value of four adjacent pixels in the original image and (x 1,y1)、(x2,y1)、(x1,y2)、(x2,y2) is the coordinates of four adjacent pixels in the original image to the new image point (x, y).
In the step (1), the image after the size adjustment is normalized by using the formula (2) to obtain a normalized image I norm;
Where I is the resized image, min (I) is the minimum value of the pixel values in the resized image, representing the darkest pixel in the image, and max (I) is the maximum value of the pixel values in the resized image, representing the brightest pixel in the image.
As a preferred embodiment of the present invention, in the step (1), the gray-scale conversion formula is as follows
Igray＝0.299×R+0.587×G+0.114×B (3)；
R, G, B are pixel values of red, green, and blue channels, respectively, and I gray is a gray-scale converted image.
As a preferred embodiment of the present invention, in the step (1), the binarization processing formula is
T is a threshold value for determining whether a pixel is white or black, I binary (x, y) is an image after binarization, 1 is that the pixel is set to white in the image after binarization, and 0 is that the pixel is set to black in the image after binarization.
In the step (1), as a preferred embodiment of the present invention, the denoising process is performed using the formula (5),
Ifiltered＝median(Ilocal) (5)；
I filtered is the denoised image, mean (I local) is the median of the local areas of the image, and I local is the local area in the image.
As a preferred embodiment of the present invention, in the steps (2) to (3), the image feature F extraction is performed using formula (6);
F＝CNN(Iprocessed) (6)；
I preessed is a preprocessed image, CNN represents a multi-layer convolution network, F is an image feature, and is a text character feature, a text direction feature or a target character feature.
As a preferred embodiment of the present invention, in the step (2), the formula for determining the boundary of the text region of the tag using the binarization technique is:
S(x，y)＝σ(Conv(F)) (7)
Wherein S (x, y) represents the probability that each pixel (x, y) belongs to the tag text region, and the probability is more than 0.6, which represents that the pixel (x, y) belongs to the tag text region; σ is a Sigmoid activation function for converting the output of the multi-layer convolutional network convolutional layer into a probability value, conv represents the convolutional operation, and F is the text character feature.
In the step (3), the method uses a multi-layer convolutional neural network to extract, classify, decide and correct the text direction characteristics of the label text region image, and specifically comprises the following steps:
s3-1: inputting the label text region image into a deep learning model, and extracting text direction characteristics in the label text region image by adopting a multi-layer convolution network;
S3-2: then using a multi-layer convolution network to make a text direction classification decision, determining a text direction through text direction feature extraction and direction classification, and outputting a classification decision result to enable a text region image to be subjected to subsequent processing in a correct direction;
s3-3: and according to the classification decision result, performing image rotation according to the angle through an OpenCV image processing library.
As a preferred embodiment of the present invention, the classification decision is formulated as
Pdirection＝softmax(FC1·F+b1) (8)；
θ＝argmax(Pdirection) (9)；
FC 1 is a full-connection layer weight matrix of the multi-layer convolution network, softmax function is a normalization function to convert output into probability distribution, F is text direction characteristics extracted from the convolution layer, b 1 is a bias vector, and P direction is probability distribution of text direction; argmax is the operation of taking the maximum value in the probability distribution, and θ is the final result of the classification decision, i.e., the rotation angle of the text region image.
As a preferred embodiment of the present invention, in the step (3), the target character feature extraction is performed on the corrected text region image according to the formula (10) to obtain a probability distribution:
P＝Softnax(Wfc·Ffinal+bfc) (10)；
Wherein F final represents the target character characteristics of the output of the last convolution layer of the multi-layer convolution neural network, W fc and b fc are the weights and offsets of the full connection layers of the multi-layer convolution neural network, and a Softmax function is used for converting the output into probability distribution; p is the target character feature probability distribution output by Softmax.
As a preferred embodiment of the present invention, the target character features are at least two of a name, a flight number, a date, a destination, a package, and an identification.
As a preferred embodiment of the present invention, in the step (3), the matching of the target character feature and the identifier is performed using formula (11):
Wherein v 1 is a target character feature vector representing a feature representation mapped by a target character feature probability distribution P, v 1 =w·p+b, W is a weight matrix for converting the target character feature probability distribution P into a feature vector, b is a bias vector for adjusting the value of the converted target character feature vector v 1; v 2 is an identifier vector, S represents a similarity score of character features and identifiers, the similarity score is above 0.7, and on matching the target character features with the identifiers, it is stated that the target character features are identifiers in airport baggage tags.
The identifier is at least two of a name, a flight number, a date, a destination, a package, and an identification.
As a preferred embodiment of the present invention, the tag text field is transformed into
Wherein datum _point is a reference point, and is a starting point of a label text region boundary in a label text region image to be transformed; datum _point is the rectangular vertex coordinates of the label text region in the label text region image of the target label to be transformed; the transformed_box is a transformed bounding box; θ is the rotation angle of the text region image; val [0] is an array containing specific measured values, wherein the specific measured values are the width and the height of the border of the text area of the tag; zoom [0] is an array of scaling parameters for adjusting the size of the frame of the image or label text region.
As a preferred embodiment of the present invention, the coordinates of the tag text region obtained based on the tag text region transformation are:
Wherein the method comprises the steps of And the coordinates of the four vertexes of the label after the label text region transformation are respectively.
As a preferred embodiment of the present invention, the self-perspective correction algorithm specifically includes:
S4-1: converting the label text region into a label text region image of a standard visual angle through perspective transformation matrixes of formulas (14) and (15) according to four vertex coordinates of the label after the label text region transformation,
sign，sinθ，cosθ＝trigonometric_function(point1，point2) (15)；
Wherein rect is a coordinate point matrix after perspective transformation; (x ', y') is coordinates after perspective transformation, (x, y) is coordinates before perspective transformation, a, b, c, d, e, f, g, h, i in the matrix are transformation parameters calculated according to image content, wherein a, b, c are parameters affecting the horizontal position and scaling of the image, d, e, f are parameters affecting the vertical position and scaling of the image; g, h, i are parameters affecting image perspective and deformation; point1, point2 is coordinates of two points for calculating an angle, sign is a flag bit, a direction representing the angle, and θ is a rotation angle of the text region image; trigonometric _function is a trigonometric function used to calculate the angle;
s4-2: then, perspective correction is carried out on the label text area image with the standard visual angle through a formula (16), each corrected label image is cut,
dst＝cv.getPerspectiveTransform(rect，maxWidth，maxHeight)
(16)；
Wherein, rect is an array containing four vertex coordinates of a label, which defines an area in the image to be subjected to perspective correction; maxWidth is the label image width after perspective correction; maxHeight is the label image height after perspective correction; dst is the perspective corrected output image, which is the label image.
As a preferred embodiment of the present invention, the extracting key information specifically includes:
s5-1: repeating step (1) based on each label image;
S5-2: then, performing secondary identification on the preprocessed image obtained in the step S5-1 by using a convolution cyclic neural network;
s5-3: and finally, extracting key information by matching the regular expression with a specific mode.
As a preferred embodiment of the present invention, the secondary identification specifically includes: firstly, extracting features from the preprocessed image obtained in the step S5-1 by using a convolution layer of a convolution cyclic neural network by using a formula (17) to obtain a feature map; the feature map includes character shapes; then the characteristic diagram is subjected to formula (18) to obtain sequence characteristic data; finally, inputting the sequence characteristic data into a transcription layer for decoding, and outputting a finally recognized text label sequence through a formula (19);
H＝Iprocessed (17)；
S＝RNN(H) (18)；
Y＝CTCDecode(S) (19)；
Wherein I processed is the preprocessed image obtained in step S5-1, and H is an advanced feature representation obtained from the multi-layer convolutional network; s is a sequence feature, is obtained after being processed by a convolutional cyclic neural network, and contains time sequence information of each part in the text; RNN represents a convolutional recurrent neural network; y is the final recognized text label sequence; CTC decoding maps sequence features to the final text sequence.
As a preferred embodiment of the present invention, the regular expression matching specific patterns is:
Info＝Regex(Y) (20)；
result＝spaCy_NER(Info) (21)；
wherein Regex represents matching key information by applying regular expressions, spaCy _NER represents named entity recognition by using spaCy, and result is a character recognition result in the picture and comprises specific information contents of names, flight numbers, dates, destinations, packages and identifiers.
Compared with the prior art, the invention has the beneficial effects that:
The invention greatly improves the operation fineness and the function definition of the whole system through image preprocessing, primary recognition, image classification, text detection, text recognition, key information extraction and the like. In the image preprocessing stage, image size processing, normalization processing, binarization processing, gray level conversion and noise removal are adopted to improve the image quality, so that a foundation is laid for subsequent recognition. Next, the objects in the image are initially classified through a multi-layer convolution network, and areas which possibly contain texts are screened out. Then, the potential text areas are subjected to image classification, and the position of the text is further determined. In the text detection stage, a convolutional recurrent neural network technology is adopted to accurately extract the text in the image, so that the accuracy of recognition is ensured. In the text recognition link, a high-performance recognition algorithm is used for converting the extracted text content into readable words for further processing. Finally, the key information extraction extracts data with important value for the airport luggage processing system aiming at the identification result.
Drawings
Fig. 1 is a flowchart of preprocessing and initial identification processing in a key information extraction method of an airport baggage tag.
Fig. 2 is a flowchart of the secondary identification in the key information extraction method of the airport baggage tag.
Fig. 3 is a flowchart of preprocessing in the key information extraction method of airport baggage tag.
Fig. 4 is a flow chart of primary text recognition.
Fig. 5 is a flow chart of image segmentation perspective transformation rectification.
Fig. 6 is a key information extraction flow chart.
Detailed Description
For a better description of the objects, technical solutions and advantages of the present invention, the present invention will be further described with reference to the following specific examples.
Example 1
A key information extraction method of airport luggage labels comprises the following steps:
(1) The image preprocessing is a key first step of an image recognition process, in this embodiment, an original image of an airport luggage tag is obtained by scanning after photographing, and the original image is sequentially subjected to size adjustment, normalization processing, gray level conversion, binarization processing and denoising processing to obtain a preprocessed image.
S1-1: the image size adjustment is to adjust an input image to a specific size so as to meet the input requirement of a deep learning model, and calculate a weighted average value of adjacent pixel points by using an interpolation method to adjust the size of the image through a formula (4), so that the image is ensured not to be distorted in the zooming process, and a new picture is obtained:
Where I (x, y) is the pixel value ,I(x1,y1),I(x2,y1),I(x1,y2),I(x2,y2) of the new image at pixel (x, y) is the pixel value of four adjacent pixels in the original image and (x 1,y1)、(x2,y1)、(x1,y2)、(x2,y2) is the coordinates of four adjacent pixels in the original image to the new image point (x, y).
S1-2: carrying out normalization processing on the image with the adjusted size by adopting a formula (2) to obtain a normalized image I norm; the normalization processing of the embodiment aims at normalizing the pixel value of the image to be within the [0,1] interval, and is realized by subtracting the minimum pixel value and dividing the minimum pixel value by the pixel value range, wherein the normalized pixel value can reduce the numerical value instability in the model training process, accelerate convergence, and is beneficial to the numerical value stability and the performance optimization when the subsequent algorithm processes the image;
Where I is the resized image, min (I) is the minimum value of the pixel values in the resized image, representing the darkest pixel in the image, and max (I) is the maximum value of the pixel values in the resized image, representing the brightest pixel in the image.
S1-3: converting the color image into a gray image by adopting a gray conversion formula (3), reducing the complexity of processing data, and simultaneously maintaining the sensitivity to text characteristics;
Igray＝0.299×R+0.587×G+0.114×B (3)；
R, G, B are pixel values of red, green and blue channels in the normalized image, respectively, and I gray is a gray-scale converted image.
S1-4: in the binarization processing process, converting the image after gray level conversion from gray level state to an image only containing black and white two colors, and setting a global threshold value through a formula (4);
t is a threshold value for determining whether a pixel is white or black, I binary (x, y) is an image after binarization, 1 is that the pixel is set to white in the image after binarization, and 0 is that the pixel is set to black in the image after binarization.
S1-5: the noise removal removes random noise points in the image by applying a Gaussian filter or a median filter, so that the quality of the image can be improved, the text edge is clearer, and the subsequent recognition processing is facilitated. In the embodiment, a median filter is adopted, and denoising treatment is carried out through a formula (5), so that salt and pepper noise can be effectively removed;
Ifiltered＝median(Ilocal) (5)；
I filtered is the denoised image, mean (I local) is the median of the local areas of the image, and I local is the local area in the image.
(2) In order to screen out images containing text character features, the invention firstly uses a multilayer convolution network to extract text character features from the preprocessed images and reject images without text character features to obtain text images containing text characters, then uses a binarization technology to determine the boundary of a label text region in the text images, finally uses a morphological processing method to expand and corrode the optimized boundary to obtain the label text region images, and realizes accurate detection and positioning of the text regions, and the method specifically comprises the following steps:
S2-1: in the text character feature extraction stage, a deep learning model (DB model) adopts a multi-layer convolution network to extract text character feature from an input preprocessed image, so as to lay a foundation for accurate text positioning. Through this step, the DB model can extract useful information about the text from the image, thereby providing powerful support for subsequent text detection.
In the step S2-1, extracting text character features by adopting a formula (9);
F＝CNN(Iprocessed) (6)；
I processed is the preprocessed image, CNN represents the multi-layer convolutional network, and F is the text character feature.
S2-2: the DB model processes text images using a differentiable binarization layer. The binarization layer dynamically adjusts whether each pixel point in the text image belongs to a text region or not by using the threshold value obtained through learning, and the process is trainable, so that the DB model can adapt to different text scenes. The binarization layer uses a binarization technique to determine the boundary of the label text region as:
S(x，y)＝σ(Conv(F)) (7)
Where S (x, y) represents the probability that each pixel (x, y) belongs to the tag text region, σ is a Sigmoid activation function for converting the output of the multi-layer convolutional network convolutional layer into a probability value, conv represents the convolutional operation, and F is the text character feature. When S (x, y) is 0.6 or more, then the corresponding pixel belongs to the label text region.
S2-3: the DB model is particularly concerned with boundary optimization of text. And through post-processing steps, such as threshold adjustment, minimum region connection and other technologies, the accuracy of the text boundary is further improved, and a label text region image is obtained. This step is significant for eliminating false detection and improving text detection accuracy.
(3) Extracting target character features of text characters of the tag text region image by using a multi-layer convolutional neural network, and then matching the target character features with identifiers to obtain the target tag text region image, wherein the method specifically comprises the following steps of:
s3-1: inputting the label text region image into a deep learning model, and extracting text direction characteristics in the label text region image by adopting a multi-layer convolution network;
The extraction formula of the target character features in the label text region image is as follows:
F＝CNN(Iprocessed) (6)；
I processed is the preprocessed image, CNN represents the multi-layer convolutional network, and F is the target character feature.
S3-2: then using a multi-layer convolution network to make a text direction classification decision, determining a text direction through text direction feature extraction and direction classification, outputting a classification decision result, enabling a text region image to be subjected to subsequent processing in a correct direction,
The classification decision formula is
Pdirection＝softmax(FC1·F+b1) (8)
θ＝arg max(Pdirection) (9)
FC 1 is a full-connection layer weight matrix of the multi-layer convolution network, softmax function is a normalization function to convert output into probability distribution, F is text direction characteristics extracted from the convolution layer, b 1 is a bias vector, and P direction is probability distribution of text direction; argmax is an operation of taking the maximum value in the probability distribution, and θ is the result of classification decision, that is, the rotation angle of the text region image.
S3-3: and according to the classification decision result (rotation angle), performing corresponding image rotation according to the angle through an image processing library (OpenCV).
The formula for extracting the target character characteristics in the label text region image by adopting the multilayer convolutional neural network is as follows:
F＝CNN(Iprocessed) (6)；
I processed is the preprocessed image, CNN represents the multi-layer convolutional network, and F is the target character feature.
P＝Softmax(Wfc·Ffinal+bfc) (10)；
Wherein F final is the characteristic of the output of the last convolutional layer of the multi-layer convolutional neural network, W fc and b fc are the weights and offsets of the full-connection layers of the multi-layer convolutional neural network, and a Softmax function is used for converting the output into probability distribution; the target character features are at least two of name, flight number, date, destination, package and identification; p is the target character feature probability distribution output by Softmax;
s3-2: character features are matched with the identifiers by adopting a formula (11):
Wherein v 1 is a target character feature vector representing a feature representation mapped by a target character feature probability distribution P, v 1 =w·p+b, W is a weight matrix for converting the target character feature probability distribution P into a feature vector, b is a bias vector for adjusting the value of the converted target character feature vector v 1; v 2 is an identifier vector, S represents a similarity score of the character feature and the identifier, and when S is above 0.7, the target character feature is described as the target character of the airport luggage tag of the invention on the matching of the target character feature and the identifier. The identifier is at least two of a name, a flight number, a date, a destination, a package, and an identification.
(4) Transforming the target label text region image through a label text region to obtain label text region coordinates, and then dividing through a self-perspective correction algorithm to obtain each label image, wherein the method specifically comprises the following steps of:
s4-1: performing label text region transformation on the target label text region image by adopting a formula (12) to calculate label text region coordinates, converting the image from an original form to a standard form,
Wherein datum _point is a reference point, and is a starting point of a label text region boundary in a label text region image to be transformed; datum _point is the rectangular vertex coordinates of the label text region in the label text region image of the target label to be transformed; the transformed_box is a transformed bounding box; θ is the rotation angle of the text region image; val [0] is an array containing specific measured values, wherein the specific measured values are the width and the height of the border of the text area of the tag; zoom [0] is an array of scaling parameters for adjusting the size of the frame of the image or label text region.
S4-2: coordinates in the tag text region are established to facilitate subsequent text analysis and processing, enabling more accurate localization to specific locations in the text. The tag text region coordinates include four coordinate values, up, down, left, right, as in equation (13), which uniquely determine the location of a text region in the coordinate system.
Wherein the method comprises the steps ofAnd the coordinates of the four vertexes of the label after the label text region transformation are respectively.
S4-3: calculating a perspective transformation matrix according to the four vertex coordinates of the label after the transformation of the text region of the label through formulas (14) and (15), the matrix transforming the text region from the current viewing angle to a standard perspective transformation matrix which is more suitable for the recognition,
sign，sinθ，cosθ＝trigonometric_function(point1，point2) (15)；
Wherein rect is a coordinate point matrix after perspective transformation; (x ', y') is coordinates after perspective transformation, (x, y) is coordinates before perspective transformation, a, b, c, d, e, f, g, h, i in the matrix are transformation parameters calculated according to image content, wherein a, b, c are parameters affecting the horizontal position and scaling of the image, d, e, f are parameters affecting the vertical position and scaling of the image; g, h, i are parameters affecting image perspective and deformation; point1, point2 is coordinates of two points for calculating an angle, sign is a flag bit, a direction representing the angle, and θ is a rotation angle of the text region image; trigonometric _function is a trigonometric function used to calculate the angle.
S4-2: then, perspective correction is carried out on the label text area image with the standard visual angle through a formula (16), each corrected label image is cut,
dst＝cv.getPerspectiveeTransform(rect，maxWidth，maxHeight)
(16)；
Wherein, rect is an array containing four vertex coordinates of a label, which defines an area in the image to be subjected to perspective correction; maxWidth is the label image width after perspective correction; maxHeight is the label image height after perspective correction; dst is the perspective corrected output image, which is the label image.
(5) Performing secondary identification on each label image by using a convolutional cyclic neural network, and then extracting key information, wherein the method specifically comprises the following steps:
s5-1: repeating step (1) based on each label image;
S5-2: then, performing secondary identification on the preprocessed image obtained in the step S5-1 by using a convolution cyclic neural network;
firstly, extracting features from the preprocessed image obtained in the step S5-1 by using a convolution layer of a convolution cyclic neural network by using a formula (17) to obtain a feature map; the feature map contains character shapes and positions; then the characteristic diagram is subjected to formula (18) to obtain sequence characteristic data; finally, inputting the sequence characteristic data into a transcription layer for decoding, and outputting a finally recognized text label sequence through a formula (19);
H＝Iprocessed (17)；
S＝RNN(H) (18)；
Y＝CTCDecode(S) (19)；
Wherein I processed is the preprocessed image obtained in step S5-1, and H is an advanced feature representation obtained from the multi-layer convolutional network; s is a sequence feature, is obtained after being processed by a convolutional cyclic neural network, and contains time sequence information of each part in the text; RNN represents a convolutional recurrent neural network; y is the final recognized text sequence; CTC decoding maps sequence features to the final text sequence.
S5-3: matching the finally identified text label sequence (shown in fig. 6) with a specific pattern through a regular expression to extract key information;
the regular expression matches a specific pattern as:
Info＝Regex(Y) (20)；
result＝spaCy_NER(Info) (21)；
wherein Regex represents matching key information by applying regular expressions, spaCy _NER represents named entity recognition by using spaCy, and result is a character recognition result in the picture and comprises specific information contents of names, flight numbers, dates, destinations, packages and identifiers.
First, regular expressions are used to quickly scan a sequence of characters, identify and extract structured data, such as flight numbers, dates, passenger names, and the like. Since the information on the luggage tag has a predefined format, the regular expression can effectively identify these formatted text fragments by defining a series of character matching rules;
Then, spaCy firstly carries out word segmentation and part-of-speech tagging on the text, then uses a built-in entity recognition model to determine that named entities in the text process the text which is preliminarily extracted by the regular expression, and executes deeper semantic analysis and entity recognition tasks;
Finally, by completing all the steps as shown in fig. 6, the segmentation of the airport luggage sorting table and the extraction of the image key information are finally realized, and the method effectively automates the luggage processing process and improves the precision and the efficiency of data processing. After the method of the present invention is implemented, information extracted from the baggage tag image may be presented in a structured manner in table 1. Table 1 contains key information extracted from each baggage tag such as passenger name, flight number, destination, etc. This information, which has been precisely identified and extracted, can now be used directly in the baggage sorting system of the airport.
TABLE 1
The invention optimizes the quality of the original image by utilizing image preprocessing technologies such as dynamic threshold adjustment, local contrast enhancement and the like, and provides a clear image foundation for subsequent text recognition. The application of perspective transformation solves the problem of image distortion caused by shooting angles, and ensures accurate identification of texts. In the primary recognition stage, a Convolutional Neural Network (CNN) is used for effective text region detection and direction determination, and the step is a precondition for key information extraction. In the secondary recognition stage, a convolutional cyclic neural network (CRNN) is adopted for deep text recognition, and the CRNN combines the characteristic extraction capability of the convolutional neural network and the sequence processing advantage of the cyclic neural network, so that high-precision text recognition is provided. Therefore, by automatically identifying and extracting key information on the luggage label, such as passenger name, flight number and the like, the invention obviously reduces the luggage mishandling event caused by human error, the accuracy of automatically identifying and extracting the key information on the luggage label is more than 93.75%, which is higher than the accuracy of extracting the key information of the luggage label by adopting a Convolutional Neural Network (CNN) in the primary identification and the secondary identification stages, and simultaneously reduces the computational complexity and quickens the handling capacity.
Finally, it should be noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the scope of the present invention, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention may be modified or substituted equally without departing from the spirit and scope of the technical solution of the present invention.

Claims (10)

1.一种机场行李标签的关键信息提取方法，其特征在于，包括如下步骤：1. A method for extracting key information from an airport luggage tag, comprising the following steps: (1)获取机场行李标签的原始图像，将原始图像进行尺寸调整、归一化处理、灰度转换、二值化处理以及去噪声处理得到预处理后的图像；(1) Obtaining the original image of the airport baggage tag, resizing, normalizing, grayscale converting, binarizing, and denoising the original image to obtain a preprocessed image; (2)使用多层卷积网络对预处理后的图像进行文本字符特征提取得到文本图像，然后将文本图像进行二值化处理，确定图像中标签文本区域的边界，最后通过形态学处理方法膨胀和腐蚀优化边界，得到标签文本区域图像；(2) Using a multi-layer convolutional network to extract text character features from the preprocessed image to obtain a text image, then binarizing the text image to determine the boundary of the label text area in the image, and finally dilating and corroding the boundary through morphological processing methods to obtain the label text area image; (3)使用多层卷积神经网络对标签文本区域图像的文本方向特征进行提取、分类决策和校正，然后对校正后的文本区域图像进行目标字符特征提取得到概率分布，基于概率分布将目标字符特征与标识符进行匹配，得到目标标签文本区域图像；(3) Using a multi-layer convolutional neural network to extract, classify and correct the text direction features of the label text area image, then extract the target character features from the corrected text area image to obtain a probability distribution, and match the target character features with the identifier based on the probability distribution to obtain a target label text area image; (4)将目标标签文本区域图像通过标签文本区域变换得到标签文本区域坐标，然后通过自透视矫正算法进行分割，得到各标签图像；(4) The target label text region image is transformed into the label text region coordinates, and then segmented using a self-perspective correction algorithm to obtain each label image; (5)将各标签图像使用卷积循环神经网络进行二次识别，然后提取关键信息。(5) Each label image is re-recognized using a convolutional recurrent neural network, and then the key information is extracted. 2.权利要求1所述机场行李标签的关键信息提取方法，其特征在于，所述步骤(1)中，使用插值方法进行原始图片的尺寸调整，使调整后的新图片尺寸满足公式(4)：2. The method for extracting key information from an airport baggage tag according to claim 1, characterized in that, in step (1), an interpolation method is used to adjust the size of the original image so that the size of the adjusted new image satisfies formula (4): 其中I(x，y)是新图像在像素点(x，y)的像素值，是原始图像中相邻四个像素点的像素值，(x1，y1)、(x2，y1)、(x1，y2)、(x2，y2)是原始图像中与新图像点(x，y)相邻的四个像素点的坐标；Where I(x, y) is the pixel value of the new image at pixel (x, y), are the pixel values of four adjacent pixels in the original image, (x 1 , y 1 ), (x 2 , y 1 ), (x 1 , y 2 ), (x 2 , y 2 ) are the coordinates of the four pixels in the original image adjacent to the new image point (x, y); 采用公式(2)对尺寸调整后的图像进行归一化处理，得到归一化后的图像Inorm；The resized image is normalized using formula (2) to obtain a normalized image I norm ; 其中，是尺寸调整后的图像，mmin(I)是尺寸调整后的图像中像素值的最小值，表示图像中最暗的像素，ma((I)是尺寸调整后的图像中像素值的最大值，表示图像中最亮的像素；Where, is the resized image, mmin(I) is the minimum pixel value in the resized image, indicating the darkest pixel in the image, and ma((I) is the maximum pixel value in the resized image, indicating the brightest pixel in the image; 灰度转换公式为The grayscale conversion formula is: Igray＝0.299×R+0.587×G+0.114×B (3)；I gray =0.299×R+0.587×G+0.114×B (3); R，G，B分别是红色、绿色和蓝色通道的像素值，Igray为灰度转换后的图像；R, G, B are the pixel values of the red, green and blue channels respectively, and I gray is the image after grayscale conversion; 二值化处理公式为 The binarization formula is: T是阈值，用于确定像素点是变为白色或黑色，Ibinary(x，y)为二值化处理后的图像，1为像素点在二值化处理后的图像中被设为白色，0为像素点在二值化处理后的图像中被设为黑色，I(x，y)是像素点(x，y)的像素值；T is a threshold value used to determine whether a pixel point becomes white or black, I binary (x, y) is the image after binary processing, 1 means that the pixel point is set to white in the image after binary processing, 0 means that the pixel point is set to black in the image after binary processing, and I (x, y) is the pixel value of the pixel point (x, y); 采用公式(5)进行去噪声处理，Formula (5) is used for denoising: Ifiltered＝median(Ilocal) (5)；I filtered =median(I local ) (5); Ifiltered为去噪声处理后的图像，meedian(Ilocal)为图像局部区域的中值，Ilocal为图像中的局部区域。I filtered is the image after denoising, meedian (I local ) is the median of the local area of the image, and I local is the local area in the image. 3.如权利要求1所述机场行李标签的关键信息提取方法，其特征在于，所述步骤(2)-(3)中，采用公式(6)进行图像特征F提取；3. The key information extraction method of airport baggage tags as claimed in claim 1, characterized in that in the steps (2)-(3), formula (6) is used to extract the image feature F; F＝CNN(Iprocessed) (6)；F = CNN (I processed ) (6); Iprocessed是预处理后的图像，CNN代表多层卷积网络，F是图像特征，为文本字符特征、文本方向特征或目标字符特征。I processed is the preprocessed image, CNN stands for multi-layer convolutional network, and F is the image feature, which can be text character feature, text direction feature or target character feature. 4.如权利要求3所述机场行李标签的关键信息提取方法，其特征在于，所述步骤(2)中，使用二值化技术确定标签文本区域的边界的公式为：4. The method for extracting key information from an airport baggage tag according to claim 3, wherein in step (2), the formula for determining the boundary of the tag text area using binarization technology is: S(x，y)＝σ(Conv(F)) (7)S(x, y) = σ(Conv(F)) (7) 式中，S(x，y)表示每个像素点(x，y)属于标签文本区域的概率，概率为0.6以上表示像素点(x，y)属于标签文本区域；σ是Sigmoid激活函数，用于将多层卷积网络卷积层的输出转换成概率值，Conv表示卷积操作，F为文本字符特征。Where S(x, y) represents the probability that each pixel (x, y) belongs to the labeled text area. A probability of 0.6 or above indicates that the pixel (x, y) belongs to the labeled text area. σ is the Sigmoid activation function, which is used to convert the output of the convolutional layer of the multi-layer convolutional network into a probability value. Conv represents the convolution operation, and F is the text character feature. 5.如权利要求3所述机场行李标签的关键信息提取方法，其特征在于，所述步骤(3)中，使用多层卷积网络对文本区域图像的方向进行分类决策，输出分类决策的结果，分类决策的公式为5. The key information extraction method of airport baggage tags as claimed in claim 3 is characterized in that in the step (3), a multi-layer convolutional network is used to classify and decide the direction of the text area image, and the classification decision result is output, and the classification decision formula is: Pdirection＝softmax(FC1·F+b1) (8)P direction =softmax(FC 1 ·F+b 1 ) (8) θ＝argmax(Pdirection) (9)θ＝argmax(P direction ) (9) FC1为多层卷积网络的全连接层权重矩阵，Softmax函数为归一化函数将输出转化为概率分布，F为从卷积层提取的文本方向特征，b1为偏置向量，Pdirection为文本方向的概率分布；argmax是取概率分布中最大值的操作，θ是分类决策的结果，即文本区域图像的旋转角度。FC 1 is the weight matrix of the fully connected layer of the multi-layer convolutional network, the Softmax function is a normalization function that converts the output into a probability distribution, F is the text direction feature extracted from the convolutional layer, b 1 is the bias vector, P direction is the probability distribution of the text direction; argmax is the operation of taking the maximum value in the probability distribution, and θ is the result of the classification decision, that is, the rotation angle of the text area image. 6.如权利要求1所述机场行李标签的关键信息提取方法，其特征在于，所述步骤(3)中，根据公式(10)对校正后的文本区域图像进行目标字符特征提取得到概率分布：6. The method for extracting key information from an airport baggage tag according to claim 1, wherein in step (3), the target character features are extracted from the corrected text region image according to formula (10) to obtain a probability distribution: P＝Softmax(Wfc·Ffinal+bfc) (10)；P=Softmax(W fc ·F final +b fc ) (10); 其中，Ffinal是多层卷积神经网络最后一个卷积层输出的目标字符特征，Wfc和bfc是多层卷积神经网络全连接层的权重和偏置，Softmax函数用于将输出转换为概率分布；所述目标字符特征为姓名、航班号、日期、目的地、包裹和标识中的至少两种；P为Softmax输出的目标字符特征概率分布；Wherein, F final is the target character feature output by the last convolution layer of the multi-layer convolutional neural network, W fc and b fc are the weights and biases of the fully connected layer of the multi-layer convolutional neural network, and the Softmax function is used to convert the output into a probability distribution; the target character feature is at least two of the name, flight number, date, destination, package and logo; P is the probability distribution of the target character feature output by Softmax; 采用公式(11)进行目标字符特征与标识符进行匹配：Formula (11) is used to match the target character features with the identifier: 其中，v1是目标字符特征向量，表示通过目标字符特征概率分布P映射后的特征表示，υ1＝W·P+b，W是权重矩阵，用于将目标字符特征概率分布P转换为特征向量，b是偏置向量，用于调整转换后的目标字符特征向量v1的值；v2是标识符向量，S表示字符特征和标识符的相似度分数；所述标识符为姓名、航班号、日期、目的地、包裹和标识中的至少两种。Among them, v1 is the target character feature vector, which represents the feature representation after mapping through the target character feature probability distribution P, υ1 =W·P+b, W is the weight matrix, which is used to convert the target character feature probability distribution P into a feature vector, b is the bias vector, which is used to adjust the value of the converted target character feature vector v1 ; v2 is the identifier vector, S represents the similarity score between the character feature and the identifier; the identifier is at least two of the name, flight number, date, destination, package and logo. 7.如权利要求1所述机场行李标签的关键信息提取方法，其特征在于，所述标签文本区域变换为7. The key information extraction method of airport luggage tags as claimed in claim 1, characterized in that the tag text area is transformed into 式中datum_point是基准点，是需要变换的目标标签文本区域图像中标签文本区域边界的起始点；datum_point是需要变换的目标标签文本区域图像中标签文本区域矩形顶点坐标；transformed_box是变换后的边界框；θ是文本区域图像的旋转角度；val[0]是包含特定测量值的数组，特定测量值为标签文本区域边框宽度和高度；zoom[0]是缩放参数数组，用于调整图像或标签文本区域边框的大小；Where datum_point is the reference point, which is the starting point of the label text area boundary in the target label text area image to be transformed; datum_point is the coordinates of the vertices of the label text area rectangle in the target label text area image to be transformed; transformed_box is the transformed bounding box; θ is the rotation angle of the text area image; val[0] is an array containing specific measurement values, which are the width and height of the label text area border; zoom[0] is an array of zoom parameters used to adjust the size of the image or label text area border; 所述标签文本区域坐标为：The label text area coordinates are: 其中分别为标签文本区域变换后的标签四个顶点坐标。in They are the coordinates of the four vertices of the label after the label text area is transformed. 8.如权利要求7所述机场行李标签的关键信息提取方法，其特征在于，所述自透视矫正算法，具体包括：8. The method for extracting key information from an airport baggage tag according to claim 7, wherein the self-perspective correction algorithm specifically comprises: S4-1：根据标签文本区域变换后的标签四个顶点坐标，通过公式(14)和(15)的透视变换矩阵，将标签文本区域变换后的图像转换为标准视角的标签文本区域图像，S4-1: According to the coordinates of the four vertices of the label after the transformation of the label text area, the transformed image of the label text area is converted into the label text area image of the standard perspective through the perspective transformation matrix of formulas (14) and (15). sign，sinθ，cosθ＝trigonometric_function(point1，point2) (15)；sign, sinθ, cosθ=trigonometric_function(point1, point2) (15); 其中，rect是透视变换后的坐标点矩阵；(x′，y′)是透视变换后的坐标，(x，y)是透视变换前的坐标；矩阵中的a，b，c，d，e，f，g，h，i是根据图像内容计算得到的变换参数，其中，a，b，c为影响图像水平位置和缩放的参数，d，e，f为影响图像垂直位置和缩放的参数；g，h，i为影响图像透视和变形的参数；point1，point2是用于计算角度的两个点的坐标，sign是标志位，表示角度的方向，θ为文本区域图像的旋转角度；trigonometric_function为用于计算角度的三角函数；Wherein, rect is the coordinate point matrix after perspective transformation; (x′, y′) is the coordinate after perspective transformation, (x, y) is the coordinate before perspective transformation; a, b, c, d, e, f, g, h, i in the matrix are transformation parameters calculated according to the image content, where a, b, c are parameters affecting the horizontal position and scaling of the image, d, e, f are parameters affecting the vertical position and scaling of the image; g, h, i are parameters affecting the perspective and deformation of the image; point1, point2 are the coordinates of the two points used to calculate the angle, sign is the flag bit, indicating the direction of the angle, θ is the rotation angle of the text area image; trigonometric_function is the trigonometric function used to calculate the angle; S4-2：然后通过公式(16)对标准视角的标签文本区域图像进行透视矫正，切割出矫正后的各标签图像，S4-2: Then, the perspective correction is performed on the label text area image of the standard viewing angle by using formula (16), and the corrected label images are cut out. dst＝cv.getPerspectiveTransform(rect，maxWidth，maxHeight) (16)；dst=cv.getPerspectiveTransform(rect, maxWidth, maxHeight) (16); 其中，式中maxWidth是透视矫正后的标签图像宽度；maxHeight是透视矫正后的标签图像高度；dst是透视矫正后的输出图像，为标签图像。Among them, maxWidth is the width of the label image after perspective correction; maxHeight is the height of the label image after perspective correction; dst is the output image after perspective correction, which is the label image. 9.如权利要求1所述机场行李标签的关键信息提取方法，其特征在于，所述提取关键信息具体包括：9. The method for extracting key information from an airport luggage tag according to claim 1, wherein the extracting key information specifically comprises: S5-1：基于各标签图像，重复步骤(1)；S5-1: Repeat step (1) based on each label image; S5-2：然后将S5-1得到的预处理后的图像使用卷积循环神经网络进行二次识别；S5-2: Then the preprocessed image obtained in S5-1 is subjected to secondary recognition using a convolutional recurrent neural network; S5-3：最后通过正则表达式匹配特定模式提取关键信息。S5-3: Finally, key information is extracted by matching specific patterns with regular expressions. 10.如权利要求9所述机场行李标签的关键信息提取方法，其特征在于，所述二次识别具体包括：首先，使用公式(17)利用卷积循环神经网络的卷积层从步骤S5-1得到的预处理后的图像中提取文本字符特征，得到特征图；然后将特征图通过公式(18)得到序列特征数据；最后将序列特征数据输入转录层来解码，通过公式(19)，输出最终识别的文本标签序列；10. The method for extracting key information of airport baggage tags as claimed in claim 9, characterized in that the secondary recognition specifically comprises: first, using formula (17) to extract text character features from the preprocessed image obtained in step S5-1 using the convolutional layer of the convolutional recurrent neural network to obtain a feature map; then using the feature map to obtain sequence feature data through formula (18); finally, inputting the sequence feature data into the transcription layer for decoding, and outputting the final recognized text label sequence through formula (19); H＝Iprocessed (17)；H = I processed (17); S＝RNN(H) (18)；S = RNN(H) (18); Y＝CTCDecode(S) (19)；Y = CTCDecode(S) (19); 其中，Iprocessed是步骤S5-1得到的预处理后的图像，H是从多层卷积网络得到的高级特征表示；S是序列特征，由卷积循环神经网络处理后得到，包含了文本中每个部分的时间序列信息；RNN表示卷积循环神经网络；Y是最终识别的文本标签序列；CTC解码将序列特征映射到最终的文本序列；Among them, I processed is the preprocessed image obtained in step S5-1, H is the high-level feature representation obtained from the multi-layer convolutional network; S is the sequence feature, which is obtained after processing by the convolutional recurrent neural network and contains the time series information of each part of the text; RNN represents the convolutional recurrent neural network; Y is the final recognized text label sequence; CTC decoding maps the sequence features to the final text sequence; 所述正则表达式匹配特定模式为：The regular expression matches a specific pattern: Info＝Regex(Y) (20)；Info = Regex(Y) (20); result＝spaCy_NER(Info) (21)；result=spaCy_NER(Info) (21); 其中，Regex代表应用正则表达式匹配关键信息，spaCy_NER代表使用spaCy进行命名实体识别，result为图片中文字识别结果，包括姓名、航班号、日期、目的地、包裹、标识的具体信息内容，Info是通过正则表达式Regex(Y)从文本标签序列Y中提取出的关键信息。Among them, Regex represents the application of regular expressions to match key information, spaCy_NER represents the use of spaCy for named entity recognition, result is the result of text recognition in the image, including the specific information content of name, flight number, date, destination, package, and logo, and Info is the key information extracted from the text label sequence Y through the regular expression Regex(Y).
CN202410906373.9A 2024-07-08 2024-07-08 A method for extracting key information from airport luggage tags Pending CN118865426A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410906373.9A CN118865426A (en) 2024-07-08 2024-07-08 A method for extracting key information from airport luggage tags

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410906373.9A CN118865426A (en) 2024-07-08 2024-07-08 A method for extracting key information from airport luggage tags

Publications (1)

Publication Number Publication Date
CN118865426A true CN118865426A (en) 2024-10-29

Family

ID=93161241

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410906373.9A Pending CN118865426A (en) 2024-07-08 2024-07-08 A method for extracting key information from airport luggage tags

Country Status (1)

Country Link
CN (1) CN118865426A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119494598A (en) * 2025-01-20 2025-02-21 厦门民航凯亚有限公司 A management planning method for airport baggage
CN119649358A (en) * 2025-02-17 2025-03-18 合肥晶合集成电路股份有限公司 An intelligent wafer packaging identification method, system and device

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119494598A (en) * 2025-01-20 2025-02-21 厦门民航凯亚有限公司 A management planning method for airport baggage
CN119649358A (en) * 2025-02-17 2025-03-18 合肥晶合集成电路股份有限公司 An intelligent wafer packaging identification method, system and device
CN119649358B (en) * 2025-02-17 2025-06-17 合肥晶合集成电路股份有限公司 Intelligent wafer package identification method, system and equipment

Similar Documents

Publication Publication Date Title
CN112686812B (en) Bank card tilt correction detection method, device, readable storage medium and terminal
CN115331245B (en) Table structure identification method based on image instance segmentation
CN109598241B (en) Recognition method of ships at sea based on satellite imagery based on Faster R-CNN
CN112307919B (en) Improved YOLOv 3-based digital information area identification method in document image
CN110543837A (en) A Visible Light Airport Aircraft Detection Method Based on Potential Target Points
CN112085024A (en) A method for character recognition on the surface of a tank
CN109948566B (en) Double-flow face anti-fraud detection method based on weight fusion and feature selection
CN111626292B (en) A text recognition method for building indication signs based on deep learning technology
CN106610969A (en) Multimodal information-based video content auditing system and method
CN118865426A (en) A method for extracting key information from airport luggage tags
CN111680690A (en) Character recognition method and device
CN111027538A (en) Container detection method based on instance segmentation model
CN115359562B (en) A Sign Language Spelling Recognition Method Based on Convolutional Neural Networks
CN113158977A (en) Image character editing method for improving FANnet generation network
CN115063802A (en) PSENet-based circular seal identification method, device and medium
CN112418210B (en) Intelligent classification method for tower inspection information
CN115393748A (en) Method for detecting infringement trademark based on Logo recognition
CN114359538A (en) Water meter reading positioning and identifying method
CN112101283A (en) Intelligent identification method and system for traffic signs
CN116740572A (en) A maritime ship target detection method and system based on improved YOLOX
CN119152502A (en) Landscape plant image semantic segmentation method based on weak supervision
CN108022245A (en) Photovoltaic panel template automatic generation method based on upper thread primitive correlation model
CN112686265A (en) Hierarchic contour extraction-based pictograph segmentation method
CN116596838B (en) A feature-aware component surface defect detection method
CN115035390B (en) Aerial image detection method based on GAN and feature enhancement

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination