CN110796032A - Video fence and early warning method based on human posture assessment - Google Patents

Abstract

The present invention relates to a video fence and an early warning method based on human body posture assessment. The early warning method includes the following steps: S1, setting a video shooting range, and delimiting a forbidden area within the shooting range; S2, acquiring persons within the shooting range The coordinate information of the ankle key point and its confidence level; S3, determine whether the obtained ankle coordinates are in the forbidden area, if it is found that there is an ankle key point in the forbidden area, go to step S4; S4, send an early warning signal. The video fence based on human posture assessment can locate the coordinates of the person's ankle through machine vision recognition technology, more accurately determine whether the person has entered the designated area in the video, and can recognize the person when a small part other than the human foot is blocked. Whether to enter the designated forbidden area.

Description

Video fence and early warning method based on human posture assessment

technical field

The invention relates to the technical field of machine vision recognition, and relates to a video fence and an early warning method based on human body posture assessment.

Background technique

Nowadays, security cameras are widely used, especially in scenes such as construction sites where the site environment is complex and there are many people.

There will be some areas in the construction site. Due to the stacking of dangerous objects, or accidents are prone to occur in the area, the identity of the entering personnel is limited or no one is allowed to enter. This requires security surveillance cameras to monitor a specific area 24 hours a day and independently determine whether someone invades the area.

Most of the existing video recognition technologies use a rectangular frame to select the position of a person in a two-dimensional space.

The traditional method identifies the position of the entire human body on the image, and selects it with a rectangular frame on the image. Since the image reflects a three-dimensional space on a two-dimensional plane, the portrait overlaps with the frame selection area in the figure. When the upper body overlaps with the designated area, it will be misjudged that someone enters the area, but the person's feet are not actually standing in the designated area.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide a video fence and an early warning method based on human body posture evaluation, aiming at the above-mentioned inaccurate judgment defect of the prior art.

The technical solution adopted by the present invention to solve the technical problem is: constructing a video fence early warning method based on human body posture assessment, comprising the following steps:

S1. Set a video shooting range, and delimit a forbidden area within the shooting range;

S2. Obtain the coordinate information and the confidence level of the key points of the ankle of the person within the shooting range;

S3, determine whether the obtained ankle coordinates are in the forbidden area, and if it is found that there is a key point of the ankle in the forbidden area, go to step S4;

S4. Send an early warning signal.

Preferably, in the step S2, the convolutional neural network based on deep learning obtains the information of each frame of the video, and combines the heat map analysis and the affinity analysis to obtain the coordinate information and confidence of the ankle key points.

Preferably, in the step S2, if there is a person in the shooting range, the coordinate information of all the key points of human skeleton joints in the image on the two-dimensional image corresponding to the shooting range and the confidence of the key points of the skeleton joints can be obtained degree, select the coordinate information of the key points of the human ankle and its confidence.

Preferably, in the step S1, the forbidden area is delineated on the screen obtained from the shooting range, including the following steps:

Extract the image frame, manually select at least one polygon frame in the image and set it as a forbidden area, and set the vertex coordinates of each polygon frame as a set P={[[x1,y1],[x2,y2],.. .,[xn,yn]],[[x'1,y'1],[x'2,y'2],...,[x'n,y'n]],...} Form storage, where x and y are the horizontal and vertical coordinates of the polygon vertices respectively, each polygon frame is composed of n vertices, and the vertices are connected in sequence in the order of the set to form a polygon frame, and the inside of the frame is the forbidden area.

Preferably, in the step S3, the key point of the ankle to be judged is taken as the endpoint, and a ray is emitted along the horizontal direction. If the number of intersections between the ray and a polygonal frame is an even number, the key point of the ankle to be judged is in the forbidden area. In addition, if the number of intersections between the ray and a polygon frame is odd, the ankle key point to be determined is in the forbidden area.

Preferably, the step S4 further includes uploading the warning information to the cloud, and displaying the real-time image of the shooting range.

A video fence based on human pose assessment, including:

an area delimitation module, used for delimiting a forbidden area within the video shooting range according to the video shooting range;

The identification module is used to obtain the coordinate information and confidence level of the key points of the ankle of the person within the shooting range;

A judgment module for judging whether the obtained ankle coordinates are within the forbidden area, and

An early warning module is used to send an early warning signal when the judgment module finds that a key point of the ankle is in the forbidden area.

Preferably, the identification module obtains the information of each frame of the video based on a deep learning convolutional neural network, and obtains the coordinate information and confidence of the ankle key points in combination with heat map analysis and affinity analysis.

Preferably, the area delineation module extracts image frames from the captured video, manually selects at least one polygonal frame in the image and sets it as a forbidden area, and generates coordinate information of the forbidden area;

The judging module takes the key point of the ankle to be judged as the endpoint, and sends out a ray along the horizontal direction. If the number of intersections between the ray and a polygonal frame is an even number, the key point of the ankle to be judged is outside the forbidden area. The number of intersection points of a polygon frame is an odd number, and the ankle key point to be determined is within the forbidden area.

Preferably, when the judgment module finds that the key point of the ankle is in the forbidden area and stays for more than a certain time, the early warning module sends the coordinate information of the key point of the human ankle on the image, and the key point of the ankle appears in the forbidden area time, and the image taken when the key point of the ankle appears in the forbidden area is sent to the cloud, and an early warning signal is issued.

Implementing the video fence and early warning method based on human body posture assessment of the present invention has the following beneficial effects: the video fence based on human body posture assessment can locate the coordinates of the person's ankle through machine vision recognition technology, and more accurately judge whether the person has entered the video. The designated area in the device can identify whether a person enters the designated forbidden area when a small part other than the human foot is blocked.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and embodiments, in which:

Fig. 1 is the module structure schematic diagram of the video fence based on human body posture assessment in the embodiment of the present invention;

Fig. 2 is the monitoring schematic diagram when the ankle of a person appears in the forbidden area;

Figure 3 is a schematic diagram of monitoring when extracting key points of human ankle and knee;

Figure 4 is a schematic diagram of monitoring when the human skeleton is inferred and the key points of the ankle are extracted;

Fig. 5 is the schematic diagram when the ray method is used to judge whether there is an ankle key point in the forbidden area;

6 is a schematic flowchart of a video fence early warning method based on human posture assessment;

FIG. 7 is a schematic diagram when a forbidden area is framed.

Detailed ways

In order to have a clearer understanding of the technical features, objects and effects of the present invention, the specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

As shown in FIG. 1 , a video fence based on human posture assessment in a preferred embodiment of the present invention includes a region delimitation module 1 , an identification module 2 , a judgment module 3 , and an early warning module 4 .

As shown in FIG. 2 , the area delimitation module 1 is used to demarcate the forbidden area B within the video shooting range A according to the video shooting range A, and the shooting direction, shooting angle, focal length, etc. can be adjusted according to the monitoring needs, and preset Determine the video shooting range A, and at the same time, according to the monitoring needs, delimit the forbidden area B within the video shooting range A.

The identification module 2 is used to obtain the coordinate information of the key point C of the ankle of the person in the shooting range A and its confidence level, as a basis for judging whether the person enters the forbidden area B, the coordinate position of the ankle is easy to judge, and can accurately reflect the personnel standing position.

The judgment module 3 is used for judging whether the obtained ankle coordinates are within the forbidden area B according to the coordinate information and the confidence degree of the ankle key point C.

The early warning module 4 is configured to send an early warning signal to remind to take countermeasures when the judgment module 3 finds that the ankle key point C is in the forbidden area B.

The video fence based on human posture assessment can locate the coordinates of the person's ankle through machine vision recognition technology, and more accurately judge whether the person has entered the designated area in the video, and can recognize the person when a small part other than the human foot is blocked. Whether to enter the designated forbidden area B.

The area delineation module 1 manually selects and draws at least one polygonal frame in the image according to the image captured by the camera shooting range A to set it as the forbidden area B, and transmits the two-dimensional coordinate information of the forbidden area B on the image. For the judgment module 3, the number of polygonal boxes is not fixed.

In other embodiments, image recognition technology can also be used to automatically delineate the area delineated on the ground where dangerous objects are stacked and there are potential safety hazards as a no-entry area, and the area delimited on the ground can be an area with a frame drawn. Also available in areas painted with special colors.

The recognition module 2 obtains the information of each frame of the video based on the convolutional neural network of deep learning, identifies whether there is a person in the image, and obtains the coordinate information and confidence of the key point C of the ankle of the person in combination with heat map analysis and affinity analysis. .

Recognition module 2 first preprocesses the image, such as image size scaling, etc., and then uses image recognition technology to identify whether there is a person in the specified area of the image. Among them, we use the scheme of identifying the position of the key point C of the human ankle to identify the person in the picture. ankle.

The recognition process uses the method of human pose estimation. First, a Convolutional Neural Network (CNN) model is used to extract the information of the image. In this method, the MobileNet_v2 network model (a lightweight convolutional neural network) is selected.

The MobileNet_v2 network model is compared with existing models, such as Inception, VGG, ResNet (Residual Neural Network), and MobileNet series network models use Depth-wise Separable Convolution, which greatly reduces the amount of parameters. This greatly reduces the amount of computation we need to process pictures, and we analyze a frame in the video faster, so that the frame rate of our video recognition results will not be too low, and the recognition process is more real-time.

As shown in Figure 3, through the MobileNet_v2 model, we can obtain a set of feature maps, which are divided into two CNN network models, and the confidence maps (Part ConfidenceMaps) of the positions of the human ankle key point C and knee key point D are extracted respectively, and A two-dimensional vector field (Part AffinityFields) of the partial affinity between the human ankle key point C and the knee key point D.

As shown in Figure 4, then all the key points of a human body are associated through greedy reasoning, and connected in a reasonable way to form the skeleton of the human body. We extract the coordinate points of the human skeleton in the figure, only take the key point C of the human ankle, and record its coordinates and confidence. After associating other skeleton key points, take the ankle key point C to make the position of the ankle key point C more accurate and avoid misjudgment.

As shown in Figure 5, we use the ray method to judge whether a point in the captured image is in the forbidden area B of the polygon. The judgment module 3 takes the ankle key point C to be judged as the endpoint, sends out a ray along the horizontal direction, and calculates the number of intersections between the ray and the polygon edge. If the number of intersections between the ray and a polygon frame is an even number, the ankle key point to be judged C is outside the forbidden area B; if the number of intersections between the ray and a polygon frame is odd, the ankle key point C to be judged is within the forbidden area B.

Under the confidence threshold setting, relatively credible ankle points are screened out, and the ray method is used to judge whether the ankle key point C is within the polygon previously framed in the two-dimensional image. If it is within the polygon, an early warning signal is issued.

When the judgment module 3 finds that the key point C of the ankle is in the forbidden area B, and stays for more than a certain time, such as more than 1S, 5S, etc., it is equivalent to a person entering the forbidden area B, and the early warning module 4 sends the human ankle. The coordinate information of the key point C on the image, the time when the ankle key point C appears in the forbidden area B, and the image captured when the ankle key point C appears in the forbidden area B are sent to the cloud 5, and an early warning signal is issued.

2 to 6, the video fence early warning method based on human body posture assessment in a preferred embodiment of the present invention includes the following steps:

S1. Set a video shooting range A, and define a forbidden area B within the shooting range A. According to the picture captured by the camera, a designated area can be manually drawn on the screen and designated as the forbidden area B. In other embodiments, image recognition technology can also be used to automatically delineate the area delineated on the ground where dangerous objects are stacked and there are potential safety hazards as a no-entry area, and the area delimited on the ground can be an area with a frame drawn. Also available in areas painted with special colors.

S2, obtaining the coordinate information and the confidence level of the key point C of the ankle of the person within the shooting range A;

S3, determine whether the obtained ankle coordinates are in the forbidden area B, if it is found that there is an ankle key point C in the forbidden area B, then go to step S4;

S4. Send an early warning signal.

In some embodiments, in step S1, the forbidden area B is demarcated on the screen obtained by the shooting range A, including the following steps:

Extracting image frames, the monitoring personnel can manually select at least one polygonal frame in the image and set it as the forbidden area B, and set the vertex coordinates of each polygonal frame as a set P={[[x1,y1],[x2,y2 ],...,[xn,yn]],[[x'1,y'1],[x'2,y'2],...,[x'n,y'n]],. ..}, where x and y are the horizontal and vertical coordinates of the polygon vertices respectively, each polygon frame is composed of n vertices, and the vertices are connected in sequence in the order of the set to form a polygon frame, and the inside of the frame is forbidden. area B.

As shown in Figure 7, the monitoring personnel can manually select the position of the forbidden area B in the image, first click the clear button to clear the original frame selection area, and then click the screen to set several vertices of the forbidden area B, Line segments will be automatically connected between the vertices to complete the box selection of the forbidden area B. The monitoring personnel can click the submit button in the figure to save the information of the forbidden area B.

In the step S2, the deep learning-based convolutional neural network obtains the information of each frame of the video, and combines the heat map analysis and the affinity analysis to obtain the coordinate information and confidence of the ankle key point C.

As shown in FIG. 2 to FIG. 4 , in step S2, if there is a person in the shooting range A, it can be obtained that all 18 key points of human skeleton joints in the image are located in the two-dimensional corresponding to the shooting range A. The coordinate information on the image and the confidence of the key points of the bones and joints are selected from the coordinate information of the key point C of the human ankle and its confidence. After associating other skeleton key points, take the ankle key point C to make the position of the ankle key point C more accurate and avoid misjudgment.

Through the key point C of the person's ankle, the accurate positioning of the person's position is realized, so even if the person's body covers the frame selection area, it can be judged that no one enters the forbidden area B, because the person's current position is at the ankle. The key point C is outside the frame selection area.

It realizes the identification of the key point C of the human ankle, and draws the key point C of the ankle of the left and right feet on the image, and accurately recognizes that the person is standing in the forbidden area B.

In the step S3, taking the ankle key point C to be judged as the endpoint, a ray is emitted along the horizontal direction, if the number of intersections between the ray and a polygonal frame is an even number, the key point to be judged is outside the forbidden area B, If the number of intersections between the ray and a polygonal frame is an odd number, the key point to be determined is in the forbidden area B.

In the step S4, it also includes uploading the warning information to the cloud 5, and displaying the real-time image of the shooting range A.

The cloud 5 receives the warning information, restores the video screenshot, and sends out the alarm information. Cloud 5 receives the sent information, converts the video screenshot from base64 format to RGB format, displays it, and issues an alarm message.

It can be understood that the above technical features can be used in any combination without limitation.

The above descriptions are only the embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied to other related technologies Fields are similarly included in the scope of patent protection of the present invention.

Claims (10)

1. a video fence early warning method based on human body posture assessment, is characterized in that, comprises the following steps: S1, setting a video shooting range (A), and delimiting a forbidden area (B) within the shooting range (A); S2, obtaining the coordinate information and the confidence level of the ankle key point (C) of the person within the shooting range (A); S3, determine whether the obtained ankle coordinates are in the forbidden area (B), if it is found that there is an ankle key point (C) in the forbidden area (B), then go to step S4; S4. Send an early warning signal. 2. the video fence early warning method based on human body posture assessment according to claim 1, is characterized in that, in described step S2, the convolutional neural network based on deep learning obtains the information of video every frame image, combines heat map analysis and Affinity analysis obtains the coordinate information and confidence of the ankle key point (C). 3. the video fence early warning method based on human body posture assessment according to claim 1, is characterized in that, in described step S2, if there is personnel in described shooting range (A), can obtain all human skeleton joints in the image The coordinate information of the key point located on the two-dimensional image corresponding to the shooting range (A) and the confidence level of the key point of the skeleton joint are selected from the coordinate information and confidence level of the key point (C) of the human ankle. 4. The video fence early warning method based on human body posture assessment according to any one of claims 1 to 3, wherein in the step S1, the forbidden area (B) is in the shooting range (A) ) is delineated on the screen obtained, including the following steps, Extract image frames, manually select at least one polygonal frame in the image and set it as the forbidden area (B), and set the vertex coordinates of each polygonal frame as a set P={[[x ₁ ,y ₁ ],[ x ₂ ,y ₂ ],...,[x _n ,y _n ]],[[x' ₁ ,y' ₁ ],[x' ₂ ,y' ₂ ],...,[x' _n , y' _n ]],...}, where x, y are the horizontal and vertical coordinates of the polygon vertices respectively, each polygon frame is composed of n vertices, and the vertices are connected in sequence in the order of the set to form a polygon box, inside the box is the forbidden area (B). 5. the video fence early warning method based on human body posture assessment according to claim 4, is characterized in that, in described step S3, with the ankle key point (C) to be judged as the end point, sends out a ray along the horizontal direction, if The number of intersections between the ray and a polygonal frame is an even number, and the ankle key point (C) to be judged is outside the forbidden area (B). If the number of intersections between the ray and a polygonal frame is an odd number, the The ankle key point (C) is within the no-go zone (B). 6. The video fence early warning method based on human body posture assessment according to any one of claims 1 to 3, wherein in the step S4, it also includes uploading early warning information to the cloud (5), displaying the shooting range (A) Live image. 7. a video fence based on human body posture assessment, is characterized in that, comprises: an area delineation module (1), configured to demarcate a forbidden area (B) within the shooting range (A) according to the video shooting range (A); An identification module (2), used for acquiring coordinate information and confidence of the key point (C) of the ankle of the person within the shooting range (A); A judgment module (3) for judging whether the obtained ankle coordinates are within the forbidden area (B), and An early warning module (4) is configured to send an early warning signal when the judgment module (3) finds that a key point (C) of the ankle is in the forbidden area (B). 8. the video fence based on human body posture assessment according to claim 7, is characterized in that, described recognition module (2) obtains the information of video every frame image based on the convolutional neural network of deep learning, combines heat map analysis and affinity The coordinate information and confidence of the ankle key point (C) are obtained by analysis. 9. The video fence based on the assessment of human body posture according to claim 7, wherein the region delimiting module (1) extracts image frames from the captured video, and manually selects at least one polygonal frame in the image Set as the forbidden area (B), and generate coordinate information of the forbidden area (B); The judging module (3) takes the ankle key point (C) to be judged as the endpoint, and sends out a ray along the horizontal direction, if the number of intersections between the ray and a polygonal frame is an even number, the ankle key point (C) to be judged Outside the forbidden area (B), if the number of intersections between rays and a polygonal frame is odd, the ankle key point (C) to be determined is within the forbidden area (B). 10. the video fence based on human body posture assessment according to claim 7, is characterized in that, in described judging module (3) it is found that there is ankle key point (C) in described forbidden area (B), and stay When a specific time is exceeded, the early warning module (4) sends the coordinate information of the human ankle key point (C) on the image, the time when the ankle key point (C) appears in the forbidden area (B), and the ankle key point. The image captured when the point (C) appears in the forbidden area (B) is sent to the cloud (5), and an early warning signal is issued.

Images