CN106502570B - Method, device and vehicle-mounted system for gesture recognition - Google Patents

Abstract

The embodiment of the invention discloses a gesture recognition method, which judges whether a current gesture is a touch gesture or a three-dimensional gesture by judging whether an area where the current gesture operation is located in acquired image information is located on a preset touch plane area. If the gesture is a touch gesture, identifying an operation gesture according to the touch position information of the touch gesture; and if the current gesture is a three-dimensional gesture, calling a pre-established three-dimensional gesture model library, identifying an operation gesture of the three-dimensional gesture, and generating a corresponding operation instruction according to the operation gesture. According to the scheme, the three-dimensional gesture and the two-dimensional gesture are recognized integrally, the use by a user is facilitated, and the control comfort of human-computer interaction of the user is improved. In addition, the embodiment of the invention also provides a corresponding implementation device and a vehicle-mounted system aiming at the gesture recognition method, so that the method has higher practicability, and the device and the system have corresponding advantages.

Description

Method, device and vehicle-mounted system for gesture recognition

technical field

The invention relates to the fields of human-computer interaction and automobile manufacturing, in particular to a gesture recognition method, and also to a gesture recognition device and a vehicle-mounted system.

Background technique

With the development of computer technology, human-computer interaction technology has also developed rapidly. From pattern recognition, such as speech recognition, Chinese character recognition, etc., where it becomes possible for operators and computers to interact at a level similar to natural language or restricted natural language, to human-computer interaction through graphics, more and more fields applied to human-computer interaction technology.

The application potential in the field of human-computer interaction technology has begun to show, such as action recognition technology applied to wearable computers, stealth technology, immersive games, etc., tactile interaction technology applied to virtual reality, remote control robots and telemedicine, etc. Voice recognition technology for call routing, home automation and voice dialing, etc. Human-computer interaction solution providers continue to introduce various innovative technologies, such as fingerprint recognition technology, side sliding fingerprint recognition technology, pressure touch technology, etc. The application and development of hot technology is both an opportunity and a challenge. Vision-based gesture recognition is currently a hot spot in human-computer interaction.

Gesture recognition technology is a technology that realizes control through the movement of various parts of the human body, but generally refers to the movement of the face and hands. Gesture recognition is mainly divided into two-dimensional gesture recognition and three-dimensional gesture recognition. At present, gesture control is another technology that is sought after by many car manufacturers after voice control. Gesture control allows users to simply wave their hands or make a few actions while driving to control in-vehicle functions. It is more accurate and more convenient.

At present, the existing implementation principles of gesture recognition include infrared sensor-based, camera-based detection, resistive-based, and capacitive-based. Generally, three-dimensional gesture recognition is based on infrared sensors and cameras, and two-dimensional gesture recognition is based on resistive and capacitive methods. For example, the touch panel of the touch-sensitive liquid crystal display of the existing automobile central control panel is usually resistive or capacitive, and the user can only use fingers or a touch pen to perform operations such as clicking on the touch screen, and cannot be far away from the touch screen. operation, that is, to realize the recognition of two-dimensional gestures; for example, when infrared sensors are used in automobiles, simple three-dimensional gesture recognition can be realized, such as detecting waving or approaching, moving away, etc., but two-dimensional gestures cannot be recognized.

It can be seen that the control and recognition of three-dimensional gestures and two-dimensional gestures in the prior art are realized by two systems respectively. For those who only recognize two-dimensional gestures, the user cannot leave the operation screen to perform operations, which will cause great inconvenience during use, especially when the touch screen is broken, the whole gesture recognition cannot be used; for those who can only recognize three-dimensional gestures , due to the limited actions it recognizes, greatly restricts the application of gesture control, and also brings inconvenience to user operations.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present invention is to provide a gesture recognition method, device and vehicle-mounted system, which realizes the integration of 3D gesture recognition and 2D gesture recognition, is more convenient for users to use, and improves the user's man-machine interaction control comfort.

In order to solve the above-mentioned technical problems, the embodiments of the present invention provide the following technical solutions:

An aspect of an embodiment of the present invention provides a method for gesture recognition, including:

Obtain the image information collected by the image collection device;

detecting the current gesture in the image information;

Determine the operation area where the current gesture is located;

When the operation area where the current gesture is located is located in the preset touch plane area, determine that the current gesture is a touch gesture, and identify the operation gesture according to the touch position information of the touch gesture;

When the operation area where the current gesture is located is not located in the preset touch plane area, determine that the current gesture is a three-dimensional gesture, call a pre-established three-dimensional gesture model library, and identify the operation gesture of the three-dimensional gesture;

A corresponding operation instruction is generated according to the recognized operation gesture.

Preferably, when the image information is two-dimensional plane information, the step of judging the operation area where the current gesture is located includes:

Detect whether the button on the preset touch plane area for indicating whether there is a touch signal is turned on, if so, determine that the operation area where the current gesture is located is located in the preset touch plane area; if not, determine that it is not located in the preset touch plane area.

Preferably, when the image information includes two-dimensional plane information and one-dimensional depth image information, the step of judging the operation area where the current gesture is located includes:

calculating the depth values of the image acquisition device and each pixel in the preset touch plane area, and determining the minimum depth value L _min ;

Calculate the distance L between the touch point of the current gesture and the image acquisition device;

If LL _min ≤ 15mm, it is determined that the operation area where the current gesture is located is not located in the preset touch plane area, otherwise, it is determined that the operation area is located in the preset touch plane area.

Preferably, the current gesture is any one or any combination of the following:

A gesture of touching or clicking with the center of the finger, the side of the finger, or the fingernail.

Preferably, the calling of the pre-established 3D gesture model library is:

Continuously acquire multiple frames of image information including the current gesture in time series. If the current gestures in the multiple frames of images are the same and the position of the current gesture has not changed, the current gesture is a static gesture, and the call 3D static gesture model library; otherwise, call the 3D dynamic gesture model library.

Another aspect of the embodiments of the present invention provides a gesture recognition device, including:

an image acquisition module for acquiring image information collected by an image acquisition device;

an image detection module for detecting the current gesture in the image information;

a judging area module for judging the operating area where the current gesture is located;

An information recognition module for recognizing the current gesture:

When the operation area where the current gesture is located is located in the preset touch plane area, and when it is determined that the gesture information is a touch gesture, the operation gesture is identified according to the touch position of the touch gesture;

When the operation area where the current gesture is located is not located in the preset touch plane area, determine that the gesture information is a three-dimensional gesture, call a pre-established three-dimensional gesture model library, and identify the operation gesture of the three-dimensional gesture;

The instruction generation module is configured to generate a corresponding operation instruction according to the recognized operation gesture.

Preferably, the judging area module includes:

a first determination unit, which detects whether the button on the preset touch plane area used to indicate whether there is a touch signal is turned on, and if so, determines that the operation area where the current gesture is located is located in the preset touch plane area; if No, it is determined that it is not located in the preset touch plane area; and/or

a second judging unit, calculating the depth values of the image acquisition device and each pixel in the preset touch plane area, and determining the minimum depth value L _min ;

Calculate the distance L between the touch point of the current gesture and the image acquisition device;

If LL _min ≤ 15mm, it is determined that the operation area where the current gesture is located is not located in the preset touch plane area, otherwise, it is determined that the operation area is located in the preset touch plane area.

Embodiments of the present invention further provide a vehicle-mounted system for gesture recognition, including an image acquisition device, a touch panel, and any of the above gesture recognition devices.

Preferably, the touch panel includes a touch panel and/or a touch switch, and when the image capturing device is a 2D camera, the touch panel includes:

Panel unit, circuit board unit, key unit and support unit;

The touch switch includes:

With beacon panel unit, circuit board unit, light guide plate unit, key unit, setting light unit and support unit unit.

Preferably, the touch panel includes a touch panel and/or a touch switch, and when the image capture device is a 3D camera, the touch panel includes:

Panel units and support units;

The touch switch includes:

Panel unit with beacon, light guide plate unit, circuit board unit, setting light unit and supporter unit.

An embodiment of the present invention provides a gesture recognition method, which determines whether the current gesture is a touch gesture or a three-dimensional gesture by judging whether the area where the current gesture operation is located in the captured image information is located in a preset touch plane area. If it is a touch gesture, the operation gesture is recognized according to the touch position information of the touch gesture; if the current gesture is a three-dimensional gesture, the pre-established three-dimensional gesture model library is called to identify the operation gesture of the three-dimensional gesture, and the three-dimensional gesture is realized. Gesture and 2D gesture recognition integration.

If the gesture recognition is single, that is, only for 2D gesture recognition or only for 3D gesture recognition, for only 2D gesture recognition, it can only be operated on the touch screen with sensing function, and cannot be operated far from the screen; For three-dimensional gesture recognition, the application of gesture control is greatly limited due to the limited recognition actions of three-dimensional gestures. However, the present application can realize the integration of 3D gesture and 2D gesture recognition, which can realize not only the recognition of 3D gestures in the air, but also the recognition of 2D gestures in the touch plane area, which is more convenient for users to use and improves the user's comfort in man-machine interaction. sex.

In addition, the embodiments of the present invention also provide a corresponding implementation device and an in-vehicle system for the gesture recognition method, which further makes the method more practical, and the device and system have corresponding advantages.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the following will briefly introduce the accompanying drawings used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only For some embodiments of the present invention, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a structural block diagram of a vehicle-mounted system for gesture recognition provided by an embodiment of the present invention;

FIG. 2 is a schematic flowchart of a gesture recognition method according to an embodiment of the present invention;

3 is a partial three-dimensional gesture of a three-dimensional dynamic model library provided by an embodiment of the present invention;

4 is a schematic diagram of image processing and gesture recognition according to an embodiment of the present invention;

FIG. 5 is a schematic flowchart of another gesture recognition method provided by an embodiment of the present invention;

FIG. 6 is a schematic diagram of a gesture operation mode and position according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of positioning of a current gesture according to an embodiment of the present invention;

8 is a schematic diagram of determining current gesture position information according to an embodiment of the present invention;

FIG. 9 is a schematic flowchart of another gesture recognition method provided by an embodiment of the present invention;

10 is a structural diagram of a gesture recognition device according to an embodiment of the present invention;

11 is a structural block diagram of a vehicle-mounted system for gesture recognition provided by an embodiment of the present invention;

12 is a schematic structural diagram of a 2D camera-based touch panel according to an embodiment of the present invention;

FIG. 13 is a schematic structural diagram of a 2D camera-based touch switch according to an embodiment of the present invention;

14 is a schematic structural diagram of a touch panel based on a 3D camera according to an embodiment of the present invention;

15 is a schematic structural diagram of a touch switch based on a 3D camera according to an embodiment of the present invention;

FIG. 16 is a structural block diagram of a vehicle-mounted system for gesture recognition control provided by an embodiment of the present invention.

Detailed ways

In order to make those skilled in the art better understand the solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The terms "first", "second", "third", "fourth", etc. in the description and claims of the present application and the above drawings are used to distinguish different objects, rather than to describe a specific order. . Furthermore, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or elements is not limited to the listed steps or elements, but may include unlisted steps or elements.

The inventors of the present application have found through research that the prior art gesture recognition is unique, that is, it is only for two-dimensional gesture recognition or only three-dimensional gesture recognition. For operation, it cannot be operated far from the screen; and for the recognition of only three-dimensional gestures, it is also necessary to assist the touch screen with the sensing function to recognize two-dimensional gestures. In view of this, the present application realizes the integration of 3D gestures and 2D gesture recognition, which can realize the recognition of 3D gestures in the air and the 2D gestures in the touch area at the same time, which is more convenient for users to use and improves the user's comfort in man-machine interaction. To a certain extent, it saves the user's cost of use.

Based on the technical solutions of the above embodiments of the present invention, some possible application scenarios involved in the technical solutions of the embodiments of the present invention are firstly introduced with reference to FIG. 1 . FIG. 1 is a vehicle-mounted system structure for gesture recognition provided by an embodiment of the present invention. block diagram.

As shown in FIG. 1 , 101 is a camera installed in a car dome light module, 102 is a sensing area of the camera, 103 is a gesture, and 104 is a center control panel, which includes a rectangular touchpad ABCD and a touch switch EF.

In the prior art, 101 is generally an infrared camera as a three-dimensional recognition gesture sensor, and 104 is a touch panel with a sensing unit. In this way, the detection and recognition of air gestures are detected and recognized by the infrared camera, and the two-dimensional touch gestures of the center control panel are recognized by the touch unit.

The solution provided by the present application is that the camera can be a 2D camera or a 3D camera as a gesture recognition sensor, and the center control panel does not need to carry an inductive touch unit. All gestures (2D, 3D) are recognized by the camera.

In this way, not only the operating comfort of the user is greatly improved, but also the structure of the control panel is simplified, which can save a certain manufacturing cost.

It should be noted that the above application scenarios are only shown for the convenience of understanding the idea and principle of the present application, and the embodiments of the present application are not limited in this respect. Rather, the embodiments of the present application can be applied to any scenario where applicable.

After introducing the technical solutions of the embodiments of the present invention, various non-limiting implementations of the present application are described in detail below.

Example 1:

Referring first to FIG. 2, FIG. 2 is a schematic flowchart of a gesture recognition method provided by an embodiment of the present invention. The embodiment of the present invention may include the following contents:

S201: Acquire image information collected by an image collection device.

S202: Detect the current gesture in the image information.

S203: Determine the operation area where the current gesture is located.

S204: When the operation area where the current gesture is located is located in the preset touch plane area, determine that the current gesture is a touch gesture, and identify the operation gesture according to the touch position information of the touch gesture.

S205: When the operation area where the current gesture is located is not located in the preset touch plane area, determine that the current gesture is a three-dimensional gesture, call a pre-established three-dimensional gesture model library, and identify the operation gesture of the three-dimensional gesture.

S206: Generate a corresponding operation instruction according to the recognized operation gesture.

Among them, S201 is specifically:

The image acquisition device is a 2D camera or a 3D camera, and the corresponding image information is different. The image information captured by the 2D camera is two-dimensional grayscale plane image information or two-dimensional color plane image information; the image information captured by the 3D camera includes two-dimensional grayscale plane image information and one-dimensional depth image information.

Among them, S202 is specifically:

When detecting the image information collected by the image acquisition device, it is necessary to determine whether the image information contains hand information. When there is no hand, the acquired image information is discarded and the next image is judged; when the detected image information contains incomplete hand information When the information is received, it can be reminded, the user can be reminded by voice, and the angle of the image acquisition device can be automatically adjusted. At present, other methods can be used to realize the reminder function, so that the sensing area contains the current hand information; when the detected image contains complete When the gesture information is displayed, enter S203.

It should be noted that the current gesture may be a static gesture or a dynamic gesture. For example, in the touch area, the static gesture may be a key action, and the dynamic gesture may be a swipe action. The image information including the current gesture can be continuously acquired in multiple frames in time series. If the current gesture in the multi-frame images is the same and the position of the current gesture has not changed, the current gesture is a static gesture; otherwise, It is a dynamic gesture.

Among them, S203 is specifically:

Since the image acquisition device captures picture information in a certain area, it can be a chamfered pyramid area, and the area refers to a preset touch plane area and an aerial sensing area, and the aerial sensing area is the aerial part of the chamfered pyramid area. The preset touch plane area is a plane area operated by a two-dimensional gesture, generally a touch pad, and the touch gesture (two-dimensional gesture) can be a touch gesture on the touch pad, or a touch gesture on a touch switch.

Wherein, the touch gesture can be performed by touching or clicking on the center of the finger, the side of the finger, the fingernail or any combination thereof. Of course, the touch gesture can also be performed by any part of the body. If the traditional capacitive touch screen wants to operate with the side of the finger and the fingernail, it often requires high precision, especially the operation of the fingernail; while for the resistive touch screen, although the operation of the fingernail can be recognized, the fingernail is not suitable for the operation of the fingernail. The damage of the touch screen is too great, which leads to inaccurate recognition of the touch position of the touch screen and shortened life span, which is not worth the loss. In the present application, a touch screen without sensing function can be used. The touch screen can be a plastic panel, a metal panel, or a liquid crystal display. Of course, a panel of any other material can also be used. There will be requirements for its accuracy, and the selection of suitable materials will inevitably damage the life of the screen.

The method used to specifically determine the operation area where the current gesture is located is different depending on the image acquisition device. For a specific introduction, refer to Embodiment 2 and Embodiment 3, which will not be repeated here.

Among them, S204 is specifically:

The preset touch plane area can be a touch pad, a touch switch, or a combination of the two, and the number of touch pads and touch switches is not limited, and can be one or any Several pieces are spliced, and the switch can be one or more. The material of the touch panel and the touch switch can be a plastic panel, a metal panel, or a liquid crystal display screen. Of course, other materials can also be used, and if necessary, a combination of the above can also be used.

The touch position information can be coordinate information or specific touch point position information, which is determined according to the camera used and the specific touch area. The coordinate information feedback of the point can also directly send the information of the blocking position according to the position where the touch point is blocked. For example, if the a button of the touch switch is detected to be blocked, the position of the a button is sent, indicating that the user's operation at the moment is to press the button. a key. Because the operation instructions corresponding to different positions are different, this can be preset by the manufacturer or preset by the user according to the needs. For example, the operation command represented by the a key is to open the sunroof. When the user presses the a key, the sunroof is opened according to the detected position information or coordinate information of the a key. Therefore, the operation gesture can be recognized according to the position information.

Among them, S205 is specifically:

The three-dimensional gestures in the air can be static gestures or dynamic gestures. For example, a gesture of directly making 'OK' is a static gesture, and a gesture that changes from a palm to a fist is a dynamic gesture. Correspondingly, a static gesture corresponds to a three-dimensional static gesture. Gesture model library, dynamic gesture calls 3D dynamic gesture model library. In this way, it is necessary to first identify whether the gesture is dynamic or static when calling. You can obtain multiple frames of image information including the current gesture continuously in time series. If the current gestures in the multi-frame images are the same and the current gesture The position of the gesture does not change, the current gesture is a static gesture, and the three-dimensional static gesture model library is called; otherwise, the three-dimensional dynamic gesture model library is called.

The model library is a series of preset gestures, and the operation instructions corresponding to the preset gesture actions can be generic by default, or can be customized according to the user's requirements. operation to achieve human-computer interaction. For example, Figure 3 lists some common gesture actions in the 3D dynamic model library. 301 is the gesture from the index finger extended state to the index finger retracted into a fist state; 302 is the counterclockwise rotation of the index finger in the extended state; 303 304 is the gesture of changing the hand from the palm to a fist; 305 is the waving gesture of the palm from left to right; 306 is the swiping gesture of the palm from right to left. Of course, the gestures listed in Figure 3 above are only some gestures in the model library. There are many gestures in practical applications, and different gestures correspond to different operation instructions. , open the palm and wave it to the right to hang up the call and cancel the operation, the V-shaped gesture means pause, play and custom operations, turn one finger clockwise to increase the volume and reduce the navigation map, and turn one finger counterclockwise to decrease , Zoom in on the navigation map, move the five fingers together in a circle to the right to indicate the next song and return to the main menu, and move the five fingers in a circle to the left to indicate the previous song and return to the main menu. For the action of sweeping the hand, the motor vehicle will determine the range of motion, and then determine the degree of car-hailing from the window according to the range of the gesture. Snap your finger near the steering wheel to activate the wipers or indicator lights, twist your wrist in front of the car dashboard to turn on the air conditioner or radio; point your finger to the sunroof to automatically open, and press your index finger to your lips to turn down the volume of your phone. A hand open into a horn means using a mobile phone to make a call, a chin-like action like thinking means "I want to retrieve information", and a like action means an "agree" action. Of course, it is not limited to the gestures and operations corresponding to the gestures described above, and can be added or modified according to the needs of users in actual implementation.

It should be noted that, the information in the model library may be the three-dimensional gesture information, or the execution command information corresponding to the three-dimensional gesture, or both. For example, in the model library, only the gesture of opening the palm and swiping to the right may be stored, or only the commands to be executed corresponding to the gesture of opening the palm and swiping to the right are hanging up the phone, canceling the operation, or both can be saved.

Among them, S206 is specifically:

The operation instruction can be position information, that is, the position information can be directly sent to the electronic function module through the interface unit, and the electronic function module further executes the action to be performed according to the preset corresponding relationship between the instruction and the position; The action command to be identified by the location information can be directly sent to the electronic function module through the interface unit, and the electronic function module directly executes the action command. Correspondingly, the preset and identification of the correspondence between the position information and the operation may be preset in the electronic function module or in the information identification module directly. For example, when the operation gesture is at the touch switch in the preset touch plane area, the operation instruction may be the coordinate information or specific position information of the a key, or the operation instruction represented by the a key, such as opening the sunroof.

Specifically, for the logic of image processing and gesture recognition, please refer to FIG. 3 , which is a schematic diagram of image processing and gesture recognition provided by this application. Among them, 401 is the real-time image information captured by the system. For a 2D camera, it can be a two-dimensional grayscale plane image signal or a two-dimensional color plane image signal; for a 3D camera, it can be a two-dimensional grayscale plane image signal and a one-dimensional depth image. Signal diagram.

402 is the preprocessing of the image, usually processing the plane image signal, that is, by extracting the contour, convexity, angle or skeleton of the hand in the image, and of course other hand features.

403 is image processing, that is, further processing the features of the gesture obtained in real time by image preprocessing, such as compression, optimization, and specifically doing different processing according to different types of gestures and different gesture features, such as extracting the outline of the hand for static gestures. When the gesture features are optimized, the dynamic gesture needs to extract the contours of multiple hands in continuous time frames, which needs to be compressed first.

Step 404 is to acquire a sample model, that is, to acquire a sample model for the current gesture according to different types of gestures after image processing. For example, the static gesture model can adopt the same image processing and feature extraction methods as in steps 402 and 403 according to the collected samples, and the sample model of the static gesture obtained by the sample training operation; similarly, the dynamic gesture model can use the static gesture model The method can continuously extract gesture contour samples from the time series, and train a sample model of dynamic gestures obtained by the sample training operation.

405 is gesture recognition and tracking. For static gestures, only recognition is required. For dynamic gestures, tracking is also required because multiple frames of images in a continuous time need to be collected. After processing the gesture feature signal obtained in real time at 403, together with the gesture sample model 404, real-time identification can be carried out through a specific algorithm, and the identification algorithm can be any one of the following or a combination of any of them: Boosting algorithm , Random Forest, Support Vector Machine (SVM), Neural Network, Deep Learning, Hidden Markov Model (Hidden Markov Model) HMM), of course, other algorithms can also be used if necessary.

406 is to send a gesture command, that is, send it to the execution unit according to the generated operation command, for example, the currently recognized gesture signal can be sent to a specific electronic function module in real time through the interface unit, and the corresponding gesture command is executed, thereby realizing the human-machine operation. interact.

An embodiment of the present invention provides a gesture recognition method, which determines whether the current gesture is a touch gesture or a three-dimensional gesture by judging whether the area where the current gesture operation is located in the captured image information is located in a preset touch plane area. If it is a touch gesture, the operation gesture is recognized according to the touch position information of the touch gesture; if the current gesture is a three-dimensional gesture, the pre-established three-dimensional gesture model library is called to identify the operation gesture of the three-dimensional gesture, and the three-dimensional gesture is realized. Gesture and 2D gesture recognition are integrated, that is, it can realize not only the recognition of 3D gestures in the air, but also the 2D gestures in the touch area, which is more convenient for users to use, and improves the freedom and comfort of user man-machine interaction.

In the actual operation process, if the 2D camera is used as the identification sensor, in order to be more convenient to use, the first embodiment is further improved.

Embodiment 2:

Referring to FIG. 5, FIG. 5 is a schematic flowchart of another gesture recognition method provided by an embodiment of the present invention, which may specifically include the following content:

S501: Acquire image information collected by a 2D camera.

S502: Detect the current gesture in the image information.

S503: Determine whether the operation area of the current gesture is a preset touch plane area, and if so, go to S504; otherwise, go to S505.

S504: Determine that the current gesture is a touch gesture, and identify the operation gesture according to the touch position information of the touch gesture.

S505: Determine that the current gesture is a three-dimensional gesture, call a pre-established three-dimensional gesture model library, and identify the operation gesture of the three-dimensional gesture.

S506: Generate a corresponding operation instruction according to the recognized operation gesture.

When it is detected that there are complete gestures in the image information collected by the 2D camera, when the preset touch area is divided into touch pads and touch switches, the current gestures are the three types shown in Figure 6, and 601 is the 2D camera, marked as S is the center of the sensing surface of the camera; 602 is the three-dimensional gesture operation, Q is the operating position of the finger in the air; 603 is the touch pad ABCD with the back of the hand facing the touch area, and the fingernails are used to click, slide and other touch operations, P is the finger is touching The touch position on the board ABCD; 604 is the touch operation such as finger click and slide of the touch switch EF with the palm facing the touch area, and M is the touch position of the finger on the touch switch EF.

The specific method for judging which operation area the current gesture is located in is:

Determine whether the area where the current gesture operation is located in the image information overlaps with the preset touch plane area, and if not, determine that the area where the gesture is located is the air area where the three-dimensional gesture is located; otherwise, further detect the Whether the button of the circuit board in the preset touch plane area is turned on, if so, it is determined that the area where the gesture is located is the preset touch plane area, otherwise, it is determined that the area where the current gesture is located is the air area where the three-dimensional gesture is located . Wherein, the button is a switch on the circuit board in the preset touch plane area. When there is a touch action in the preset touch plane area, the switch is turned on, and the button can be a micro-stroke non-self-locking elastic switch , of course, it can also be other devices, as long as the function of sending a corresponding signal when there is a touch action in the preset touch plane area can be realized. Of course, the preset touch plane area may be the touch pad ABCD, or the touch switch EF.

After it is determined whether the current gesture is located in the preset touch plane area, it can be determined whether the current gesture is a touch gesture or a three-dimensional gesture. When it is determined that the current gesture is a touch gesture, the operation gesture can be realized by determining the touch position information. To identify the specific location information, you can first locate the current gesture, that is, to determine the touch point or location point, as follows:

For example, please refer to FIG. 7 . FIG. 7 is a schematic diagram of the positioning of the current gesture. The current gesture may be a three-dimensional gesture operation above the preset touch plane area or a touch operation. 701 is a single-finger operation gesture using the index finger of a finger, such as gesture 301 , gesture 302 or gesture 303 in the model library in FIG. 3 . 702 is a gesture other than the index finger operation gesture, such as gesture 304, gesture 305 or gesture 306 in the model library in FIG. 3 .

When using a single finger to operate, that is, the gesture represented by 701, the location of the touch point is as follows: T in the figure is the vertex (highest point) of the finger of gesture 701 in the whole image, and L and T are pixels respectively. When the point T moves down △, the edge pixels on the left and right sides of the finger, G in the figure is the center position of the pixel points L and T in the same row, that is, the position point or touch point of the current gesture. Among them, the value range of △ is set at 5mm-15mm in this device, because the size and width of fingers of different people or people from different countries and regions are different. For the gesture represented by 702, H is the vertex (highest pixel point) of the gesture in the entire image, that is, the position point or touch point of the current gesture.

The position information of the current gesture can be determined by establishing a two-dimensional Cartesian coordinate system between the image information and the preset touch plane area, and calculating the two-dimensional coordinate value of the touch point of the current gesture according to the image information. Corresponding to the actual coordinate value of the preset touch plane area, the actual coordinate value is the touch position information of the current gesture. When the touch gesture is the touch gesture of touching the touch switch in the preset touch plane area, calculating the touch position information can also be performed by detecting the position where the touch switch is blocked, the blocked position is the touch position information of the touch gesture. Specifically as follows:

Please refer to FIG. 8 , which is a schematic diagram of determining the current gesture position information. 801 is the image of the preset touch area captured by the camera; 802 is the actual preset touch plane area, wherein the preset touch plane area can be divided into the touch pad ABCD and the touch switch EF. The rectangular area A'B'C'D' in 801 corresponds to the touchpad ABCD of the actual preset touch plane area; E'F' corresponds to the touch switch EF of the actual preset touch plane area; the length of the touchpad in the image is The dimensions of and width are W ₁ ' and H' respectively, while the length and width dimensions of the actual touchpad are W ₁ and H; the length of the touch switch in the image is set to W ₂ ', and the actual length of the touch switch is set to W ₂ . In order to facilitate the determination of the touch position information, in the image 801, two coordinate systems O'X'Y' and O ₁ 'X ₁ 'Y ₁ ' are respectively established, where O'X'Y' is the touch panel A' The upper left corner vertex A' of B'C'D' is used as the coordinate origin O', the row where the pixel of the coordinate origin O' is located is used as the X' axis, and the column where the pixel of the coordinate origin O' is located is used as the Y' axis. O ₁ 'X ₁ 'Y ₁ ' takes the center point E' of the left edge of the touch switch E'F' area in the image as the coordinate origin O ₁ ', and takes the line where the pixel of the coordinate origin O ₁ ' is located as the X ₁ ' axis , which is established with the column where the coordinate origin O ₁ ' pixel is located as the Y ₁ 'axis; correspondingly, on the actual preset touch plane area, two coordinate systems OXY and O ₁ X ₁ Y ₁ are also set, where The OXY coordinate system is established with the upper left corner vertex A of the touchpad ABCD as the coordinate origin O, the upper horizontal edge as the X axis, and the left edge as the Y axis; the O ₁ X ₁ Y ₁ coordinate system is based on the touch switch The center point of the left side line of EF is taken as the origin O ₁ , the horizontal center line is taken as the X ₁ axis, and the left side line is taken as the Y ₁ axis. (x', y') in the image 801 are the image coordinates of the current gesture touch point in the O'X'Y' coordinate system, and the corresponding coordinates in the actual preset touch plane area 802 are (x, y) Representation; (x ₁ ', 0) in the image 801 is the image coordinates of the current gesture touch point in the O ₁ 'X ₁ 'Y ₁ ' coordinate system, and the corresponding coordinates in the actual preset touch plane area 802 are represented by (x ₁ , 0) represents. The specific calculation method is as follows:

For the touchpad, first calculate the coordinates (x', y') of the touch point of the current gesture in the O'X'Y' coordinate system and the length W ₁ ' of the touchpad A'B'C'D' in the image 801 and width H'; because the image of the touchpad captured by the camera and the touchpad in the actual preset touch plane area have only a small graphic distortion, so approximately press the formula to calculate the touch point relative to OXY in the actual touchpad ABCD Coordinates of the coordinate system:

but

For the touch switch, a method can be expressed by the above-mentioned coordinates, that is, firstly, the coordinates (x ₁ ', 0) of the touch point of the current gesture in the O ₁ 'X ₁ 'Y ₁ ' coordinate system and the touch point in the image 801 are calculated. Switch length W ₂ '; then the coordinates of the touch point in the actual touch switch relative to the O ₁ X ₁ Y ₁ coordinate system:

Another method is to directly use the information of the beacon as the position information of the current gesture to feed back the beacon information of the specific switch in the E'F' area of the touch panel switch E'F' in the image 801. For example, in the figure If the switch whose beacon is 'e' is blocked, it is determined that the actual switch of the touch point of the current gesture is the switch whose beacon is 'e'.

When it is determined that the current gesture is a three-dimensional gesture, it can be further determined whether the three-dimensional static gesture model library or the three-dimensional dynamic gesture model library needs to be called by first determining whether the current gesture is a static gesture or a dynamic gesture. The operation gesture of the three-dimensional gesture is recognized according to the model library. For details, refer to S205, which will not be repeated here.

As can be seen from the above, an embodiment of the present invention provides a gesture recognition method, by judging whether the area where the current gesture operation is located in the image information captured by the 2D camera is located in the preset touch plane area, and determining that the current gesture is a touch gesture. Or three-dimensional gestures. If it is a touch gesture, the operation gesture is recognized according to the touch position information of the touch gesture; if the current gesture is a three-dimensional gesture, the pre-established three-dimensional gesture model library is called to identify the operation gesture of the three-dimensional gesture, and the three-dimensional gesture is realized. Gesture and 2D gesture recognition are integrated, which is convenient for users to use and improves the freedom and comfort of user's human-computer interaction.

In the actual operation process, if the 3D camera is used as the identification sensor, in order to be more convenient to use, the first embodiment is further improved.

Embodiment three:

Referring to FIG. 9, FIG. 9 is a schematic flowchart of another gesture recognition method provided by an embodiment of the present invention, which may specifically include the following content:

S901: Acquire image information collected by a 3D camera.

S902: Detect the current gesture in the image information.

S903: Determine whether the operation area of the current gesture is a preset touch plane area, if so, go to S904, otherwise, go to S905.

S904: When the operation area where the current gesture is located is located in the preset touch plane area, determine that the current gesture is a touch gesture, and identify the operation gesture according to the touch position information of the touch gesture.

S905: When the operation area where the current gesture is located is not located in the preset touch plane area, determine that the current gesture is a three-dimensional gesture, call a pre-established three-dimensional gesture model library, and identify the operation gesture of the three-dimensional gesture.

S906: Generate a corresponding operation instruction according to the recognized operation gesture.

Compared with the second embodiment, the 2D camera is replaced by a 3D camera, that is, the collected image information has one more dimension of depth information, and the corresponding S903 judgment method for judging the operation area where the current gesture is located is different, and other steps are the same as the embodiment. The two are the same, so only the S903 is described in detail here, and the others will not be repeated.

Determine whether the area where the current gesture operation is located in the image information overlaps with the preset touch plane area, and if not, determine that the area where the gesture is located is the air area where the three-dimensional gesture is located; otherwise, continue with the following method for further judgment.

Calculate the depth values of the image acquisition device and each pixel in the preset touch plane area, and determine the minimum depth value L _min . Of course, the preset touch plane area may be a touch pad and/or a touch switch. For example, first calculate the depth value of each pixel point in the rectangular touchpad ABCD and the depth value La, Lb, Lc, Ld, Le, Lf, Lg, Lh, Li of each switch center point in the touch switch EF area, and The minimum depth value Lmin between the camera and the preset touch plane area is determined according to the above depth information. For example, among the numerous depth values, if La is the smallest value, it is determined that Lmin=La.

The touch point of the current gesture is determined by the method of positioning the current gesture in the second embodiment, and the distance L between the touch point of the current gesture and the image acquisition device is calculated. For example, when the touch point is Q, the distance L between the touch point of the current gesture and the camera is calculated. distance L _sQ ;

Because the value range of △ is set at 5mm-15mm, if the condition is satisfied when △ takes the maximum value, any value within the range can satisfy the condition, so there are:

If LL _min ≤ 15mm, it is determined that the operation area where the current gesture is located is not located in the preset touch plane area, otherwise, it is determined that the operation area is located in the preset touch plane area. For example, if L _sQ ≤L _min -15mm, it is determined that the current gesture is a three-dimensional gesture, otherwise, it is a touch gesture.

As can be seen from the above, an embodiment of the present invention provides a method for gesture recognition. By determining whether the area where the current gesture operation is located in the image information captured by the 3D camera is located in the preset touch plane area, it is determined that the current gesture is a touch gesture. Or three-dimensional gestures. If it is a touch gesture, the operation gesture is recognized according to the touch position information of the touch gesture; if the current gesture is a three-dimensional gesture, the pre-established three-dimensional gesture model library is called to identify the operation gesture of the three-dimensional gesture, and the three-dimensional gesture is realized. Gesture and 2D gesture recognition are integrated, which is convenient for users to use and improves the user's comfort of human-computer interaction.

The embodiment of the present invention also provides a corresponding implementation device for the gesture recognition method, which further makes the method more practical. The gesture recognition device provided by the embodiment of the present invention is introduced below. The gesture recognition device described below and the gesture recognition method described above can be referred to each other correspondingly.

Embodiment 4:

Referring to FIG. 10, FIG. 10 is a structural diagram of a gesture recognition device provided by an embodiment of the present invention, and the device may include:

The image acquisition module 1001 is used for acquiring image information acquired by an image acquisition device.

The image detection module 1002 is configured to detect the current gesture in the image information.

The judgment area module 1003 is used for judging the operation area where the current gesture is located.

The information recognition module 1004 is used to recognize the current gesture:

When the operation area where the current gesture is located is located in the preset touch plane area, and when it is determined that the gesture information is a touch gesture, the operation gesture is identified according to the touch position of the touch gesture;

When the operation area where the current gesture is located is not located in the preset touch plane area, it is determined that the gesture information is a three-dimensional gesture, and a pre-established three-dimensional gesture model library is called to identify the operation gesture of the three-dimensional gesture.

The instruction generation module 1005 is configured to generate a corresponding operation instruction according to the recognized operation gesture.

Preferably, the judging area module 1003 may include:

a first determination unit, which detects whether the button on the preset touch plane area used to indicate whether there is a touch signal is turned on, and if so, determines that the operation area where the current gesture is located is located in the preset touch plane area; if No, it is determined that it is not located in the preset touch plane area; and/or

a second judging unit, calculating the depth values of the image acquisition device and each pixel in the preset touch plane area, and determining the minimum depth value L _min ;

Calculate the distance L between the touch point of the current gesture and the image acquisition device;

If LL _min ≤ 15mm, it is determined that the operation area where the current gesture is located is not located in the preset touch plane area, otherwise, it is determined that the operation area is located in the preset touch plane area.

Optionally, in other implementations of this embodiment, the apparatus may further include, for example:

The infrared light supplement module is used to emit specific modulated infrared light, and cooperate with the image acquisition equipment to obtain clear image signals. In this way, when the current ambient lighting is insufficient, a clear image can still be obtained, which is helpful for better gesture recognition.

The functions of each functional module of the gesture recognition device according to the embodiment of the present invention may be specifically implemented according to the methods in the foregoing method embodiments, and the specific implementation process may refer to the relevant descriptions of the foregoing method embodiments, which will not be repeated here.

As can be seen from the above, an embodiment of the present invention provides a gesture recognition device, which determines whether the current gesture is a touch gesture by judging whether the area where the current gesture operation is located in the image information collected by the image acquisition device is located in the preset touch plane area. It is also a three-dimensional gesture, which realizes the integration of three-dimensional gesture and two-dimensional gesture recognition, which is more convenient for users to use and improves the control comfort of user-computer interaction.

Embodiment 5:

Referring to FIG. 11 , an embodiment of the present invention further provides a vehicle-mounted system, which may include:

an image acquisition device 1101 for acquiring image information;

touch panel 1102, used to realize the operation of two-dimensional gestures;

The gesture recognition device 1103 is used to recognize the current gesture, and is the gesture recognition device described in the fourth embodiment.

Wherein, the image capture device can be installed on the roof light module, and the image capture device can be placed on the roof of the car to expand its sensing area, and the image capture device can be installed on the dome light to avoid the cost of installing the image capture device in a separate location. Of course, it may not be installed on the car. The lamp module can be set according to the user's preference. It should be noted that the image capture device can also be set to rotate, that is, to capture images in 360 degrees without dead ends, but the problem caused by this is that the user needs to track the rotation angle of the image capture device to make gestures, which will bring For some inconvenience, preferably, the image acquisition device can be fixed, and the user only needs to perform gesture operation in its sensing area. The image acquisition device can use a 3D camera or a 2D camera, of course, other image acquisition devices can be used when necessary.

The touch panel 1102 may include a touch panel and/or a touch switch, and the touch panel and the touch switch may be integrally and seamlessly disposed on the touch panel, that is to say, there is no gap between the touch panel and the touch switch, and the whole is a board , so you can avoid the trouble of cleaning the touchpad, and it is beautiful and beautiful.

Optionally, in some implementations of this embodiment, when the image acquisition device is a 2D camera, the touch panel includes, for example, a touch pad and a touch switch, please refer to FIG. 12 and FIG. 13 , and FIG. 12 is the present invention 13 is a schematic structural diagram of a 2D camera-based touch switch provided by an embodiment of the present invention. The touchpad and the touch switch may include:

The touchpad includes:

The panel unit 1201 can be a liquid crystal display or a plastic panel or a metal panel or a panel of any other material;

circuit board unit 1202;

key unit 1203;

The supporter unit 1204, the supporter unit includes a supporter and a casing, and the material of which is preferably plastic, but of course other materials are also possible.

Among them, 1205 represents a touch operation such as clicking and sliding with the palm of the hand facing the touchpad ABCD; the numeral 1206 in the figure represents a touch operation such as clicking and sliding with the fingernail of the back of the hand facing the touchpad ABCD.

The touch switch includes:

With a beacon panel unit 1301, it can be a liquid crystal display or a plastic panel or a metal panel or a panel of any other material;

The light guide plate unit 1302 can preferably be made of transparent plastic material, of course, it can also be made of other materials, but preferably it should be transparent;

circuit board unit 1303, on which buttons and lights are arranged;

The button unit 1304 can be a micro-stroke non-self-locking elastic switch;

A lamp unit 1305 is provided, and the lamp is generally a lamp with a small volume and low power consumption, such as an LED lamp;

The supporter unit 1306, the supporter unit includes a supporter and a casing, and the material of which is preferably plastic, but of course other materials are also possible.

Among them, 1307 represents a touch operation such as clicking and sliding with the palm of the hand toward the touch switch EF; the numeral 1308 in the figure represents a touch operation such as clicking and sliding with the fingernail of the back of the hand toward the touch switch EF.

Optionally, in other implementations of this embodiment, when the image acquisition device is a 3D camera, the touch panel includes, for example, a touch pad and a touch switch, please refer to FIG. 14 and FIG. 15 , and FIG. 14 is this FIG. 15 is a schematic structural diagram of a 3D camera-based touch switch provided by an embodiment of the present invention. The touch panel and the touch switch may include :

The touchpad includes:

The panel unit 1401 can be a liquid crystal display or a plastic panel or a metal panel or a panel of any other material;

The supporter unit 1402, the supporter unit includes a supporter and a casing, and the material of which is preferably plastic, but of course other materials are also possible.

Compared with the structure diagram of the touchpad based on the 2D camera as the recognition sensor shown in Figure 12, it can be seen that the structure of the touchpad based on the 3D camera as the recognition sensor is simpler, and it does not need any electronic components and circuit boards. The touch panel is used, which can save the user's use cost.

The touch switch includes:

Panel unit with beacon, light guide plate unit, setting light unit and supporter unit.

With a beacon panel unit 1501, it can be a liquid crystal display or a plastic panel or a metal panel or a panel of any other material;

The light guide plate unit 1502 can preferably be made of transparent plastic material, of course, it can also be made of other materials, but preferably it should be transparent;

Circuit board unit 1503, on which lights are arranged;

The supporter unit 1504, the supporter unit includes a supporter and a casing, and its material is preferably plastic, of course, it can also be made of other materials;

A lamp unit 1505 is provided, and the lamp is generally smaller in size and consumes less power, such as an LED lamp.

Compared with the schematic structure diagram of the touch switch based on the 2D camera as the recognition sensor shown in Figure 13, it can be seen that the touch switch based on the 3D camera as the recognition sensor lacks a key unit, which simplifies the structure and function, and can save users to a certain extent. cost.

It should be noted that an in-vehicle system not only includes the above-described units, but also includes many units, and this embodiment only provides relevant detailed descriptions for units that have been improved in the prior art. The gesture recognition device is the gesture recognition device described in the fourth embodiment. For a specific implementation process, reference may be made to the related descriptions of the above method embodiments and device embodiments, which will not be repeated here.

As can be seen from the above, an embodiment of the present invention provides a vehicle-mounted system for gesture recognition, which determines whether the current gesture is a touch by judging whether the area where the current gesture operation is located in the image information collected by the image acquisition device is located in a preset touch plane area. Gestures are still three-dimensional gestures, which realizes the integration of three-dimensional gestures and two-dimensional gesture recognition, which is more convenient for users to use, and improves the control comfort of users' human-computer interaction. In addition, the structure and function of the touch panel of the present application are more simplified, saving to a certain extent. user cost.

In order to facilitate better understanding and implementation of the above solutions in Embodiment 4 and Embodiment 5 of the present invention, the following describes specific application scenarios by way of example. Please refer to FIG. 16 . FIG. 16 is a structural block diagram of an in-vehicle system capable of gesture recognition and control according to an embodiment of the present invention.

The control unit 1601 includes a gesture recognition device, which is the control core of the in-vehicle system. The control unit 1601 can be used to acquire the image information of the camera 1602, and according to the acquired information, detect the operator's gesture in real time, and use the gesture recognition unit to detect the gesture of the operator. The detected gesture is recognized;

Wherein, when the camera is a 2D camera, the control unit 1601 can be used for real-time detection of the key signal 1608 inside the touch panel of the central control panel and the internal key signal of the touch switch 1607 of the central control panel, and judge the current finger touch according to the key signal The region where the operation is located.

The control unit 1601 controls the infrared supplementary light module 1603 to emit specific modulated infrared light, and cooperates with the image acquisition device to obtain clear image signals. In this way, when the current ambient lighting is insufficient, a clear image can still be obtained, which is helpful for better gesture recognition.

The control unit 1601 can also be used to detect the car headlight signal 1606, when it is detected that the car headlight is turned on, the corresponding control and lighting of the touch switch backlight 1605 of the center control panel; when it is detected that the car headlight 1606 is turned off, Correspondingly control and turn off the touch switch backlight 1605 of the center control panel.

The control unit 1601 can also control the interface unit 1605, and is used to send the real-time detected three-dimensional gesture and touch gesture related information such as position information to a specific electronic function module, execute corresponding gesture instructions, and realize human-computer interaction. The interface unit module, usually a CAN bus, a Lin bus or an analog level signal interface, is an interface used to match a specific electronic function module, and an interface used to communicate with a specific electronic function module.

By implanting a gesture recognition device in the control unit of the in-vehicle system, the gesture recognition device is the gesture recognition device described in the above method embodiment, device and system embodiment, and can recognize the user's three-dimensional gesture or two-dimensional gesture during the driving process of the user. Dimensional gestures greatly improve the comfort of the user's human-computer interaction operation.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments may be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method.

Professionals may further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two, in order to clearly illustrate the possibilities of hardware and software. Interchangeability, the above description has generally described the components and steps of each example in terms of functionality. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

The steps of a method or algorithm described in conjunction with the embodiments disclosed herein may be directly implemented in hardware, a software module executed by a processor, or a combination of the two. A software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other in the technical field. in any other known form of storage medium.

The method, device and vehicle-mounted system for gesture recognition provided by the present invention are described above in detail. The principles and implementations of the present invention are described herein by using specific examples, and the descriptions of the above embodiments are only used to help understand the method and the core idea of the present invention. It should be pointed out that for those skilled in the art, without departing from the principle of the present invention, several improvements and modifications can also be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims (6)

1. a method for gesture recognition, is characterized in that, comprises: Obtain the image information collected by the image collection device; detecting the current gesture in the image information; Determine the operation area where the current gesture is located; When the operation area where the current gesture is located is located in the preset touch plane area, determine that the current gesture is a touch gesture, and identify the operation gesture according to the touch position information of the touch gesture; When the operation area where the current gesture is located is not located in the preset touch plane area, determine that the current gesture is a three-dimensional gesture, call a pre-established three-dimensional gesture model library, and identify the operation gesture of the three-dimensional gesture; Generate a corresponding operation instruction according to the identified operation gesture; The current gesture is any one or any combination of the following: Gestures of touching or clicking with the center of the finger, the side of the finger, or the fingernail; The calling of the pre-established 3D gesture model library is: Continuously acquire multiple frames of image information including the current gesture in time series. If the current gestures in the multiple frames of images are the same and the position of the current gesture has not changed, the current gesture is a static gesture, and the call 3D static gesture model library; otherwise, call the 3D dynamic gesture model library; When the image information is two-dimensional plane information, the step of judging the operation area where the current gesture is located includes: Detect whether the button on the preset touch plane area for indicating whether there is a touch signal is turned on, if so, determine that the operation area where the current gesture is located is located in the preset touch plane area; if not, determine that it is not located in the preset touch plane area. 2. The method according to claim 1, wherein when the image information includes two-dimensional plane information and one-dimensional depth image information, the step of judging the operation area where the current gesture is located comprises: Determine the minimum depth value Lmin according to the image acquisition device and the depth value of each pixel in the preset touch plane area; Calculate the distance L between the touch point of the current gesture and the image acquisition device; If L-Lmin≤15mm, it is determined that the operation area where the current gesture is located is not located in the preset touch plane area, otherwise, it is determined to be located in the preset touch plane area. 3. A device for gesture recognition, comprising: an image acquisition module for acquiring image information collected by an image acquisition device; an image detection module for detecting the current gesture in the image information; a judging area module for judging the operating area where the current gesture is located; The information recognition module is used to recognize the current gesture: when the operation area where the current gesture is located is located in the preset touch plane area, when it is determined that the gesture information is a touch gesture, the operation gesture is determined according to the touch position of the touch gesture. Recognize; when the operation area where the current gesture is located is not located in the preset touch plane area, determine that the gesture information is a three-dimensional gesture, call a pre-established three-dimensional gesture model library, and identify the operation gesture of the three-dimensional gesture; an instruction generation module, configured to generate a corresponding operation instruction according to the identified operation gesture; The information recognition module is also specifically used to: the current gesture recognized is any one or any combination of the following: a gesture of touching or clicking on the center of a finger, the side of a finger, or a fingernail; calling a pre-established three-dimensional gesture model The library is: continuous acquisition of multiple frames of image information including the current gesture in time series, if the current gesture in the multi-frame images is the same and the position of the current gesture does not change, the current gesture is a static gesture , call the 3D static gesture model library; otherwise, call the 3D dynamic gesture model library; The judging area module includes: a first determination unit, configured to detect whether the button on the preset touch plane area used to indicate whether there is a touch signal is turned on, and if so, determine that the operation area where the current gesture is located is located in the preset touch plane area ; if not, it is determined not to be located in the preset touch plane area; and/or The second judgment unit is configured to calculate the depth value of the image acquisition device and each pixel in the preset touch plane area, and determine the minimum depth value Lmin; calculate the touch point of the current gesture and the image acquisition device. Distance L; if L-Lmin≤15mm, it is determined that the operation area where the current gesture is located is not located in the preset touch plane area, otherwise, it is determined to be located in the preset touch plane area. 4 . A vehicle-mounted system for gesture recognition, comprising an image acquisition device, a touch panel, and the gesture recognition device according to claim 3 . 5. The vehicle-mounted system according to claim 4, wherein the touch panel comprises a touch panel and/or a touch switch, and when the image acquisition device is a 2D camera, the touch panel comprises: Panel unit, circuit board unit, key unit and support unit; The touch switch includes: With beacon panel unit, circuit board unit, light guide plate unit, key unit, setting light unit and support unit unit. 6. The vehicle-mounted system according to claim 4, wherein the touch panel comprises a touch panel and/or a touch switch, and when the image acquisition device is a 3D camera, the touch panel comprises: Panel units and support units; The touch switch includes: Panel unit with beacon, circuit board unit, light guide plate unit, setting light unit and supporter unit.

Images