CN104935698A - A shooting method, shooting device and smart phone of an intelligent terminal - Google Patents
A shooting method, shooting device and smart phone of an intelligent terminal Download PDFInfo
- Publication number
- CN104935698A CN104935698A CN201510350524.8A CN201510350524A CN104935698A CN 104935698 A CN104935698 A CN 104935698A CN 201510350524 A CN201510350524 A CN 201510350524A CN 104935698 A CN104935698 A CN 104935698A
- Authority
- CN
- China
- Prior art keywords
- face
- photo
- shooting
- front camera
- prompt
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Studio Devices (AREA)
Abstract
Description
技术领域technical field
本发明涉及智能终端的拍摄领域，尤其涉及一种智能终端的拍摄方法、拍摄装置及智能手机。The present invention relates to the field of photographing of intelligent terminals, in particular to a photographing method of an intelligent terminal, a photographing device and a smart phone.
背景技术Background technique
如今一般手机等智能终端的后置摄像头的像素普遍高于前置摄像头的像素，通过前置摄像头拍摄的照片效果不好，通过后置摄像头的照片效果好，但通过后置摄像头进行自拍或视频操作，往往会使拍摄的照片或视频中人脸的位置不理想，很难拍摄出满意的照片或视频。Nowadays, the pixels of the rear camera of general mobile phones and other smart terminals are generally higher than the pixels of the front camera. Operation, often make the position of the face in the captured photo or video unsatisfactory, and it is difficult to take a satisfactory photo or video.
发明内容Contents of the invention
针对上述缺陷，本发明的目的在于提供一种智能终端的拍摄方法、拍摄装置及智能手机，以通过后置摄像头拍摄时提示用户理想的自拍位置。In view of the above defects, the object of the present invention is to provide a shooting method, a shooting device and a smart phone of an intelligent terminal, so as to prompt the user of an ideal self-portrait position when shooting through a rear camera.
为达此目的，本发明提供了一种手机的拍摄方法，包括以下步骤：To achieve this purpose, the invention provides a method for photographing a mobile phone, comprising the following steps:
通过前置摄像头拍摄带有人脸的照片，获取所述照片中人脸位置；Take a photo with a human face through the front camera, and obtain the position of the human face in the photo;
通过后置摄像头拍摄模式对所述后置摄像头获取的实时图像进行人脸位置识别，识别当前图像中人脸位置与所述前置摄像头拍摄的照片中人脸位置匹配，则发出拍摄提示。Face position recognition is performed on the real-time image acquired by the rear camera through the rear camera shooting mode, and if the face position in the current image matches the face position in the photo taken by the front camera, a shooting prompt is issued.
优选地，所述获取所述照片中人脸位置，包括：Preferably, said obtaining the face position in said photo includes:
识别所述照片中人脸的双眼坐标；identifying the coordinates of the eyes of the face in the photo;
确定所述照片中人脸的双眼距离和所述照片中的人脸方框位置；Determine the distance between the eyes of the face in the photo and the frame position of the face in the photo;
所述通过后置摄像头拍摄模式对所述后置摄像头获取的实时图像进行人脸位置识别，包括：The real-time image obtained by the rear camera through the rear camera shooting mode is carried out to face position recognition, including:
识别所述实时图像中人脸的双眼坐标；Recognize the binocular coordinates of the human face in the real-time image;
确定所述实时图像中人脸的双眼距离和实时图像中的人脸方框位置。Determine the binocular distance of the face in the real-time image and the frame position of the face in the real-time image.
优选地，识别当前图像中人脸位置与所述前置摄像头拍摄的照片中人脸位置匹配，则发出拍摄提示，具体为，识别所述照片的双眼距离与所述当前图像中的双眼距离相匹配，且所述照片中的人脸方框位置与所述当前图像中人脸方框位置的重合面积与所述照片中的人脸方框位置面积的比值大于预置比例，则发出声音提示。Preferably, if the position of the face in the current image matches the position of the face in the photo taken by the front camera, a shooting prompt is issued, specifically, identifying the distance between the eyes of the photo and the distance between the eyes in the current image. match, and the ratio of the overlapping area of the face frame position in the photo to the face frame position in the current image to the area of the face frame position in the photo is greater than the preset ratio, a sound prompt will be issued .
优选地，所述通过前置摄像头拍摄带有人脸的照片，具体为：通过前置摄像头拍摄带有人脸的照片并通过重力传感器获取前置摄像头的倾斜角度；Preferably, said taking a photo with a human face through the front camera is specifically: taking a photo with a human face through the front camera and obtaining the tilt angle of the front camera through a gravity sensor;
所述获取拍摄指令，拍摄照片或视频，还包括：获取当前后置摄像头的倾斜角度与所述前置摄像头的倾斜角度相匹配，则拍摄照片或视频优选地，所述拍摄提示为声音提示、灯光提示和震动提示中的一种或几种。The acquisition of shooting instructions, taking photos or videos, also includes: obtaining that the tilt angle of the current rear camera matches the tilt angle of the front camera, then taking photos or videos. Preferably, the shooting prompt is a sound prompt, One or more of light prompts and vibration prompts.
优选地，所述声音提示的音调和/或音量随着当前图像中人脸位置与所述前置摄像头拍摄的照片中人脸位置的匹配度增加而升高或降低，所述灯光提示的亮度和/或闪烁频率随着当前图像中人脸位置与所述前置摄像头拍摄的照片中人脸位置的匹配度增加而升高或降低，所述震动提示的强度随着当前图像中人脸位置与所述前置摄像头拍摄的照片中人脸位置的匹配度增加而升高或降低。Preferably, the tone and/or volume of the sound prompt increases or decreases as the matching degree between the face position in the current image and the face position in the photo taken by the front camera increases, and the brightness of the light prompt And/or the flickering frequency increases or decreases as the matching degree between the position of the face in the current image and the position of the face in the photo taken by the front camera increases, and the intensity of the vibration prompt increases or decreases with the position of the face in the current image. The degree of matching with the position of the face in the photo taken by the front camera increases or decreases.
本发明还提供了一种拍摄装置，包括：The present invention also provides a photographing device, comprising:
拍摄照片单元，用于通过前置摄像头拍摄带有人脸的照片，获取所述照片中人脸位置；A photo taking unit is used to take a photo with a human face through the front camera, and obtain the position of the human face in the photo;
提示单元，通过后置摄像头拍摄模式对所述后置摄像头获取的实时图像进行人脸位置识别，识别当前图像中人脸位置与所述前置摄像头拍摄的照片中人脸位置匹配，则发出拍摄提示。The prompting unit performs face position recognition on the real-time image acquired by the rear camera through the rear camera shooting mode, recognizes that the face position in the current image matches the face position in the photo taken by the front camera, and sends a shooting hint.
优选地，所述拍摄照片单元中包括：Preferably, the photographing unit includes:
第一拍摄模块，用于通过前置摄像头拍摄带有人脸的照片；The first shooting module is used to take photos with human faces through the front camera;
第一识别双眼坐标模块，用于识别所述照片中人脸的双眼坐标；The first module for identifying binocular coordinates is used to identify the binocular coordinates of the face in the photo;
第一确定人脸位置模块，用于确定所述照片中人脸的双眼距离和所述照片中的人脸方框位置；The first module for determining the face position is used to determine the distance between the eyes of the face in the photo and the position of the face frame in the photo;
提示单元中包括：Included in the prompt unit are:
第二识别双眼坐标模块，用于识别所述实时图像中人脸的双眼坐标；The second module for identifying binocular coordinates is used to identify the binocular coordinates of the face in the real-time image;
第二确定人脸位置模块，用于确定所述实时图像中人脸的双眼距离和实时图像中的人脸方框位置；The second module for determining the position of the human face is used to determine the distance between the eyes of the human face in the real-time image and the position of the human face frame in the real-time image;
人脸识别模块，用于识别当前图像中人脸位置与所述前置摄像头拍摄的照片中的人脸位置匹配，则发出拍摄提示。The face recognition module is used to identify that the position of the face in the current image matches the position of the face in the photo taken by the front camera, and then issue a shooting prompt.
优选地，所述人脸识别模块，用于：Preferably, the face recognition module is used for:
识别所述照片的双眼距离与所述当前图像中的双眼距离相匹配，且所述照片中的人脸方框位置与所述当前图像中人脸方框位置的重合面积与所述照片中的人脸方框位置面积的比值大于预置比例，则发出拍摄提示。It is recognized that the distance between the eyes of the photo matches the distance between the eyes in the current image, and the overlapping area of the position of the human face frame in the photo and the position of the human face frame in the current image is the same as that in the photo If the ratio of the area of the face frame is greater than the preset ratio, a shooting prompt will be issued.
优选地，所述拍摄模块，用于：通过前置摄像头拍摄带有人脸的照片并通过重力传感器获取前置摄像头的倾斜角度，获取所述照片中人脸位置；Preferably, the shooting module is configured to: take a photo with a human face through the front camera and obtain the tilt angle of the front camera through the gravity sensor, and obtain the position of the face in the photo;
所述拍摄单元，用于发出拍摄提示后，获取当前后置摄像头的倾斜角度与所述前置摄像头的倾斜角度相匹配，则拍摄照片或视频。The shooting unit is configured to acquire a match between the tilt angle of the current rear camera and the tilt angle of the front camera after sending out a shooting prompt, and then take photos or videos.
一种智能手机，包括用于实现通讯的通讯信号处理单元以及处理和存储信息的信息处理单元，所述信息处理单元连接所述通讯信号处理单元，其特征在于，所述信息处理单元中还包括权利要求7至10中任意一项所述的拍摄照片单元、提示单元和拍摄单元。A smart phone, including a communication signal processing unit for realizing communication and an information processing unit for processing and storing information, the information processing unit is connected to the communication signal processing unit, and it is characterized in that the information processing unit also includes The photographing unit, prompting unit and photographing unit described in any one of claims 7 to 10.
本发明采用上述技术方案，与现有的技术方案相比，具有以下有益的效果：The present invention adopts above-mentioned technical scheme, compares with existing technical scheme, has following beneficial effect:
本发明可以通过使用前置摄像头拍摄照片并通过识别人脸位置来提示用户，实现快捷准确的用后置摄像头来自拍或摄像，使人脸在通过后置摄像头自拍的照片或视频中的位置理想；通过得到前置摄像头拍照时的倾斜角度，可以在后置摄像头开启拍摄模式后通过检测倾斜角度来自动拍摄，使拍摄角度理想，拍摄效果更佳；通过变化音量、音调、灯光闪烁频率、灯光亮度或震动强度来提示用户人脸位置的准确度，更加直观，方便用户根据提示变化调整移动手机的位置，更快找到理想拍摄位置。The present invention can use the front camera to take pictures and prompt the user by identifying the position of the face, so as to realize fast and accurate self-portrait or video recording with the rear camera, so that the position of the face in the photo or video self-portraited by the rear camera is ideal ;By obtaining the tilt angle of the front camera when taking pictures, you can automatically shoot by detecting the tilt angle after the rear camera turns on the shooting mode, so that the shooting angle is ideal and the shooting effect is better; by changing the volume, tone, light flashing frequency, light Brightness or vibration intensity is used to remind the user of the accuracy of the face position, which is more intuitive, and it is convenient for the user to adjust the position of the mobile phone according to the change of the prompt, and find the ideal shooting position faster.
附图说明Description of drawings
图1是本发明实施例一中手机的拍摄方法的流程图一；Fig. 1 is the flow chart one of the photographing method of mobile phone in the embodiment one of the present invention;
图2是本发明实施例一中手机的拍摄方法的流程图二；Fig. 2 is the flowchart two of the shooting method of the mobile phone in the first embodiment of the present invention;
图3是本发明实施例一中手机的拍摄方法的流程图三；Fig. 3 is a flowchart three of the photographing method of the mobile phone in Embodiment 1 of the present invention;
图4是本发明实施例二中手机的拍摄方法的流程图；Fig. 4 is the flow chart of the photographing method of mobile phone in the second embodiment of the present invention;
图5是本发明实施例三和实施例四中手机的拍摄装置的结构框图一；Fig. 5 is a structural block diagram 1 of the photographing device of the mobile phone in Embodiment 3 and Embodiment 4 of the present invention;
图6是本发明实施例三中手机的拍摄装置的结构框图二；Fig. 6 is a structural block diagram 2 of the photographing device of the mobile phone in Embodiment 3 of the present invention;
图7是本发明实施例三中手机的拍摄装置的结构框图三；Fig. 7 is a structural block diagram 3 of the photographing device of the mobile phone in Embodiment 3 of the present invention;
图8是本发明实施例五中智能手机的结构框图。FIG. 8 is a structural block diagram of a smart phone in Embodiment 5 of the present invention.
具体实施方式Detailed ways
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.
实施例一Embodiment one
附图1-3为本发明实施例一的一种手机的拍摄方法的步骤流程图，该方法包括以下步骤：Accompanying drawing 1-3 is the flow chart of the steps of a mobile phone shooting method according to Embodiment 1 of the present invention, and the method includes the following steps:
100：通过前置摄像头拍摄带有人脸的照片，获取照片中人脸位置。100: Use the front camera to take a photo with a face, and obtain the position of the face in the photo.
本实施例中，具体为，101：通过前置摄像头拍摄带有人脸的照片。In this embodiment, specifically, 101: Take a photo with a human face through a front camera.
例如，先打开前置摄像头或通过触发指令开启前置摄像头，并通过前置摄像头捕捉图像，识别到人脸后，用户点击快门，则拍摄成功，如果没有识别到人脸或识别到人脸不完整时用户便进行拍摄，则提示拍摄失败，需要重新拍摄。For example, first turn on the front-facing camera or turn on the front-facing camera through a trigger command, and capture an image through the front-facing camera. After the face is recognized, the user clicks the shutter, and the shooting is successful. If the user shoots when it is complete, it prompts that the shooting failed and needs to be shot again.
102：识别照片中人脸的双眼坐标。例如，识别照片中的双眼坐标(x1,y1)、(x2,y2)。102: Identify the coordinates of the eyes of the face in the photo. For example, the binocular coordinates (x 1 , y 1 ), (x 2 , y 2 ) in the photo are recognized.
103：确定照片中人脸的双眼距离和人脸方框位置。人脸方框位置为通过双眼坐标进行标准化计算得到的人脸方框所在的位置。103: Determine the distance between the eyes of the face in the photo and the position of the frame of the face. The position of the human face frame is the position of the human face frame obtained by standardizing and calculating the binocular coordinates.
如上例中，双眼距离为(x1,y1)和(x2,y2)通过计算得出，根据双眼坐标来确定人脸方框的左上顶角和右下顶角的坐标分别为(X1,Y1),(X2,Y2)，下面为详细计算过程：As in the above example, the distance between the eyes is (x 1 ,y 1 ) and (x 2 ,y 2 ) by It is calculated that the coordinates of the upper left corner and the lower right corner of the face frame are determined according to the coordinates of the eyes, respectively (X 1 , Y 1 ), (X 2 , Y 2 ), and the following is the detailed calculation process:
Weyes＝x2-x1；W eyes = x 2 -x 1 ;
Wface＝Weyes/a；W face = W eyes /a;
X1＝x1-(Wface-Weyes)/2；X 1 =x 1 -(W face -W eyes )/2;
X2＝X1+Wface；X 2 =X 1 +W face ;
Heyes＝(y1+y2)/2；H eyes = (y 1 +y 2 )/2;
Hface＝Wface；H face = W face ;
Y1＝Heyes-Hface/b；Y 1 =H eyes -H face /b;
Y2＝Y1+Hface；Y 2 =Y 1 +H face ;
其中，a和b均为经验常数，a取值为2，b取值为3.5，由此得到的人脸方框的尺寸随着双眼距离Weyes的变化而变化。Among them, a and b are empirical constants, the value of a is 2, and the value of b is 3.5. The size of the obtained face frame changes with the change of the distance W eyes between the eyes.
200：通过后置摄像头拍摄模式对后置摄像头获取的实时图像进行人脸识别，识别当前图像中人脸位置与前置摄像头拍摄的照片中的人脸位置匹配，则发出拍摄提示。200: Use the rear camera shooting mode to perform face recognition on the real-time images acquired by the rear camera, recognize that the face position in the current image matches the face position in the photo taken by the front camera, and issue a shooting prompt.
本实施例中，具体为，201：开启后置摄像头拍摄模式，本实施例中，识别照片中人脸的位置后，便自动开启后置摄像头并切换到后置摄像头拍摄模式。In this embodiment, specifically, 201: Turn on the shooting mode of the rear camera. In this embodiment, after the position of the face in the photo is recognized, the rear camera is automatically turned on and switched to the shooting mode of the rear camera.
202：识别实时图像中人脸的双眼坐标。202: Identify the binocular coordinates of the face in the real-time image.
203：确定实时图像中人脸的双眼距离和实时图像中的人脸方框位置。可以采用上述步骤103中的方法来确定实时图像中人脸的双眼距离和实时图像中的人脸方框位置。203: Determine the distance between the eyes of the face in the real-time image and the position of the frame of the face in the real-time image. The method in step 103 above can be used to determine the distance between the eyes of the face in the real-time image and the position of the frame of the face in the real-time image.
204：识别当前图像中人脸位置与前置摄像头拍摄的照片中的人脸位置匹配，则发出拍摄提示。204: Identify that the position of the face in the current image matches the position of the face in the photo taken by the front camera, and issue a shooting prompt.
拍摄提示为声音提示、灯光提示和震动提示中的一种或几种。声音提示的音调和/或音量随着当前图像中人脸位置与前置摄像头拍摄的照片中人脸位置的匹配度增加而升高或降低，灯光提示的亮度和/或闪烁频率随着当前图像中人脸位置与前置摄像头拍摄的照片中人脸位置的匹配度增加而升高或降低，震动提示的强度随着当前图像中人脸位置与前置摄像头拍摄的照片中人脸位置的匹配度增加而升高或降低。本发明通过使用前置摄像头拍摄照片并通过识别人脸位置来提示用户，实现快捷准确的用后置摄像头来自拍或摄像，使人脸在通过后置摄像头自拍的照片或视频中的位置理想。The shooting prompt is one or more of sound prompt, light prompt and vibration prompt. The tone and/or volume of the sound prompt increases or decreases with the matching degree of the face position in the current image and the face position in the photo taken by the front camera, and the brightness and/or flashing frequency of the light prompt increases or decreases with the current image The matching degree of the face position in the current image and the face position in the photo taken by the front camera increases or decreases, and the strength of the vibration prompt increases with the match between the face position in the current image and the face position in the photo taken by the front camera increase or decrease with increasing degrees. The present invention uses the front camera to take photos and prompts the user by identifying the position of the face, so as to realize fast and accurate self-portrait or video recording with the rear camera, so that the position of the face in the photo or video self-portrait through the rear camera is ideal.
本实施例中，判断当前图像中的双眼距离与照片的双眼距离相匹配，且照片中的人脸方框位置与当前图像中人脸方框位置的重合面积M与照片中的人脸方框位置面积的比值大于预置比例，则发出声音提示。其中，重合面积M与照片中的人脸方框位置面积的比值以及双眼距离的识别中不需要设定识别顺序，只需要识别其中一项不符合标准，则不发出声音提示。In this embodiment, it is determined that the distance between the eyes in the current image matches the distance between the eyes in the photo, and the overlapping area M between the position of the face frame in the photo and the position of the face frame in the current image is the same as the face frame in the photo If the ratio of the location area is greater than the preset ratio, a sound prompt will be issued. Among them, the ratio of the overlapping area M to the area of the face frame in the photo and the recognition of the distance between the eyes do not need to set the recognition order, only one of them needs to be recognized if it does not meet the standard, and no sound prompt will be issued.
本实施例中，预置比例可以取75％，当前图像中的双眼距离与照片的双眼距离相匹配，具体为，当前图像的双眼距离与照片的双眼距离相同，或是当前图像的双眼距离为1-c至1+c倍的照片双眼距离，其中c可以取10％-20％。重合面积M与所述照片中的人脸方框位置面积的比值即为当前图像中人脸位置与前置摄像头拍摄的照片中人脸位置的匹配度，声音提示的音调和/或音量随着重合面积M与所述照片中的人脸方框位置面积的比值的增加而升高以提示用户人脸位置的准确度，更加直观，方便用户根据声音变化调整移动手机的位置，更快找到理想拍照位置。In this embodiment, the preset ratio can be 75%. The distance between the eyes in the current image matches the distance between the eyes in the photo. Specifically, the distance between the eyes in the current image is the same as the distance between the eyes in the photo, or the distance between the eyes in the current image is 1-c to 1+c times the distance between the eyes of the photo, where c can take 10%-20%. The ratio of the overlapping area M to the area of the face frame position in the photo is the matching degree of the face position in the current image and the face position in the photo taken by the front camera. The ratio of the overlapping area M to the area of the face frame in the photo increases to remind the user of the accuracy of the face position, which is more intuitive and convenient for the user to adjust the position of the mobile phone according to the sound change, and find the ideal position faster. photo location.
300：获取拍摄指令，拍摄照片或视频。300: Obtain a shooting instruction, and take a photo or video.
拍摄指令可以为用户输入的按键或触屏指令，也可以为动作指令、语言指令或角度位置指令，角度位置指令作为拍摄指令的详细介绍见实施例二。为了便捷操作，发出拍摄提示后，动作指令可以为识别当前人脸为笑脸，则对焦当前图像中人脸方框位置后拍照或摄像。The shooting instruction can be a button or a touch screen instruction input by the user, and can also be an action instruction, a language instruction, or an angle position instruction. For a detailed introduction of the angle position instruction as a shooting instruction, see Embodiment 2. For convenient operation, after sending out the shooting prompt, the action command can be to recognize the current face as a smiling face, then focus on the position of the face frame in the current image and then take a photo or video.
实施例二Embodiment two
附图4为本发明实施例二的一种拍摄装置的结构框图，该方法包括实施例一中的步骤100、步骤200和步骤300，其中该方法的步骤100中进一步为：通过前置摄像头拍摄带有人脸的照片并通过重力传感器获取前置摄像头的倾斜角度，获取所述照片中人脸位置。Accompanying drawing 4 is the structural block diagram of a kind of photographing device of embodiment 2 of the present invention, and this method comprises step 100, step 200 and step 300 in embodiment 1, wherein step 100 of this method further is: through the front camera A photo with a human face is used to obtain the tilt angle of the front camera through the gravity sensor, and obtain the position of the human face in the photo.
为了使拍照角度理想，步骤300具体为获取当前后置摄像头的倾斜角度与所述前置摄像头的倾斜角度相匹配，则拍摄照片或视频。本实施例中将获取当前后置摄像头的倾斜角度与所述前置摄像头的倾斜角度相匹配作为拍摄指令，当识别当前后置摄像头的倾斜角度与所述前置摄像头的倾斜角度相匹配，则对焦当前图像中人脸方框位置后拍照或摄影，拍照时可以对焦的哪个区图像中人脸方框位置中的某一部位，如对焦人脸方框位置的中点、双眼或双眼中点等，且在拍照同时发出拍照声音，提示用户拍照成功。In order to make the photographing angle ideal, step 300 is specifically to acquire a match between the current tilt angle of the rear camera and the tilt angle of the front camera, and then take a photo or video. In this embodiment, matching the inclination angle of the current rear camera with the inclination angle of the front camera is obtained as a shooting instruction, and when it is recognized that the inclination angle of the current rear camera matches the inclination angle of the front camera, then After focusing on the position of the face frame in the current image and taking pictures or photography, which area in the image can be focused on when taking a photo, such as the midpoint of the face frame, the eyes, or the midpoint of both eyes etc., and a sound is emitted while taking a photo, prompting the user to take a photo successfully.
实施例三Embodiment Three
附图5-7为本发明实施例三的一种拍摄装置的结构框图，包括拍摄照片单元501、提示单元502以及拍摄单元503。5-7 are structural block diagrams of a photographing device according to Embodiment 3 of the present invention, including a photographing unit 501 , a prompting unit 502 and a photographing unit 503 .
拍摄照片单元501，用于通过前置摄像头拍摄带有人脸的照片，获取照片中人脸的位置。The photo taking unit 501 is configured to take a photo with a human face through the front camera, and acquire the position of the human face in the photo.
本实施例中，拍摄照片单元501中包括第一拍照模块511、第一识别双眼坐标模块512以及第一确定人脸位置模块513。第一拍照模块511，用于通过前置摄像头拍摄带有人脸的照片。例如，先打开前置摄像头或通过触发指令开启前置摄像头，并通过前置摄像头捕捉图像，识别到人脸后，用户点击快门，则拍摄成功，如果没有识别到人脸或识别到人脸不完整时用户便进行拍摄，则提示拍摄失败，需要重新拍摄。In this embodiment, the photographing unit 501 includes a first photographing module 511 , a first identifying binocular coordinates module 512 and a first determining human face position module 513 . The first photographing module 511 is configured to take photographs with human faces through the front camera. For example, first turn on the front-facing camera or turn on the front-facing camera through a trigger command, and capture an image through the front-facing camera. After the face is recognized, the user clicks the shutter, and the shooting is successful. If the user shoots when it is complete, it prompts that the shooting failed and needs to be shot again.
第一识别双眼坐标模块512，用于识别照片中人脸的双眼坐标。例如，识别照片中的双眼坐标(x1,y1)，(x2,y2)。The first identifying binocular coordinates module 512 is configured to identify the binocular coordinates of the face in the photo. For example, the coordinates (x 1 , y 1 ) and (x 2 , y 2 ) of the eyes in the photo are recognized.
第一确定人脸位置模块513，用于确定照片中人脸的双眼距离和人脸方框位置。The first face position determination module 513 is used to determine the distance between the eyes of the face in the photo and the position of the frame of the face.
如上例中，双眼距离为(x1,y1)和(x2,y2)通过计算得出，根据双眼坐标来确定人脸方框的左上顶角和右下顶角的坐标分别为(X1,Y1),(X2,Y2)，下面为详细计算过程：As in the above example, the distance between the eyes is (x 1 ,y 1 ) and (x 2 ,y 2 ) by It is calculated that the coordinates of the upper left corner and the lower right corner of the face frame are determined according to the coordinates of the eyes, respectively (X 1 , Y 1 ), (X 2 , Y 2 ), and the following is the detailed calculation process:
Weyes＝x2-x1；W eyes = x 2 -x 1 ;
Wface＝Weyes/a；W face = W eyes /a;
X1＝x1-(Wface-Weyes)/2；X 1 =x 1 -(W face -W eyes )/2;
X2＝X1+Wface；X 2 =X 1 +W face ;
Heyes＝(y1+y2)/2；H eyes = (y 1 +y 2 )/2;
Hface＝Wface；H face = W face ;
Y1＝Heyes-Hface/b；Y 1 =H eyes -H face /b;
Y2＝Y1+Hface；Y 2 =Y 1 +H face ;
其中，a和b均为经验常数，a取值为2，b取值为3.5，由此得到的人脸方框的尺寸随着双眼距离Weyes的变化而变化。Among them, a and b are empirical constants, the value of a is 2, and the value of b is 3.5. The size of the obtained face frame changes with the change of the distance W eyes between the eyes.
提示单元502，通过后置摄像头拍摄模式对所述后置摄像头获取的实时图像进行人脸位置识别，识别当前图像中人脸位置与所述前置摄像头拍摄的照片中人脸位置匹配，则发出拍摄提示。The prompting unit 502, through the rear camera shooting mode, performs face position recognition on the real-time image acquired by the rear camera, recognizes that the face position in the current image matches the face position in the photo taken by the front camera, and sends out Shooting Tips.
本实施例中，提示单元502中包括后置摄像头开启模块521第二识别双眼坐标模块522、第二确定人脸位置模块523、人脸识别模块524。In this embodiment, the prompting unit 502 includes a rear camera opening module 521 , a second identifying binocular coordinates module 522 , a second determining face position module 523 , and a face recognition module 524 .
后置摄像头开启模块521，开启后置摄像头拍摄模式，本实施例中，识别照片中人脸的位置后，便自动开启后置摄像头并切换到后置摄像头拍摄模式。The rear camera opening module 521 is used to open the rear camera shooting mode. In this embodiment, after recognizing the position of the face in the photo, the rear camera is automatically turned on and switched to the rear camera shooting mode.
第二识别双眼坐标模块522，用于识别实时图像中人脸的双眼坐标。The second binocular coordinate recognition module 522 is configured to recognize the binocular coordinates of the human face in the real-time image.
第二确定人脸位置模块523，用于确定实时图像中人脸的双眼距离和实时图像中的人脸方框位置。可以采用上述第一确定人脸位置模块513中的算法来确定实时图像中人脸的双眼距离和实时图像中的人脸方框位置。The second determining module 523 of the face position is used to determine the distance between the eyes of the face in the real-time image and the position of the frame of the face in the real-time image. The algorithm in the above-mentioned first determining human face position module 513 can be used to determine the binocular distance of the human face in the real-time image and the position of the human face frame in the real-time image.
人脸识别模块524，用于识别当前图像中人脸位置与前置摄像头拍摄的照片中的人脸位置匹配，则发出拍摄提示。The face recognition module 524 is used to identify that the position of the face in the current image matches the position of the face in the photo taken by the front camera, and then issue a shooting prompt.
拍摄提示为声音提示、灯光提示和震动提示中的一种或几种。声音提示的音调和/或音量随着当前图像中人脸位置与前置摄像头拍摄的照片中人脸位置的匹配度增加而升高或降低，灯光提示的亮度和/或闪烁频率随着当前图像中人脸位置与前置摄像头拍摄的照片中人脸位置的匹配度增加而升高或降低，震动提示的强度随着当前图像中人脸位置与前置摄像头拍摄的照片中人脸位置的匹配度增加而升高或降低。本发明通过使用前置摄像头拍摄照片并通过识别人脸位置来提示用户，实现快捷准确的用后置摄像头来自拍或摄像，使人脸在通过后置摄像头自拍的照片或视频中的位置理想。The shooting prompt is one or more of sound prompt, light prompt and vibration prompt. The tone and/or volume of the sound prompt increases or decreases with the matching degree of the face position in the current image and the face position in the photo taken by the front camera, and the brightness and/or flashing frequency of the light prompt increases or decreases with the current image The matching degree of the face position in the current image and the face position in the photo taken by the front camera increases or decreases, and the strength of the vibration prompt increases with the match between the face position in the current image and the face position in the photo taken by the front camera increase or decrease with increasing degrees. The present invention uses the front camera to take photos and prompts the user by identifying the position of the face, so as to realize fast and accurate self-portrait or video recording with the rear camera, so that the position of the face in the photo or video self-portrait through the rear camera is ideal.
本实施例中，识别照片的双眼距离与当前图像中的双眼距离相匹配，且照片中的人脸方框位置与当前图像中人脸方框位置的重合面积M与照片中的人脸方框位置面积的比值大于预置比例，则发出声音提示。其中，重合面积M与照片中的人脸方框位置面积的比值以及双眼距离的识别中不需要设定识别顺序，只需要识别其中一项不符合标准，则不发出声音提示。In this embodiment, the distance between the eyes of the identification photo matches the distance between the eyes in the current image, and the overlapping area M between the position of the face frame in the photo and the position of the face frame in the current image is the same as the face frame in the photo If the ratio of the location area is greater than the preset ratio, a sound prompt will be issued. Among them, the ratio of the overlapping area M to the area of the face frame in the photo and the recognition of the distance between the eyes do not need to set the recognition order, only one of them needs to be recognized if it does not meet the standard, and no sound prompt will be issued.
本实施例中，第一比例可以取75％，当前图像中的双眼距离与照片的双眼距离相匹配，具体为，当前图像的双眼距离与照片的双眼距离相同，或是当前图像的双眼距离为1-c至1+c倍的照片双眼距离，其中c可以取10％-20％。重合面积M与所述照片中的人脸方框位置面积的比值即为当前图像中人脸位置与前置摄像头拍摄的照片中人脸位置的匹配度，声音提示的音调或音量随着重合面积M与所述照片中的人脸方框位置面积的比值的增加而升高，以提示用户人脸位置的准确度，更加直观，方便用户根据声音变化调整移动手机的位置，更快找到理想拍照位置。In this embodiment, the first ratio can be 75%. The distance between the eyes in the current image matches the distance between the eyes in the photo. Specifically, the distance between the eyes in the current image is the same as the distance between the eyes in the photo, or the distance between the eyes in the current image is 1-c to 1+c times the distance between the eyes of the photo, where c can take 10%-20%. The ratio of the overlapping area M to the area of the face frame in the photo is the matching degree between the face position in the current image and the face position in the photo taken by the front camera. The ratio of M to the position area of the face frame in the photo increases to remind the user of the accuracy of the face position, which is more intuitive, convenient for the user to adjust the position of the mobile phone according to the sound change, and find the ideal photo faster. Location.
拍摄单元503，用于获取拍摄指令，拍摄照片或视频。The shooting unit 503 is configured to obtain a shooting instruction, and take photos or videos.
拍摄指令可以为用户输入的按键或触屏指令，也可以为动作指令、语言指令或角度位置指令，角度位置指令作为拍摄指令的详细介绍见实施例五。为了便捷操作，发出拍摄提示后，动作指令可以为识别当前人脸为笑脸，则对焦当前图像中人脸方框位置后拍照或摄像。The shooting instruction can be a button or a touch screen instruction input by the user, and can also be an action instruction, a language instruction, or an angle position instruction. For a detailed introduction of the angle position instruction as a shooting instruction, see Embodiment 5. For convenient operation, after sending out the shooting prompt, the action command can be to recognize the current face as a smiling face, then focus on the position of the face frame in the current image and then take a photo or video.
为了便捷操作，还可以包括笑脸判断单元，用于发出声音提示后，识别当前人脸为笑脸，则对焦当前图像中人脸方框位置后拍照。For the convenience of operation, a smile judging unit may also be included, which is used to identify the current face as a smile after a sound prompt is issued, and then focus on the position of the face frame in the current image and then take a photo.
实施例四Embodiment Four
附图5为本发明实施例四的一种拍摄装置的结构框图，该装置包括实施例三中的拍摄照片单元501、提示单元502以及拍摄单元503。进一步地，拍摄照片单元501用于通过前置摄像头拍摄带有人脸的照片并通过重力传感器获取前置摄像头的倾斜角度，获取照片中人脸位置。Fig. 5 is a structural block diagram of a photographing device according to Embodiment 4 of the present invention. The device includes a photographing unit 501, a prompting unit 502 and a photographing unit 503 in Embodiment 3. Further, the photo taking unit 501 is used to take a photo with a human face through the front camera, obtain the tilt angle of the front camera through the gravity sensor, and obtain the position of the human face in the photo.
为了使拍照角度理想，拍摄单元503具体用于：获取当前后置摄像头的倾斜角度与所述前置摄像头的倾斜角度相匹配，则拍摄照片或视频。In order to make the camera angle ideal, the photographing unit 503 is specifically configured to: acquire a match between the current tilt angle of the rear camera and the tilt angle of the front camera, and then take a photo or video.
本实施例中将获取当前后置摄像头的倾斜角度与所述前置摄像头的倾斜角度相匹配作为拍摄指令，当识别当前后置摄像头的倾斜角度与所述前置摄像头的倾斜角度相匹配，则对焦当前图像中人脸方框位置后拍照或摄影，拍照时可以对焦的哪个区图像中人脸方框位置中的某一部位，如对焦人脸方框位置的中点、双眼或双眼中点等，且在拍照同时发出拍照声音，提示用户拍照成功。In this embodiment, matching the inclination angle of the current rear camera with the inclination angle of the front camera is obtained as a shooting instruction, and when it is recognized that the inclination angle of the current rear camera matches the inclination angle of the front camera, then After focusing on the position of the face frame in the current image and taking pictures or photography, which area in the image can be focused on when taking a photo, such as the midpoint of the face frame, the eyes, or the midpoint of both eyes etc., and a sound is emitted while taking a photo, prompting the user to take a photo successfully.
实施例五Embodiment five
附图8为本发明实施例五提供的一种智能手机的结构框图，包括用于实现通讯的通讯信号处理单元610以及处理和存储信息的信息处理单元620，信息处理单元620连接通讯信号处理单元610。其中，通讯信号处理单元610中主要包括天线、滤波器、调制解调电路等，均为现有技术，在此不再赘述。信息处理单元620中主要包括信息输入装置(如摄像头、触屏、键盘等)、信息处理装置(如CPU)、信息存储装置(如存储硬盘、存储器、存储卡等)等，信息输入装置、信息存储装置均连接信息处理装置，上述均为现有技术，在此不再赘述。本实施例中，信息处理单元620还包括拍摄照片模块621、提示模块622和拍摄模块623。Figure 8 is a structural block diagram of a smart phone provided by Embodiment 5 of the present invention, including a communication signal processing unit 610 for realizing communication and an information processing unit 620 for processing and storing information, and the information processing unit 620 is connected to the communication signal processing unit 610. Wherein, the communication signal processing unit 610 mainly includes an antenna, a filter, a modulation and demodulation circuit, etc., all of which are in the prior art, and will not be repeated here. The information processing unit 620 mainly includes an information input device (such as a camera, a touch screen, a keyboard, etc.), an information processing device (such as a CPU), an information storage device (such as a storage hard disk, a memory, a memory card, etc.), and the information input device, information The storage devices are all connected to the information processing device, and the above are all prior art, and will not be repeated here. In this embodiment, the information processing unit 620 further includes a photographing module 621 , a prompting module 622 and a photographing module 623 .
拍摄照片模块621，用于通过前置摄像头拍摄带有人脸的照片，获取照片中人脸位置。其功能详述见实施例三装置中的拍摄照片单元501。The photo taking module 621 is used to take a photo with a human face through the front camera, and obtain the position of the human face in the photo. For details of its functions, refer to the photographing unit 501 in the device of the third embodiment.
提示模块622，用于通过后置摄像头拍摄模式对所述后置摄像头获取的实时图像进行人脸位置识别，识别当前图像中人脸位置与前置摄像头拍摄的照片中人脸位置匹配，则发出拍摄提示。其功能详述见实施例三和实施例四的装置中的提示单元502。The prompting module 622 is used to perform face position recognition on the real-time image acquired by the rear camera through the rear camera shooting mode, recognize that the face position in the current image matches the face position in the photo taken by the front camera, and then send Shooting Tips. See the prompt unit 502 in the devices of the third and fourth embodiments for details of its functions.
拍摄模块623，用于提示模块622发出拍摄提示后，获取拍摄指令，拍摄照片或视频。其功能详述见实施例三和实施例四的装置中的拍摄单元503。The photographing module 623 is configured to obtain a photographing instruction and take a photo or video after the prompting module 622 issues a photographing prompt. For detailed descriptions of its functions, refer to the photographing unit 503 in the devices of Embodiment 3 and Embodiment 4.
以上仅是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以作出若干改进和润饰，这些改进和润饰也应视为本发明的保护范围。The above are only preferred embodiments of the present invention, and it should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, and these improvements and modifications should also be regarded as protection scope of the present invention.
Claims (11)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510350524.8A CN104935698B (en) | 2015-06-23 | 2015-06-23 | Photographing method, photographing device and smart phone of an intelligent terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510350524.8A CN104935698B (en) | 2015-06-23 | 2015-06-23 | Photographing method, photographing device and smart phone of an intelligent terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104935698A true CN104935698A (en) | 2015-09-23 |
| CN104935698B CN104935698B (en) | 2019-02-22 |
Family
ID=54122660
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510350524.8A Active CN104935698B (en) | 2015-06-23 | 2015-06-23 | Photographing method, photographing device and smart phone of an intelligent terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104935698B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106445328A (en) * | 2016-08-31 | 2017-02-22 | 维沃移动通信有限公司 | Unlocking method of mobile terminal screen, and mobile terminal |
| CN106534684A (en) * | 2016-11-15 | 2017-03-22 | 极翼机器人(上海)有限公司 | Terminal control system and operation method thereof |
| CN106572299A (en) * | 2016-10-31 | 2017-04-19 | 北京小米移动软件有限公司 | Camera switching-on method and device |
| WO2017128750A1 (en) * | 2016-01-28 | 2017-08-03 | 中兴通讯股份有限公司 | Image collection method and image collection device |
| CN107168294A (en) * | 2017-06-26 | 2017-09-15 | 西安热工研究院有限公司 | A kind of unmanned inspection monitoring method of thermoelectricity water system equipment |
| CN107657218A (en) * | 2017-09-12 | 2018-02-02 | 广东欧珀移动通信有限公司 | Face identification method and Related product |
| CN108833794A (en) * | 2018-08-27 | 2018-11-16 | 维沃移动通信有限公司 | A shooting method and mobile terminal |
| CN109257537A (en) * | 2018-09-05 | 2019-01-22 | 广东小天才科技有限公司 | Photographing method and device based on intelligent pen, intelligent pen and storage medium |
| CN110516630A (en) * | 2019-08-30 | 2019-11-29 | 广东智媒云图科技股份有限公司 | A kind of LED display picture-drawing method, device, equipment and storage medium |
| CN113747074A (en) * | 2021-09-17 | 2021-12-03 | 广东小天才科技有限公司 | Tracking shooting method and device and wearable device |
| WO2023198073A1 (en) * | 2022-04-15 | 2023-10-19 | 华为技术有限公司 | Facial feature detection method, and readable medium and electronic device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102799877A (en) * | 2012-09-11 | 2012-11-28 | 上海中原电子技术工程有限公司 | Method and system for screening face images |
| CN102946514A (en) * | 2012-11-08 | 2013-02-27 | 广东欧珀移动通信有限公司 | Method and device for self-shooting of mobile terminal |
| CN103391361A (en) * | 2013-07-05 | 2013-11-13 | 中科创达软件股份有限公司 | Automatic reminding method and device for self-timer composition of intelligent terminal |
| WO2014190628A1 (en) * | 2013-05-28 | 2014-12-04 | 中兴通讯股份有限公司 | Photographing method and device with photographing function |
| CN104580909A (en) * | 2015-01-09 | 2015-04-29 | 上海小蚁科技有限公司 | Image acquisition method and device |
| CN104639819A (en) * | 2013-11-07 | 2015-05-20 | 中兴通讯股份有限公司 | Portable terminal and method for taking pictures using same |
-
2015
- 2015-06-23 CN CN201510350524.8A patent/CN104935698B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102799877A (en) * | 2012-09-11 | 2012-11-28 | 上海中原电子技术工程有限公司 | Method and system for screening face images |
| CN102946514A (en) * | 2012-11-08 | 2013-02-27 | 广东欧珀移动通信有限公司 | Method and device for self-shooting of mobile terminal |
| CN102946514B (en) * | 2012-11-08 | 2014-12-17 | 广东欧珀移动通信有限公司 | Method and device for self-shooting of mobile terminal |
| WO2014190628A1 (en) * | 2013-05-28 | 2014-12-04 | 中兴通讯股份有限公司 | Photographing method and device with photographing function |
| CN103391361A (en) * | 2013-07-05 | 2013-11-13 | 中科创达软件股份有限公司 | Automatic reminding method and device for self-timer composition of intelligent terminal |
| CN104639819A (en) * | 2013-11-07 | 2015-05-20 | 中兴通讯股份有限公司 | Portable terminal and method for taking pictures using same |
| CN104580909A (en) * | 2015-01-09 | 2015-04-29 | 上海小蚁科技有限公司 | Image acquisition method and device |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017128750A1 (en) * | 2016-01-28 | 2017-08-03 | 中兴通讯股份有限公司 | Image collection method and image collection device |
| CN107018304A (en) * | 2016-01-28 | 2017-08-04 | 中兴通讯股份有限公司 | A kind of image-pickup method and image collecting device |
| CN106445328B (en) * | 2016-08-31 | 2020-06-16 | 维沃移动通信有限公司 | A mobile terminal screen unlocking method and mobile terminal |
| CN106445328A (en) * | 2016-08-31 | 2017-02-22 | 维沃移动通信有限公司 | Unlocking method of mobile terminal screen, and mobile terminal |
| CN106572299A (en) * | 2016-10-31 | 2017-04-19 | 北京小米移动软件有限公司 | Camera switching-on method and device |
| CN106534684A (en) * | 2016-11-15 | 2017-03-22 | 极翼机器人(上海)有限公司 | Terminal control system and operation method thereof |
| CN107168294A (en) * | 2017-06-26 | 2017-09-15 | 西安热工研究院有限公司 | A kind of unmanned inspection monitoring method of thermoelectricity water system equipment |
| CN107168294B (en) * | 2017-06-26 | 2024-01-19 | 西安热工研究院有限公司 | An unmanned inspection and monitoring method for thermal power and water system equipment |
| CN107657218B (en) * | 2017-09-12 | 2021-03-09 | Oppo广东移动通信有限公司 | Face recognition method and related product |
| CN107657218A (en) * | 2017-09-12 | 2018-02-02 | 广东欧珀移动通信有限公司 | Face identification method and Related product |
| CN108833794B (en) * | 2018-08-27 | 2020-11-10 | 维沃移动通信有限公司 | Shooting method and mobile terminal |
| CN108833794A (en) * | 2018-08-27 | 2018-11-16 | 维沃移动通信有限公司 | A shooting method and mobile terminal |
| CN109257537A (en) * | 2018-09-05 | 2019-01-22 | 广东小天才科技有限公司 | Photographing method and device based on intelligent pen, intelligent pen and storage medium |
| CN110516630A (en) * | 2019-08-30 | 2019-11-29 | 广东智媒云图科技股份有限公司 | A kind of LED display picture-drawing method, device, equipment and storage medium |
| CN113747074A (en) * | 2021-09-17 | 2021-12-03 | 广东小天才科技有限公司 | Tracking shooting method and device and wearable device |
| WO2023198073A1 (en) * | 2022-04-15 | 2023-10-19 | 华为技术有限公司 | Facial feature detection method, and readable medium and electronic device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104935698B (en) | 2019-02-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104935698A (en) | A shooting method, shooting device and smart phone of an intelligent terminal | |
| CN105554389B (en) | Shooting method and device | |
| CN106572299B (en) | Camera opening method and device | |
| CN103167230B (en) | Electronic equipment and its method taken pictures according to gesture control | |
| CN105578026B (en) | A kind of image pickup method and user terminal | |
| CN104754216B (en) | A kind of photographic method and device | |
| WO2020259073A1 (en) | Image processing method and apparatus, electronic device, and storage medium | |
| WO2021027537A1 (en) | Method and apparatus for taking identification photo, device and storage medium | |
| CN104917959A (en) | Photographing method and terminal | |
| CN105915782A (en) | Picture obtaining method based on face identification, and mobile terminal | |
| CN102413282A (en) | A self-portrait guiding method and device | |
| CN106971164B (en) | Face shape matching method and device | |
| CN113840070B (en) | Shooting method, shooting device, electronic equipment and medium | |
| CN108600612B (en) | Photographing method and mobile terminal | |
| CN108650457A (en) | Automatic photographing method and device, storage medium and mobile terminal | |
| CN104639839B (en) | Shooting method and device | |
| CN104867112B (en) | Photo processing method and device | |
| CN107766820A (en) | Image classification method and device | |
| WO2017000491A1 (en) | Iris image acquisition method and apparatus, and iris recognition device | |
| WO2015024367A1 (en) | Shot image processing method and apparatus | |
| CN104639819A (en) | Portable terminal and method for taking pictures using same | |
| CN105306819B (en) | A kind of method and device taken pictures based on gesture control | |
| CN111797746A (en) | Face recognition method, device and computer-readable storage medium | |
| CN114445298A (en) | Image processing method and device, electronic equipment and storage medium | |
| CN103873755B (en) | Jump portrait system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |