CN108509866B - A face contour extraction method - Google Patents

Abstract

The invention discloses a face contour extraction method, which comprises the following steps: 1) after an image is input, extracting a region of a human face by using a human face detection algorithm, and finding a rough region of a human face contour by using a key characteristic point positioning algorithm; 2) sampling a rough area along the face contour to generate a series of dense squares, and containing the whole face contour area; 3) in each local square area, extracting a parabola to guide a local contour curve based on gradient information, and forming a local result set consisting of the local contour curves; 4) and fusing the dense redundant local contour curves into a global contour curve result through a global fusion algorithm based on PCA (principal component analysis), so as to obtain a complete face contour line. The method has the advantages of high precision, high speed, full automation and user interaction, and the finally obtained human face contour curve has high precision at the pixel level and conforms to the parabolic characteristic of the human face contour.

Description

A face contour extraction method

technical field

The invention relates to the technical field of image processing, in particular to a face contour extraction method.

Background technique

In the related application of face image, it is often necessary to automatically locate the key feature points of the face, such as eyes, nose tip, mouth corner points and the contour of the face, according to the input face image. Existing methods can precisely locate prominent features such as eyes, nose tip, and mouth corners, while facial contours located on cheeks and chin are challenging. For most of the current face alignment methods based on feature points, only a few feature points are set at the edge of the face, and it is difficult to extract the curved shape details of the face contour. But these detailed features are very important in many computer vision applications, such as face recognition, expression recognition and 3D face reconstruction. Improving the accuracy of face contours is of great significance and value in the fields of digital image processing and computer vision research, but it is a very challenging problem to extract pixel-level accurate continuous face contour curves.

Some earlier methods of face contour detection simply use a parabola to fit the chin area of the face, but this method is too simple and can only be used for the chin area, and a parabola cannot show the curved features of the face contour. The active contour model proposed by Kass et al. [M.Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active contour models," IJCV, vol.1, no.4, pp.321–331, 1988.], can It is used to describe the target contour curve and is mainly used for shape-based target segmentation. Some researchers [V.Perlibakas,"Automatical detectionof face features and exact face contour,"Pattern Recognition Letters,vol.24,no.16,pp.2977–2985,2003.] applied this method to face extraction Face contour lines. The active contour model is sensitive to its parameters, and will converge to different local optimal solutions due to different initialization positions, resulting in unsatisfactory results. Therefore, the active contour method usually requires continuous parameter adjustment and accurate initialization, which is difficult to achieve in the natural environment. At the same time, the active contour model iteratively fits the curve from the initial position to the target contour, and this process is usually easily disturbed by noise, so that it often stops in the middle or at the local optimum.

Face segmentation is also a method that can extract face contours, such as Wang et al. [B.Wang,X.Chang,andC.Liu, "Skin detection and segmentation of human face in color images," International Journal of Intelligent Engineering and Systems , vol.4, no.1, pp.10–17, 2011.] proposed skin detection and face segmentation method, usually this kind of method usually divides the image into face area and non-face area, so the two parts The boundary between them can be used as the face contour. However, these methods deal with the face boundary relatively rough, so that the boundary curve is too tortuous and does not fit the actual contour of the face. At the same time, a part of the neck area and the upper part of the hair close to the face are often included in the face area. , and can not get a good face contour curve.

The method of face alignment is to find predefined face key points on face photos, such as eyes, nose tip, mouth corner points, eyebrows, and contour points of various parts of the face, which is a strategy based on supervised learning. In recent years, cascaded shape regression models have performed well in the problem of key feature point localization, such as Kazemi et al. [V.Kazemi and J.Sullivan, "One millisecond face alignment with an ensemble of regression trees," in CVPR, 2014, pp. 1867–1874.] used the regression model to directly learn the mapping function from the face image to the face shape model, which is simple and efficient, and obtained good experimental results in both laboratory and natural scenes. The face alignment method greatly improves the robustness of face key point location. However, the number of key points on the face contour is scarce, and due to the complex situation in the real situation, there will be inward or outward situations. Therefore, such sparse points are difficult to show the different features of different people's facial contours.

The above methods are too limited by parameters and initialization, and cannot be robust to normal operation on natural pictures, or can find the approximate position of the face contour and cannot accurately describe the curved features of the face contour. In the current explosion of various visual applications, a method that can obtain high-precision face contours robustly and quickly is required, which provides the key for upper-layer applications such as face recognition, facial expression recognition, and 3D face reconstruction. contour information.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the defects and deficiencies of the prior art, and propose a face contour extraction method, which can quickly find a complete continuous face contour curve accurate to the pixel level when a face image is input arbitrarily.

In order to achieve the above purpose, the technical solution provided by the present invention is: a method for extracting human face contour, comprising the following steps:

1) Face image preprocessing

After inputting an image, the face detection algorithm is used to extract the area of the face, and the approximate area of the face contour is found through the key feature point positioning algorithm;

2) Local square area sampling

Generate a series of dense squares by sampling the approximate area along the face contour, including the entire face contour area;

3) Local contour curve extraction

In each local square area, a parabola is extracted to guide the local contour curve based on the gradient information to form a local result set composed of the local contour curve;

4) Global contour curve fusion

Through the PCA-based global fusion algorithm, the dense and redundant local contour curves are fused into a global contour curve result, and a complete face contour line is obtained.

In step 1), the approximate area of the face contour is found through the key feature point positioning algorithm, which is robust to the initialization position, as follows:

Use the face detection algorithm to extract the area where the face is located in the picture, obtain the bounding box of the face area, and obtain the position and scale information of the face in this picture at the same time; then use the face alignment algorithm to get the face contour. Several rough key points are used as initialization to form an initialization curve representing the contour area of the face;

In step 2), a series of dense squares are generated by sampling the approximate area along the face contour, and the entire face contour area is included therein, as follows:

In step 1), an initialization curve representing the face contour area has been obtained. Along this curve, the entire face contour area is divided into multiple overlapping square areas, and the center point of these square areas is located on the initialization curve. The size of the square area is set to the preset magnification of the face bounding box size extracted by the face detection algorithm. These overlapping square areas should cover the entire face contour area, and the direction of the square area is set to the tangent direction of the initialization curve of the current center point. The result of the local contour curve always starts from the top of the square area and ends at the bottom of the square. The initialization curve is located near the real face contour, and these overlapping square areas cover the entire real face contour; The method of sampling the face contour area can obtain a large number of overlapping and redundant local contour curve results, which meets the needs of cross-validation in step 4) the global contour curve extraction step;

In step 3), in each local square area, a parabola is extracted to guide the local contour curve based on the gradient information to form a local result set composed of the local contour curve, as follows:

In step 2), a large number of overlapping square areas are obtained, each square area contains a real face contour curve, and the direction of each square area, that is, the vertical direction of any square side, is the same as the direction of the initialization curve. Consistent; in this step, the parabolic-guided gradient information-based local contour curve is obtained according to the following method:

The direction of each square is consistent with the direction of the initialization curve, then the side of the square area is used as a rectangular coordinate system to look at this square area alone, and the local face contour line C must start from the top of the square area and go to the square The bottom of the region ends; this square region is an N×N matrix of pixels, then the local contour curve C is represented as a collection of N pixels from different rows:

C=<p ₁ ,p ₂ ,...,p _i ,...,p _N >

Among them, C is the local contour curve, p _i = (i, j) is the ith point on the local contour curve C, and the position of this point is the ith row and jth column of the square area, where i The range of and j are both [1, N]; p ₁ is the first point, p _N is the last point; for the two adjacent points p _i and p _i+1 on the local face contour line, it is required to The difference between the number of adjacent two columns is not greater than 1 to ensure smoothness between pixels;

The parabola guides the local contour curve based on the gradient information to determine its location through the following energy function:

是优化得到最优的局部轮廓曲线，G(C)表示局部轮廓曲线C的梯度值，而S(C)表示局部轮廓曲线C的曲率，α值是调节这条局部轮廓曲线C的平滑程度；上面的能量函数难以直接求解，因此使用动态规划的方法来求解这个问题，同时没有直接求解整条局部轮廓曲线的平滑度S(C)，而是利用贪心算法引导结果更像一条抛物线；C^*(i,j)是使用动态规划算法得到的从第0行到第i行区域内以(i,j)点终止的局部轮廓曲线，那么它一定来自于之前的{C^*(i-1,j-1)，C^*(i-1,j)，C^*(i-1,j+1)}三条轮廓线中的一个，使用d_i-1,j+Δ(i,j)表示点(i,j)到抛物线C^*(i-1,j+Δ)，Δ＝{-1，0，1}的距离，使用

表示偏离抛物线的误差，动态规划算法的过程如下表示：in,

is the optimal local contour curve obtained by optimization, G(C) represents the gradient value of the local contour curve C, and S(C) represents the curvature of the local contour curve C, and the α value is to adjust the smoothness of the local contour curve C; The above energy function is difficult to solve directly, so the dynamic programming method is used to solve this problem. At the same time, the smoothness S(C) of the entire local contour curve is not directly solved, but the greedy algorithm is used to guide the result to be more like a parabola; C ^* (i,j) is the local contour curve from the 0th line to the i-th line that terminates at the point (i,j) obtained by using the dynamic programming algorithm, then it must come from the previous {C ^* (i-1, j-1), C ^* (i-1,j), C ^* (i-1,j+1)} one of the three contour lines, use d _i-1,j+Δ (i,j) to represent the point The distance from (i,j) to the parabola C ^* (i-1,j+Δ), Δ={-1,0,1}, use

Representing the deviation from the parabola, the process of the dynamic programming algorithm is expressed as follows:

Among them, M represents the matrix of dynamic programming, M(i-1,j+Δ) is the energy of the corresponding point in a row on the matrix, g(i,j) represents the gradient value at this point, i is the number of rows, j is the column Number, e _i-1,j+Δ (i,j) represents the deviation from the parabola, the α value is a parameter to control the smoothness, different α values have an impact on the results of the local contour curve C, and an appropriate value needs to be obtained through experiments .

In step 4), described by the PCA-based global fusion algorithm, the dense and redundant local contour curves are fused into a global contour curve result, and a complete face contour line is obtained, as follows:

表示第k条局部轮廓曲线上的第i个点；M表示局部人脸轮廓线的数目，N表示每条局部人脸轮廓线的长度；如此所有的局部轮廓曲线通过一系列的点表示

因为步骤2)对人脸轮廓区域的密集采样，P点的集合大量冗余，不仅包含真是人脸轮廓上的点，而且还有失败的局部轮廓曲线上的点，在全局轮廓曲线融合中找出一条单像素宽度精确的全局轮廓曲线；Through step 2), a set of square regions is obtained by dense sampling on the face contour region, and then a local contour curve is obtained in each square region through step 3); now, a set of overlapping local contour curves is obtained; let

Represents the i-th point on the k-th local contour curve; M represents the number of local face contour lines, and N represents the length of each local face contour line; thus all local contour curves are represented by a series of points

Because of the dense sampling of the face contour area in step 2), the set of P points is a lot of redundancy, including not only the points on the real face contour, but also the points on the failed local contour curve, which are found in the global contour curve fusion. Generate a global contour curve with a single pixel width accurate;

The face contour curve usually contains a large number of curved details, which cannot be represented in a simple parametric form; therefore, a queue of a series of points Q={q _l } _l∈[1,L] needs to be used to represent the final global contour curve result , where L is the length of the global contour curve, and the points in Q all come from P to maintain the curved details of the global contour curve;

At the beginning, the queue Q is empty, obviously q ₀ is the first point p ₀ on the first local contour curve, and then the PCA algorithm is used to find the next point each time;

处的PCA方向，首先计算点

对于P中所有其他点的协方差矩阵如下：To calculate points

PCA direction at , first calculate the point

The covariance matrix for all other points in P is as follows:

就是点

的协方差矩阵，

表示第k条局部轮廓曲线上的第i个点，

表示其他的点，并且i'∈[1,N]\{i},k'∈[1,M]\{k}，M表示局部人脸轮廓线的数目，N表示每条局部人脸轮廓线的长度；函数θ(·)表示

r是两点之间的距离，h是一个固定阈值；当点之间的距离越远，则之间的影响越小；所以PCA并不是作用在整个点的集合上的，使用一个阈值h来加速计算；通过这步计算就得到协方差矩阵的特征值和对应最大特征值的特征向量就是所需要的点

处的主方向

in,

just point

The covariance matrix of ,

represents the i-th point on the k-th local contour curve,

Represent other points, and i'∈[1,N]\{i},k'∈[1,M]\{k}, M denotes the number of local face contour lines, N denotes each local face contour The length of the line; the function θ( ) represents

r is the distance between two points, and h is a fixed threshold; when the distance between the points is farther, the influence between them is smaller; so PCA does not act on the entire set of points, using a threshold h to Accelerate the calculation; through this step of calculation, the eigenvalue of the covariance matrix and the eigenvector corresponding to the largest eigenvalue are the required points

main direction

为了获取下一个点q_l+1,先使用K近邻算法找到离点q_l最近的K个点

在此将K设为7，如果所有K个点都是属于第k条局部轮廓曲线，则

沿着这条局部轮廓曲线往下走，如果有不属于第k条局部轮廓曲线的点，那么需要找到点

处最符合PCA方向

的一个点，通过下面公式计算：When the last point in the queue Q is q _l , the corresponding point in P is

In order to obtain the next point q _l+1 , first use the K-nearest neighbor algorithm to find the K points closest to the point q _l

Here K is set to 7, if all K points belong to the kth local contour curve, then

Go down this local contour curve, if there is a point that does not belong to the kth local contour curve, then you need to find the point

most compatible with PCA orientation

A point of , calculated by the following formula:

是两个方向间的内积，点

处最符合PCA方向

的一个点为通过求极值k^*,

得到的

这个过程不断迭代，直到没有点再加入队列Q中，而Q所有点构成的曲线就是所需的全局轮廓曲线。in,

is the inner product between the two directions, the point

most compatible with PCA orientation

A point of , by finding the extreme value k ^* ,

owned

This process iterates continuously until no points are added to the queue Q, and the curve formed by all the points of Q is the desired global contour curve.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

1. The present invention adopts a dense sampling method on the initial face contour area. The square area obtained by different sampling contains different image information, and different local contour curve results will be found. In one of the areas, the contour curve of the real face cannot be fitted, and an accurate result may be obtained in another area. This cross-validation mechanism ensures the accuracy and robustness of the algorithm.

2. The present invention adopts the method of local contour curve guided by parabola to generate local face contour line, makes full use of the parabolic characteristic of real face contour line, and cleverly uses dynamic programming algorithm to find the optimal solution of the layout. The parabolic guidance method not only speeds up the algorithm, but also makes the global contour curve fit the real situation.

3. The present invention adopts three steps from local to global to obtain high-quality continuous face curves, the calculation process is simple, the accuracy is high, the accuracy is pixel-level, and it is not easily disturbed by changes in picture illumination.

4. In practical applications, it can be fully automatic or user-interactive, and is robust to the initialized position of face contour extraction.

Description of drawings

FIG. 1 is a flowchart of a method for extracting face contours.

Figure 2 is an example diagram of the results of each step of the face contour extraction method.

Figure 3 is an example diagram of preprocessing and sampling square regions.

FIG. 4 is a comparison diagram of the changes of the local contour curves with different α values.

Figure 5 is a comparison of (a)(b) with parabolic guidance and (c)(d) without parabolic guidance.

FIG. 6 is an example diagram of a comparison between the present invention and methods such as ERT.

FIG. 7 is a comparison diagram of the results of the present invention (g)(h)(i) and the ACM method (d)(e)(f) at different initialization positions.

Detailed ways

The present invention will be further described below in conjunction with specific embodiments.

As shown in FIG. 1 and FIG. 2 , the method for extracting a face contour provided by this embodiment includes the following steps:

1) Face image preprocessing: After inputting an image, use the algorithm of face detection to extract the area of the face, and find the approximate area of the face contour through the key feature point positioning algorithm;

2) Local square area sampling, generating a series of dense squares in the approximate area along the face contour, including the entire face contour area;

3) Local contour curve extraction, in each local square area, extract a parabola to guide the local contour curve based on the gradient information to form a local result set composed of local contour curves;

4) Global contour curve fusion, through the PCA-based global fusion algorithm, the dense and redundant local contour curves are fused into a global contour curve result, and a complete face contour line is obtained.

In step 1), input an image, first preprocess it, extract the area where the face part is located in the image, find the approximate area of the face contour, and initialize the face contour extraction algorithm.

First, the face detection algorithm is used to extract the area where the face is located in the picture, and the bounding box of the face area is obtained, and the size of the face scale in this picture can be determined at the same time. Then, any kind of face alignment algorithm is used to provide the face contour extraction algorithm of the present invention with several rough key points on the face contour as initialization, which are connected to form an initialization curve representing the face contour area.

In the present invention, the Ensembles of Regression Trees (ERT ) algorithm, the face contour extraction algorithm of the present invention does not require high accuracy of initialization, but only needs to indicate the approximate area where the face contour is located. In order to speed up the whole algorithm, the present invention uses the ERT algorithm to detect 5 key feature points near the contour of the face. They are located on the sides of the eyes, the sides of the mouth, and the tip of the chin. Figure 3(a) shows an example of the positions of five key feature points, and it can be seen from the figure that the present invention only uses the input of very rough initialization positions.

Then, these five points are formed into an initialization curve by the curve fitting algorithm, which represents the face contour area, and provides the square center position and direction information for the next local square area sampling. There are only 5 key points, which are very sparse. In order to ensure that the fitting curve passes through all the key points without deviating from the face area, the present invention uses the Catmull-Rom algorithm. Figure 3(b) shows the fitting of 5 key feature points into an initialization curve using the Catmull-Rom curve.

In step 2), the method for the described local square area sampling is as follows:

In the above step 1), an initialization curve representing the face contour area has been obtained. Along this curve, we can divide the entire face contour area into many smaller overlapping square areas. The center point of these square areas is located on the initialization curve, the size of the square area is set to a certain magnification of the size of the face bounding box extracted by the face detection algorithm (the range is between 0 and 0.5), and the present invention is set to 0.2 of the size of the face bounding box times. The direction of the square area is set to the tangent direction of the initialization curve at the current center point, so that the resulting local contour curve always starts from the top of the square area and ends at the bottom of the square.

The initialization curve is located near the real face contour, and these overlapping square areas can cover the entire real face contour. For the accuracy of the result, the method of densely sampling the face contour area is adopted, and the default value of the present invention is set to 70 square areas. In this way, a large number of overlapping and redundant local contour curve results can be obtained, which meets the requirement of obtaining high-precision global contour curve results in the step 4) global contour curve extraction step. Figure 3(c) shows a densely sampled square region.

In step 3), the method for the described local contour curve extraction is as follows:

In the previous step, a large number of overlapping square areas were obtained, each square area contains a real face contour curve, and the direction of each square area (the vertical direction of any square side) is the same as that of the initialization curve. the same direction. In this step, the local contour curve of the parabolic guidance based on the gradient information will be found.

The direction of each square is consistent with the direction of the initialization curve, then the side of the square area is used as a rectangular coordinate system to look at this square area alone, and the local face contour line C must start from the top of the square area and go to the square The bottom end of the area. This curve not only has the largest gradient, but also has the smoothest possible parabola. This square area is an N×N pixel matrix, then the local contour curve C can be represented as N sets of pixels from different rows:

C=<p ₁ ,p ₂ ,...,p _i ,...,p _N >

where p _i =(i,j) is the i-th point on the local contour curve C, and the position of this point is the i-th row and j-th column of the square area, where the ranges of i and j are both [ 1,N]. p ₁ is the first point and p _N is the first point. For the two adjacent points _pi and pi ₊₁ on the local face contour line, the present invention requires that the difference in the number of the adjacent two columns is not greater than 1 to ensure smoothness between pixels.

The local contour curve has a large gradient value and has a smoothness like a parabola. The parabola in the present invention guides the local contour curve based on the gradient information to determine the location through the following energy function:

是优化得到最优的局部轮廓曲线，G(C)表示局部轮廓曲线C的梯度值，而S(C)表示局部轮廓曲线C的曲率，α值是调节这条局部轮廓曲线C的平滑程度。上面的能量函数难以直接求解，本发明使用动态规划的方法来求解这个问题。同时为了便利本发明并没有求解整条局部轮廓曲线的平滑度S(C)，而是利用了贪心算法引导结果更像一条抛物线。in

is the optimal local contour curve obtained by optimization, G(C) represents the gradient value of the local contour curve C, and S(C) represents the curvature of the local contour curve C, and the α value is to adjust the smoothness of the local contour curve C. The above energy function is difficult to solve directly, and the present invention uses the method of dynamic programming to solve this problem. At the same time, for convenience, the present invention does not solve the smoothness S(C) of the entire local contour curve, but uses a greedy algorithm to guide the result to be more like a parabola.

表示偏离抛物线的误差。动态规划算法的过程如下表示：C ^* (i,j) is the local contour curve that ends at point (i,j) in the area from line 0 to line i obtained using the dynamic programming algorithm, then it must come from the previous {C ^* (i- 1,j-1), C ^* (i-1,j), C ^* (i-1,j+1)} one of the three contour lines, use d _i-1,j+Δ (i,j) Represents the distance from point (i, j) to the parabola C ^* (i-1, j+Δ), Δ={-1, 0, 1}, using

Indicates the deviation from the parabola. The process of dynamic programming algorithm is expressed as follows:

where M represents the dynamic programming matrix, M(i-1,j+Δ) is the energy of the corresponding point in a row on the matrix, g(i,j) represents the gradient value at this point, i is the number of rows, and j is the number of columns . e _i-1,j+Δ (i,j) represents the deviation from the parabola. The α value is a parameter that controls the smoothness, and Figure 4 shows the effect of different α values on the results of the local profile curve C. In the present invention, the α value of the smoothing parameter is set to 0.7. Figure 5 shows the different results obtained by a purely gradient-based dynamic programming algorithm with and without parabolic guidance.

In step 4), the described method for global contour curve fusion is as follows:

表示第k条局部轮廓曲线上的第i个点。M表示局部人脸轮廓线的数目，N表示每条局部人脸轮廓线的长度。如此所有的局部轮廓曲线可以通过一系列的点表示

步骤2)对人脸轮廓区域的密集采样，P点的集合大量冗余，不仅包含真是人脸轮廓上的点，而且还有失败的局部轮廓曲线上的点。在全局轮廓曲线融合中本发明找出一条单像素宽度精确的全局轮廓曲线。The previous step 2) obtains a set of square regions through dense sampling on the face contour region, and then obtains a local contour curve in each square region through step 3). A collection of many shorter and overlapping local contour curves is now obtained. make

represents the i-th point on the k-th local contour curve. M represents the number of local face contour lines, and N represents the length of each local face contour line. So all local contour curves can be represented by a series of points

Step 2) Intensive sampling of the face contour area, the set of P points is largely redundant, including not only the points on the real face contour, but also the points on the failed local contour curve. In the global contour curve fusion, the present invention finds an accurate global contour curve with a single pixel width.

Face contour curves usually contain a large number of curved details, which cannot be represented in a simple parametric form. Therefore, a sequence of points Q={q _l } _l∈[1,L] needs to be used to represent the final global contour curve result, where L is the length of the global contour curve. At the same time, the points in Q are all derived from P to maintain the curved details of the global contour curve.

At the beginning, the queue Q is empty, obviously q ₀ is the first point p ₀ on the first local contour curve, and then the PCA algorithm is used to find the next point each time;

处的PCA方向，本发明首先计算了点

对于P中所有其他点的协方差矩阵如下：To calculate points

The PCA direction at , the present invention first calculates the point

The covariance matrix for all other points in P is as follows:

就是点

的协方差矩阵。

表示第k条局部轮廓曲线上的第i个点，

表示其他的点，并且i'∈[1,N]\{i},k'∈[1,M]\{k}，M表示局部人脸轮廓线的数目，N表示每条局部人脸轮廓线的长度。函数θ(·)表示

r是两点之间的距离，h是一个固定阈值。当点之间的距离越远，则之间的影响越小。所以PCA并不是作用在整个点的集合上的，本发明使用一个阈值h来加速计算。然后就获得了协方差矩阵的特征值和对应最大特征值的特征向量就是所需要的点

处的主方向

in

just point

The covariance matrix of .

represents the i-th point on the k-th local contour curve,

Represent other points, and i'∈[1,N]\{i},k'∈[1,M]\{k}, M denotes the number of local face contour lines, N denotes each local face contour the length of the line. The function θ( ) represents

r is the distance between two points and h is a fixed threshold. When the distance between the points is farther, the influence between them is smaller. Therefore, PCA does not act on the entire set of points, and the present invention uses a threshold h to speed up the calculation. Then the eigenvalues of the covariance matrix and the eigenvectors corresponding to the largest eigenvalues are obtained, which are the required points

main direction

为了获取下一个点q_l+1,先使用K近邻算法找到离点q_l最近的K个点

本发明中将K设为7。如果所有K个点都是属于第k条局部轮廓曲线，则

沿着这条局部轮廓曲线往下走，如果有不属于第k条局部轮廓曲线的点，那么需要找到点

处最符合PCA方向

的一个点，通过下面公式计算：When the last point in the queue Q is q _l , the corresponding point in P is

In order to obtain the next point q _l+1 , first use the K-nearest neighbor algorithm to find the K points closest to the point q _l

In the present invention, K is set to 7. If all K points belong to the kth local contour curve, then

Go down this local contour curve, if there is a point that does not belong to the kth local contour curve, then you need to find the point

most compatible with PCA orientation

A point of , calculated by the following formula:

是两个方向间的内积。点

处最符合PCA方向

的一个点为通过求极值k^*,

得到的

这个过程不断迭代，直到没有点再加入队列Q中，而Q所有点构成的曲线就是所需的全局轮廓曲线。in

is the inner product between the two directions. point

most compatible with PCA orientation

A point of , by finding the extreme value k ^* ,

owned

This process iterates continuously until no points are added to the queue Q, and the curve formed by all the points of Q is the desired global contour curve.

To sum up, the method of the present invention has high precision and high speed, can be fully automatic in practical application, and can also be user-interactive, and the finally obtained face contour curve has pixel-level accuracy and conforms to the parabolic characteristics of the face contour. . The present invention has been compared with the existing method, and the results shown in Figure 6 show that the present invention has superior performance. At the same time, the method of the present invention is robust to the initialized position of face contour extraction, and some changes in the original initialized position will not affect the result. As shown in Figure 7, three initialized positions are used. The ACM method did not get the correct result.

The above-mentioned embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of implementation of the present invention. Therefore, any changes made according to the shape and principle of the present invention should be included within the protection scope of the present invention.

Claims (3)

1. a face contour extraction method, is characterized in that, comprises the following steps: 1) Face image preprocessing After inputting an image, the face detection algorithm is used to extract the area of the face, and the approximate area of the face contour is found through the key feature point positioning algorithm; 2) Local square area sampling Generate a series of dense squares by sampling the approximate area along the face contour, including the entire face contour area; 3) Local contour curve extraction In each local square area, a parabola is extracted to guide the local contour curve based on the gradient information to form a local result set composed of the local contour curve; 4) Global contour curve fusion Through the PCA-based global fusion algorithm, the dense and redundant local contour curves are fused into a global contour curve result, and a complete face contour line is obtained. 2. a kind of human face contour extraction method according to claim 1, is characterized in that, in step 1), described finds the rough area of human face contour by key feature point positioning algorithm, has robustness to initialization position sex, as follows: Use the face detection algorithm to extract the area where the face is located in the picture, obtain the bounding box of the face area, and obtain the position and scale information of the face in this picture at the same time; then use the face alignment algorithm to get the face contour. Several rough key points are used as initialization to form an initialization curve representing the contour area of the face; In step 2), a series of dense squares are generated by sampling the approximate area along the face contour, and the entire face contour area is included therein, as follows: In step 1), an initialization curve representing the face contour area has been obtained. Along this curve, the entire face contour area is divided into multiple overlapping square areas, and the center point of these square areas is located on the initialization curve. The size of the square area is set to the preset magnification of the face bounding box size extracted by the face detection algorithm. These overlapping square areas should cover the entire face contour area, and the direction of the square area is set to the tangent direction of the initialization curve of the current center point. The result of the local contour curve always starts from the top of the square area and ends at the bottom of the square. The initialization curve is located near the real face contour, and these overlapping square areas cover the entire real face contour; The method of sampling the face contour area can obtain a large number of overlapping and redundant local contour curve results, which meets the needs of cross-validation in step 4) the global contour curve extraction step; In step 3), in each local square area, a parabola is extracted to guide the local contour curve based on the gradient information to form a local result set composed of the local contour curve, as follows: In step 2), a large number of overlapping square areas are obtained, each square area contains a real face contour curve, and the direction of each square area, that is, the vertical direction of any square side, is the same as the direction of the initialization curve. Consistent; in this step, the parabolic-guided gradient information-based local contour curve is obtained according to the following method: The direction of each square is consistent with the direction of the initialization curve, then the side of the square area is used as a rectangular coordinate system to look at this square area alone, and the local face contour line C must start from the top of the square area and go to the square The bottom of the region ends; this square region is an N×N matrix of pixels, then the local contour curve C is represented as a collection of N pixels from different rows: C=<p ₁ ,p ₂ ,...,p _i ,...,p _N > Among them, C is the local contour curve, p _i = (i, j) is the ith point on the local contour curve C, and the position of this point is the ith row and jth column of the square area, where i The range of and j are both [1, N]; p ₁ is the first point, p _N is the last point; for the two adjacent points p _i and p _i+1 on the local face contour line, it is required to The difference between the number of adjacent two columns is not greater than 1 to ensure smoothness between pixels; The parabola guides the local contour curve based on the gradient information to determine its location through the following energy function:

in,

is the optimal local contour curve obtained by optimization, G(C) represents the gradient value of the local contour curve C, and S(C) represents the curvature of the local contour curve C, and the α value is to adjust the smoothness of the local contour curve C; The above energy function is difficult to solve directly, so the dynamic programming method is used to solve this problem. At the same time, the smoothness S(C) of the entire local contour curve is not directly solved, but the greedy algorithm is used to guide the result to be more like a parabola; C ^* (i,j) is the local contour curve from the 0th line to the i-th line that terminates at the point (i,j) obtained using the dynamic programming algorithm, then it must come from the previous {C ^* (i-1, j-1), C ^* (i-1,j), C ^* (i-1,j+1)} one of the three contour lines, use d _i-1,j+Δ (i,j) to represent the point The distance from (i,j) to the parabola C ^* (i-1,j+Δ), Δ={-1,0,1}, use

Representing the deviation from the parabola, the process of the dynamic programming algorithm is expressed as follows:

Among them, M represents the matrix of dynamic programming, M(i-1,j+Δ) is the energy of the corresponding point in a row on the matrix, g(i,j) represents the gradient value at this point, i is the number of rows, j is the column Number, e _i-1,j+Δ (i,j) represents the deviation from the parabola, the α value is a parameter to control the smoothness, different α values have an impact on the results of the local contour curve C, and an appropriate value needs to be obtained through experiments . 3. a kind of human face contour extraction method according to claim 1 is characterized in that: in step 4), described by the global fusion algorithm based on PCA, the local contour curve of dense redundancy is fused into a global Contour curve results to obtain a complete face contour line, as follows: Through step 2), a set of square regions is obtained by dense sampling on the face contour region, and then a local contour curve is obtained in each square region through step 3); now, a set of overlapping local contour curves is obtained; let

Represents the i-th point on the k-th local contour curve; M represents the number of local face contour lines, and N represents the length of each local face contour line; thus all local contour curves are represented by a series of points

Because of the dense sampling of the face contour area in step 2), the set of P points is a lot of redundancy, including not only the points on the real face contour, but also the points on the failed local contour curve, which are found in the global contour curve fusion. Generate a global contour curve with a single pixel width accurate; The face contour curve usually contains a large number of curved details, which cannot be represented in a simple parametric form; therefore, a queue of a series of points Q={q _l } _l∈[1,L] needs to be used to represent the final global contour curve result , where L is the length of the global contour curve, and the points in Q all come from P to maintain the curved details of the global contour curve; At the beginning, the queue Q is empty, obviously q ₀ is the first point p ₀ on the first local contour curve, and then the PCA algorithm is used to find the next point each time; To calculate points

PCA direction at , first calculate the point

The covariance matrix for all other points in P is as follows:

in,

just point

The covariance matrix of ,

represents the i-th point on the k-th local contour curve,

Represent other points, and i'∈[1,N]\{i},k'∈[1,M]\{k}, M denotes the number of local face contour lines, N denotes each local face contour The length of the line; the function θ( ) represents

r is the distance between two points, and h is a fixed threshold; when the distance between the points is farther, the influence between them is smaller; so PCA does not act on the entire set of points, using a threshold h to Accelerate the calculation; through this step of calculation, the eigenvalue of the covariance matrix and the eigenvector corresponding to the largest eigenvalue are the required points

main direction

When the last point in the queue Q is q _l , the corresponding point in P is

In order to obtain the next point q _l+1 , first use the K-nearest neighbor algorithm to find the K points closest to the point q _l

Here K is set to 7, if all K points belong to the kth local contour curve, then

Go down this local contour curve, if there is a point that does not belong to the kth local contour curve, then you need to find the point

most compatible with PCA orientation

A point of , calculated by the following formula:

in,

is the inner product between the two directions, the point

most compatible with PCA orientation

A point of is determined by finding the extreme value

owned

This process iterates continuously until no points are added to the queue Q, and the curve formed by all the points of Q is the desired global contour curve.

Images