CN109872720B - Re-recorded voice detection algorithm for different scene robustness based on convolutional neural network - Google Patents

Abstract

The invention discloses a re-recorded speech detection algorithm based on a convolutional neural network that is robust to different scenarios, and specifically relates to the field of speech detection algorithms. By inputting a speech time-frequency graph into the algorithm model, the algorithm model includes seven layers, each layer Contains a convolutional layer and a pooling layer. The output of the convolutional layer passes through a linear rectification function, and a residual connection is added between the layers. Finally, the final feature is extracted through global pooling, and the detection result is predicted through sigmoid. The present invention adopts the time-frequency diagram as the data input form of the network in the present invention. Compared with directly inputting voice data, the time-frequency diagram has a relatively dense distribution for the feature information introduced by the re-recording device, which is more conducive to the feature extraction of the neural network, thereby speeding up training to improve accuracy.

Description

A Re-recorded Speech Detection Algorithm Robust to Different Scenes Based on Convolutional Neural Network

technical field

The present invention relates to the field of speech detection algorithms, and more specifically, the present invention relates to a re-recorded speech detection algorithm robust to different scenarios based on a convolutional neural network.

Background technique

Studies have proved that deceptive voices such as Voice Conversion (Voice Conversion, VC), Speech Synthesis (SS) and re-recorded voice can effectively deceive the Automatic Speaker Recognition (ASV) system, thereby impersonating others to log in to the system , the re-recording of the voice will cause the ASV system to have a higher false acceptance rate, which poses a serious threat to social security. Among them, VC and SS require more voice information and features of the target speaker, and the existing algorithms are not yet fully mature, so the cost and difficulty of implementation are relatively high; while the re-recorded voice can be easily obtained with cheap recording equipment, and The re-recorded voice basically contains all the characteristics of the target character's voice, so it is more threatening than VC and SS. For this reason, the detection of re-recorded speech should be paid attention to.

SV (Automatic Speaker Recognition) systems are used more and more in practice, such as: access control systems, telephone banking, military and other fields. Since the speaker verification process does not require any face-to-face contact, ASV systems are highly vulnerable to spoofed speech. The spoofed voice generated by the audio equipment will pose a threat to the ASV (Automatic Speaker Recognition) system and affect the security performance of the system. In the past ten years, audio digital products have not only emerged in an endless stream, but also integrated functions of various products have become more and more powerful. The same or similar effects can now be achieved with relatively cheap devices such as a personal computer with audio processing software installed or a PDA with audio processing capabilities. For example, a high-quality, low-cost recording device—smartphone—forms spoofed speech, which poses a risk to the ASV system. Deceptive speech includes replay attacks, speech conversion, speech synthesis, etc. Attackers will use deceptive voice to forge feature data to obtain illegal identity access to the system, and then the user's file data and privacy will be stolen, causing many irreparable losses. Among them, replay attacks are more threatening than speech conversion and speech synthesis. A replay attack is a speech sample taken from the actual target speaker in the form of successive pre-recorded speech samples. The replay-based spoofing attack does not require any technical processing of the speech. The speech of the actual target speaker and the replay speech have exactly the same frequency spectrum and advanced features. It is the easiest type of speech attack. Compared with the voice of the actual target speaker, synthetic speech and deformed speech have certain errors and changes, and are not exactly the same. Therefore, the detection of replay attacks is more difficult than synthetic speech and deformed speech.

Contents of the invention

In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a re-recorded speech detection algorithm based on convolutional neural networks that is robust to different scenarios. Compared with directly inputting voice data, the time-frequency graph has a relatively dense distribution of feature information introduced by re-recording equipment, which is more conducive to neural network feature extraction, thereby speeding up training and improving accuracy. The detection of recorded voice has a high accuracy.

In order to achieve the above object, the present invention provides the following technical solution: a re-recorded speech detection algorithm based on a convolutional neural network that is robust to different scenarios, specifically comprising the following steps:

a. Use the recording equipment to collect the original voice, and convert it through DA/AD to obtain the re-recorded voice;

b. The original speech will be distorted during the conversion process, and the distortion data of the original speech is calculated through the distortion model, wherein the expression of the distortion model is:

y(t) is the re-recorded speech, x(t) is the original speech, λ is the amplitude conversion factor, α is the time axis linear scaling factor, and η is the superimposed noise;

The corresponding frequency domain change expression:

Y(jω), X(jω), and N(jω) are the frequency domain representations of y(t), x(t), and η respectively. For a fixed recording device, their characteristics are very stable, that is, λ and α are constant;

c. Re-recording the voice and producing the voice time-frequency map by short-time Fourier transform;

d. The speech time-frequency graph is input into the algorithm model. The algorithm model consists of seven layers, each layer includes a convolution layer and a pooling layer. The output of the convolution layer passes through a linear rectification function, and residuals are added between layers. Poor connections, and finally extract the final features through global pooling, and predict the detection results through sigmoid.

In a preferred embodiment, when the re-recorded speech is transformed, the short-time Fourier transform adopts a 126-length Hamming (Hanning) window, the step size is 50, and the size of the time-frequency map is (64x62).

In a preferred embodiment, the algorithm model adopts convolution in the frequency dimension and pooling in the time dimension. The specific setting is to use a 3x1 convolution kernel and 1x2 pooling, and it can match the feature distribution characteristics of the time-frequency map. The distribution characteristics of the frequency map are independent between adjacent speech frames and consistent in a specific frequency band.

In a preferred embodiment, the algorithmic model uses deep learning as a data-driven technique.

In a preferred embodiment, the re-recording device will introduce changes in the frequency domain of the original speech signal, and the deep learning model uses the original audio signal as the input data of the network.

In a preferred embodiment, when the algorithm model performs convolution in the frequency dimension, the correlation of the time dimension is not considered, and when the convolution is performed in the frequency dimension, pooling is performed in the time dimension at the same time.

In a preferred embodiment, the parameters of the convolution kernel can be shared, and the characteristic information of the equipment with the same distribution in the time dimension repeatedly trains the parameters of the convolution kernel. The pooling layer adopts the pooling of the time dimension (1x2), and the frequency dimension does not perform pooling.

Technical effect and advantage of the present invention:

1. The present invention uses a time-frequency diagram as the data input form of the network in the present invention. Compared with directly inputting voice data, the time-frequency diagram has a relatively dense distribution for the feature information introduced by the re-recording device, which is more conducive to neural network feature extraction. Thereby speeding up training and improving accuracy;

2. The present invention uses convolution in the frequency dimension and pooling in the time dimension. Specifically, it is set to use a 3x1 convolution kernel and 1x2 pooling. Convolution is only performed in the frequency dimension without considering the correlation of the time dimension, which can greatly reduce The number of convolution kernel parameters makes the model more resistant to overfitting and reduces excessive dependence on the amount of data. At the same time, during the training process, due to the sharing of convolution kernel parameters, the time dimension has the characteristic information of devices with the same distribution Repeatedly training the parameters of the convolution kernel can make the training more sufficient;

3. The present invention does not need to manually select one or more specific features like traditional machine learning methods and then classify them with a classifier, and can spontaneously extract relevant features including some shallow edge features and deep features and then Further classification simplifies the entire process and achieves better results;

4. The algorithm of the present invention has high accuracy in the detection of re-recorded voices with different recording devices, recording environments and recording distances.

Description of drawings

Fig. 1 is a structural schematic diagram of the algorithm model of the present invention.

Fig. 2 is a schematic diagram of the speech re-recording process of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Example 1

As shown in Figure 1, a re-recorded speech detection algorithm based on convolutional neural network robust to different scenarios, the algorithm model has 7 layers, each layer contains a convolutional layer and a pooling layer, the output of the convolutional layer is passed through Linear rectification function, and add residual connections between layers, and finally extract the final features through global pooling, and predict the detection results through sigmoid, using convolution in the frequency dimension and pooling in the time dimension, the specific setting is to use 3x1 convolution Accumulation of cores, 1x2 pooling, maximizing the reduction of model capacity, greatly reducing the risk of overfitting, reducing the dependence of the model on the amount of data and highly consistent with the characteristic distribution characteristics of the time-frequency map, and assigning training parameters to more reasonable place, use more effective features to train more compact parameters;

Speech time-frequency map is generated by short-time Fourier transform. Compared with directly inputting speech data, time-frequency map has a relatively dense distribution of feature information introduced by re-recording equipment, which is more conducive to neural network feature extraction, thereby speeding up training. To improve the accuracy, the re-recording equipment will introduce changes in the frequency domain of the original voice signal. The performance of the deep learning model has a high dependence on the data. The original audio signal is used as the input data of the network, and its feature distribution is too sparse, which greatly improves It reduces the difficulty of extracting effective features by neural network;

Example 2

As shown in Figure 2, a re-recording speech detection algorithm based on convolutional neural network robust to different scenarios, re-recording leads to a certain degree of distortion of speech data, including amplitude distortion and linear scaling on the time axis, where the distortion model The expression is:

y(t) is the re-recorded speech, x(t) is the original speech, λ is the amplitude conversion factor, α is the time axis linear scaling factor, and η is the superimposed noise;

The corresponding frequency domain change expression:

Y(jω), X(jω), and N(jω) are the frequency domain representations of y(t), x(t), and η respectively. For a fixed recording device, their characteristics are very stable, that is, λ and α are constant;

Example 3

In this implementation, the speech segment of 0.2 seconds is used as the experimental data, the short-time Fourier transform adopts a 126-length Hamming (hanning) window, the step size is 50, and the size of the time-frequency map is (64x62);

Further, in the above technical solution, convolution is performed in the frequency dimension, and pooling is performed in the time dimension at the same time. Convolution is only performed in the frequency dimension, regardless of the correlation of the time dimension, which can greatly reduce the convolution kernel parameters. The amount makes the model have stronger anti-overfitting ability and reduces the excessive dependence on the amount of data. At the same time, due to the parameter sharing of the convolution kernel during the training process, the feature information of the same distribution equipment in the time dimension has repeated training convolution. The kernel parameters can make the training more sufficient. The pooling layer adopts the pooling of the time dimension (1x2), and the frequency dimension does not perform pooling. Scaling and deformation are more robust. For the time-frequency graph, there is no stretching and deformation in the feature distribution, and it is only pooled in the time dimension, which not only reduces the feature dimension, but also does not cause the loss of frequency dimension features. Through multiple Layer convolution and pooling calculation, the feature dimension finally becomes one-dimensional, and the length is the same as the frequency of the time-frequency map;

Further, in the above technical solution, the original voice database consists of 30,000 pieces of voice recorded by 60 people, with a sampling frequency of 16 kHz and a quantization precision of 16 bits;

The voices of 10 speakers are randomly selected as test data, and the voices of the remaining 50 people are used for training to ensure the independence of training data and test data, and avoid the recording of the same speaker from appearing in different data sets;

The specific recording process is as follows: For the training set, the original speech library was re-recorded 4 times under different distances and equipment combinations in a quiet environment, thus obtaining 4 re-recorded speech libraries, which respectively contained 25,000 segments of speech, from the 4 speech libraries A total of 25,000 segments of speech are randomly extracted as negative samples, and together with the original speech, they form a training data set of 50,000 segments. The original voice is played through a laptop Lenovo Y40-70AT-IFI; the re-recording device is a laptop Dell Inspion 14 (Ins14VD-258) and a smartphone Xiaomi 2S;

The situation of the 4 recordings is shown in Table 1:

Table 1 recording voice

For the test data, the same recording settings as in Table 2 are used. In order to verify the robustness of the model to the interference of random noise in the environment, recordings were made in a quiet environment and in an environment with a certain amount of random noise. The test set contains a total of 4 speech libraries , each voice library contains a total of 10,000 test voices in a quiet environment and ambient noise in the recording mode of the library;

Further, in the above technical solution, the network error function is a cross-entropy loss function, the Adam optimization algorithm is used for training, the initial learning rate is set to 0.001, and the learning rate is dynamically adjusted during the training process, and the learning rate is reduced by It is twice as small, and the batch size of each training is 32. In order to monitor the training effect during the training process, 2000 pieces of data are randomly selected from the training data for verification. By comparing the loss function of the training data and the loss function of the verification data, add regularization to the loss function The normalization item and setting the regularization coefficient to 0.0001 can effectively prevent overfitting;

Table 2 lists some important hyperparameter settings during the training process. Under this setting, the network has rapid convergence during the training process, and finally achieved a fairly high accuracy;

Table 2 Hyperparameters (β ₁ and β ₂ are Adam optimizer parameters respectively)

Further, in the above technical solution, this embodiment includes 4 experimental tests, which are respectively for different recording devices and different recording distances. The experimental results obtained in each experiment are shown in Table 3:

Table 3 Experimental results

The accuracy of test experiments under different conditions can reach more than 99.8%, which ensures that the experimental model has good versatility.

Finally: the above is only a preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the present invention within the scope of protection.

Claims (5)

1. A re-recorded voice detection algorithm based on a convolutional neural network and robust to different scenes is characterized by comprising the following steps:

a. acquiring original voice by using a recording device, and obtaining re-recorded voice through DA/AD conversion;

b. the original voice can generate distortion in the transformation process, and the distortion data of the original voice is calculated through a distortion model, wherein the expression of the distortion model is as follows:

y (t) is the re-recorded speech, x (t) is the original speech, λ is the amplitude transformation factor, α is the time axis linear scaling factor, η is the superposition noise;

the corresponding frequency domain variation expression:

Y(jω)＝λX(jαω)+N(jω)，

y (j omega), X (j omega) and N (j omega) are respectively frequency domain representations of Y (t), X (t) and eta, and are characterized by being very stable for a fixed recording device, namely lambda and alpha are constants;

c. re-recording the voice and producing a voice time-frequency graph by short-time Fourier transform;

d. inputting a voice time-frequency graph into an algorithm model, wherein the algorithm model comprises seven layers, each layer comprises a convolution layer and a pooling layer, the output of the convolution layer passes through a linear rectification function, residual connection is added between the layers, and finally, the final characteristics are extracted through global pooling, and the detection result is predicted through sigmoid;

convolution layers of the algorithm model are only convoluted in a frequency dimension without considering the correlation of a time dimension, and pooling layers are only pooled in the time dimension without pooling in the frequency dimension; the specific setting is that 3x1 convolution kernel is adopted, 1x2 pooling is adopted, and the feature distribution characteristics of the speech time-frequency diagram can be matched, and the distribution characteristics of the speech time-frequency diagram have independence between adjacent speech frames and consistency in a specific frequency band.

2. The convolutional neural network-based re-recorded speech detection algorithm robust to different scenarios as claimed in claim 1, wherein: when the re-recorded speech is transformed, a 126-length hamming (hanning) window is used for short-time fourier transform, the step size is 50, and the size of the time-frequency diagram is (64 × 62).

3. The convolutional neural network-based re-recorded speech detection algorithm robust to different scenarios as claimed in claim 1, wherein: the algorithmic model employs deep learning as a data-driven technique.

4. The convolutional neural network-based re-recorded speech detection algorithm robust to different scenarios as claimed in claim 1, wherein: the algorithm model does not consider the correlation of the time dimension when performing convolution in the frequency dimension, and performs pooling of the time dimension when performing convolution in the frequency dimension.

5. The convolutional neural network-based re-recorded speech detection algorithm robust to different scenarios as claimed in claim 1, wherein: the convolution kernel can be shared by parameters, the time dimension has characteristic information of the same distributed equipment to repeatedly train the parameters of the convolution kernel, the pooling layer adopts pooling 1x2 of the time dimension, and the frequency dimension does not carry out pooling.

Images