CN106228240B - Deep convolution neural network implementation method based on FPGA - Google Patents

Abstract

The method of the invention belongs to the technical fields of digital image processing and pattern recognition. Specifically, it is a deep convolutional neural network implementation method based on FPGA. The hardware platform realized by the invention is Xilinx ZYNQ-7030 programmable on-chip SoC, and the hardware platform has a built-in FPGA and an ARM Cortex A9 processor. The present invention firstly loads the trained network model parameters to the FPGA side, then preprocesses the input data on the ARM side, and then transmits the result to the FPGA side, and realizes the convolution calculation and downsampling of the deep convolutional neural network on the FPGA side , form a data feature vector and transmit it to the ARM side to complete the feature classification calculation. The invention utilizes the fast parallel processing of FPGA and the high-efficiency computing characteristics of extremely low power consumption to realize the convolutional computing part with the highest complexity in the deep convolutional neural network model. On the premise of ensuring the accuracy of the algorithm, the algorithm efficiency is greatly improved, Reduce power consumption.

Description

Implementation method of deep convolutional neural network based on FPGA

technical field

The invention belongs to the technical field of digital image processing and pattern recognition, and particularly relates to a method for implementing a deep convolutional neural network model on an FPGA hardware platform.

Background technique

With the current rapid development of computer technology and the Internet, the scale of data is growing explosively, and the intelligent analysis and processing of massive data has become the key to effectively utilizing the value of data. Artificial intelligence technology is an effective means to discover valuable information from massive data. In recent years, breakthroughs have been made in the fields of computer vision, speech recognition and natural language processing. The deep learning algorithm model based on deep convolutional neural network is a typical representative.

Convolutional Neural Networks (CNN) are inspired by neuroscience research. After more than 20 years of evolution, remarkable theoretical research and practical application results have been achieved in pattern recognition, human-machine confrontation and other fields. In the famous human-machine Go competition, the artificial intelligence system AlphaGo based on CNN + Monte Carlo search tree algorithm, He defeated the world Go champion Lee Sedol by 4:1. A typical CNN algorithm model consists of two parts: a feature extractor and a classifier. The feature extractor is responsible for generating the low-dimensional feature vector of the input data, which has better robustness to the data. This vector is used as the input data of the classifier (usually based on the traditional artificial neural network) to classify, and the classification result of the input data is obtained.

In the implementation of the convolutional neural network algorithm model, the convolution calculation accounts for 90% of the calculation amount of the entire algorithm model ^[1] . Therefore, the efficient calculation of the convolution layer is the key to greatly improve the calculation efficiency of the CNN algorithm model. The convolution is realized through hardware acceleration. Computing is an efficient way.

At present, GPU clusters are widely used in the industry to implement deep learning algorithm models, and deep neural network models are implemented through large-scale parallel computing, which has achieved remarkable high efficiency and high performance results. However, the high power consumption of GPU also restricts its large-scale application. , and then become the bottleneck of the practical promotion and application of the deep convolutional neural network algorithm model. FPGA has the advantages of high-performance parallel computing and ultra-low power consumption. Implementing deep learning algorithm models on FPGA is an inevitable development direction in this field.

At present, there are three main schemes for implementing CNN using FPGA:

(1) Use the soft-core CPU to realize the control part, and cooperate with the FPGA to realize the algorithm acceleration;

(2) Use the hard-core ARM Cortex A9 CPU embedded in the hard-core SoC to realize the control part, and cooperate with the FPGA to realize the algorithm acceleration;

(3) Use cloud server with FPGA to achieve algorithm acceleration.

The three schemes have their own advantages and disadvantages, and different acceleration schemes can be selected according to different application scenarios.

In the deep convolutional neural network, the calculation of the convolutional layer occupies more than 90% of the calculation amount, and is a key link in the entire network model, and its computational efficiency directly affects the performance of the model algorithm implementation. However, it is very difficult to implement convolution calculation on FPGA, which is mainly reflected in the following aspects:

(1) The deep learning algorithm model is still basically in the academic research stage, and there are still many algorithms and model optimization work for large-scale industrial applications. Therefore, the algorithm model needs to be continuously optimized to adapt to different application scenarios. A very in-depth understanding of theories and algorithms;

(2) The research and development of FPGA is based on the underlying hardware language, which is suitable for the situation that the algorithm model is relatively stable, and the constantly changing deep learning algorithm model brings great difficulty to its implementation on FPGA;

(3) Implementing deep convolutional neural networks on FPGA requires rich experience in FPGA engineering implementation. The operating clock frequency of the FPGA and the output delay of the multiplier and other modules used are contradictory. The higher the clock frequency, the longer the output delay of the module, and the lower the clock frequency, the shorter the output delay of the module. It is necessary to find relatively balanced parameters through manual experiments with the help of engineering experience.

SUMMARY OF THE INVENTION

The purpose of the method of the present invention is to provide a method for implementing a deep convolutional neural network model with high efficiency and low power consumption, so as to solve the problems of high power consumption and low efficiency of the current deep learning model based on GPU or CPU.

The invention optimizes the FPGA hardware design, effectively reduces resource consumption, and can implement a deep convolutional neural network model on a low-end FPGA hardware platform.

The method for realizing a deep convolutional neural network model provided by the present invention is realized by a Xilinx ZYNQ-7030 programmable on-chip SoC, and the hardware platform has a built-in FPGA and an ARM Cortex A9 processor. The present invention firstly loads the trained network model parameters to the FPGA side, then preprocesses the input data on the ARM side, and then transmits the result to the FPGA side, and realizes the convolution calculation and downsampling of the deep convolutional neural network on the FPGA side , form a data feature vector and transmit it to the ARM side to complete the feature classification calculation. Specifically, it includes 4 processes: model parameter loading process, input data preprocessing operation process, convolution and downsampling calculation process, and classification calculation process:

1. The model parameter loading process is:

(1) Offline training of deep convolutional neural network models;

(2) ARM side loads training model parameters;

(3) Transfer the model parameters to the FPGA;

2. The input data preprocessing operation process is as follows:

(1) Normalization processing;

(2) Transfer the processing results to the FPGA;

(3) Store to Block RAM on the FPGA side;

3. The calculation process of convolution and downsampling is:

(1) Initialize the convolution pipeline;

(2) Convolution calculation;

(3) Pooled downsampling calculation;

(4) Reinitialize the convolution pipeline and perform multi-layer convolution downsampling calculations;

4. The classification calculation process is as follows:

(1) Send the feature vector back to the ARM side;

(2) Calculated by classification model;

(3) Output the classification result.

The details are as follows:

Step 1. Load training model parameters

(1) Load the parameters of the deep convolutional neural network model trained offline on the ARM side;

(2) Transfer the training model parameters to the FPGA side;

(3) The FPGA side is stored in the Block RAM (block random access memory) after being buffered by the FIFO;

Step 2. Preprocessing the deep convolutional neural network model

(1) Normalize the input data to make it meet the requirements of the model convolution operation;

(2) Use the APB bus to transmit the normalized data from the ARM side to the FPGA side;

(3) The FPGA side stores the normalized data into the Block RAM after passing through the FIFO buffer;

Step 3. Convolution and downsampling calculations

For the calculation of the convolutional layer with the largest amount of computation in the deep convolutional neural network model, a deep pipeline implementation mode is designed. Suppose the network model has H convolutional layers and pooling layers. The input of the hth (h=1,2,...,H) convolutional layer is T m×m floating-point (32-bit) matrices, and the output is S (m-n+1)×(m-n+1 ) Floating-point number (32-bit) matrix, the convolution kernel is K n×n floating-point number (32-bit) matrix (n≤m), the input data sliding window scale is n×n, the horizontal sliding step is 1, and the vertical sliding The step size is 1.

(1) Initialize the convolution operation pipeline

Define n+1 data buffer registers P ₀ , P ₁ , . . . , P _n-1 , P _n , and each register stores m pieces of data. where n registers (P _{(i-1)%(n+1)+0} , P _{(i-1)%(n+1)+1} , ..., P _{(i-1)%(n+1)+ n-1} ) stores the i-th (i=1,2,...,m-n+1) sub-matrix (n×m) data of the t-th (t=1,2,…,T) input data matrix, Among them, % means taking the remainder, if (i-1)%(n+1)+x>n, then (i-1)%(n+1)+x=0, (i-1)%(n+1 )+x+1=1,…, where x=0,1,…,n-1. If n<m, the P _{(i-1)%(n+1)+n} register stores the i+nth row data in the input data matrix, and implements parallel initialization during the convolution calculation process to reduce FPGA idle cycles and improve Computational efficiency.

Define a convolution kernel matrix cache register W to store the kth (k=1, 2, ..., K) n×n convolution kernel matrix weight data.

(2) Calculation of the hth convolutional layer

The convolution calculation of the t-th input data matrix and the k-th convolution kernel of the h-th convolutional layer of the network is completed, and the activation of the calculation result is realized by the Sigmoid function.

Specifically, while performing each convolution calculation, initialize the i+nth data buffer register P _{(i-1)%(n+1)+n} as the i+1th submatrix convolution in the convolution Computed buffered input data to implement circular convolution.

。具体步骤为：On the FPGA side, a Sigmoid function is constructed through a floating-point IP (Floating-point IP) core to realize the activation of the convolution calculation result; the expression of the Sigmoid function is:

. The specific steps are:

As mentioned above, the input data is an m×m floating-point number matrix, the convolution kernel is an n×n floating-point number matrix, the sliding window scale is n×n, the horizontal sliding step is 1, and the vertical sliding step is 1, then the convolution The result of the product is a (m-n+1)×(m-n+1) floating-point matrix. Each element of the matrix is added with the offset b11 (offline training model parameter). After activation by the Sigmoid function, the result is ( The floating-point number matrix of m-n+1)×(m-n+1) is stored in the Block RAM.

After completing one convolution calculation, re-initialize the convolution kernel matrix cache register W, perform the next convolution calculation, and reciprocate the circular convolution calculation. The calculation result is S (m-n+1)×(m-n+1 ) Floating point matrix, stored in Block RAM.

(3) Calculation of the hth pooling layer

The pooling calculation of the calculation result of the hth convolutional layer is realized, and the result is S [(m-n+1)/2]×[(m-n+1)/2] floating-point number matrices, which are stored in Block RAM. The specific steps are: set the sliding window scale of the convolution calculation result data to be 2×2 and the step size to be 2, and use the average downsampling method to realize pooling, that is, add 2×2 floating-point number matrices one by one, take the average of the calculation results, and obtain S A [(m-n+1)/2]×[(m-n+1)/2] floating-point matrix is used as the input matrix for the calculation of the h+1th convolutional layer.

Step 4. Classification calculation

The convolution calculation and pooling calculation results are sent back to the ARM side for classification operations. The specific steps are: the FPGA side transfers the convolution pooling calculation result matrix in the Block RAM to the ARM side through the FIFO buffer and the APB bus. The ARM side uses the Softmax operation to complete the data classification calculation, obtains the classification result of the input data and outputs it.

The main features of the method of the present invention are:

(1) Implemented a deep convolutional neural network model on a low-end FPGA;

(2) The convolution calculation acceleration in the deep convolutional neural network model is realized by the pipeline calculation method;

(3) The control chip is implemented by Soc embedded ARM processor, which has the characteristics of small size, low power consumption and high efficiency, and can be widely used in the field of embedded systems.

The invention utilizes the fast parallel processing of FPGA and the high-efficiency computing characteristics of extremely low power consumption to realize the convolution computing part with the highest complexity in the deep convolutional neural network model, and greatly improves the algorithm efficiency on the premise of ensuring the accuracy of the algorithm. Compared with the traditional method for implementing deep convolutional neural network based on CPU or GPU, the method of the present invention greatly reduces power consumption while effectively improving the calculation speed of the algorithm, and effectively solves the problems caused by using CPU or GPU to implement deep convolutional neural network. The problem of long operation time or high power consumption.

Description of drawings

Figure 1 is a flow chart of FPGA-based deep convolutional neural network implementation.

Figure 2 MNIST database (section).

Figure 3 Schematic diagram of matrix transpose.

Figure 4 Schematic diagram of pipeline calculation.

Figure 5 Schematic diagram of convolution calculation.

Figure 6. Structure diagram of deep convolutional neural network.

Figure 7 Schematic diagram of downsampling calculation.

Figure 8. Simulation results of the FPGA-based deep convolutional neural network model.

Figure 9 Measured classification results of the digit "7" (MNIST database).

Detailed ways

The specific implementation of the method of the present invention is explained below in conjunction with the accompanying drawings, and the deep convolutional neural network model is used on the FPGA hardware platform to realize the specific implementation of the handwritten character recognition algorithm. (The deep convolutional neural network model consists of input layer I, first convolutional layer C1, first downsampling layer S1, second convolutional layer C2, second downsampling layer S2 and full link layer Softmax The input image size is 28×28, the first convolutional layer contains 1 convolution kernel of size 5×5, and the second convolutional layer contains 3 convolution kernels of size 5×5).

The specific operation steps implemented on the FPGA by the handwritten character recognition algorithm using the deep convolutional neural network model are shown in Figure 1.

1. Load the trained model parameters

First, refer to the function of CNN in DeepLearnToolbox-master, and make certain modifications (rewrite the convolution function, and change the number of neural network layers to 5 layers, one input layer, two convolution layers, and two downsampling layers; The first convolutional layer has 1 convolution kernel of size 5×5, the second convolutional layer has 3 convolution kernels of size 5×5, the sliding step size of the two downsampling layers is 2, and the sliding window 2 × 2 matrix, the number of training is set to 10), use Matlab to train the deep convolutional neural network, then load the trained weight parameters and bias parameters on the ARM side, and finally transfer the trained model parameters to the FPGA side. FIFO is buffered and stored in Block RAM.

2. Preprocessing

The MNIST handwriting image shown in Figure 2 is read into memory, each pixel is divided by 255 for normalization, and then transposed as shown in Figure 3.

3. Transfer the preprocessing results to the FPGA

Through the APB bus on the ZYNQ-7030 Soc, the preprocessing results are transmitted to the FPGA side, and stored in the Block RAM after being buffered by the FIFO.

4. Initialize the convolution operation pipeline

As shown in FIG. 4 , six data cache registers P ₀ , P ₁ , P ₂ , P ₃ , P ₄ , and P ₅ are defined, and each register can store 28 floating-point data. Among them, 5 registers (P _{(i-1)%(5+1)+0} , P _{(i-1)%(5+1)+1} , ..., P _{(i-1)%(5+1)+ 5-1} ) Store the data of the i-th (i=1, 2,..., 24) sub-matrix (5×28) of the input image matrix, where % means taking the remainder. If (i-1)%(5+1)+x>5, then (i-1)%(5+1)+x=0, (i-1)%(5+1)+x+1= 1,…,where x=0,1,…,4. The P _{(i-1)%(5+1)+5} register stores the i+5th row data in the input image matrix.

Define a convolution kernel matrix cache register W, which stores a 5×5 convolution kernel matrix weight data of the first convolution layer.

5. Perform the first convolutional layer calculation

Complete the convolution calculation of the input image matrix of the first convolution layer of the network and the first convolution kernel of the first convolution layer, and activate the calculation result through the Sigmoid function.

While performing the convolution calculation, initialize the i+5th data buffer register P _{(i-1)%(5+1)+5} as the buffered input data for the i+1th submatrix convolution calculation in the convolution, Implement circular convolution, as shown in Figure 5.

。On the FPGA side, a sigmoid function is constructed through a floating-point IP (Floating-point IP) core to realize the activation of the convolution calculation result. The expression of the sigmoid function is:

.

The specific steps are:

As mentioned above, the input image is a 28×28 floating-point matrix, the convolution kernel is a 5×5 floating-point matrix, the sliding window scale is 5×5, the horizontal sliding step is 1, and the vertical sliding step is 1, then the convolution The result of the product is a 24×24 floating-point number matrix. Each element of the matrix is added with the offset b11 (offline training model parameter). After activation by the Sigmoid function, the result is a 24×24 floating-point number matrix, which is stored in the Block RAM.

After completing one convolution calculation, the calculation result is a 24×24 floating-point number matrix, which is stored in the Block RAM.

6. Perform the first pooling layer calculation

Implement the pooling calculation of the calculation result of the first convolutional layer, as shown in Figure 6, the result is a 12 × 12 floating-point number matrix, which is stored in the Block RAM. The specific steps are: the sliding window scale of the convolution calculation result data is 2 × 2, the step size is 2, and the average downsampling method is used to realize the pooling, that is, the 2 × 2 floating-point number matrices are added one by one, and the calculation results are averaged to obtain 1 The 12×12 floating-point number matrix is used as the input matrix for the calculation of the second convolutional layer, as shown in Figure 7.

7. Reinitialize the convolution pipeline

As shown in FIG. 4 , 6 data buffer registers P ₀ , P ₁ , P ₂ , P ₃ , P ₄ , and P ₅ are reinitialized, and each register stores 12 floating-point data. Among them, 5 registers (P _{(i-1)%(5+1)+0} , P _{(i-1)%(5+1)+1} , ..., P _{(i-1)%(5+1)+ 5-1} ) Store the data of the i-th (i=1,2,...,8) submatrix (5×12) of the input matrix, where % means taking the remainder. If (i-1)%(5+1)+x>5, then (i-1)%(5+1)+x=0, (i-1)%(5+1)+x+1= 1,…,where x=0,1,…,4. The P _{(i-1)%(5+1)+5} register stores the i+5th row data in the input matrix.

Reinitialize the convolution kernel matrix cache register W, and store the first 5×5 convolution kernel matrix weight data of the second convolution layer.

8. Perform the second convolutional layer calculation

Complete the convolution calculation of the input data matrix of the second convolution layer of the network and the first convolution kernel of the second convolution layer, and activate the calculation result through the Sigmoid function.

Reinitialize the convolution kernel matrix cache register W, store the second 5 × 5 convolution kernel matrix weight data of the second convolution layer, and complete the input data matrix of the second convolution layer of the network and the second convolution The convolution calculation of the second convolution kernel of the layer realizes the activation of the calculation result through the Sigmoid function.

Reinitialize the convolution kernel matrix cache register W, store the third 5×5 convolution kernel matrix weight data of the second convolution layer, and complete the input data matrix of the second convolution layer of the network and the second convolution The convolution calculation of the third convolution kernel of the layer realizes the activation of the calculation result through the Sigmoid function.

While performing each convolution calculation, initialize the i+5th data buffer register P _{(i-1)%(5+1)+5} as the buffer input for the i+1th submatrix convolution calculation in the convolution data, implement circular convolution, as shown in Figure 5.

The specific steps are: as mentioned above, the input image is a 12×12 floating-point matrix, the convolution kernel is three 5×5 floating-point matrices, the sliding window scale is 5×5, the horizontal sliding step is 1, and the vertical sliding step is 1. If the length is 1, the convolution result is three 8×8 floating-point number matrices. Each element of the three matrices is added with offsets b21, b22, b23 (offline training model parameters), and after activation by the Sigmoid function, The result is three 8×8 floating-point number matrices, which are stored in Block RAM.

After completing the two convolution calculations, the calculation results are three 8×8 floating-point number matrices, which are stored in the Block RAM.

9. Perform the second pooling layer calculation

Implement the pooling calculation of the calculation results of the second convolution layer, as shown in Figure 6, the results are three 4×4 floating point number matrices, which are stored in the Block RAM. The specific steps are: the sliding window scale of the convolution calculation result data is 2 × 2, the step size is 2, and the average downsampling method is used to realize the pooling, that is, the 2 × 2 floating-point number matrices are added one by one, and the calculation results are averaged to obtain 3 The 4×4 floating point number matrix is used as the input matrix of the Softmax layer, as shown in Figure 7.

10. Classification calculation

The convolution calculation and pooling calculation results are sent back to the ARM side for classification operations. The specific steps are: the FPGA side transfers the convolution pooling calculation result matrix in the Block RAM to the ARM side through the FIFO buffer and the APB bus. The ARM side uses the Softmax operation to complete the data classification calculation, and obtains the classification result of the input image and outputs it.

The simulation result of the above method processing the digital picture "7" in the MNIST database is shown in Figure 8.

Figure 9 shows the measured classification results of the above method processing the digital picture "7" in the MNIST database.

references

[1] Cong J, Xiao B. Minimizing Computation in Convolutional NeuralNetworks[M]// Artificial Neural Networks and Machine Learning – ICANN 2014. Springer International Publishing, 2014:33-7.

[2] Farabet C, Poulet C, Han J Y, et al. CNP: An FPGA-based processor for Convolutional Networks[J]. International Conference on Field ProgrammableLogic & Applications, 2009:32-37.

[3] Gokhale V, Jin J, Dundar A, et al. A 240 G-ops/s MobileCoprocessor for Deep Neural Networks[C]// IEEE Embedded Vision Workshop.2014:696-701.

[4] Zhang C, Li P, Sun G, et al. Optimizing FPGA-based AcceleratorDesign for Deep Convolutional Neural Networks[C]// Acm/sigda InternationalSymposium. 2015:161-170.

[5] Krizhevsky A, Sutskever I, Hinton G E. ImageNet Classification with Deep Convolutional Neural Networks[J]. Advances in Neural Information Processing Systems, 2012, 25(2):2012.

[6] Farabet C, Martini B, Corda B, et al. NeuFlow: A runtimereconfigurable dataflow processor for vision[J]. 2011, 9(6):109-116.

[7] Matai J, Irturk A, Kastner R. Design and Implementation of an FPGA-Based Real-Time Face Recognition System[C]// IEEE, International Symposium on Field-Programmable Custom Computing Machines. 2011:97-100.

[8] Sankaradas M, Jakkula V, Cadambi S, et al. A Massively Parallel Coprocessor for Convolutional Neural Networks[C]// IEEE International Conference on Application-Specific Systems, Architectures and Processors. IEEE Computer Society, 2009:53-60.

Claims (1)

1. a deep convolutional neural network implementation method based on FPGA, is characterized in that concrete steps are: Step 1. Load training model parameters (1) Load the parameters of the deep convolutional neural network model trained offline on the ARM side; (2) Transfer the training model parameters to the FPGA side; (3) The FPGA side is stored in the block random access memory after being buffered by the FIFO; Step 2. Preprocessing the deep convolutional neural network model (1) Normalize the input data to make it meet the requirements of the model convolution operation; (2) Use the APB bus to transmit the normalized data from the ARM side to the FPGA side; (3) The FPGA side stores the normalized data into the block random access memory after passing through the FIFO buffer; Step 3. Convolution and downsampling calculations Suppose the network model has H convolutional layers and H pooling layers, the input of the hth convolutional layer is T m×m 32-bit floating-point number matrices, h=1,2,…,H; the output is S ( m-n+1)×(m-n+1) 32-bit floating-point number matrix, the convolution kernel is K n×n 32-bit floating-point number matrices, n≤m, the input data sliding window scale is n×n, horizontal The sliding step is 1, and the vertical sliding step is 1; (1) Initialize the convolution operation pipeline Define n+1 data cache registers P ₀ , P ₁ ,..., P _n-1 , P _n , each register stores m data, of which n registers P _{(i-1)%(n+1)+0} , P _{(i-1)%(n+1)+1} ,...,P _{(i-1)%(n+1)+n-1} stores the i-th submatrix n×m data of the t-th input data matrix , t=1,2,...,T,i=1,2,...,m-n+1; use % to represent the remainder, if (i-1)%(n+1)+x>n, then ( i-1)%(n+1)+x=0, (i-1)%(n+1)+x+1=1,..., where x=0,1,...,n-1; if n <m, P _{(i-1)%(n+1)+n} register stores the i+nth row data in the input data matrix, and implements parallel initialization during the convolution calculation process to reduce FPGA idle cycles and improve computational efficiency ; Define a convolution kernel matrix cache register W to store the kth n×n convolution kernel matrix weight data, k=1,2,...,K; (2) The calculation of the h-th convolutional layer Complete the convolution calculation of the t-th input data matrix and the k-th convolution kernel of the h-th convolution layer of the network, and activate the calculation result through the Sigmoid function; While performing each convolution calculation, initialize the data buffer register P _{(i-1)%(n+1)+n} as the buffered input data for the i+1th sub-matrix convolution calculation in the convolution to realize the circular volume product; The Sigmoid function is constructed on the FPGA side through the floating-point IP core to realize the activation of the convolution calculation result. The expression of the Sigmoid function is:

The specific steps are: As mentioned above, the input data is an m×m floating-point number matrix, the convolution kernel is an n×n floating-point number matrix, the sliding window scale is n×n, the horizontal sliding step is 1, and the vertical sliding step is 1, then the convolution The result of the product is a (m-n+1)×(m-n+1) floating-point number matrix. Each element of the matrix is added with the offset b11, that is, the offline training model parameters. After activation by the Sigmoid function, the result is ( m-n+1)×(m-n+1) floating-point number matrix, stored in Block RAM; After completing one convolution calculation, re-initialize the convolution kernel matrix cache register W, perform the next convolution calculation, and reciprocate the convolution calculation. The calculation result is S (m-n+1)×(m-n+1 ) Floating point matrix, stored in Block RAM; (3) The calculation of the h-th pooling layer Implement the pooling calculation of the calculation result of the hth convolutional layer, and the result is S [(m-n+1)/2]×[(m-n+1)/2] floating-point number matrices, which are stored in Block RAM; The specific steps are: set the sliding window scale of the convolution calculation result data to be 2×2 and the step size to be 2, and use the average downsampling method to realize pooling, that is, add 2×2 floating-point number matrices one by one, take the average of the calculation results, and obtain S A [(m-n+1)/2]×[(m-n+1)/2] floating-point number matrix is used as the input matrix calculated by the h+1th convolutional layer; Step 4. Classification calculation Send the convolution calculation and pooling calculation results back to the ARM side for classification operations; the specific steps are: the FPGA side transmits the convolution pooling calculation result matrix in the BlockRAM to the ARM side through the FIFO buffer, APB bus, and the ARM side uses Softmax. The operation completes the data classification calculation, obtains the classification result of the input data and outputs it.

Images