WO2025218403A1 - 数据处理方法、处理器、芯片及电子设备 - Google Patents

数据处理方法、处理器、芯片及电子设备

Info

Publication number
WO2025218403A1
WO2025218403A1 PCT/CN2025/082601 CN2025082601W WO2025218403A1 WO 2025218403 A1 WO2025218403 A1 WO 2025218403A1 CN 2025082601 W CN2025082601 W CN 2025082601W WO 2025218403 A1 WO2025218403 A1 WO 2025218403A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
processed
control logic
block data
loop
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2025/082601
Other languages
English (en)
French (fr)
Inventor
高坤
文思
李晓森
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Moore Threads Technology Co Ltd
Original Assignee
Moore Threads Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Moore Threads Technology Co Ltd filed Critical Moore Threads Technology Co Ltd
Publication of WO2025218403A1 publication Critical patent/WO2025218403A1/zh
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/76—Architectures of general purpose stored program computers
    • G06F15/78—Architectures of general purpose stored program computers comprising a single central processing unit
    • G06F15/7807—System on chip, i.e. computer system on a single chip; System in package, i.e. computer system on one or more chips in a single package
    • G06F15/781—On-chip cache; Off-chip memory
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/15—Correlation function computation including computation of convolution operations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/30003—Arrangements for executing specific machine instructions
    • G06F9/30007—Arrangements for executing specific machine instructions to perform operations on data operands
    • G06F9/3001—Arithmetic instructions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/30003—Arrangements for executing specific machine instructions
    • G06F9/30007—Arrangements for executing specific machine instructions to perform operations on data operands
    • G06F9/30029—Logical and Boolean instructions, e.g. XOR, NOT
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/30003—Arrangements for executing specific machine instructions
    • G06F9/30007—Arrangements for executing specific machine instructions to perform operations on data operands
    • G06F9/30036—Instructions to perform operations on packed data, e.g. vector, tile or matrix operations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/30003—Arrangements for executing specific machine instructions
    • G06F9/3004—Arrangements for executing specific machine instructions to perform operations on memory
    • G06F9/30047—Prefetch instructions; cache control instructions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/30098—Register arrangements
    • G06F9/3012—Organisation of register space, e.g. banked or distributed register file
    • G06F9/30134—Register stacks; shift registers
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]

Definitions

  • the present disclosure relates to the field of computer technology, and in particular to a data processing method, a processor, a chip, and an electronic device.
  • Neural network algorithms are a recently popular type of machine learning algorithm in artificial intelligence (AI), achieving remarkable results in a variety of fields, such as image recognition, speech recognition, and natural language processing.
  • AI artificial intelligence
  • processors such as central processing units (CPUs) and graphics processing units (GPUs) consumes significant computational time and energy.
  • Processors and AI accelerators incorporate numerous proprietary modules to accelerate training and inference. Matrix multiplication and convolution are common operators in AI algorithms.
  • SIMD Single Instruction Multiple Data
  • SIMT Single Instruction Multiple Thread
  • the present disclosure proposes a data processing technology solution.
  • a data processing method comprising: the data processing method is applied to a processor, the processor comprising a control logic unit and a dot multiplication unit array, the dot multiplication unit array comprising a plurality of dot multiplication units for performing multiplication and accumulation operations, the method comprising: the control logic unit sequentially reading, according to an acquired control instruction, cyclic block data of data to be processed from a memory to the dot multiplication unit array; the dot multiplication unit array performing a multiplication and accumulation operation on the cyclic block data of the data to be processed received each time, and determining a cyclic block result of the data to be processed received each time; the control logic unit determining a logical operation result of the data to be processed based on the plurality of cyclic block results acquired from the dot multiplication unit array.
  • the control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the acquired control instruction, including: the control logic unit parses the access information of the data to be processed and the logical operation type of the data to be processed according to the acquired control instruction; determines the cyclic block order of the data to be processed according to the logical operation type of the data to be processed; and reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the access information and the preset size and the cyclic block order of the data to be processed, wherein the size of the cyclic block data is less than or equal to the preset size, and the preset size is determined by the number of idle dot multiplication units in the dot multiplication unit array.
  • the access information of the data to be processed includes access information of a first matrix and a second matrix
  • the number of columns of the first matrix is the same as the number of rows of the second matrix
  • the logical operation type includes a matrix multiplication operation of the first matrix and the second matrix
  • the loop block order determined by the matrix multiplication operation type includes: a first outer loop order in the direction of the number of rows of the first matrix, a second outer loop order in the direction of the number of columns of the second matrix, and an inner loop order in the direction of the number of columns of the first matrix and the number of rows of the second matrix; or, a first outer loop order in the direction of the number of columns of the second matrix, a second outer loop order in the direction of the number of rows of the first matrix, and an inner loop order in the direction of the number of columns of the first matrix and the number of rows of the second matrix.
  • the memory includes a local memory arranged outside the processor, and a reuse cache arranged within the processor for caching loop block data.
  • the loop block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the loop block order of the data to be processed, including: the control logic unit stores the first loop block data of the first matrix read from the local memory into the reuse cache according to the first outer loop order and the inner loop order, so that the control logic unit reuses the first loop block data stored in the reuse cache according to the second outer loop order; or, the control logic unit stores the second loop block data of the second matrix read from the local memory into the reuse cache according to the second outer loop order and the inner loop order, so that the control logic unit reuses the second loop block data stored in the reuse cache according to the first outer loop order.
  • the dot multiplication unit array performs a multiplication and accumulation operation on the circular block data of the data to be processed each time it is received, and determines the circular block result of the data to be processed each time it is received, including: the dot multiplication unit array performs a multiplication and accumulation operation on the first circular block data and the second circular block data obtained each time, and obtains the circular block result corresponding to each multiplication and accumulation operation; the control logic unit determines the logical operation result of the data to be processed based on the multiple circular block results obtained from the dot multiplication unit array, including: the control logic unit performs an accumulation operation on the multiple circular block results obtained from the dot multiplication unit array in any round of the circular block sequence to obtain a logical operator result; the control logic unit writes the logical operator result into a register stack; the control logic unit determines the logical operation result based on the multiple logical operator results obtained from the register stack.
  • the access information of the data to be processed includes access information of the data to be convolved and the convolution kernel
  • the logical operation type includes the convolution operation of the data to be convolved and the convolution kernel
  • the circular block order determined by the convolution operation type includes the coordinate order of the elements in the convolution kernel
  • the circular block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the circular block order of the data to be processed, including: the control logic unit forms a third circular block data with multiple elements with the same coordinates read from multiple convolution kernels in the memory each time according to the coordinate order of the elements in the convolution kernel, and, according to the coordinates of the current element in each convolution kernel and the access information, reads multiple elements from the data to be convolved in the memory to form a fourth circular block data; the control logic unit writes the third circular block data and the fourth circular block data into the dot product unit array.
  • the memory includes a local memory arranged outside the processor, and a reuse cache arranged within the processor for caching circular block data.
  • the method also includes: in the process of calculating the fourth circular block data corresponding to the same row in the convolution output result, the control logic unit writes M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column of the convolution kernel into the reuse cache according to the coordinate order of the elements in the convolution kernel, where r, s, and M are positive integers; the control logic unit reads M-1 elements from the reuse cache, reads 1 element from the data to be convolved in the local memory, and determines the fourth circular block data corresponding to the coordinates of the element in the rth row and s+1th column of the convolution kernel.
  • the method further includes: in the process of calculating the fourth circular block data corresponding to the adjacent rows in the convolution output result, in response to the control logic unit calculating the fourth circular block data of the current row of the convolution output result, M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column in the convolution kernel are written into the reuse cache, where r, s, and M are positive integers; in response to the control logic unit calculating the fourth circular block data of the next row of the current row of the convolution output result, M elements are read from the reuse cache to constitute the fourth circular block data corresponding to the coordinates of the element in the r-1th row and sth column in the convolution kernel.
  • the memory includes a local memory arranged outside the processor, and a reuse cache arranged within the processor for caching cyclic block data
  • the control logic unit according to the coordinate order of the elements in the convolution kernel, converts the multiple elements with the same coordinates read from the multiple convolution kernels in the memory each time into third cyclic block data, including: the control logic unit reads the multiple convolution kernels from the local memory to the reuse cache; according to the coordinate order of the elements in the convolution kernel, the control logic unit converts the multiple elements with the same coordinates read from the multiple convolution kernels in the reuse cache each time into third cyclic block data.
  • the dot product unit array performs a multiplication and accumulation operation on the cyclic block data of the data to be processed each time received to determine the cyclic block result of the data to be processed each time received, including: the dot product unit array performs a multiplication and accumulation operation on the third cyclic block data and the fourth cyclic block data received each time to determine the cyclic block result.
  • a processor comprising a control logic unit and a dot multiplication unit array, wherein the dot multiplication unit array comprises a plurality of dot multiplication units for performing multiplication and accumulation operations, and wherein: the control logic unit sequentially reads cyclic block data of to-be-processed data from a memory to the dot multiplication unit array according to an acquired control instruction; the dot multiplication unit array performs a multiplication and accumulation operation on the cyclic block data of the to-be-processed data received each time, and determines a cyclic block result of the to-be-processed data received each time; and the control logic unit determines a logical operation result of the to-be-processed data according to the plurality of cyclic block results acquired from the dot multiplication unit array.
  • the control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the acquired control instruction, including: the control logic unit parses the access information of the data to be processed and the logical operation type of the data to be processed according to the acquired control instruction; determines the cyclic block order of the data to be processed according to the logical operation type of the data to be processed; and reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the access information and the preset size and the cyclic block order of the data to be processed, wherein the size of the cyclic block data is less than or equal to the preset size, and the preset size is determined by the number of idle dot multiplication units in the dot multiplication unit array.
  • the access information of the data to be processed includes access information of a first matrix and a second matrix
  • the number of columns of the first matrix is the same as the number of rows of the second matrix
  • the logical operation type includes a matrix multiplication operation of the first matrix and the second matrix
  • the loop block order determined by the matrix multiplication operation type includes: a first outer loop order in the direction of the number of rows of the first matrix, a second outer loop order in the direction of the number of columns of the second matrix, and an inner loop order in the direction of the number of columns of the first matrix and the number of rows of the second matrix; or, a first outer loop order in the direction of the number of columns of the second matrix, a second outer loop order in the direction of the number of rows of the first matrix, and an inner loop order in the direction of the number of columns of the first matrix and the number of rows of the second matrix.
  • the memory includes a local memory arranged outside the processor, and a reuse cache arranged within the processor for caching loop block data.
  • the loop block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the loop block order of the data to be processed, including: the control logic unit stores the first loop block data of the first matrix read from the local memory into the reuse cache according to the first outer loop order and the inner loop order, so that the control logic unit reuses the first loop block data stored in the reuse cache according to the second outer loop order; or, the control logic unit stores the second loop block data of the second matrix read from the local memory into the reuse cache according to the second outer loop order and the inner loop order, so that the control logic unit reuses the second loop block data stored in the reuse cache according to the first outer loop order.
  • the dot multiplication unit array performs a multiplication and accumulation operation on the circular block data of the data to be processed each time it is received, and determines the circular block result of the data to be processed each time it is received, including: the dot multiplication unit array performs a multiplication and accumulation operation on the first circular block data and the second circular block data obtained each time, and obtains the circular block result corresponding to each multiplication and accumulation operation; the control logic unit determines the logical operation result of the data to be processed based on the multiple circular block results obtained from the dot multiplication unit array, including: the control logic unit performs an accumulation operation on the multiple circular block results obtained from the dot multiplication unit array in any round of the circular block sequence to obtain a logical operator result; the control logic unit writes the logical operator result into a register stack; the control logic unit determines the logical operation result based on the multiple logical operator results obtained from the register stack.
  • the access information of the data to be processed includes access information of the data to be convolved and the convolution kernel
  • the logical operation type includes the convolution operation of the data to be convolved and the convolution kernel
  • the circular block order determined by the convolution operation type includes the coordinate order of the elements in the convolution kernel
  • the circular block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the circular block order of the data to be processed, including: the control logic unit forms a third circular block data with multiple elements with the same coordinates read from multiple convolution kernels in the memory each time according to the coordinate order of the elements in the convolution kernel, and, according to the coordinates of the current element in each convolution kernel and the access information, reads multiple elements from the data to be convolved in the memory to form a fourth circular block data; the control logic unit writes the third circular block data and the fourth circular block data into the dot product unit array.
  • the memory includes a local memory arranged outside the processor, and a reuse cache arranged within the processor for caching circular block data
  • the processor is further used to: in the process of calculating the fourth circular block data corresponding to the same row in the convolution output result, the control logic unit writes M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column of the convolution kernel into the reuse cache according to the coordinate order of the elements in the convolution kernel, where r, s, and M are positive integers; the control logic unit reads M-1 elements from the reuse cache, reads 1 element from the data to be convolved in the local memory, and determines the fourth circular block data corresponding to the coordinates of the element in the rth row and s+1th column of the convolution kernel.
  • the processor is further used to: in the process of calculating the fourth circular block data corresponding to the adjacent rows in the convolution output result, in response to the control logic unit calculating the fourth circular block data of the current row of the convolution output result, write M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column in the convolution kernel into the reuse cache, where r, s, and M are positive integers; in response to the control logic unit calculating the fourth circular block data of the next row of the current row of the convolution output result, read M elements from the reuse cache to constitute the fourth circular block data corresponding to the coordinates of the element in the r-1th row and sth column in the convolution kernel.
  • the memory includes a local memory arranged outside the processor, and a reuse cache arranged within the processor for caching cyclic block data
  • the control logic unit according to the coordinate order of the elements in the convolution kernel, converts the multiple elements with the same coordinates read from the multiple convolution kernels in the memory each time into third cyclic block data, including: the control logic unit reads the multiple convolution kernels from the local memory to the reuse cache; according to the coordinate order of the elements in the convolution kernel, the control logic unit converts the multiple elements with the same coordinates read from the multiple convolution kernels in the reuse cache each time into third cyclic block data.
  • the dot product unit array performs a multiplication and accumulation operation on the cyclic block data of the data to be processed each time received to determine the cyclic block result of the data to be processed each time received, including: the dot product unit array performs a multiplication and accumulation operation on the third cyclic block data and the fourth cyclic block data received each time to determine the cyclic block result.
  • an artificial intelligence chip comprising the processor as described above.
  • an electronic device comprising the artificial intelligence chip as described above.
  • control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the control instructions obtained, so that the dot multiplication unit array performs multiplication and accumulation operations on the cyclic block data of the data to be processed each time it is received, and determines the cyclic block result of the data to be processed each time it is received.
  • the control logic unit determines the logical operation result of the data to be processed based on the multiple cyclic block results obtained from the dot multiplication unit array.
  • the reading and logical operation of the data to be processed can be converted into the reading and logical operation of multiple cyclic block data of the data to be processed (small-sized image data), which is beneficial for processing larger-sized data while keeping the processor hardware resources unchanged, reducing the pressure on the storage bandwidth.
  • FIG1 shows a schematic diagram of a processor according to an embodiment of the present disclosure.
  • FIG2 shows a flow chart of a data processing method according to an embodiment of the present disclosure.
  • FIG3 is a schematic diagram showing a cyclic blocking sequence according to an embodiment of the present disclosure.
  • FIG4 is a schematic diagram showing a situation in which repeated reading of cyclically blocked data occurs according to an embodiment of the present disclosure.
  • FIG5 is a schematic diagram showing the third cycle of block data of a convolution kernel according to an embodiment of the present disclosure.
  • FIG6 is a schematic diagram showing fourth-cycle block data of a convolution kernel according to an embodiment of the present disclosure.
  • FIG7 shows a schematic diagram of a reuse cache according to an embodiment of the present disclosure.
  • FIG8 shows a schematic diagram of another reuse cache according to an embodiment of the present disclosure.
  • FIG9 shows a block diagram of an electronic device according to an embodiment of the present disclosure.
  • a and/or B can represent the existence of three situations: A alone, A and B simultaneously, and B alone.
  • at least one herein refers to any combination of at least two of any one or more of a plurality of items.
  • at least one of A, B, and C can represent any one or more elements selected from the set consisting of A, B, and C.
  • Figure 1 shows a schematic diagram of a processor according to an embodiment of the present disclosure.
  • the processor includes a control logic unit and a dot multiplication unit array, wherein the dot multiplication unit array includes a plurality of dot multiplication units for performing multiply-accumulate operations.
  • the dot multiplication unit may also include an operator to perform a specified operation, such as calculating multiple multiplications and performing accumulation.
  • the dot multiplication unit may include a multiplier, an adder, etc.
  • the specific structures of each dot multiplication unit may be the same or different, and this disclosure does not limit this.
  • the dot multiplication unit may also include other types of operators to accommodate various different calculation processes. This disclosure does not limit the number and type of operators included in the dot multiplication unit.
  • the control logic unit can connect each dot multiplication unit in the dot multiplication unit array.
  • the control logic unit can number each dot multiplication unit in the form of a two-dimensional matrix or a multi-dimensional matrix so that multiple dot multiplication units can be logically arranged in the form of a two-dimensional matrix or a multi-dimensional matrix, thereby better adapting to the logical operations of the matrix.
  • the memory corresponding to the processor may include a local memory arranged outside the processor, and the control logic unit in the processor can be connected to the local memory.
  • the control logic unit can be used for address calculation to load data from the memory to the dot multiplication unit array and control the dot multiplication unit array to process the data to be processed.
  • the local memory may be an on-chip cache
  • the control logic unit may load the executable program and the data to be processed (for example, the input matrix) on the off-chip flash memory into the above-mentioned local memory (on-chip cache), and then perform the subsequent logical operations on the data to be processed.
  • the local memory may store data to be processed and an executable program.
  • the executable program may include control instructions.
  • the processor executes the control instructions to implement logical operations on the data to be processed, such as matrix multiplication operations, convolution operations, and other multiplication-accumulation-related operations on the data to be processed.
  • the memory may also include a reuse buffer set inside the processor.
  • the reuse buffer may be set inside the dot multiplication unit array (not shown in Figure 1) or outside the dot multiplication unit array (see Figure 1). Compared with setting the reuse buffer outside the dot multiplication unit array and setting the reuse buffer inside the dot multiplication unit array, the dot multiplication unit array can have higher data reading efficiency.
  • the control logic unit is provided with a loader, a decoder, etc., wherein the loader can be used to load the data to be processed or part of the data to be processed in the local memory into the reuse cache in the processor.
  • the decoder can decode the control instructions for accessing data in the executable program based on the change in the storage address of the data to be processed after loading. For example, for the control instruction to access data X in the local memory, since the data X is cached in the reuse cache, the address of the data X stored in the reuse cache can be obtained through decoding.
  • the decoder can convert the control instruction to access data X in the local memory into a control instruction to access data X in the reuse cache, which is conducive to the subsequent control logic unit directly sending the data cached in the reuse cache to the dot product unit, and the dot product unit performs a multiplication and accumulation operation on it.
  • the control logic unit can also directly load data from the off-chip memory to the reuse cache, which is not limited in this disclosure.
  • a corresponding register file (Register File) can also be set for the processor for reading, storing, processing and transmitting data without being combined with a memory.
  • the register file includes memory address registers, memory data registers, instruction registers, operation code word registers, accumulators, flag registers, etc., and the present disclosure does not limit this.
  • the control logic unit can be used for address calculation so as to move data between the dot multiplication unit array and the register file.
  • the processor of the embodiment of the present disclosure may be a newly designed one, or may be obtained by improving an existing processor chip.
  • the types of processor chips may include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), a general-purpose computing on graphics processing units (GPGPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a tensor processing unit (TPU), a field programmable gate array (FPGA), or other programmable logic devices, and may also include a microprocessor or other conventional processor processor.
  • Figure 2 shows a flow chart of a data processing method according to an embodiment of the present disclosure. As shown in Figure 2, the data processing method is applied to a processor, and the data processing method may include the following steps:
  • step S11 the control logic unit reads the cyclic block data of the data to be processed from the memory to the point multiplication unit array in sequence according to the obtained control instructions.
  • the control instructions are used to indicate the access information of the data to be processed and the logical operation type of the data to be processed, such as matrix multiplication operations, convolution operations and other multiplication-accumulation related operations of the data to be processed.
  • control logic unit can obtain the address information of the data to be processed according to the control instruction, and determine the address information of each cyclic block data of the data to be processed in sequence according to the address information of the data to be processed, and read the cyclic block data of the data to be processed from the memory to the dot multiplication unit array.
  • the memory may include a local memory provided outside the processor, and the control logic unit may read the cyclic block data of the data to be processed from the local memory in sequence to the dot product unit array according to the acquired control instructions.
  • the memory may also be provided in the processor for caching the cyclic block data.
  • the control logic unit reads data from the reuse cache faster than the speed of reading data from the local memory.
  • control logic unit may also synchronously store the cyclic block data that needs to be read repeatedly into the reuse cache provided in the processor, so that the control logic unit can subsequently read the cyclic block data from the reuse cache to the dot multiplication unit array.
  • the data to be processed includes feature data in a deep learning task
  • the feature data includes at least one of image feature data, speech feature data, and text feature data.
  • the data to be processed may be image feature data, and the image feature data of the target object (such as a face feature map) may be stored in a memory; in a scenario where a deep neural network is used to perform speech recognition on a target object, the data to be processed may be speech feature data, and the speech feature data of the target object may be stored in a memory; in a scenario where a deep neural network is used to perform text recognition on a target document, the data to be processed may be text feature data, and the text feature data of the target document may be stored in a memory; the embodiments of the present application do not impose any restrictions on the type of data to be processed.
  • control logic unit may obtain data structure information of the data to be processed and address information of the data to be processed through control instructions; wherein the data structure information of the data to be processed includes, for example, the dimension of the data to be processed, the size of the data to be processed, the data type of the elements in the data to be processed (for example, integer type, single-precision floating-point type, double-precision floating-point type, character type, etc.), and other information used to describe the data to be processed; the address information of the data to be processed includes, for example, the base address (Base Address) of the data to be processed in the memory, the addressing space, and other address-related information.
  • the data structure information of the data to be processed includes, for example, the dimension of the data to be processed, the size of the data to be processed, the data type of the elements in the data to be processed (for example, integer type, single-precision floating-point type, double-precision floating-point type, character type, etc.), and other information used to describe the data to be processed
  • step S12 the dot product unit array performs a multiplication and accumulation operation on the cyclic block data of the to-be-processed data received each time, and determines the cyclic block result of the to-be-processed data received each time.
  • the dot multiplication unit array may include tile_M ⁇ tile_N dot multiplication units, each of which may calculate tile_K multiplications and accumulate them, where tile_M, tile_N, and tile_K are positive integers.
  • tile_M, tile_N, and tile_K are positive integers.
  • the present disclosure does not impose any restrictions on the specific values of tile_M, tile_N, and tile_K, and they may be set according to actual application scenarios.
  • the dot product array performs matrix multiplication or convolution operations by performing multiplication and accumulation operations on the received blocks of data to be processed.
  • the dot product array can multiply a matrix of size tile_M ⁇ tile_K by a matrix of size tile_K ⁇ tile_N.
  • the dot product array can process tile_M convolved elements and tile_N convolution kernel elements at a time, where the number of channels of tile_M convolved elements and tile_N convolution kernel elements is tile_K.
  • step S13 the control logic unit determines the logic operation result of the data to be processed according to the multiple loop block results obtained from the dot product unit array.
  • control logic unit may concatenate and/or add the obtained multiple cyclic block results to determine the logical operation result of the data to be processed.
  • the control logic unit may concatenate and/or add each received cyclic block data with the previously received cyclic block data.
  • the control logic unit may also concatenate and/or add all received cyclic block data, and the present disclosure does not limit this.
  • the control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the control instruction obtained for indicating the logical operation of the data to be processed, so that the dot multiplication unit array performs multiplication and accumulation operations on the cyclic block data of the data to be processed each time it is received, and determines the cyclic block result of the data to be processed each time it is received.
  • the control logic unit determines the logical operation result of the data to be processed based on the multiple cyclic block results obtained from the dot multiplication unit array.
  • the reading and logical operation of the data to be processed can be converted into the reading and logical operation of multiple cyclic block data of the data to be processed (small-sized image data), which is beneficial for processing larger-sized data while keeping the processor hardware resources unchanged, reducing the pressure on the storage bandwidth.
  • step S11 the control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot product unit array according to the acquired control instruction.
  • the memory may include a local memory disposed outside the processor; or, in addition to the local memory disposed outside the processor, the memory may also include a reuse cache disposed within the processor for caching cyclic block data, and the embodiments of the present disclosure do not impose specific restrictions on this.
  • step S11 may include: the control logic unit parses the access information of the data to be processed and the logical operation type of the data to be processed according to the acquired control instruction; determines the cyclic blocking order of the data to be processed according to the logical operation type of the data to be processed; and reads the cyclic blocking data of the data to be processed from the memory in sequence to the dot multiplication unit array according to the access information and the preset size and the cyclic blocking order of the data to be processed, wherein the size of the cyclic blocking data is less than or equal to the preset size, and the preset size is determined by the number of idle dot multiplication units in the dot multiplication unit array.
  • the control logic unit parses the acquired control instruction and can parse out the access information of the data to be processed and the logical operation type of the data to be processed.
  • the access information includes, for example, size information, address information, and layout information of the data to be processed.
  • the layout information can be a row-major sequence (RowMajor) or a column-major sequence (ColMajor), where the row-major sequence indicates that the elements of the same row of the data to be processed are adjacent in the memory, and the column-major sequence indicates that the elements of the same column of the data to be processed are adjacent in the memory.
  • the access information may include size information, address information (such as a starting address), layout information, etc. of the first matrix and the second matrix.
  • the control logic unit can determine the loop block order of the data to be processed based on the logical operation type of the data to be processed; for example, if the logical operation type of the data to be processed is a matrix multiplication operation of a first matrix and a second matrix, the control logic unit can select a pre-stored loop block order that matches the matrix multiplication operation.
  • the control logic unit can determine the size of the data to be processed and, based on the size of the data to be processed, determine whether to perform cyclic block processing on the data to be processed. If the size of the data to be processed is less than or equal to the preset size, the number of idle multiplication units in the multiplication unit array is sufficient to process the data to be processed.
  • the control logic unit can directly read all the data to be processed from the memory into the multiplication unit array, so that the multiplication unit array can perform multiplication and addition operations on the data to be processed.
  • the control logic unit can determine the cyclic blocking order of the data to be processed according to the logical operation type of the data to be processed; and determine the cyclic blocking data of the data to be processed according to the access information and the preset size of the data to be processed, and then read the cyclic blocking data of the data to be processed from the memory in sequence according to the cyclic blocking order of the data to be processed to the dot multiplication unit array, and realize the multiplication and addition operation of the data to be processed by the dot multiplication unit array on multiple cyclic blocking data.
  • each dot multiplication unit can calculate tile_K multiplications and accumulate them.
  • the dot multiplication unit array can complete the multiplication operation of a matrix of size tile_M ⁇ tile_K with a matrix of size tile_K ⁇ tile_N.
  • the control logic unit is responsible for loop control, and reads the loop block data of the required size from the data to be processed stored in the memory in the order of the loop block to the dot multiplication unit array.
  • control logic unit parses the obtained control instruction and can parse out the access information of the data to be processed, which includes, for example, the size information of the data to be convolved and the convolution kernel, address information, convolution description information (for example, including the stride of the convolution kernel and the amount of padding), and the dimension information of the convolution output result.
  • the access information may include the convolution description information (for example, including the stride of the convolution kernel and the amount of padding), the dimension information of the convolution output result, the size information of the data to be convolved and the convolution kernel, address information, layout information, etc.
  • the control logic unit can determine the circular blocking order of the data to be processed based on the logical operation type of the data to be processed; for example, if the logical operation type of the data to be processed is a convolution operation of the data to be convolved and the convolution kernel, the control logic unit can select a pre-stored circular blocking order that matches the convolution operation.
  • the control logic unit can determine the size of the data to be convolved and the convolution kernel, and based on the size of the data to be convolved and the convolution kernel, determine whether to perform cyclic block processing on the data to be convolved and the convolution kernel. If the size of the data to be convolved and the convolution kernel is less than or equal to the preset size, it means that the number of idle point multiplication units in the point multiplication unit array can meet the requirements for convolution operation of the data to be convolved and the convolution kernel.
  • the control logic unit can directly read the data to be convolved and the convolution kernel from the memory into the point multiplication unit array, so that the point multiplication unit array can perform multiplication and addition operation on them.
  • the size of any data in the data to be convolved or the convolution kernel is larger than the preset size, it means that the number of idle point multiplication units in the point multiplication unit array cannot meet the requirements of convolution operation of the data to be convolved and the convolution kernel.
  • the control logic unit can determine the circular blocking order of the data to be processed according to the logical operation type of the data to be processed; and determine the circular blocking data from the data to be convolved and/or the convolution kernel that is larger than the preset size according to the access information and preset size of the data to be processed, and then read the corresponding circular blocking data from the memory to the point multiplication unit array in sequence according to the circular blocking order of the data to be processed, and realize the convolution operation of the data to be convolved and the convolution kernel through the multiplication and addition operation of multiple circular blocking data by the point multiplication unit array.
  • the size of the cyclic block data is less than or equal to a preset size, and the present disclosure does not impose any specific limitation on the specific size of the cyclic block data.
  • each dot multiplication unit calculates tile_K multiplications and accumulates them.
  • the dot multiplication unit array can process tile_M elements to be convolved and tile_N convolution kernel elements at a time, where the number of channels of tile_M elements to be convolved and tile_N convolution kernel elements is tile_K.
  • the control logic unit is responsible for loop control, and reads the loop block data that meets the size from the data to be processed stored in the memory in the order of loop blocks to the dot multiplication unit array.
  • the access information of the data to be processed includes access information of a first matrix and a second matrix
  • the number of columns of the first matrix is the same as the number of rows of the second matrix
  • the logical operation includes a matrix multiplication operation of the first matrix and the second matrix
  • the loop block order determined by the matrix multiplication operation type includes: a first outer loop order in the direction of the number of rows of the first matrix, a second outer loop order in the direction of the number of columns of the second matrix, and an inner loop order in the direction of the number of columns of the first matrix and the direction of the number of rows of the second matrix.
  • FIG3 illustrates a schematic diagram of a loop block order according to an embodiment of the present disclosure.
  • the control instruction received by the control logic unit may be to perform a matrix multiplication operation on a first matrix A of size M ⁇ K and a second matrix B of size K ⁇ N, where the number of columns K of the first matrix A is the same as the number of rows K of the second matrix B.
  • C represents the matrix multiplication result of the first matrix A and the second matrix B
  • tile_A represents the first loop block data of the first matrix A, whose size is tile_M ⁇ tile_K
  • tile_B represents the second loop block data of the second matrix B, whose size is tile_K ⁇ tile_N
  • tile_C represents the matrix multiplication result of the first loop block data tile_A and the second loop block data tile_B.
  • the loop block order can be a multi-layer loop (for example, including three layers of nested loops), a first outer loop order in the direction of the number of rows of the first matrix A (such as the loop in the M direction in Figure 3), a second outer loop order in the direction of the number of columns of the second matrix B (such as the loop in the N direction in Figure 3), and an inner loop order in the direction of the number of columns of the first matrix A and the number of rows of the second matrix B (such as the loop in the K direction in Figure 3).
  • a multi-layer loop for example, including three layers of nested loops
  • a first outer loop order in the direction of the number of rows of the first matrix A such as the loop in the M direction in Figure 3
  • a second outer loop order in the direction of the number of columns of the second matrix B such as the loop in the N direction in Figure 3
  • an inner loop order in the direction of the number of columns of the first matrix A and the number of rows of the second matrix B (such as the loop in the K direction in
  • step S11 the control logic unit can sequentially read the first circular block data tile_A of the first matrix A and the second circular block data tile_B of the second matrix B from the memory into the dot multiplication unit array in accordance with the circular block order, so that in step S12, the dot multiplication unit array performs a multiplication-accumulation operation on the first circular block data tile_A and the second circular block data tile_B obtained each time, to obtain a circular block result tile_C corresponding to each multiplication-accumulation operation. Furthermore, in step S13, the control logic unit performs a cumulative operation on the multiple circular block results tile_C obtained sequentially from the dot multiplication unit array in any round of the circular block order to obtain a logical operator result; the pseudo code is as follows:
  • the first outer loop (such as the loop in the M direction in Figure 3) can be looped M/tile_M times
  • the second outer loop (such as the loop in the N direction in Figure 3) can be looped N/tile_N times
  • the inner loop (such as the loop in the K direction in Figure 3) can be looped K/tile_K times, so that the first matrix A can be divided into M/tile_M rows and K/tile_K columns
  • the second matrix B can be divided into K/tile_K rows and N/tile_N columns.
  • Each row and column in the first matrix A corresponds to a first loop block data tile_A
  • tile_A mk represents the first loop block data tile_A in the mth row and kth column of the first matrix A
  • each row and column in the second matrix B corresponds to a second loop block data tile_B
  • tile_B kn represents the second loop block data tile_B in the kth row and nth column of the second matrix B.
  • tile_A mk ⁇ tile_B kn represents the matrix multiplication of the first circular tile data tile_A mk in the mth row and kth column of the first matrix A and the second circular tile data tile_B kn in the kth row and nth column of the second matrix B.
  • the loop block sequence may include m ⁇ n rounds of loops.
  • the mth time of any first outer loop (such as the loop in the M direction in Figure 3) and the nth time of the second outer loop (such as the loop in the N direction in Figure 3) correspond to all K/tile_K times of inner loops (such as the loop in the K direction in Figure 3) as a round of loops, referred to as the mth and nth rounds.
  • the control logic unit can accumulate the multiple loop block results tile_A mk ⁇ tile_B kn obtained from the dot product unit array in any round of loop block sequence to obtain the logical operator results of the mth and nth rounds.
  • the loop block order determined by the matrix multiplication operation type may also include: a first outer loop order in the direction of the number of columns of the second matrix, a second outer loop order in the direction of the number of rows of the first matrix, and an inner loop order in the direction of the number of columns of the first matrix and the direction of the number of rows of the second matrix.
  • the loop block order may be a first outer loop order in the direction of the number of columns of the second matrix B (such as the loop in the N direction in Figure 3), a second outer loop order in the direction of the number of rows of the first matrix A (such as the loop in the M direction in Figure 3), and an inner loop order in the direction of the number of columns of the first matrix A and the direction of the number of rows of the second matrix B (such as the loop in the K direction in Figure 3).
  • step S11 the control logic unit can sequentially read the first circular block data tile_A of the first matrix A and the second circular block data tile_B of the second matrix B from the memory into the dot multiplication unit array in accordance with the circular block order, so that in step S12, the dot multiplication unit array performs a multiplication-accumulation operation on the first circular block data tile_A and the second circular block data tile_B obtained each time, to obtain a circular block result tile_C corresponding to each multiplication-accumulation operation. Furthermore, in step S13, the control logic unit performs a cumulative operation on the multiple circular block results tile_C obtained sequentially from the dot multiplication unit array in any round of the circular block order to obtain a logical operator result; the pseudo code is as follows:
  • the previous loop block result can be stored in an accumulator cache (for example, it can be part of a reused cache).
  • the previous loop block result can be read from the accumulator cache and added to the current loop block data, and the addition result can be used to update the current loop block data as the next loop block data.
  • an accumulator cache for storing intermediate data (for example, loop block data) of a memory loop (for example, a loop in the K direction), it is helpful to reduce the number of memory accesses and reduce the pressure on the memory bandwidth. In this way, multiple loop block results are accumulated based on the accumulator cache to obtain a logical operator result.
  • the control logic unit writes the logical operator result into the register file; the control logic unit determines the logical operation result based on the multiple logical operator results obtained from the register file. For example, the control logic unit can write the logical operator result C mn obtained after each round of inner loops, which is K/tile_K times, into the register file. The control logic unit then concatenates the logical operator results C mn obtained from the register file for a total of M/tile_M ⁇ N/tile_N rounds to obtain the logical operation results of the first matrix A and the first matrix B. Providing a register file further reduces the number of accesses to local memory outside the processor.
  • FIG4 illustrates a schematic diagram of repeated reading of cyclic block data according to an embodiment of the present disclosure.
  • first matrix A can be divided into 3 ⁇ 2 blocks, namely: first cyclic block data tile_A 11 , first cyclic block data tile_A 12 , first cyclic block data tile_A 21 , first cyclic block data tile_A 22 , first cyclic block data tile_A 31 , and first cyclic block data tile_A 32 .
  • the second matrix B can be divided into 2 ⁇ 2 blocks, namely: second cyclic block data tile_B 11 , second cyclic block data tile_B 12 , second cyclic block data tile_B 21 , and second cyclic block data tile_B 22 .
  • the first outer loop (for example, the loop along the rows of the first matrix A in FIG4 ) is performed for the first time
  • the second outer loop for example, the loop along the columns of the second matrix B in FIG4
  • one inner loop is performed (for example, the loop along the columns of the first matrix A and the rows of the second matrix B in FIG4 ; one inner loop may include two loops).
  • the results of each inner loop are accumulated.
  • the logical operator result C 11 tile_A 11 ⁇ tile_B 11 + tile_A 12 ⁇ tile_B 21 can be obtained.
  • the results of each inner loop are accumulated.
  • the logical operator result C 32 tile_A 31 ⁇ tile_B 12 + tile_A 32 ⁇ tile_B 22 is obtained.
  • First-loop block data tile_A 31 and first-loop block data tile_A 32 are reused, while second-loop block data tile_B 12 and second-loop block data tile_B 22 are reused.
  • the control logic unit repeatedly reads first-loop block data tile_A 31 , first-loop block data tile_A 32 , second-loop block data tile_B 12 , and second-loop block data tile_B 22 from local memory.
  • the memory includes a local memory arranged outside the processor, and a reuse cache arranged within the processor for caching the loop block data
  • the loop block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the loop block order of the data to be processed, including: the control logic unit stores the first loop block data of the first matrix read from the local memory into the reuse cache according to the first outer loop order and the inner loop order, so that the control logic unit reuses the first loop block data stored in the reuse cache according to the second outer loop order; or, the control logic unit stores the second loop block data of the second matrix read from the local memory into the reuse cache according to the second outer loop order and the inner loop order, so that the control logic unit reuses the second loop block data stored in the reuse cache according to the first outer loop order.
  • the control logic unit can read the first loop block data tile_A 11 of the first matrix A and the second loop block data tile_B 11 of the second matrix B from the local memory to obtain the loop block result tile_A 11 ⁇ tile_B 11 , and can store the first loop block data tile_A 11 of the first matrix A and the second loop block data tile_B 11 of the second matrix B into the reuse cache.
  • the first loop block data tile_A 11 can be read from the reuse cache
  • the second loop block data tile_B 12 can be read from the local memory to obtain the loop block result tile_A 11 ⁇ tile_B 12 .
  • the first loop block data tile_A 21 can be read from the local memory
  • the second loop block data tile_B 11 can be read from the reuse cache to obtain the loop block result tile_A 21 ⁇ tile_B 11 .
  • the first loop block data tile_A 31 can be read from the local memory
  • the second loop block data tile_B 11 can be read from the reuse cache to obtain the loop block result tile_A 31 ⁇ tile_B 11 .
  • the control logic unit can read the first loop block data tile_A 12 of the first matrix A and the second loop block data tile_B 21 of the second matrix B from the local memory to obtain the loop block result tile_A 12 ⁇ tile_B 21 , and can store the first loop block data tile_A 12 of the first matrix A and the second loop block data tile_B 21 of the second matrix B into the reuse cache.
  • the first loop block data tile_A 12 can be read from the reuse cache, and the second loop block data tile_B 22 can be read from the local memory to obtain the loop block result tile_A 12 ⁇ tile_B 22 .
  • the first loop block data tile_A 22 can be read from the local memory
  • the second loop block data tile_B 21 can be read from the reuse cache to obtain the loop block result tile_A 22 ⁇ tile_B 21 .
  • the first loop block data tile_A 32 can be read from the local memory
  • the second loop block data tile_B 21 can be read from the reuse cache to obtain the loop block result tile_A 32 ⁇ tile_B 21 .
  • first outer loop, the second outer loop, and the inner loop can be cycled to other times, which can be referred to above and will not be repeated here.
  • the specific method of reusing the first loop block data and the second loop block data based on the reuse cache in the multi-layer loop process can be set according to the actual application scenario, and the embodiment of the present disclosure does not limit this.
  • the reuse cache will not be set very large.
  • the size of the reuse cache can be set to cache one first loop block data and one second loop block data. Only one first loop block data and one second loop block data can be cached at the same time.
  • the loop block data can be read again from the reuse cache.
  • the control logic unit can respond to each update of the first outer loop sequence and read the first loop block data of the first matrix from the local memory and store it in the reuse cache, so that the control logic unit reuses the first loop block data stored in the reuse cache during the process of traversing the second outer loop sequence.
  • control logic unit can respond to each update of the second outer loop sequence and read the second loop block data of the second matrix from the local memory and store it in the reuse cache, so that the control logic unit reuses the second loop block data stored in the reuse cache during the process of traversing the first outer loop sequence.
  • the access information of the data to be processed includes access information of the data to be convolved and the convolution kernel
  • the logical operation includes the convolution operation of the data to be convolved and the convolution kernel
  • the circular block order determined by the convolution operation type includes the coordinate order of the elements in the convolution kernel
  • the circular block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the circular block order of the data to be processed, including: the control logic unit forms a third circular block data with multiple elements with the same coordinates read from multiple convolution kernels in the memory each time according to the coordinate order of the elements in the convolution kernel; and, according to the coordinates of the current element in each convolution kernel and the access information, reads multiple elements from the data to be convolved in the memory to form a fourth circular block data; the control logic unit writes the third circular block data and the fourth circular block data into the dot product unit array.
  • the memory may include a local memory disposed outside the processor; or, in addition to the local memory disposed outside the processor, the memory may also include a reuse cache disposed within the processor for caching cyclic block data, and the embodiments of the present disclosure do not impose specific restrictions on this.
  • the access information of the data to be convolved and the convolution kernel includes convolution description information, which can be used to characterize the dimensional information of the data to be convolved, the dimensional information of the convolution kernel, the dimensional information of the convolution output result, layout information, etc.
  • the control logic unit can read the convolution description information and perform a mapping operation on the data to be convolved and the convolution kernel according to the convolution description information.
  • Each mapping operation can read the third loop block data and the fourth loop block data once, and then associate the third loop block data and the fourth loop block data with the dot product unit array and perform calculations. In this way, traversing each element of the convolution kernel and accumulating the results can obtain the final convolution output result.
  • control logic unit can expand the matrix operation instructions to implement the convolution function according to the read convolution description information, and convert the convolution operation into matrix multiplication, making the processor more versatile.
  • FIG. 5 shows a schematic diagram of the third cycle block data of the convolution kernel according to an embodiment of the present disclosure.
  • the memory stores a convolution kernel C ⁇ R ⁇ S ⁇ K, where C represents the channel dimension of the convolution kernel, R represents the height dimension of the convolution kernel, S represents the width dimension of the convolution kernel, and K represents the number dimension of the convolution kernel.
  • Each convolution kernel may include R ⁇ S ⁇ C/tile_K convolution kernel elements, each of which occupies one unit of space in the height dimension R and the width dimension S, and may occupy tile_K units of space in the channel dimension C.
  • each third-cycle block data tile_B rsc is tile_K ⁇ tile_N, where tile_N represents the number of convolution kernels and tile_K represents the size of each convolution kernel element in the channel dimension.
  • Figure 6 shows a schematic diagram of the fourth cycle block data of the convolution kernel according to an embodiment of the present disclosure.
  • the memory stores the data to be convolved C ⁇ H ⁇ W (for example, including an input image of size C ⁇ H ⁇ W), where C represents the channel dimension of the data to be convolved, H represents the height dimension of the data to be convolved, and W represents the width dimension of the kernel to be convolved.
  • the data to be convolved may include C ⁇ H ⁇ W/tile_K elements to be convolved, each element to be convolved occupies one unit of space in the height dimension H and the width dimension W respectively, and may occupy tile_K units of space in the channel dimension C.
  • the control logic unit can calculate the coordinates of the data to be convolved according to the coordinate order of the elements in the convolution kernel, according to the coordinates (r, s, c) of the current element in each convolution kernel, and the dimension information of the convolution output result included in the access information, and read tile_M elements from the data to be convolved in the memory according to the coordinates of the data to be convolved, and map them to the fourth loop block data.
  • the number of tile_M elements can be determined by the dimension information of the convolution output result.
  • the control logic unit can write the third circular block data and the fourth circular block data corresponding to the coordinates of each element in the convolution kernel into the dot multiplication unit array, so that the dot multiplication unit array performs matrix multiplication operations on the third circular block data and the fourth circular block data.
  • the matrix multiplication operation please refer to the matrix multiplication operation above, which will not be repeated here.
  • the embodiments of the present disclosure can determine the third-loop block data of the convolution kernel and the fourth-loop block data of the data to be convolved, respectively, according to the coordinate order of the elements in the convolution kernel.
  • This eliminates the need for im2col (for example, sliding the convolution kernel on the data to be convolved, converting the data contained in each convolution kernel window into a column vector, and finally arranging them into a new matrix by column) expansion in the memory, thereby reducing the pressure on the memory bandwidth.
  • this method is conducive to reusing the matrix calculation structure to implement the convolution engine, so that the convolution description information can be used to expand the matrix operation instructions and realize the convolution function.
  • the memory includes a local memory arranged outside the processor, and a reuse cache arranged within the processor for caching circular block data.
  • the method also includes: in the process of calculating the fourth circular block data corresponding to the same row in the convolution output result, the control logic unit writes M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column of the convolution kernel into the reuse cache according to the coordinate order of the elements in the convolution kernel, where M is a positive integer; the control logic unit reads M-1 elements from the reuse cache, reads 1 element from the data to be convolved in the local memory, and determines the fourth circular block data corresponding to the coordinates of the element in the rth row and s+1th column of the convolution kernel.
  • Figure 7 shows a schematic diagram of a reuse cache according to an embodiment of the present disclosure.
  • the input data of the convolution is reused, further reducing the number of accesses to the local memory outside the processor and the bandwidth pressure on the local memory outside the processor.
  • M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column in the convolution kernel are written into the reuse cache; in response to the control logic unit calculating the fourth circular block data of the next row of the current row of the convolution output result, M elements are read from the reuse cache to constitute the fourth circular block data corresponding to the coordinates of the element in the r-1th row and sth column in the convolution kernel.
  • FIG8 is a schematic diagram of another reuse cache according to an embodiment of the present disclosure.
  • the implementation of convolution operation requires the addition of software im2col (for example, sliding the convolution kernel on the data to be convolved, and then converting the data contained in each convolution kernel window into a column vector, and finally arranging them into a new matrix by column) calculation logic, which will introduce additional overhead.
  • software im2col for example, sliding the convolution kernel on the data to be convolved, and then converting the data contained in each convolution kernel window into a column vector, and finally arranging them into a new matrix by column
  • 9 times the amount of data is required, which puts a greater pressure on the storage bandwidth.
  • the embodiment of the present disclosure can determine the third loop block data of the convolution kernel and the fourth loop block data of the data to be convolved according to the coordinate order of the elements in the convolution kernel, and can reuse the matrix calculation structure to implement the convolution engine.
  • control logic unit forms third circular block data with multiple elements having the same coordinates read from the multiple convolution kernels in the memory each time according to the coordinate order of the elements in the convolution kernel, including: the control logic unit reads multiple convolution kernels from the local memory to the reuse cache; and forms third circular block data with multiple elements having the same coordinates read from the multiple convolution kernels in the reuse cache each time according to the coordinate order of the elements in the convolution kernel.
  • the control logic unit can read the three convolution kernels [S1, S2], [S3, S4], and [S5, S6] from the local memory into the reuse cache.
  • the logic control unit reads multiple elements S1, S3, and S5 at coordinate 1 from the reuse cache for the first time according to the coordinate order of the elements in the convolution kernels [S1, S2], [S3, S4], and [S5, S6] to form the third circular block data [S1, S3, S5].
  • the logic control unit reads multiple elements S2, S4, and S6 at coordinate 2 from the reuse cache for the second time to form the third circular block data [S2, S4, S6].
  • the present disclosure only takes the convolution kernels [S1, S2], [S3, S4], and [S5, S6] as examples, and does not limit the size and number of the convolution kernels.
  • the convolution kernel has been read into the reuse cache inside the processor, and the convolution kernel stored in the reuse cache can be reused, avoiding repeated reading of the local memory outside the processor.
  • the dot product unit array performs a multiplication-accumulation operation on each received cyclic block data of the to-be-processed data to determine a cyclic block result for each received cyclic block data, including: the dot product unit array performs a multiplication-accumulation operation on each received third cyclic block data and fourth cyclic block data to determine a cyclic block result.
  • the dot product unit array performs a multiplication-accumulation operation on each received first cyclic block data and second cyclic block data, which will not be repeated here.
  • the subsequent control logic unit can perform accumulation operations on multiple loop block results obtained from the point multiplication unit array in sequence according to the loop block order to obtain the logical operator result and write it into the register file; the control logic unit can determine the convolution output result based on the multiple logical operator results obtained from the register file.
  • the control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the control instruction obtained for indicating the logical operation of the data to be processed, so that the dot multiplication unit array performs multiplication and accumulation operations on the cyclic block data of the data to be processed each time it is received, and determines the cyclic block result of the data to be processed each time it is received.
  • the control logic unit determines the logical operation result of the data to be processed based on the multiple cyclic block results obtained from the dot multiplication unit array.
  • the reading and logical operation of the data to be processed can be converted into the reading and logical operation of multiple cyclic block data of the data to be processed (small-sized image data), which is beneficial for processing larger-sized data while keeping the processor hardware resources unchanged, reducing the pressure on the storage bandwidth.
  • the present disclosure also provides a processor, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any data processing method provided by the present disclosure.
  • a processor an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any data processing method provided by the present disclosure.
  • the corresponding technical solutions and descriptions are referred to the corresponding records in the method section and will not be repeated here.
  • the processor shown in Figure 1 includes a control logic unit and a dot multiplication unit array
  • the dot multiplication unit array includes multiple dot multiplication units for performing multiplication and accumulation operations
  • the processor is used to: the control logic unit reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array in sequence according to the obtained control instructions, and the control instructions are used to indicate the logical operation of the data to be processed;
  • the dot multiplication unit array performs a multiplication and accumulation operation on the cyclic block data of the data to be processed each time it is received, and determines the cyclic block result of the data to be processed each time it is received; the control logic unit determines the logical operation result of the data to be processed based on the multiple cyclic block results obtained from the dot multiplication unit array.
  • the control logic unit sequentially reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the acquired control instruction, including: the control logic unit parses the access information of the data to be processed and the logical operation type of the data to be processed according to the acquired control instruction; determines the cyclic block order of the data to be processed according to the logical operation type of the data to be processed; and reads the cyclic block data of the data to be processed from the memory to the dot multiplication unit array according to the access information and the preset size and the cyclic block order of the data to be processed, wherein the size of the cyclic block data is less than or equal to the preset size, and the preset size is determined by the number of idle dot multiplication units in the dot multiplication unit array.
  • the access information of the data to be processed includes access information of a first matrix and a second matrix
  • the number of columns of the first matrix is the same as the number of rows of the second matrix
  • the logical operation type includes a matrix multiplication operation of the first matrix and the second matrix
  • the loop block order determined by the matrix multiplication operation type includes: a first outer loop order in the direction of the number of rows of the first matrix, a second outer loop order in the direction of the number of columns of the second matrix, and an inner loop order in the direction of the number of columns of the first matrix and the number of rows of the second matrix; or, a first outer loop order in the direction of the number of columns of the second matrix, a second outer loop order in the direction of the number of rows of the first matrix, and an inner loop order in the direction of the number of columns of the first matrix and the number of rows of the second matrix.
  • the memory includes a local memory arranged outside the processor, and a reuse cache arranged within the processor for caching loop block data.
  • the loop block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the loop block order of the data to be processed, including: the control logic unit stores the first loop block data of the first matrix read from the local memory into the reuse cache according to the first outer loop order and the inner loop order, so that the control logic unit reuses the first loop block data stored in the reuse cache according to the second outer loop order; or, the control logic unit stores the second loop block data of the second matrix read from the local memory into the reuse cache according to the second outer loop order and the inner loop order, so that the control logic unit reuses the second loop block data stored in the reuse cache according to the first outer loop order.
  • the dot multiplication unit array performs a multiplication and accumulation operation on the circular block data of the data to be processed each time it is received, and determines the circular block result of the data to be processed each time it is received, including: the dot multiplication unit array performs a multiplication and accumulation operation on the first circular block data and the second circular block data obtained each time, and obtains the circular block result corresponding to each multiplication and accumulation operation; the control logic unit determines the logical operation result of the data to be processed based on the multiple circular block results obtained from the dot multiplication unit array, including: the control logic unit performs an accumulation operation on the multiple circular block results obtained from the dot multiplication unit array in any round of the circular block sequence to obtain a logical operator result; the control logic unit writes the logical operator result into a register stack; the control logic unit determines the logical operation result based on the multiple logical operator results obtained from the register stack.
  • the access information of the data to be processed includes access information of the data to be convolved and the convolution kernel
  • the logical operation type includes the convolution operation of the data to be convolved and the convolution kernel
  • the circular block order determined by the convolution operation type includes the coordinate order of the elements in the convolution kernel
  • the circular block data of the data to be processed are read from the memory in sequence to the dot product unit array according to the circular block order of the data to be processed, including: the control logic unit forms a third circular block data with multiple elements with the same coordinates read from multiple convolution kernels in the memory each time according to the coordinate order of the elements in the convolution kernel, and, according to the coordinates of the current element in each convolution kernel and the access information, reads multiple elements from the data to be convolved in the memory to form a fourth circular block data; the control logic unit writes the third circular block data and the fourth circular block data into the dot product unit array.
  • the memory includes a local memory arranged outside the processor, and a reuse cache arranged within the processor for caching circular block data
  • the processor is further used to: in the process of calculating the fourth circular block data corresponding to the same row in the convolution output result, the control logic unit writes M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column of the convolution kernel into the reuse cache according to the coordinate order of the elements in the convolution kernel, where r, s, and M are positive integers; the control logic unit reads M-1 elements from the reuse cache, reads 1 element from the data to be convolved in the local memory, and determines the fourth circular block data corresponding to the coordinates of the element in the rth row and s+1th column of the convolution kernel.
  • the processor is further used to: in the process of calculating the fourth circular block data corresponding to the adjacent rows in the convolution output result, in response to the control logic unit calculating the fourth circular block data of the current row of the convolution output result, write M elements of the fourth circular block number corresponding to the coordinates of the element in the rth row and sth column in the convolution kernel into the reuse cache, where r, s, and M are positive integers; in response to the control logic unit calculating the fourth circular block data of the next row of the current row of the convolution output result, read M elements from the reuse cache to constitute the fourth circular block data corresponding to the coordinates of the element in the r-1th row and sth column in the convolution kernel.
  • the memory includes a local memory arranged outside the processor, and a reuse cache arranged within the processor for caching cyclic block data
  • the control logic unit according to the coordinate order of the elements in the convolution kernel, converts the multiple elements with the same coordinates read from the multiple convolution kernels in the memory each time into third cyclic block data, including: the control logic unit reads the multiple convolution kernels from the local memory to the reuse cache; according to the coordinate order of the elements in the convolution kernel, the control logic unit converts the multiple elements with the same coordinates read from the multiple convolution kernels in the reuse cache each time into third cyclic block data.
  • the dot product unit array performs a multiplication and accumulation operation on the cyclic block data of the data to be processed each time received to determine the cyclic block result of the data to be processed each time received, including: the dot product unit array performs a multiplication and accumulation operation on the third cyclic block data and the fourth cyclic block data received each time to determine the cyclic block result.
  • the functions or modules included in the processor provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiment. Its specific implementation can refer to the description of the above method embodiment. For the sake of brevity, it will not be repeated here.
  • the present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor.
  • the computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.
  • the embodiments of the present disclosure also provide an artificial intelligence chip, which includes the processor described above.
  • the present disclosure also provides an electronic device including the processor described above.
  • the electronic device may include a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, or the like.
  • UE user equipment
  • PDA personal digital assistant
  • An embodiment of the present disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code.
  • computer-readable code runs in a processor of an electronic device
  • the processor in the electronic device executes the above method.
  • the electronic device may be provided as a terminal, a server, or other forms of devices.
  • FIG9 shows a block diagram of an electronic device 1900 according to an embodiment of the present disclosure.
  • the electronic device 1900 can be provided as a server or a terminal device.
  • the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions that can be executed by the processing component 1922, such as an application.
  • the application stored in the memory 1932 can include one or more modules, each of which corresponds to a set of instructions.
  • the processing component 1922 is configured to execute instructions to perform the above method.
  • the electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958.
  • the electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Microsoft's server operating system (Windows Server TM ), Apple's graphical user interface-based operating system (Mac OS X TM ), a multi-user multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open source Unix-like operating system (FreeBSD TM ), or the like.
  • Microsoft's server operating system Windows Server TM
  • Mac OS X TM Apple's graphical user interface-based operating system
  • Unix TM multi-user multi-process computer operating system
  • Linux TM free and open source Unix-like operating system
  • FreeBSD TM open source Unix-like operating system
  • a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.
  • the present disclosure may be a system, method and/or computer program product.
  • the computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.
  • Computer-readable storage media can be a tangible device that can hold and store the instructions used by the instruction execution device.
  • Computer-readable storage media can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.
  • Non-exhaustive list of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • SRAM static random access memory
  • CD-ROM compact disc read-only memory
  • DVD digital versatile disk
  • memory stick a floppy disk
  • mechanical encoding device for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof.
  • Computer-readable storage media used herein is not interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated by waveguides or other transmission media (for example, light pulses by fiber optic cables), or electrical signals transmitted by wires.
  • the computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing/processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • the network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing/processing device.
  • the computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages.
  • Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet).
  • an electronic circuit such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions.
  • the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.
  • These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions/actions specified in one or more blocks in the flowchart and/or block diagram.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and/or other device to operate in a specific manner.
  • the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions/actions specified in one or more blocks in the flowchart and/or block diagram.
  • Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions/actions specified in one or more blocks in the flowchart and/or block diagram.
  • each box in the block diagram and/or flow chart, and the combination of the boxes in the block diagram and/or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.
  • the computer program product may be implemented in hardware, software, or a combination thereof.
  • the computer program product is implemented as a computer storage medium.
  • the computer program product is implemented as a software product, such as a software development kit (SDK).
  • SDK software development kit
  • the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process.
  • the specific execution order of each step should be determined by its function and possible internal logic.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Computer Hardware Design (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Image Processing (AREA)

Abstract

本公开涉及一种数据处理方法、处理器、芯片及电子设备，所述方法包括：控制逻辑单元根据获取的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列，点乘单元阵列对每次收到的待处理数据的循环分块数据进行乘积累加运算，确定每次收到的待处理数据的循环分块结果，控制逻辑单元根据从点乘单元阵列获取的多个循环分块结果，确定待处理数据的逻辑运算结果。本公开实施例可将对待处理数据的读取与逻辑运算，转换为对待处理数据的多个循环分块数据的读取与逻辑运算，有利于在处理器硬件资源不变的情况下，处理更大尺寸的数据，减少对存储带宽的压力。

Description

数据处理方法、处理器、芯片及电子设备
本申请要求在2024年4月18日提交中国专利局、申请号为202410472589.9、申请名称为“数据处理方法、处理器、芯片及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开涉及计算机技术领域，尤其涉及一种数据处理方法、处理器、芯片及电子设备。
背景技术
在人工智能技术领域，神经网络算法是最近非常流行的一种机器学习算法，在各种领域中都取得了非常好的效果，比如图像识别，语音识别，自然语言处理等。随着神经网络算法的发展，算法的复杂度也越来越高，为了提高识别度，模型的规模也在逐渐增大。用中央处理器(Central Processing Unit，CPU)、图形处理器(Graphic Processing Unit，GPU)等处理器处理起这些大规模的模型，要花费大量的计算时间，并且耗电量很大。
处理器以及人工智能加速器中加入了大量专有模块来加速训练和推理。其中矩阵乘和卷积是人工智能算法中常见的算子。相关技术中的处理器的单指令多数据架构(Single Instruction Multiple Data，SIMD)或者单指令多线程架构(Single Instruction Multiple Thread，SIMT)依靠乘积累加指令来实现相应算法，不同线程之间数据共用较少，需要多次读取重复数据，处理器的性能以及能耗都比较差。
发明内容
本公开提出了一种数据处理技术方案。
根据本公开的一方面，提供了一种数据处理方法，包括：所述数据处理方法应用于处理器，所述处理器包括控制逻辑单元、点乘单元阵列，所述点乘单元阵列包括多个用于执行乘积累加运算的点乘单元，所述方法包括：所述控制逻辑单元根据获取的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列；所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果；所述控制逻辑单元根据从所述点乘单元阵列获取的多个所述循环分块结果，确定所述待处理数据的逻辑运算结果。
在一种可能的实现方式中，所述控制逻辑单元根据获取的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列，包括：所述控制逻辑单元根据获取的控制指令，解析所述待处理数据的访问信息，以及所述待处理数据的逻辑运算类型；根据所述待处理数据的逻辑运算类型，确定所述待处理数据的循环分块顺序；根据所述访问信息和所述预设尺寸，按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，所述循环分块数据的尺寸小于或等于所述预设尺寸，所述预设尺寸由所述点乘单元阵列中处于空闲状态的点乘单元的数量确定。
在一种可能的实现方式中，所述待处理数据的访问信息包括第一矩阵和第二矩阵的访问信息，所述第一矩阵的列数与所述第二矩阵的行数相同，所述逻辑运算类型包括所述第一矩阵和所述第二矩阵的矩阵乘运算，所述矩阵乘运算类型确定的循环分块顺序包括：按照所述第一矩阵的行数方向顺序的第一外层循环顺序、按照所述第二矩阵的列数方向的第二外层循环顺序、按照所述第一矩阵的列数方向和所述第二矩阵的行数方向的内层循环顺序；或者，按照所述第二矩阵的列数方向的第一外层循环顺序、按照所述第一矩阵的行数方向顺序的第二外层循环顺序、按照所述第一矩阵的列数方向和所述第二矩阵的行数方向的内层循环顺序。
在一种可能的实现方式中，所述存储器包括设置在所述处理器外部的局部存储器，以及设置在所述处理器内用于缓存循环分块数据的重用缓存，所述按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，包括：所述控制逻辑单元根据所述第一外层循环顺序和所述内层循环顺序，将从所述局部存储器中读取的所述第一矩阵的第一循环分块数据存入所述重用缓存，以使所述控制逻辑单元根据所述第二外层循环顺序复用所述重用缓存中存储的第一循环分块数据；或者，所述控制逻辑单元根据所述第二外层循环顺序和所述内层循环顺序，将从所述局部存储器中读取的所述第二矩阵的第二循环分块数据存入所述重用缓存，以使所述控制逻辑单元根据所述第一外层循环顺序复用所述重用缓存中存储的第二循环分块数据。
在一种可能的实现方式中，所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果，包括：所述点乘单元阵列对每次获取的所述第一循环分块数据与所述第二循环分块数据进行乘积累加运算，得到每次乘积累加运算对应的循环分块结果；所述控制逻辑单元根据从所述点乘单元阵列获取的多个所述循环分块结果，确定所述待处理数据的逻辑运算结果，包括：所述控制逻辑单元将任一轮所述循环分块顺序中，依次从所述点乘单元阵列获取的多个所述循环分块结果进行累加运算，得到逻辑运算子结果；所述控制逻辑单元将所述逻辑运算子结果写入寄存器堆；所述控制逻辑单元根据从所述寄存器堆获取的多个所述逻辑运算子结果，确定所述逻辑运算结果。
在一种可能的实现方式中，所述待处理数据的访问信息包括待卷积数据和卷积核的访问信息，所述逻辑运算类型包括所述待卷积数据和所述卷积核的卷积运算，所述卷积运算类型确定的循环分块顺序包括所述卷积核中元素的坐标顺序，所述按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，包括：所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将每次从所述存储器中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据，以及，根据每次卷积核中当前元素的坐标和所述访问信息，从所述存储器中所述待卷积数据中读取多个元素，构成第四循环分块数据；所述控制逻辑单元将所述第三循环分块数据和所述第四循环分块数据写入所述点乘单元阵列。
在一种可能的实现方式中，所述存储器包括设置在所述处理器外部的局部存储器，以及设置在所述处理器内用于缓存循环分块数据的重用缓存，所述方法还包括：在计算卷积输出结果中相同行对应的第四循环分块数据的程中，所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将所述卷积核中第r行第s列元素坐标对应的第四循环分块数的M个元素写入重用缓存，r、s、M为正整数；所述控制逻辑单元从所述重用缓存中读取M-1个元素，从所述局部存储器中所述待卷积数据中读取1个元素，确定所述卷积核中第r行第s+1列元素坐标对应的第四循环分块数据。
在一种可能的实现方式中，所述方法还包括：在计算卷积输出结果中相邻行对应的第四循环分块数据过程中，响应于所述控制逻辑单元在计算卷积输出结果当前行的第四循环分块数据，将所述卷积核中第r行第s列元素的坐标对应的第四循环分块数的M个元素写入重用缓存，r、s、M为正整数；响应于所述控制逻辑单元计算卷积输出结果当前行的下一行的第四循环分块数据，从所述重用缓存中读取M个元素，构成所述卷积核中第r-1行第s列元素坐标对应的第四循环分块数据。
在一种可能的实现方式中，所述存储器包括设置在所述处理器外部的局部存储器，以及设置在所述处理器内用于缓存循环分块数据的重用缓存，所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将每次从所述存储器中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据，包括：所述控制逻辑单元从所述局部存储器中读取多个卷积核至所述重用缓存；按照所述卷积核中元素的坐标顺序，所述控制逻辑单元将每次从所述重用缓存中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据。
在一种可能的实现方式中，所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果，包括：所述点乘单元阵列对每次收到的所述第三循环分块数据和所述第四循环分块数据进行乘积累加运算，确定循环分块结果。
根据本公开的一方面，提供了一种处理器，所述处理器包括控制逻辑单元、点乘单元阵列，所述点乘单元阵列包括多个用于执行乘积累加运算的点乘单元，所述处理器用于：所述控制逻辑单元根据获取的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列；所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果；所述控制逻辑单元根据从所述点乘单元阵列获取的多个所述循环分块结果，确定所述待处理数据的逻辑运算结果。
在一种可能的实现方式中，所述控制逻辑单元根据获取的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列，包括：所述控制逻辑单元根据获取的控制指令，解析所述待处理数据的访问信息，以及所述待处理数据的逻辑运算类型；根据所述待处理数据的逻辑运算类型，确定所述待处理数据的循环分块顺序；根据所述访问信息和所述预设尺寸，按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，所述循环分块数据的尺寸小于或等于所述预设尺寸，所述预设尺寸由所述点乘单元阵列中处于空闲状态的点乘单元的数量确定。
在一种可能的实现方式中，所述待处理数据的访问信息包括第一矩阵和第二矩阵的访问信息，所述第一矩阵的列数与所述第二矩阵的行数相同，所述逻辑运算类型包括所述第一矩阵和所述第二矩阵的矩阵乘运算，所述矩阵乘运算类型确定的循环分块顺序包括：按照所述第一矩阵的行数方向顺序的第一外层循环顺序、按照所述第二矩阵的列数方向的第二外层循环顺序、按照所述第一矩阵的列数方向和所述第二矩阵的行数方向的内层循环顺序；或者，按照所述第二矩阵的列数方向的第一外层循环顺序、按照所述第一矩阵的行数方向顺序的第二外层循环顺序、按照所述第一矩阵的列数方向和所述第二矩阵的行数方向的内层循环顺序。
在一种可能的实现方式中，所述存储器包括设置在所述处理器外部的局部存储器，以及设置在所述处理器内用于缓存循环分块数据的重用缓存，所述按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，包括：所述控制逻辑单元根据所述第一外层循环顺序和所述内层循环顺序，将从所述局部存储器中读取的所述第一矩阵的第一循环分块数据存入所述重用缓存，以使所述控制逻辑单元根据所述第二外层循环顺序复用所述重用缓存中存储的第一循环分块数据；或者，所述控制逻辑单元根据所述第二外层循环顺序和所述内层循环顺序，将从所述局部存储器中读取的所述第二矩阵的第二循环分块数据存入所述重用缓存，以使所述控制逻辑单元根据所述第一外层循环顺序复用所述重用缓存中存储的第二循环分块数据。
在一种可能的实现方式中，所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果，包括：所述点乘单元阵列对每次获取的所述第一循环分块数据与所述第二循环分块数据进行乘积累加运算，得到每次乘积累加运算对应的循环分块结果；所述控制逻辑单元根据从所述点乘单元阵列获取的多个所述循环分块结果，确定所述待处理数据的逻辑运算结果，包括：所述控制逻辑单元将任一轮所述循环分块顺序中，依次从所述点乘单元阵列获取的多个所述循环分块结果进行累加运算，得到逻辑运算子结果；所述控制逻辑单元将所述逻辑运算子结果写入寄存器堆；所述控制逻辑单元根据从所述寄存器堆获取的多个所述逻辑运算子结果，确定所述逻辑运算结果。
在一种可能的实现方式中，所述待处理数据的访问信息包括待卷积数据和卷积核的访问信息，所述逻辑运算类型包括所述待卷积数据和所述卷积核的卷积运算，所述卷积运算类型确定的循环分块顺序包括所述卷积核中元素的坐标顺序，所述按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，包括：所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将每次从所述存储器中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据，以及，根据每次卷积核中当前元素的坐标和所述访问信息，从所述存储器中所述待卷积数据中读取多个元素，构成第四循环分块数据；所述控制逻辑单元将所述第三循环分块数据和所述第四循环分块数据写入所述点乘单元阵列。
在一种可能的实现方式中，所述存储器包括设置在所述处理器外部的局部存储器，以及设置在所述处理器内用于缓存循环分块数据的重用缓存，所述处理器还用于：在计算卷积输出结果中相同行对应的第四循环分块数据过程中，所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将所述卷积核中第r行第s列元素坐标对应的第四循环分块数的M个元素写入重用缓存，r、s、M为正整数；所述控制逻辑单元从所述重用缓存中读取M-1个元素，从所述局部存储器中所述待卷积数据中读取1个元素，确定所述卷积核中第r行第s+1列元素坐标对应的第四循环分块数据。
在一种可能的实现方式中，所述处理器还用于：在计算卷积输出结果中相邻行对应的第四循环分块数据过程中，响应于所述控制逻辑单元在计算卷积输出结果当前行的第四循环分块数据，将所述卷积核中第r行第s列元素的坐标对应的第四循环分块数的M个元素写入重用缓存，r、s、M为正整数；响应于所述控制逻辑单元计算卷积输出结果当前行的下一行的第四循环分块数据，从所述重用缓存中读取M个元素，构成所述卷积核中第r-1行第s列元素坐标对应的第四循环分块数据。
在一种可能的实现方式中，所述存储器包括设置在所述处理器外部的局部存储器，以及设置在所述处理器内用于缓存循环分块数据的重用缓存，所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将每次从所述存储器中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据，包括：所述控制逻辑单元从所述局部存储器中读取多个卷积核至所述重用缓存；按照所述卷积核中元素的坐标顺序，所述控制逻辑单元将每次从所述重用缓存中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据。
在一种可能的实现方式中，所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果，包括：所述点乘单元阵列对每次收到的所述第三循环分块数据和所述第四循环分块数据进行乘积累加运算，确定循环分块结果。
根据本公开的一方面，提供了一种人工智能芯片，所述人工智能芯片包括如上所述的处理器。
根据本公开的一方面，提供了一种电子设备，所述电子设备包括如上所述的人工智能芯片。
在本公开实施例中，控制逻辑单元根据获取的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列，以使点乘单元阵列对每次收到的待处理数据的循环分块数据进行乘积累加运算，确定每次收到的待处理数据的循环分块结果，控制逻辑单元根据从点乘单元阵列获取的多个循环分块结果，确定待处理数据的逻辑运算结果。通过这种方式，可以将对待处理数据(例如大尺寸的图像数据)的读取与逻辑运算，转换为对待处理数据的多个循环分块数据(小尺寸的图像数据)的读取与逻辑运算，有利于在处理器硬件资源不变的情况下，处理更大尺寸的数据，减少对存储带宽的压力。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本公开。根据下面参考附图对示例性实施例的详细说明，本公开的其它特征及方面将变得清楚。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。
图1示出根据本公开实施例的处理器的示意图。
图2示出根据本公开实施例的数据处理方法的流程图。
图3示出根据本公开实施例的循环分块顺序的示意图。
图4示出根据本公开实施例的循环分块数据出现重复读取情况的示意图。
图5示出根据本公开实施例的卷积核的第三循环分块数据的示意图。
图6示出根据本公开实施例的卷积核的第四循环分块数据的示意图。
图7示出根据本公开实施例的一种重用缓存的示意图。
图8示出根据本公开实施例的另一种重用缓存的示意图。
图9示出根据本公开实施例的一种电子设备的框图。
具体实施方式
以下将参考附图详细说明本公开的各种示例性实施例、特征和方面。附图中相同的附图标记表示功能相同或相似的元件。尽管在附图中示出了实施例的各种方面，但是除非特别指出，不必按比例绘制附图。
在这里专用的词“示例性”意为“用作例子、实施例或说明性”。这里作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。
本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。
另外，为了更好地说明本公开，在下文的具体实施方式中给出了众多的具体细节。本领域技术人员应当理解，没有某些具体细节，本公开同样可以实施。在一些实例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本公开的主旨。
图1示出根据本公开实施例的处理器的示意图。如图1所示，所述处理器包括控制逻辑单元、点乘单元阵列，所述点乘单元阵列包括多个用于执行乘积累加运算的点乘单元。
点乘单元中还可以包括运算器以完成指定的运算，例如计算多个乘法并进行累加。其中，点乘单元中可以包括乘法器、加法器等，各个点乘单元中的具体结构可以相同，也可以存在不同，本公开对此不作限定。点乘单元中还可以包括其他类型的运算器，以适应各种不同的运算过程，本公开对点乘单元包括的运算器的数量和类型不作限定。
控制逻辑单元可以连接点乘单元阵列中的每个点乘单元，在一些实施方式中，控制逻辑单元可以以二维矩阵或多维矩阵的形式对各点乘单元进行编号，以使多个点乘单元在逻辑上可以以二维矩阵或多维矩阵的形式排列，从而更好的适应矩阵的逻辑运算。
如图1所示，所述处理器对应的存储器可以包括设置在所述处理器外部的局部存储器，处理器中的控制逻辑单元可以连接局部存储器，控制逻辑单元可用于地址计算，以从存储器中加载数据到点乘单元阵列、并控制点乘单元阵列对待处理数据进行处理。
在一种可能的实现方式中，局部存储器可以为片上缓存，控制逻辑单元可以将片外闪存上的可执行程序以及待处理数据(例如，输入矩阵)加载到上述局部存储器(片上缓存)中，再进行之后的待处理数据的逻辑运算的过程。
在一种可能的实现方式中，局部存储器中可以存储有待处理数据和可执行程序，可执行程序中可以包括控制指令，处理器执行控制指令可以实现待处理数据的逻辑运算，例如包括待处理数据的矩阵乘运算、卷积运算等与乘累加相关的运算。
为了减少重复数据的读取，降低处理器功耗，增大处理器的通讯带宽，存储器还可以包括设置在处理器内部的重用缓存(Reuse Buffer)，可以将重用缓存设置在点乘单元阵列内部(图1未示出)，也可以将重用缓存设置在点乘单元阵列外部(见图1)，其中，对比将重用缓存设置在点乘单元阵列外部，将重用缓存设置在点乘单元阵列内部，点乘单元阵列可具有更高的数据读取效率。在控制逻辑单元中设置有加载器、译码器等，其中，加载器可以用于将局部存储器中的待处理数据或部分待处理器加载到处理器内的重用缓存，译码器可以根据加载后待处理数据的存储地址的变化，对可执行程序中访问数据的控制指令进行译码，比如说，对于访问局部存储器中数据X的控制指令，由于在重用缓存中缓存了该数据X，可通过译码获得该数据X在重用缓存中存储的地址，译码器可以将访问局部存储器中数据X的控制指令，转化为访问重用缓存中数据X的控制指令，有利于后续控制逻辑单元直接将重用缓存中缓存的数据发送给点乘单元，由点乘单元对其执行乘积累加运算。其中，控制逻辑单元也可以直接从片外内存上加载数据至重用缓存，本公开对此不作限定。
在一种可能的实现方式中，为了便于处理器处理大规模数据，如图1所示，还可以为处理器设置对应的寄存器堆(Register File)，用于在不和存储器结合的情况下，读取，存储，处理和传输数据。这样，在处理器处理大量数据时，能够大大减少指令执行时间。其中，寄存器堆包括诸如存储器地址寄存器，存储器数据寄存器，指令寄存器，操作码字寄存器，累加器，标志位寄存器等，本公开对此不作限制。在示例中，控制逻辑单元可用于地址计算，以便在点乘单元阵列与寄存器堆之间进行数据的搬运。
在一种可能的实现方式中，本公开实施例的处理器可以是全新设计的，也可以是对已有处理器芯片进行改进后得到的，处理器芯片的类型可包括但不限于：中央处理器(Central Processing Unit，CPU)、图形处理器(Graphic Processing Unit，GPU)、通用图形处理器(General-Purpose Computing on Graphics Processing Units，GPGPU)、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、张量处理器(Tensor Processing Unit，TPU)、现场可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件，还可以包括微处理器或其他常规处理器的处理器。
图2示出根据本公开实施例的数据处理方法的流程图。如图2所示，该数据处理方法应用于处理器，该数据处理方法可以包括以下步骤：
在步骤S11中，所述控制逻辑单元根据获取的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列，所述控制指令用于指示所述待处理数据的访问信息，以及所述待处理数据的逻辑运算类型，例如包括待处理数据的矩阵乘运算、卷积运算等与乘累加相关的运算。
在一种可能的实现方式中，控制逻辑单元可以根据控制指令，获取待处理数据的地址信息，并根据待处理数据的地址信息，依次确定待处理数据的各循环分块数据的地址信息，并从存储器中读取待处理数据的循环分块数据至点乘单元阵列。
可选的，所述存储器可以包括设置在所述处理器外部的局部存储器，控制逻辑单元可根据获取的控制指令，依次从局部存储器中读取待处理数据的循环分块数据至点乘单元阵列。
可选的，由于待处理数据的循环分块数据会存在被重复读取的场景，所述存储器除了包括设置在所述处理器外部的局部存储器，还可以设置在所述处理器内用于缓存循环分块数据的重用缓存。其中，控制逻辑单元从重用缓存中读取数据的速度快于从局部存储器中读取数据的速度。这样，对于待处理数据中需要重复读取的循环分块数据，响应于控制逻辑单元将从局部存储器中获取到的需要重复读取的循环分块数据传输至点乘单元阵列，控制逻辑单元还可以同步将该需要重复读取的循环分块数据存入设置在处理器内的重用缓存，以便后续控制逻辑单元可以从重用缓存中读取循环分块数据至点乘单元阵列。
在一种可能的实现方式中，所述待处理数据包括深度学习任务中的特征数据，所述特征数据包括图像特征数据、语音特征数据、文本特征数据中的至少一种。
例如，在使用深度神经网络对目标对象进行人脸识别的场景下，待处理数据可以是图像特征数据，可以在存储器中存储目标对象的图像特征数据(例如人脸特征图)；在使用深度神经网络对目标对象进行语音识别的场景下，待处理数据可以是语音特征数据，可以在存储器中存储目标对象的语音特征数据；在使用深度神经网络对目标文档进行文字识别的场景下，待处理数据可以是文本特征数据，可以在存储器中存储目标文档的文本特征数据；本申请的实施例对待处理数据的类型不作限制。
在一种可能的实现方式中，控制逻辑单元可以通过控制指令，获取待处理数据的数据结构信息和待处理数据的地址信息；其中，待处理数据的数据结构信息例如包括待处理数据的维度、待处理数据的尺寸、待处理数据中元素的数据类型(例如整数类型、单精度浮点数类型、双精度浮点数类型、字符型等)等用于描述待处理数据的信息；待处理数据的地址信息例如包括待处理数据在存储器中的基地址(Base Address)、寻址空间等与地址相关的信息。
在步骤S12中，所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果。
示例性地，点乘单元阵列可以包括tile_M×tile_N个点乘单元，每个点乘单元可以计算tile_K个乘法并进行累加，其中，tile_M、tile_N、tile_K为正整数，本公开对tile_M、tile_N、tile_K的具体取值不作限制，可根据实际的应用场景进行设置。
其中，点乘单元阵列通过对每次收到的待处理数据的循环分块数据进行乘积累加运算，可以实现矩阵乘运算或卷积运算。对于矩阵乘运算，点乘单元阵列可以完成尺寸为tile_M×tile_K的矩阵，与尺寸为tile_K×tile_N的矩阵的乘法操作。对于卷积运算，点乘单元阵列一次可以处理tile_M个待卷积元素和tile_N个卷积核元素，其中，tile_M个待卷积元素和tile_N个卷积核元素的通道数为tile_K。
在步骤S13中，所述控制逻辑单元根据从所述点乘单元阵列获取的多个所述循环分块结果，确定所述待处理数据的逻辑运算结果。
示例性地，控制逻辑单元可以将获取的多个循环分块结果进行拼接处理和/或相加处理，确定待处理数据的逻辑运算结果。其中，控制逻辑单元可以每收到一次循环分块数据，就与上一次收到的循环分块数据进行拼接处理和/或相加处理；控制逻辑单元也可以收到全部的循环分块数据，再对全部的循环分块数据进行拼接处理和/或相加处理，本公开对此不作限制。
在本公开的实施例的数据处理方法中，控制逻辑单元根据获取的用于指示待处理数据逻辑运算的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列，以使点乘单元阵列对每次收到的待处理数据的循环分块数据进行乘积累加运算，确定每次收到的待处理数据的循环分块结果，控制逻辑单元根据从点乘单元阵列获取的多个循环分块结果，确定待处理数据的逻辑运算结果。通过这种方式，可以将对待处理数据(例如大尺寸的图像数据)的读取与逻辑运算，转换为对待处理数据的多个循环分块数据(小尺寸的图像数据)的读取与逻辑运算，有利于在处理器硬件资源不变的情况下，处理更大尺寸的数据，减少对存储带宽的压力。
下面对本公开实施例的数据处理方法进行示例性说明。
在步骤S11中，所述控制逻辑单元根据获取的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列。
其中，所述存储器可以包括设置在所述处理器外部的局部存储器；或者，所述存储器除了包括设置在所述处理器外部的局部存储器，还可以设置在所述处理器内用于缓存循环分块数据的重用缓存，本公开的实施例对此不作具体限制。
在一种可能的实现方式中，步骤S11可包括：所述控制逻辑单元根据获取的控制指令，解析所述待处理数据的访问信息，以及所述待处理数据的逻辑运算类型；根据所述待处理数据的逻辑运算类型，确定所述待处理数据的循环分块顺序；根据所述访问信息和所述预设尺寸，按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，所述循环分块数据的尺寸小于或等于所述预设尺寸，所述预设尺寸由所述点乘单元阵列中处于空闲状态的点乘单元的数量确定。
示例性地，假设控制指令是用于指示待处理数据的矩阵乘运算，控制逻辑单元对获取的控制指令进行解析，可以解析出待处理数据的访问信息，以及所述待处理数据的逻辑运算类型。在该访问信息例如包括待处理数据的尺寸信息、地址信息、布局信息。其中，布局信息可以是行主序(RowMajor)或列主序(ColMajor)，行主序表示待处理数据的同一行的元素在存储器中是相邻的，列主序表示待处理数据的同一列的元素在存储器中是相邻的。例如，如果待处理数据的逻辑运算类型是第一矩阵和第二矩阵的矩阵乘运算，访问信息可包括第一矩阵和第二矩阵的尺寸信息、地址信息(例如起始地址)，布局信息等。
控制逻辑单元可根据待处理数据的逻辑运算类型，确定待处理数据的循环分块顺序；例如，如果待处理数据的逻辑运算类型是第一矩阵和第二矩阵的矩阵乘运算，控制逻辑单元可以选择预先存储的与矩阵乘运算匹配的循环分块顺序。
控制逻辑单元根据解析出的待处理数据的访问信息，可以确定待处理数据的尺寸，可根据待处理数据的尺寸，判断是否需要对待处理数据执行循环分块处理。如果待处理数据的尺寸小于或等于预设尺寸，说明点乘单元阵列中处于空闲状态的点乘单元的数量可以满足处理待处理数据的需求，控制逻辑单元可以直接从存储器中读取全部的待处理数据至点乘单元阵列，以使点乘单元阵列对待处理数据执行乘加运算。
如果待处理数据的尺寸大于预设尺寸，说明点乘单元阵列中处于空闲状态的点乘单元的数量无法满足处理待处理数据的需求，控制逻辑单元可根据待处理数据的逻辑运算类型，确定待处理数据的循环分块顺序；以及根据待处理数据的访问信息和预设尺寸，确定待处理数据的循环分块数据，再按照待处理数据的循环分块顺序，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列，通过点乘单元阵列对多个循环分块数据的乘加运算，实现对待处理数据的乘加运算。
例如，点乘单元阵列中存在tile_M×tile_N个处于空闲状态的点乘单元，每个点乘单元可以计算tile_K个乘法并进行累加。对于矩阵乘运算，点乘单元阵列可以完成尺寸为tile_M×tile_K的矩阵，与尺寸为tile_K×tile_N的矩阵的乘法操作。对于大于tile_M，tile_N和tile_K尺寸的矩阵，控制逻辑单元负责循环控制，按照循环分块顺序，依次从存储器存储的待处理数据中读取符合尺寸的循环分块数据至点乘单元阵列。
示例性地，假设控制指令是用于指示待处理数据的卷积运算，控制逻辑单元对获取的控制指令进行解析，可以解析出待处理数据的访问信息，该访问信息例如包括待卷积数据和卷积核的尺寸信息、地址信息、卷积描述信息(例如包括卷积核的步幅stride，填充数量padding)、卷积输出结果的维度信息。例如，如果待处理数据的逻辑运算类型是待卷积数据和卷积核的卷积运算，访问信息可包括卷积描述信息(例如包括卷积核的步幅stride，填充数量padding)、卷积输出结果的维度信息、待卷积数据和卷积核的尺寸信息、地址信息、布局信息等。
控制逻辑单元可根据待处理数据的逻辑运算类型，确定待处理数据的循环分块顺序；例如，如果待处理数据的逻辑运算类型是待卷积数据和卷积核的卷积运算，控制逻辑单元可以选择预先存储的与卷积运算匹配的循环分块顺序。
控制逻辑单元根据解析出的待处理数据的访问信息，可以确定待卷积数据和卷积核的尺寸，可根据待卷积数据和卷积核的尺寸，判断是否需要对待卷积数据和卷积核执行循环分块处理。如果待卷积数据和卷积核的尺寸均小于或等于预设尺寸，说明点乘单元阵列中处于空闲状态的点乘单元的数量可以满足待卷积数据与卷积核进行卷积运算的需求，控制逻辑单元可以直接从存储器中读取待卷积数据和卷积核至点乘单元阵列，以使点乘单元阵列对其执行乘加运算。
如果待卷积数据或卷积核中任一数据的尺寸大于预设尺寸，说明点乘单元阵列中处于空闲状态的点乘单元的数量无法满足待卷积数据与卷积核进行卷积运算的需求，控制逻辑单元可根据待处理数据的逻辑运算类型，确定待处理数据的循环分块顺序；以及根据待处理数据的访问信息和预设尺寸，从大于预设尺寸的待卷积数据和/或卷积核中确定循环分块数据，再按照待处理数据的循环分块顺序，依次从存储器中读取对应的循环分块数据至点乘单元阵列，通过点乘单元阵列对多个循环分块数据的乘加运算，实现对待卷积数据和卷积核的卷积运算。
其中，循环分块数据的尺寸小于或等于预设尺寸，本公开对循环分块数据的具体尺寸不作具体限制。
例如，点乘单元阵列中存在tile_M×tile_N个处于空闲状态的点乘单元，每个点乘单元以计算tile_K个乘法并进行累加。对于卷积运算，点乘单元阵列一次可以处理tile_M个待卷积元素和tile_N个卷积核元素，其中，tile_M个待卷积元素和tile_N个卷积核元素的通道数为tile_K。对于大于tile_M，tile_N和tile_K尺寸的待处理数据，控制逻辑单元负责循环控制，按照循环分块顺序，依次从存储器存储的待处理数据中读取符合尺寸的循环分块数据至点乘单元阵列。
通过这种方式，有利于将大尺寸的待处理数据，转换为满足点乘单元阵列计算结构需要尺寸的循环分块数据，提高了处理器的计算性能。
在一种可能的实现方式中，所述待处理数据的访问信息包括第一矩阵和第二矩阵的访问信息，所述第一矩阵的列数与所述第二矩阵的行数相同，所述逻辑运算包括所述第一矩阵和所述第二矩阵的矩阵乘运算，所述矩阵乘运算类型确定的循环分块顺序包括：按照所述第一矩阵的行数方向顺序的第一外层循环顺序、按照所述第二矩阵的列数方向的第二外层循环顺序、按照所述第一矩阵的列数方向和所述第二矩阵的行数方向的内层循环顺序。
由于处理器的硬件资源有限，处理器访问的数据量过大时，控制逻辑单元无法一次性将所需要的数据加载到点乘单元阵列，通过设置循环分块顺序，有利于在处理器硬件资源不变的情况下，处理更大尺寸的数据，减少对存储带宽的压力。
示例性地，图3示出根据本公开实施例的循环分块顺序的示意图。如图3所示，控制逻辑单元接收到的控制指令，可以是对尺寸为M×K的第一矩阵A，和尺寸为K×N的第二矩阵B执行矩阵乘运算，第一矩阵A的列数K与第二矩阵B的行数K相同。
其中，C代表第一矩阵A与第二矩阵B的矩阵乘运算结果，tile_A代表第一矩阵A的第一循环分块数据，其尺寸为tile_M×tile_K，tile_B代表第二矩阵B的第二循环分块数据，其尺寸为tile_K×tile_N，tile_C代表第一循环分块数据tile_A与第二循环分块数据tile_B的矩阵乘运算结果。
循环分块顺序可以是多层循环(例如包括三层循环嵌套)，按照第一矩阵A的行数方向顺序的第一外层循环顺序(如图3中M方向的循环)、按照第二矩阵B的列数方向的第二外层循环顺序(如图3中N方向的循环)、按照第一矩阵A的列数方向和第二矩阵B的行数方向的内层循环顺序(如图3中K方向的循环)。
这样，在步骤S11中，控制逻辑单元可以按照循环分块顺序，依次从存储器中读取第一矩阵A的第一循环分块数据tile_A、第二矩阵B的第二循环分块数据tile_B至点乘单元阵列，以便在步骤S12中，所述点乘单元阵列对每次获取的所述第一循环分块数据tile_A与所述第二循环分块数据tile_B进行乘积累加运算，得到每次乘积累加运算对应的循环分块结果tile_C。并在步骤S13中，所述控制逻辑单元将任一轮所述循环分块顺序中，依次从所述点乘单元阵列获取的多个所述循环分块结果tile_C进行累加运算，得到逻辑运算子结果；伪代码如下：
其中，第一外层循环(如图3中M方向的循环)可循环M/tile_M次，第二外层循环(如图3中N方向的循环)可循环N/tile_N次，内层循环(如图3中K方向的循环)可循环K/tile_K次，从而可以将第一矩阵A划分为M/tile_M行K/tile_K列，将第二矩阵B划分为K/tile_K行N/tile_N列，第一矩阵A中的每行每列对应一个第一循环分块数据tile_A，tile_Amk代表第一矩阵A中第m行第k列的第一循环分块数据tile_A，第二矩阵B中的每行每列对应一个第二循环分块数据tile_B，tile_Bkn代表第二矩阵B中第k行第n列的第二循环分块数据tile_B。
tile_Amk×tile_Bkn代表第一矩阵A中第m行第k列的第一循环分块数据tile_Amk，与第二矩阵B中第k行第n列的第二循环分块数据tile_Bkn的矩阵乘运算，点乘单元阵列对第一循环分块数据tile_Amk与第二循环分块数据tile_Bkn进行乘积累加运算，得到第m、n、k次乘积累加运算对应的循环分块结果tile_C＝tile_Amk×tile_Bkn。
循环分块顺序可包括m×n轮循环，任意第一外层循环(如图3中M方向的循环)的第m次，与第二外层循环(如图3中N方向的循环)的第n次，所对应的全部K/tile_K次的内层循环(如图3中K方向的循环)为一轮循环，简称第m、n轮，控制逻辑单元可以将任一轮循环分块顺序中，依次从点乘单元阵列获取的多个循环分块结果tile_Amk×tile_Bkn进行累加运算，得到第m、n轮的逻辑运算子结果
可选的，矩阵乘运算类型确定的循环分块顺序也可以包括：按照所述第二矩阵的列数方向的第一外层循环顺序、按照所述第一矩阵的行数方向顺序的第二外层循环顺序、按照所述第一矩阵的列数方向和所述第二矩阵的行数方向的内层循环顺序。例如，循环分块顺序可以按照第二矩阵B的列数方向的第一外层循环顺序(如图3中N方向的循环)、第一矩阵A的行数方向顺序的第二外层循环顺序(如图3中M方向的循环)、按照第一矩阵A的列数方向和第二矩阵B的行数方向的内层循环顺序(如图3中K方向的循环)。
这样，在步骤S11中，控制逻辑单元可以按照循环分块顺序，依次从存储器中读取第一矩阵A的第一循环分块数据tile_A、第二矩阵B的第二循环分块数据tile_B至点乘单元阵列，以便在步骤S12中，所述点乘单元阵列对每次获取的所述第一循环分块数据tile_A与所述第二循环分块数据tile_B进行乘积累加运算，得到每次乘积累加运算对应的循环分块结果tile_C。并在步骤S13中，所述控制逻辑单元将任一轮所述循环分块顺序中，依次从所述点乘单元阵列获取的多个所述循环分块结果tile_C进行累加运算，得到逻辑运算子结果；伪代码如下：
具体过程可以参考上文，此处不再赘叙。
通过这种方式，用户可以通过几条简单的循环代码实现循环分块顺序，减少了软件用户实现的复杂度。
在一种可能的实现方式中，可以将上一次的循环分块结果存入累加器缓存(例如可以是重用缓存的一部分)，当从所述点乘单元阵列获取到当前次的循环分块数据，可以从累加器缓存中读取上一次的循环分块结果与当前次的循环分块数据相加，并用相加结果更新当前次的循环分块数据，作为下一次的循环分块数据。通过设置用于存放内存循环(例如K方向的某次循环)的中间数据(例如循环分块数据)的累加器缓存，有利于减少对存储器的访问次数，降低对存储带宽的压力。这样，基于累加器缓存对多个循环分块结果进行累加运算，得到逻辑运算子结果。然后，内层循环结束(例如K方向循环结束)，所述控制逻辑单元将所述逻辑运算子结果写入寄存器堆；所述控制逻辑单元根据从所述寄存器堆获取的多个所述逻辑运算子结果，确定所述逻辑运算结果。例如，控制逻辑单元可以将每轮经过全部K/tile_K次的内层循环后，得到的逻辑运算子结果Cmn写入寄存器堆；控制逻辑单元根据从寄存器堆获取的共计M/tile_M×N/tile_N轮的逻辑运算子结果Cmn进行拼接，得到第一矩阵A和第一矩阵B的逻辑运算结果。通过设置寄存器堆，有利于进一步降低对设置在处理器外的局部存储器的访问次数。
可选的，如果所述存储器为设置在处理器外部的局部存储器，控制逻辑单元按照循环分块顺序，依次从局部存储器中读取待处理数据的循环分块数据至点乘单元阵列的过程中，会出现对循环分块数据重复读取的情况，图4示出根据本公开实施例的循环分块数据出现重复读取情况的示意图。如图4所示，假设第一矩阵A可分为3×2块，即：第一循环分块数据tile_A11、第一循环分块数据tile_A12、第一循环分块数据tile_A21、第一循环分块数据tile_A22、第一循环分块数据tile_A31、第一循环分块数据tile_A32。
假设第二矩阵B可分为2×2块，即：第二循环分块数据tile_B11、第二循环分块数据tile_B12、第二循环分块数据tile_B21、第二循环分块数据tile_B22。
按照循环分块顺序，在第一外层循环(例如图4中第一矩阵A行数方向的循环)到第1次，第二外层循环(例如图4中第二矩阵B列数方向的循环)到第1次，经过一轮内层循环顺序(例如图4中第一矩阵A列数方向与第二矩阵B的行数方向的循环，一轮内层循环可包括2次循环)，将每次内层循环的结果进行累加，经过2次内循环后，可以得到逻辑运算子结果C11＝tile_A11×tile_B11+tile_A12×tile_B21。
在第一外层循环到第1次，第二外层循环到第2次，经过一轮内层循环顺序，将每次内层循环的结果进行累加，经过2次内循环后，可以得到逻辑运算子结果C12＝tile_A11×tile_B12+tile_A12×tile_B22。其中，第一循环分块数据tile_A11和第一循环分块数据tile_A12会被重用，控制逻辑单元会从局部存储器中重复读取第一循环分块数据tile_A11和第一循环分块数据tile_A12。
在第一外层循环到第2次，第二外层循环到第1次，经过一轮内层循环顺序，将每次内层循环的结果进行累加，经过2次内循环后，可以得到逻辑运算子结果C21＝tile_A21×tile_B11+tile_A22×tile_B21。其中，第二循环分块数据tile_B11和第二循环分块数据tile_B21会被重用，控制逻辑单元会从局部存储器中重复读取第二循环分块数据tile_B11和第二循环分块数据tile_B21。
在第一外层循环到第2次，第二外层循环到第2次，经过一轮内层循环顺序，将每次内层循环的结果进行累加，经过2次内循环后，可以得到逻辑运算子结果C22＝tile_A21×tile_B12+tile_A22×tile_B22。其中，第一循环分块数据tile_A21和第一循环分块数据tile_A22会被重用，第二循环分块数据tile_B12和第二循环分块数据tile_B22会被重用，控制逻辑单元会从局部存储器中重复读取第一循环分块数据tile_A21、第一循环分块数据tile_A22、第二循环分块数据tile_B12、第二循环分块数据tile_B22。
在第一外层循环到第3次，第二外层循环到第1次，经过一轮内层循环顺序，将每次内层循环的结果进行累加，经过2次内循环后，可以得到逻辑运算子结果C31＝tile_A31×tile_B11+tile_A32×tile_B21。其中，第二循环分块数据tile_B11和第二循环分块数据tile_B21会被重用，控制逻辑单元会从局部存储器中重复读取第二循环分块数据tile_B11和第二循环分块数据tile_B21。
在第一外层循环到第3次，第二外层循环到第2次，经过一轮内层循环顺序，将每次内层循环的结果进行累加，经过2次内循环后，可以得到逻辑运算子结果C32＝tile_A31×tile_B12+tile_A32×tile_B22。其中，第一循环分块数据tile_A31和第一循环分块数据tile_A32会被重用，第二循环分块数据tile_B12和第二循环分块数据tile_B22会被重用，控制逻辑单元会从局部存储器中重复读取第一循环分块数据tile_A31、第一循环分块数据tile_A32、第二循环分块数据tile_B12、第二循环分块数据tile_B22。
可选的，为了进一步节省处理器的访存带宽，减少对处理器外局部存储器中相同循环分块数据的重复访问，提高处理器的计算性能，在一种可能的实现方式中，所述存储器包括设置在所述处理器外部的局部存储器，以及设置在所述处理器内用于缓存循环分块数据的重用缓存，所述按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，包括：所述控制逻辑单元根据所述第一外层循环顺序和所述内层循环顺序，将从所述局部存储器中读取的所述第一矩阵的第一循环分块数据存入所述重用缓存，以使所述控制逻辑单元根据所述第二外层循环顺序复用所述重用缓存中存储的第一循环分块数据；或者，所述控制逻辑单元根据所述第二外层循环顺序和所述内层循环顺序，将从所述局部存储器中读取的所述第二矩阵的第二循环分块数据存入所述重用缓存，以使所述控制逻辑单元根据所述第一外层循环顺序复用所述重用缓存中存储的第二循环分块数据。
示例性地，如图4所示，按照循环分块顺序，在第一外层循环(例如图4中第一矩阵A行数方向的循环)到第1次，第二外层循环(例如图4中第二矩阵B列数方向的循环)到第1次，内层循环(例如图4中第一矩阵A列数方向与第二矩阵B的行数方向的循环，一轮内层循环可包括2次循环)到第1次(例如M＝1，N＝1，K＝1)，控制逻辑单元可以从局部存储器中读取第一矩阵A的第一循环分块数据tile_A11、第二矩阵B的第二循环分块数据tile_B11，得到循环分块结果tile_A11×tile_B11，可以将第一矩阵A的第一循环分块数据tile_A11和第二矩阵B的第二循环分块数据tile_B11存入重用缓存。
这样，在第一外层循环到第1次，第二外层循环到第2次，内层循环到第1次(例如M＝1，N＝2，K＝1)，可以从重用缓存中读取第一循环分块数据tile_A11，从局部存储器中读取第二循环分块数据tile_B12，得到循环分块结果tile_A11×tile_B12。
在第一外层循环到第2次，第二外层循环到第1次，内层循环到第1次(例如M＝2，N＝1，K＝1)，可以从局部存储器中读取第一循环分块数据tile_A21，从重用缓存中读取第二循环分块数据tile_B11，得到循环分块结果tile_A21×tile_B11。
在第一外层循环到第3次，第二外层循环到第1次，内层循环到第1次(例如M＝3，N＝1，K＝1)，可以从局部存储器中读取第一循环分块数据tile_A31，从重用缓存中读取第二循环分块数据tile_B11，得到循环分块结果tile_A31×tile_B11。
类似的，在第一外层循环到第1次，第二外层循环到第1次，内层循环到第2次(例如M＝1，N＝1，K＝2)，控制逻辑单元可以从局部存储器中读取第一矩阵A的第一循环分块数据tile_A12、第二矩阵B的第二循环分块数据tile_B21，得到循环分块结果tile_A12×tile_B21，可以将第一矩阵A的第一循环分块数据tile_A12和第二矩阵B的第二循环分块数据tile_B21存入所述重用缓存。
这样，在第一外层循环到第1次，第二外层循环到第2次，内层循环到第2次(例如M＝1，N＝2，K＝2)，可以从重用缓存中读取第一循环分块数据tile_A12，从局部存储器中读取第二循环分块数据tile_B22，得到循环分块结果tile_A12×tile_B22。
在第一外层循环到第2次，第二外层循环到第2次，内层循环到第2次(例如M＝2，N＝2，K＝2)，可以从局部存储器中读取第一循环分块数据tile_A22，从重用缓存中读取第二循环分块数据tile_B21，得到循环分块结果tile_A22×tile_B21。
在第一外层循环到第3次，第二外层循环到第2次，内层循环到第2次(例如M＝3，N＝2，K＝2)，可以从局部存储器中读取第一循环分块数据tile_A32，从重用缓存中读取第二循环分块数据tile_B21，得到循环分块结果tile_A32×tile_B21。
应当理解，第一外层循环、第二外层循环、内层循环分别循环到其他次数，可以参考上文，此处不再赘叙。具体如何在多层循环过程中基于重用缓存复用第一循环分块数据和第二循环分块数据，可以根据实际的应用场景进行设置，本公开实施例对此不作限制。
例如，在硬件资源比较紧张的场景下，为了减小对硬件资源的消耗，重用缓存不会设置的很大，例如可以将重用缓存的大小，设置为缓存一个第一循环分块数据和一个第二循环分块数据，同一时刻只能缓存一个第一循环分块数据和第二循环分块数据，在每次切换第一外层循环或第二外层循环的次序时，可以重新从重用缓存中去读取循环分块数据。例如，控制逻辑单元可以响应于每次更新第一外层循环顺序，从局部存储器中读取第一矩阵的第一循环分块数据存入重用缓存，以使控制逻辑单元在遍历第二外层循环顺序的过程中，复用重用缓存中存储的第一循环分块数据。或者，控制逻辑单元可以响应于每次更新第二外层循环顺序，从局部存储器中读取第二矩阵的第二循环分块数据存入重用缓存，以使控制逻辑单元在遍历第一外层循环顺序的过程中，复用重用缓存中存储的第二循环分块数据。
通过在处理器内部设置重用缓存，有利于减少对处理器外的局部存储器的访问次数，降低对存储带宽的压力。
在一种可能的实现方式中，所述待处理数据的访问信息包括待卷积数据和卷积核的访问信息，所述逻辑运算包括所述待卷积数据和所述卷积核的卷积运算，所述卷积运算类型确定的循环分块顺序包括所述卷积核中元素的坐标顺序，所述按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，包括：所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将每次从所述存储器中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据；以及，根据每次卷积核中当前元素的坐标和所述访问信息，从所述存储器中所述待卷积数据中读取多个元素，构成第四循环分块数据；所述控制逻辑单元将所述第三循环分块数据和所述第四循环分块数据写入所述点乘单元阵列。
其中，所述存储器可以包括设置在所述处理器外部的局部存储器；或者，所述存储器除了包括设置在所述处理器外部的局部存储器，还可以设置在所述处理器内用于缓存循环分块数据的重用缓存，本公开的实施例对此不作具体限制。
示例性地，待卷积数据和卷积核的访问信息包括卷积描述信息，该卷积描述信息可用于表征待卷积数据的维度信息、卷积核维度信息、卷积输出结果的维度信息、布局信息等。控制逻辑单元可读取卷积描述信息，并根据卷积描述信息对待卷积数据和卷积核进行映射操作，每次映射操作可读取一次第三循环分块数据和第四循环分块数据，然后将第三循环分块数据和第四循环分块数据与点乘单元阵列关联，并进行计算。按照这种方式，遍历卷积核的每个元素，将结果进行累加可以得到最终的卷积输出结果。
通过这种方式，控制逻辑单元可根据读取的卷积描述信息，扩展矩阵运算指令实现卷积功能，将卷积运算转换成矩阵乘法，使处理器具有更强的通用性。
图5示出根据本公开实施例的卷积核的第三循环分块数据的示意图。如图5所示，存储器中存储了卷积核C×R×S×K，C代表卷积核的通道维度，R代表卷积核的高度维度，S代表卷积核的宽度维度，K代表卷积核的数量维度。其中，每个卷积核可包括R×S×C/tile_K个卷积核元素，每个卷积核元素在高度维度R和宽度维度S分别占用一个单位的空间，在通道维度C可占用tile_K个单位的空间。
控制逻辑单元可按照卷积核中元素的坐标顺序，例如从卷积核元素(r＝0,s＝0,c＝0)至(r＝R-1,s＝S-1,c＝C/tile_K-1)的坐标顺序，第一次从存储器中读取每个卷积核在通道维度为0～tile_K的第一行第一列的元素，将8个(K＝0～7)卷积核中具有相同坐标(r＝0,s＝0,c＝0)的8个(K＝0～7)元素，映射为第一个第三循环分块数据tile_B000；第二次从存储器中读取每个卷积核在通道维度为0～tile_K的第一行第二列的元素，将8个(K＝0～7)卷积核中具有相同坐标(r＝0,s＝1,c＝0)的8个(K＝0～7)元素，映射为第二个第三循环分块数据tile_B010；以此类推，最后一次从存储器中读取每个卷积核在通道维度为(C-tile_K)～C的最后一行最后一列的元素，将从存储器中8个(K＝0～7)卷积核中读取的具有相同坐标(r＝R-1,s＝S-1,c＝C/tile_K-1)的8个(K＝0～7)元素，映射为最后一个第三循环分块数据tile_B(R-1)(S-1)(C/tile_K-1)。
如图5所示，每个第三循环分块数据tile_Brsc的尺寸为tile_K×tile_N，tile_N代表卷积核的数量，tile_K代表每个卷积核元素在通道维度的尺寸。
与此同时，图6示出根据本公开实施例的卷积核的第四循环分块数据的示意图。如图6所示，存储器中存储了待卷积数据C×H×W(例如包括尺寸为C×H×W的输入图像)，C代表待卷积数据的通道维度，H代表待卷积数据的高度维度，W代表待卷积核的宽度维度。其中，待卷积数据可包括C×H×W/tile_K待卷积元素，每个待卷积元素在高度维度H和宽度维度W分别占用一个单位的空间，在通道维度C可占用tile_K个单位的空间。
由于卷积输出结果的维度是由待卷积数据和卷积核在待卷积数据中滑动的次数确定的，基于此，控制逻辑单元可按照卷积核中元素的坐标顺序，根据每次卷积核中当前元素的坐标(r,s,c)，和访问信息中包括的卷积输出结果的维度信息，计算待卷积数据的坐标，并根据待卷积数据的坐标，从存储器中待卷积数据中读取tile_M个元素，映射为第四循环分块数据。其中，tile_M个元素的数量可以由卷积输出结果的维度信息确定。
例如，卷积核中当前元素的坐标为(r＝0,s＝0,c＝0)，卷积输出结果中第一行第一列元素对应待卷积数据中第一行第一列元素，其坐标为(h＝0，w＝0，c＝0)，可以将待卷积数据中坐标为(h＝0，w＝0，c＝0)的元素作为第四循环分块数据的tile_M个元素中的第一个元素。以此类推，直至从存储器中待卷积数据中读取tile_M个元素，映射为第四循环分块数据。
控制逻辑单元可以将卷积核中每个元素的坐标对应的第三循环分块数据和第四循环分块数据写入点乘单元阵列，以使点乘单元阵列对第三循环分块数据和第四循环分块数据执行矩阵乘运算，具体可参见上文的矩阵乘运算，此处不再赘述。
本公开的实施例，可以按照卷积核中元素的坐标顺序，分别确定卷积核的第三循环分块数据，和待卷积数据的第四循环分块数据，无需im2col(例如，在待卷积数据上滑动卷积核，再将每个卷积核窗口内含有的数据转为列向量，最后按列排成新的矩阵)在存储器的展开，降低了对存储带宽的压力。并且，通过这种方式，有利于复用矩阵计算结构实现卷积引擎，以便利用卷积描述信息来扩展矩阵运算指令，实现卷积功能。
在一种可能的实现方式中，所述存储器包括设置在所述处理器外部的局部存储器，以及设置在所述处理器内用于缓存循环分块数据的重用缓存，所述方法还包括：在计算卷积输出结果中相同行对应的第四循环分块数据过程中，所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将所述卷积核中第r行第s列元素坐标对应的第四循环分块数的M个元素写入重用缓存，M为正整数；所述控制逻辑单元从所述重用缓存中读取M-1个元素，从所述局部存储器中所述待卷积数据中读取1个元素，确定所述卷积核中第r行第s+1列元素坐标对应的第四循环分块数据。
图7示出根据本公开实施例的一种重用缓存的示意图。如图7所示，假设卷积核的步长为1，控制逻辑单元可以将卷积核中第r＝1行第s＝0列元素的坐标对应次的第四循环分块数据[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]写入重用缓存。
对于卷积核中第r＝1行第s＝1列元素的坐标对应次的第四循环分块数据，控制逻辑单元只需要从局部存储器中待卷积数据中多读取一个元素，其他15个元素可以复用重用缓存中的[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]，构成第四循环分块数据[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]；其中，控制逻辑单元将多读取的元素16写入重用缓存，使重用缓存继续存储[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]。
对于卷积核中第r＝1行第s＝2列元素的坐标对应次的第四循环分块数据，控制逻辑单元只需要从局部存储器中待卷积数据中多读取一个元素，其他15个元素可以复用重用缓存中的[2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]，构成第四循环分块数据[2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]；其中，控制逻辑单元将多读取的元素17写入重用缓存，得到[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]。
通过在处理器内部设置重用缓存，对卷积实现输入数据的复用，进一步减少对处理器外局部存储器的访问次数和对处理器外局部存储器的带宽压力。
在一种可能的实现方式中，在计算卷积输出结果中相邻行对应的第四循环分块数据过程中，响应于所述控制逻辑单元计算卷积输出结果当前行的第四循环分块数据，将所述卷积核中第r行第s列元素的坐标对应的第四循环分块数的M个元素写入重用缓存；响应于所述控制逻辑单元计算卷积输出结果当前行的下一行的第四循环分块数据，从所述重用缓存中读取M个元素，构成所述卷积核中第r-1行第s列元素坐标对应的第四循环分块数据。
图8示出根据本公开实施例的另一种重用缓存的示意图。如图8所示，假设卷积核的步长为1，重用缓存中存储了数据[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]，其中，[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]是在计算卷积输出结果的第一行时(见图8中卷积结果的第一行oh_0)，根据待卷积数据的第二行(见图8中ih_1)，由卷积核中第r＝1行第s＝0列元素的坐标确定的第四循环分块数据；[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]是在计算卷积输出结果的第一行时(见图8中卷积结果的第一行oh_0)，根据待卷积数据的第二行(见图8中ih_1)，由卷积核中第r＝1行第s＝1列元素的坐标确定的第四循环分块数据，[2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]是在计算卷积输出结果的第一行时(见图8中卷积结果的第一行oh_0)，根据待卷积数据的第二行(见图8中ih_1)，由卷积核中第r＝1行第s＝2列元素的坐标确定的第四循环分块数据。
应当理解，由卷积核中第r＝1行第s＝1列元素的坐标确定的第四循环分块数据[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]，可以利用重用缓存，复用由卷积核中第r＝1行第s＝0列元素的坐标确定的第四循环分块数据[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]；类似的，由卷积核中第r＝1行第s＝2列元素的坐标确定的第四循环分块数据[2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]，可以利用重用缓存，复用由卷积核中第r＝1行第s＝1列元素的坐标确定的第四循环分块数据[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]，具体可参见与图7相关的描述，此处不再赘叙。
考虑到在卷积输出结果中相邻的两行会有数据复用，例如，在计算卷积输出结果的第二行时(参见图8中卷积结果的第二行oh_1)，卷积核中r＝0行元素对应的待卷积元素(见图8中待卷积数据的第二行ih_1)，与计算卷积输出结果中前一行(参见图8中卷积结果的第一行oh_0)的卷积核r＝1所对应的待卷积元素一致(同样是图8中待卷积数据的第二行ih_1)，可以进行复用，减少读取局部存储器的数据量。
例如，在计算卷积输出结果的第二行时(参见图8中卷积结果的第二行oh_1)，卷积核中第r＝0行第s＝0列元素对应的待卷积元素(见图8中待卷积数据的第二行ih_1)，与计算卷积输出结果中前一行(参见图8中卷积结果的第一行oh_0)的卷积核第r＝1行第s＝0列所对应的待卷积元素一致(同样是图8中待卷积数据的第二行ih_1)。可以利用重用缓存，复用计算卷积输出结果中第一行(参见图8中卷积结果的第一行oh_0)时，由卷积核中第r＝1行第s＝0列确定的第四循环分块数据[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]，作为计算卷积输出结果的第二行时(参见图8中卷积结果的第二行oh_1)，由卷积核中第r＝0行第s＝0列元素确定的第四循环分块数据[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]。
类似的，可以利用重用缓存，复用计算卷积输出结果中第一行(参见图8中卷积结果的第一行oh_0)时，由卷积核中第r＝1行第s＝1列确定的第四循环分块数据[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]，作为计算卷积输出结果的第二行时(参见图8中卷积结果的第二行oh_1)，由卷积核中第r＝0行第s＝1列元素确定的第四循环分块数据[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]。以及，复用计算卷积输出结果中第一行(参见图8中卷积结果的第一行oh_0)时，由卷积核中第r＝1行第s＝2列确定的第四循环分块数据[2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]，作为计算卷积输出结果的第二行时(参见图8中卷积结果的第二行oh_1)，由卷积核中第r＝0行第s＝2列元素确定的第四循环分块数据[2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]。
对比相关技术中实现卷积运算需要加入软件im2col(例如在待卷积数据上滑动卷积核，再将每个卷积核窗口内含有的数据转为列向量，最后按列排成新的矩阵)的计算逻辑，会引入额外的开销，例如，对于常见的3x3大小的卷积核需要9倍的数据量，对存储带宽有较大的压力。本公开实施例可以按照卷积核中元素的坐标顺序，分别确定卷积核的第三循环分块数据，和待卷积数据的第四循环分块数据，可以复用矩阵计算结构实现卷积引擎，无需im2col在存储器的展开，降低了对存储带宽的压力。并通过在处理器内设置重用缓存，对卷积实现输入数据的复用，进一步减少对处理器外局部存储器的访问次数和对处理器外局部存储器的带宽压力。
在一种可能的实现方式中，所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将每次从所述存储器中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据，包括：所述控制逻辑单元从所述局部存储器中读取多个卷积核至所述重用缓存；按照所述卷积核中元素的坐标顺序，所述控制逻辑单元将每次从所述重用缓存中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据。
示例性的，假设局部存储器中存储有卷积核[S1,S2]、[S3,S4]、[S5,S6]，控制逻辑单元可以从局部存储器中，将这三个卷积核[S1,S2]、[S3,S4]、[S5,S6]读取至重用缓存；这样，逻辑控制单元按照卷积核[S1,S2]、[S3,S4]、[S5,S6]中元素的坐标顺序，第一次从重用缓存中读取坐标1处的多个元素S1,S3,S5，构成第三循环分块数据[S1,S3,S5]；第二次从重用缓存中读取坐标2处的多个元素S2,S4,S6，构成第三循环分块数据[S2,S4,S6]。应当理解，本公开仅以卷积核[S1,S2]、[S3,S4]、[S5,S6]作为示例，对卷积核的尺寸和数量不作限制。
通过这种方式，卷积核已经读取到处理器内部的重用缓存，可以对重用缓存中存储的卷积核复用，避免了重复读取处理器外部的局部存储器。
在一种可能的实现方式中，所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果，包括：所述点乘单元阵列对每次收到的所述第三循环分块数据和所述第四循环分块数据进行乘积累加运算，确定循环分块结果。具体可参见上文点乘单元阵列对每次获取的第一循环分块数据与第二循环分块数据进行乘积累加运算，此处不再赘述。
后续控制逻辑单元可以按照循环分块顺序，对依次从点乘单元阵列获取的多个循环分块结果进行累加运算，得到逻辑运算子结果，并将其写入寄存器堆；控制逻辑单元可以根据从寄存器堆获取的多个所述逻辑运算子结果，确定卷积输出结果。
通过这种方式，有利于复用矩阵计算结构实现卷积引擎，有利于后续减少了软件用户实现的复杂度。
在本公开的实施例的数据处理方法中，控制逻辑单元根据获取的用于指示待处理数据逻辑运算的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列，以使点乘单元阵列对每次收到的待处理数据的循环分块数据进行乘积累加运算，确定每次收到的待处理数据的循环分块结果，控制逻辑单元根据从点乘单元阵列获取的多个循环分块结果，确定待处理数据的逻辑运算结果。通过这种方式，可以将对待处理数据(例如大尺寸的图像数据)的读取与逻辑运算，转换为对待处理数据的多个循环分块数据(小尺寸的图像数据)的读取与逻辑运算，有利于在处理器硬件资源不变的情况下，处理更大尺寸的数据，减少对存储带宽的压力。
可以理解，本公开提及的上述各个方法实施例，在不违背原理逻辑的情况下，均可以彼此相互结合形成结合后的实施例，限于篇幅，本公开不再赘述。本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。
此外，本公开还提供了处理器、电子设备、计算机可读存储介质、程序，上述均可用来实现本公开提供的任一种数据处理方法，相应技术方案和描述和参见方法部分的相应记载，不再赘述。
在一种可能的实现方式中，如图1所示的处理器，所述处理器包括控制逻辑单元、点乘单元阵列，所述点乘单元阵列包括多个用于执行乘积累加运算的点乘单元，所述处理器用于：所述控制逻辑单元根据获取的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列，所述控制指令用于指示所述待处理数据的逻辑运算；所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果；所述控制逻辑单元根据从所述点乘单元阵列获取的多个所述循环分块结果，确定所述待处理数据的逻辑运算结果。
在一种可能的实现方式中，所述控制逻辑单元根据获取的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列，包括：所述控制逻辑单元根据获取的控制指令，解析所述待处理数据的访问信息，以及所述待处理数据的逻辑运算类型；根据所述待处理数据的逻辑运算类型，确定所述待处理数据的循环分块顺序；根据所述访问信息和所述预设尺寸，按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，所述循环分块数据的尺寸小于或等于所述预设尺寸，所述预设尺寸由所述点乘单元阵列中处于空闲状态的点乘单元的数量确定。
在一种可能的实现方式中，所述待处理数据的访问信息包括第一矩阵和第二矩阵的访问信息，所述第一矩阵的列数与所述第二矩阵的行数相同，所述逻辑运算类型包括所述第一矩阵和所述第二矩阵的矩阵乘运算，所述矩阵乘运算类型确定的循环分块顺序包括：按照所述第一矩阵的行数方向顺序的第一外层循环顺序、按照所述第二矩阵的列数方向的第二外层循环顺序、按照所述第一矩阵的列数方向和所述第二矩阵的行数方向的内层循环顺序；或者，按照所述第二矩阵的列数方向的第一外层循环顺序、按照所述第一矩阵的行数方向顺序的第二外层循环顺序、按照所述第一矩阵的列数方向和所述第二矩阵的行数方向的内层循环顺序。
在一种可能的实现方式中，所述存储器包括设置在所述处理器外部的局部存储器，以及设置在所述处理器内用于缓存循环分块数据的重用缓存，所述按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，包括：所述控制逻辑单元根据所述第一外层循环顺序和所述内层循环顺序，将从所述局部存储器中读取的所述第一矩阵的第一循环分块数据存入所述重用缓存，以使所述控制逻辑单元根据所述第二外层循环顺序复用所述重用缓存中存储的第一循环分块数据；或者，所述控制逻辑单元根据所述第二外层循环顺序和所述内层循环顺序，将从所述局部存储器中读取的所述第二矩阵的第二循环分块数据存入所述重用缓存，以使所述控制逻辑单元根据所述第一外层循环顺序复用所述重用缓存中存储的第二循环分块数据。
在一种可能的实现方式中，所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果，包括：所述点乘单元阵列对每次获取的所述第一循环分块数据与所述第二循环分块数据进行乘积累加运算，得到每次乘积累加运算对应的循环分块结果；所述控制逻辑单元根据从所述点乘单元阵列获取的多个所述循环分块结果，确定所述待处理数据的逻辑运算结果，包括：所述控制逻辑单元将任一轮所述循环分块顺序中，依次从所述点乘单元阵列获取的多个所述循环分块结果进行累加运算，得到逻辑运算子结果；所述控制逻辑单元将所述逻辑运算子结果写入寄存器堆；所述控制逻辑单元根据从所述寄存器堆获取的多个所述逻辑运算子结果，确定所述逻辑运算结果。
在一种可能的实现方式中，所述待处理数据的访问信息包括待卷积数据和卷积核的访问信息，所述逻辑运算类型包括所述待卷积数据和所述卷积核的卷积运算，所述卷积运算类型确定的循环分块顺序包括所述卷积核中元素的坐标顺序，所述按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，包括：所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将每次从所述存储器中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据，以及，根据每次卷积核中当前元素的坐标和所述访问信息，从所述存储器中所述待卷积数据中读取多个元素，构成第四循环分块数据；所述控制逻辑单元将所述第三循环分块数据和所述第四循环分块数据写入所述点乘单元阵列。
在一种可能的实现方式中，所述存储器包括设置在所述处理器外部的局部存储器，以及设置在所述处理器内用于缓存循环分块数据的重用缓存，所述处理器还用于：在计算卷积输出结果中相同行对应的第四循环分块数据过程中，所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将所述卷积核中第r行第s列元素坐标对应的第四循环分块数的M个元素写入重用缓存，r、s、M为正整数；所述控制逻辑单元从所述重用缓存中读取M-1个元素，从所述局部存储器中所述待卷积数据中读取1个元素，确定所述卷积核中第r行第s+1列元素坐标对应的第四循环分块数据。
在一种可能的实现方式中，所述处理器还用于：在计算卷积输出结果中相邻行对应的第四循环分块数据过程中，响应于所述控制逻辑单元计算卷积输出结果当前行的第四循环分块数据，将所述卷积核中第r行第s列元素的坐标对应的第四循环分块数的M个元素写入重用缓存，r、s、M为正整数；响应于所述控制逻辑单元计算卷积输出结果当前行的下一行的第四循环分块数据，从所述重用缓存中读取M个元素，构成所述卷积核中第r-1行第s列元素坐标对应的第四循环分块数据。
在一种可能的实现方式中，所述存储器包括设置在所述处理器外部的局部存储器，以及设置在所述处理器内用于缓存循环分块数据的重用缓存，所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将每次从所述存储器中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据，包括：所述控制逻辑单元从所述局部存储器中读取多个卷积核至所述重用缓存；按照所述卷积核中元素的坐标顺序，所述控制逻辑单元将每次从所述重用缓存中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据。
在一种可能的实现方式中，所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果，包括：所述点乘单元阵列对每次收到的所述第三循环分块数据和所述第四循环分块数据进行乘积累加运算，确定循环分块结果。
在一些实施例中，本公开实施例提供的处理器具有的功能或包含的模块可以用于执行上文方法实施例描述的方法，其具体实现可以参照上文方法实施例的描述，为了简洁，这里不再赘述。
本公开实施例还提出一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述方法。计算机可读存储介质可以是易失性或非易失性计算机可读存储介质。
本公开实施例还提出一种人工智能芯片，所述芯片包括如上所述的处理器。
本公开实施例还提出一种电子设备，所述电子设备包括如上所述的处理器。其中，所述电子设备可以包括用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字助理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等。
本公开实施例还提供了一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当所述计算机可读代码在电子设备的处理器中运行时，所述电子设备中的处理器执行上述方法。
电子设备可以被提供为终端、服务器或其它形态的设备。
图9示出根据本公开实施例的一种电子设备1900的框图。例如，电子设备1900可以被提供为一服务器或终端设备。参照图9，电子设备1900包括处理组件1922，其进一步包括一个或多个处理器，以及由存储器1932所代表的存储器资源，用于存储可由处理组件1922的执行的指令，例如应用程序。存储器1932中存储的应用程序可以包括一个或一个以上的每一个对应于一组指令的模块。此外，处理组件1922被配置为执行指令，以执行上述方法。
电子设备1900还可以包括一个电源组件1926被配置为执行电子设备1900的电源管理，一个有线或无线网络接口1950被配置为将电子设备1900连接到网络，和一个输入输出(I/O)接口1958。电子设备1900可以操作基于存储在存储器1932的操作系统，例如微软服务器操作系统(Windows ServerTM)，苹果公司推出的基于图形用户界面操作系统(Mac OS XTM)，多用户多进程的计算机操作系统(UnixTM),自由和开放原代码的类Unix操作系统(LinuxTM)，开放原代码的类Unix操作系统(FreeBSDTM)或类似。
在示例性实施例中，还提供了一种非易失性计算机可读存储介质，例如包括计算机程序指令的存储器1932，上述计算机程序指令可由电子设备1900的处理组件1922执行以完成上述方法。
本公开可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本公开的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是(但不限于)电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本公开操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本公开的各个方面。
这里参照根据本公开实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本公开的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本公开的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
该计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一个可选实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
上文对各个实施例的描述倾向于强调各个实施例之间的不同之处，其相同或相似之处可以互相参考，为了简洁，本文不再赘述。
本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。
以上已经描述了本公开的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。

Claims (13)

  1. 一种数据处理方法，其特征在于，所述数据处理方法应用于处理器，所述处理器包括控制逻辑单元、点乘单元阵列，所述点乘单元阵列包括多个用于执行乘积累加运算的点乘单元，所述方法包括：
    所述控制逻辑单元根据获取的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列；
    所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果；
    所述控制逻辑单元根据从所述点乘单元阵列获取的多个所述循环分块结果，确定所述待处理数据的逻辑运算结果。
  2. 根据权利要求1所述的方法，其特征在于，所述控制逻辑单元根据获取的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列，包括：
    所述控制逻辑单元根据获取的控制指令，解析所述待处理数据的访问信息，以及所述待处理数据的逻辑运算类型；
    根据所述待处理数据的逻辑运算类型，确定所述待处理数据的循环分块顺序；
    根据所述访问信息和预设尺寸，按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，所述循环分块数据的尺寸小于或等于所述预设尺寸，所述预设尺寸由所述点乘单元阵列中处于空闲状态的点乘单元的数量确定。
  3. 根据权利要求2所述的方法，其特征在于，所述待处理数据的访问信息包括第一矩阵和第二矩阵的访问信息，所述第一矩阵的列数与所述第二矩阵的行数相同，所述逻辑运算类型包括所述第一矩阵和所述第二矩阵的矩阵乘运算，
    所述矩阵乘运算类型确定的循环分块顺序包括：按照所述第一矩阵的行数方向顺序的第一外层循环顺序、按照所述第二矩阵的列数方向的第二外层循环顺序、按照所述第一矩阵的列数方向和所述第二矩阵的行数方向的内层循环顺序；
    或者，按照所述第二矩阵的列数方向的第一外层循环顺序、按照所述第一矩阵的行数方向顺序的第二外层循环顺序、按照所述第一矩阵的列数方向和所述第二矩阵的行数方向的内层循环顺序。
  4. 根据权利要求3所述的方法，其特征在于，所述存储器包括设置在所述处理器外部的局部存储器，以及设置在所述处理器内用于缓存循环分块数据的重用缓存，所述按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，包括：
    所述控制逻辑单元根据所述第一外层循环顺序和所述内层循环顺序，将从所述局部存储器中读取的所述第一矩阵的第一循环分块数据存入所述重用缓存，以使所述控制逻辑单元根据所述第二外层循环顺序复用所述重用缓存中存储的第一循环分块数据；
    或者，
    所述控制逻辑单元根据所述第二外层循环顺序和所述内层循环顺序，将从所述局部存储器中读取的所述第二矩阵的第二循环分块数据存入所述重用缓存，以使所述控制逻辑单元根据所述第一外层循环顺序复用所述重用缓存中存储的第二循环分块数据。
  5. 根据权利要求4所述的方法，其特征在于，所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果，包括：
    所述点乘单元阵列对每次获取的所述第一循环分块数据与所述第二循环分块数据进行乘积累加运算，得到每次乘积累加运算对应的循环分块结果；
    所述控制逻辑单元根据从所述点乘单元阵列获取的多个所述循环分块结果，确定所述待处理数据的逻辑运算结果，包括：
    所述控制逻辑单元将任一轮所述循环分块顺序中，依次从所述点乘单元阵列获取的多个所述循环分块结果进行累加运算，得到逻辑运算子结果；
    所述控制逻辑单元将所述逻辑运算子结果写入寄存器堆；
    所述控制逻辑单元根据从所述寄存器堆获取的多个所述逻辑运算子结果，确定所述逻辑运算结果。
  6. 根据权利要求2所述的方法，其特征在于，所述待处理数据的访问信息包括待卷积数据和卷积核的访问信息，所述逻辑运算类型包括所述待卷积数据和所述卷积核的卷积运算，
    所述卷积运算类型确定的循环分块顺序包括所述卷积核中元素的坐标顺序，所述按照所述待处理数据的循环分块顺序，依次从存储器中读取所述待处理数据的循环分块数据至点乘单元阵列，包括：
    所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将每次从所述存储器中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据，以及，根据每次卷积核中当前元素的坐标和所述访问信息，从所述存储器中所述待卷积数据中读取多个元素，构成第四循环分块数据；
    所述控制逻辑单元将所述第三循环分块数据和所述第四循环分块数据写入所述点乘单元阵列。
  7. 根据权利要求6所述的方法，其特征在于，所述存储器包括设置在所述处理器外部的局部存储器，以及设置在所述处理器内用于缓存循环分块数据的重用缓存，所述方法还包括：
    在计算卷积输出结果中相同行对应的第四循环分块数据过程中，所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将所述卷积核中第r行第s列元素坐标对应的第四循环分块数的M个元素写入重用缓存，r、s、M为正整数；
    所述控制逻辑单元从所述重用缓存中读取M-1个元素，从所述局部存储器中所述待卷积数据中读取1个元素，确定所述卷积核中第r行第s+1列元素坐标对应的第四循环分块数据。
  8. 根据权利要求6所述的方法，其特征在于，所述方法还包括：在计算卷积输出结果中相邻行对应的第四循环分块数据过程中，响应于所述控制逻辑单元计算卷积输出结果当前行的第四循环分块数据，将所述卷积核中第r行第s列元素的坐标对应的第四循环分块数的M个元素写入重用缓存，r、s、M为正整数；
    响应于所述控制逻辑单元计算卷积输出结果当前行的下一行的第四循环分块数据，从所述重用缓存中读取M个元素，构成所述卷积核中第r-1行第s列元素坐标对应的第四循环分块数据。
  9. 根据权利要求6所述的方法，其特征在于，所述存储器包括设置在所述处理器外部的局部存储器，以及设置在所述处理器内用于缓存循环分块数据的重用缓存，所述控制逻辑单元按照所述卷积核中元素的坐标顺序，将每次从所述存储器中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据，包括：
    所述控制逻辑单元从所述局部存储器中读取多个卷积核至所述重用缓存；
    按照所述卷积核中元素的坐标顺序，所述控制逻辑单元将每次从所述重用缓存中多个所述卷积核中读取的具有相同坐标的多个元素，构成第三循环分块数据。
  10. 根据权利要求6-9中任一项所述的方法，其特征在于，所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果，包括：
    所述点乘单元阵列对每次收到的所述第三循环分块数据和所述第四循环分块数据进行乘积累加运算，确定循环分块结果。
  11. 一种处理器，其特征在于，所述处理器包括控制逻辑单元、点乘单元阵列，所述点乘单元阵列包括多个用于执行乘积累加运算的点乘单元，
    所述处理器用于：
    所述控制逻辑单元根据获取的控制指令，依次从存储器中读取待处理数据的循环分块数据至点乘单元阵列；
    所述点乘单元阵列对每次收到的所述待处理数据的循环分块数据进行乘积累加运算，确定每次收到的所述待处理数据的循环分块结果；
    所述控制逻辑单元根据从所述点乘单元阵列获取的多个所述循环分块结果，确定所述待处理数据的逻辑运算结果。
  12. 一种人工智能芯片，其特征在于，所述人工智能芯片包括如权利要求11所述的处理器。
  13. 一种电子设备，其特征在于，所述电子设备包括如权利要求12所述的人工智能芯片。
PCT/CN2025/082601 2024-04-18 2025-03-14 数据处理方法、处理器、芯片及电子设备 Pending WO2025218403A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202410472589.9 2024-04-18
CN202410472589.9A CN118277328B (zh) 2024-04-18 2024-04-18 数据处理方法、处理器、芯片及电子设备

Publications (1)

Publication Number Publication Date
WO2025218403A1 true WO2025218403A1 (zh) 2025-10-23

Family

ID=91640010

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2025/082601 Pending WO2025218403A1 (zh) 2024-04-18 2025-03-14 数据处理方法、处理器、芯片及电子设备

Country Status (3)

Country Link
CN (1) CN118277328B (zh)
TW (1) TW202542747A (zh)
WO (1) WO2025218403A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN121166072A (zh) * 2025-11-20 2025-12-19 上海壁仞科技股份有限公司 用于计算矩阵乘法的方法、人工智能芯片、计算装置、介质和程序产品
CN121597137A (zh) * 2026-01-30 2026-03-03 之江实验室 基于众核处理器的反向权重梯度计算数据复用方法和装置

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118277328B (zh) * 2024-04-18 2025-04-18 摩尔线程智能科技(北京)股份有限公司 数据处理方法、处理器、芯片及电子设备
CN118626294B (zh) * 2024-08-13 2025-09-02 北京壁仞科技开发有限公司 数据处理方法、装置、电子设备和计算机可读存储介质
CN119512979B (zh) * 2025-01-16 2025-04-25 山东云海国创云计算装备产业创新中心有限公司 确定逻辑块地址访问序列的方法及装置、程序产品

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200159809A1 (en) * 2018-11-15 2020-05-21 Imec Vzw Convolution Engine for Neural Networks
CN112862059A (zh) * 2019-11-28 2021-05-28 华为技术有限公司 长短期记忆lstm网络计算设备以及计算设备
US20210265015A1 (en) * 2020-02-20 2021-08-26 Illumina, Inc. Hardware Execution and Acceleration of Artificial Intelligence-Based Base Caller
CN114707114A (zh) * 2022-04-25 2022-07-05 上海壁仞智能科技有限公司 分块方法及装置、卷积运算的方法及装置、存储介质
CN115982530A (zh) * 2023-03-13 2023-04-18 苏州浪潮智能科技有限公司 加速器运算控制方法、系统、存储介质、装置及设备
CN118277328A (zh) * 2024-04-18 2024-07-02 摩尔线程智能科技(北京)有限责任公司 数据处理方法、处理器、芯片及电子设备

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106445471B (zh) * 2016-10-13 2018-06-01 北京百度网讯科技有限公司 处理器和用于在处理器上执行矩阵乘运算的方法
CN113989169B (zh) * 2020-07-08 2024-12-10 北京硅升科技有限公司 一种膨胀卷积加速计算方法及装置
CN113641952B (zh) * 2021-10-14 2022-02-08 北京壁仞科技开发有限公司 卷积设备、卷积方法、矩阵拆聚装置以及矩阵拆聚方法
CN114970849B (zh) * 2022-06-28 2024-08-13 西安交通大学 一种硬件加速器多阵列并行计算方法及系统
CN115357854B (zh) * 2022-08-30 2025-09-09 无锡江南计算技术研究所 一种高效的矩阵乘运算加速装置及方法
CN117787365B (zh) * 2023-12-29 2024-11-26 中科南京智能技术研究院 一种卷积数据流的调度方法、装置、介质及设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200159809A1 (en) * 2018-11-15 2020-05-21 Imec Vzw Convolution Engine for Neural Networks
CN112862059A (zh) * 2019-11-28 2021-05-28 华为技术有限公司 长短期记忆lstm网络计算设备以及计算设备
US20210265015A1 (en) * 2020-02-20 2021-08-26 Illumina, Inc. Hardware Execution and Acceleration of Artificial Intelligence-Based Base Caller
CN114707114A (zh) * 2022-04-25 2022-07-05 上海壁仞智能科技有限公司 分块方法及装置、卷积运算的方法及装置、存储介质
CN115982530A (zh) * 2023-03-13 2023-04-18 苏州浪潮智能科技有限公司 加速器运算控制方法、系统、存储介质、装置及设备
CN118277328A (zh) * 2024-04-18 2024-07-02 摩尔线程智能科技(北京)有限责任公司 数据处理方法、处理器、芯片及电子设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN121166072A (zh) * 2025-11-20 2025-12-19 上海壁仞科技股份有限公司 用于计算矩阵乘法的方法、人工智能芯片、计算装置、介质和程序产品
CN121597137A (zh) * 2026-01-30 2026-03-03 之江实验室 基于众核处理器的反向权重梯度计算数据复用方法和装置

Also Published As

Publication number Publication date
CN118277328B (zh) 2025-04-18
CN118277328A (zh) 2024-07-02
TW202542747A (zh) 2025-11-01

Similar Documents

Publication Publication Date Title
CN109284130B (zh) 神经网络运算装置及方法
WO2025218403A1 (zh) 数据处理方法、处理器、芯片及电子设备
CN111310910B (zh) 一种计算装置及方法
US12046028B1 (en) Compiler system for deploying CNN models to FPGA-based high-performance accelerators
US10140251B2 (en) Processor and method for executing matrix multiplication operation on processor
US11531540B2 (en) Processing apparatus and processing method with dynamically configurable operation bit width
WO2020047823A1 (en) Convolution over sparse and quantization neural networks
CN111651205B (zh) 一种用于执行向量内积运算的装置和方法
US20230259578A1 (en) Configurable pooling processing unit for neural network accelerator
CN106846235B (zh) 一种利用NVIDIA Kepler GPU汇编指令加速的卷积优化方法及系统
CN111814957A (zh) 神经网络运算方法及相关设备
CN113724127A (zh) 一种图像矩阵卷积的实现方法、计算设备及储存介质
CN120087414A (zh) 注意力机制计算的优化方法、设备、存储介质及程序产品
CN116888591A (zh) 一种矩阵乘法器、矩阵计算方法及相关设备
CN116842304A (zh) 一种不规则稀疏矩阵的计算方法及系统
US11941383B1 (en) Compilation with caching of code analysis result
EP4073628B1 (en) Column data driven arithmetic expression evaluation
WO2025139521A1 (zh) 数据处理方法及装置、电子设备和存储介质
US12395187B2 (en) Computer architecture with data decompression support for neural network computing
US12292838B2 (en) Host device performing near data processing function and accelerator system including the same
KR102722476B1 (ko) 증가된 정밀도의 뉴럴 프로세싱 요소
TWI922046B (zh) 數據處理方法及裝置、電子設備和存儲媒體
US20230195660A1 (en) Memory expansion device performing near data processing function and accelerator system including the same
KR20230095795A (ko) Ndp 기능을 포함하는 호스트 장치 및 이를 포함하는 가속기 시스템
WO2026035679A1 (en) Exploiting input and output sparsity for in-memory machine-learning computations

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 25789273

Country of ref document: EP

Kind code of ref document: A1