CN114463163B - Heterogeneous multi-core image processing method and device - Google Patents
Heterogeneous multi-core image processing method and device Download PDFInfo
- Publication number
- CN114463163B CN114463163B CN202210053900.7A CN202210053900A CN114463163B CN 114463163 B CN114463163 B CN 114463163B CN 202210053900 A CN202210053900 A CN 202210053900A CN 114463163 B CN114463163 B CN 114463163B
- Authority
- CN
- China
- Prior art keywords
- operating system
- image
- data
- buffer area
- linux operating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/60—Memory management
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
- G06F15/161—Computing infrastructure, e.g. computer clusters, blade chassis or hardware partitioning
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
- G06F15/163—Interprocessor communication
- G06F15/167—Interprocessor communication using a common memory, e.g. mailbox
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
The invention provides a heterogeneous multi-core image processing method and device, wherein the method comprises the following steps: A. the Linux operating system receives an acquisition instruction of a user, writes the acquisition instruction into the class shared memory, and triggers the IPI to interrupt to the FreeRTOS operating system; B. FreeRTOS the operating system reads the acquisition instruction from the class shared memory and controls the USB3.0 industrial camera to acquire images; C. the Linux operating system puts the images into an image buffer area according to frames; D. the image processor reads the image in the image buffer area, processes the image, and puts the processing result into the FIFO stack; E. the image processor carries the processing result in the FIFO stack to a result buffer area, and the Linux operating system sends the data of the result buffer area to a user. The invention realizes effective complementation of instantaneity and operability, can complete image acquisition and processing at high speed, enables the USB3.0 industrial camera to be directly accessed, and greatly reduces the research cost and the technical threshold of a high-speed image acquisition and processing system.
Description
Technical Field
The invention relates to a heterogeneous multi-core image processing method and device.
Background
The image acquisition interfaces in the current high-speed high-performance image processing system are mostly realized by FPGA, so that the existing USB3.0 industrial camera in the market cannot be directly used, a CCD or CMOS camera needs to be built by itself, a data communication protocol is realized, the development period is long, and the labor cost is high.
The reasons why the existing USB3.0 industrial camera in the market cannot be adopted mainly include the following points:
if the system adopts the existing USB3.0 industrial camera in the market, ARM is generally needed to acquire image data, the image data is transmitted to the PC end through a gigabit network or other interfaces, then the PC end performs image processing (frame processing), and finally the required processing result is output. The complete process needs two communication links, and the PC image processing needs to be started after receiving the complete image, so the processing speed is far lower than the frame rate of an industrial camera, and the requirements of high speed and high performance are not met.
Meanwhile, in order to realize communication with an industrial camera, a Linux system is generally adopted as a software operating system at an ARM end, and the Linux system is a non-real-time operating system, so that a certain delay exists in response to hardware signals, and delay of high real-time signals is easy to cause. Meanwhile, the problem of large data volume data interaction between kernel mode and user mode of the Linux system exists.
Disclosure of Invention
The invention provides a heterogeneous multi-core image processing method and device, which realize effective complementation of instantaneity and operability, can complete image acquisition and processing at high speed, can enable a USB3.0 industrial camera to be directly accessed, and greatly reduce the research cost and the technical threshold of a high-speed image acquisition and processing system without independently developing data acquisition and processing software of an FPGA.
The invention is realized by the following technical scheme:
the heterogeneous multi-core image processing method is based on Zynq UltraScale +MPSoC chip and is used for controlling a USB3.0 industrial camera to collect images and process the collected images, and comprises the following steps:
A. After receiving an acquisition instruction of a user, the Linux operating system writes the acquisition instruction into a class shared memory and triggers an IPI to interrupt to the FreeRTOS operating system, and the Linux operating system and the FreeRTOS operating system respectively run on four A53 hard cores and two R5 hard cores of a Zynq UltraScale +MPSOC chip;
B. C, the FreeRTOS operating system responds to the interrupt in the step A, reads an acquisition instruction from the class shared memory, and controls the USB3.0 industrial camera to acquire images according to the acquisition instruction;
C. Outputting the image acquired by the USB3.0 industrial camera to a Linux operating system, wherein the Linux operating system puts the image into an image buffer area according to a current writing address by frames, recalculates a new current writing address, and takes an initial address of the image buffer area as the new current writing address when the new current writing address exceeds a preset boundary;
D. The image processor reads the image in the image buffer area according to the read address according to the read enabling signal sent by the Linux operating system, processes the image according to the frame, puts the processed result into the FIFO stack, and the reading and processing processes of the image processor are concurrent execution processes, and the image processor operates on the PL unit of the Zynq UltraScale +MPSOC chip;
E. The image processor carries the processing result in the FIFO stack to a result buffer area according to a write enabling signal sent by a Linux operating system, the Linux operating system sends the data of the result buffer area to a user, and the data transmission process is as follows: and taking the data with the set frame number and continuous frame number as one group of data, after the data of the group is transmitted, if the communication result replied by the user is successful, continuing to transmit the next group of data, otherwise, retransmitting the data of the group.
Further, the image buffer area in the step C includes a first level buffer area and a second level buffer area, the first level buffer area and the second level buffer area both include a first read-write control area and a first data area, the first data area of the first level buffer area stores frame numbers, image information and image data of images, the first read-write control area of the second level buffer area stores processing states, the first data area stores image data, the Linux operating system places images into the first level buffer area according to a current writing address, and when the current writing address data of the second level buffer area has been processed, copies the image data of the current reading address of the first level buffer area to the current writing address of the second level buffer area, and the image processor reads the image data in the second level buffer area according to the reading address.
Further, the class shared memory is composed of DDR4 memory areas unused by the Linux operating system and FreeRTOS operating system.
Further, in the step a, after receiving the acquisition instruction of the user, the Linux operating system checks the acquisition instruction by using the CRC16, if the check is correct, analyzes the frame type corresponding to the acquisition instruction, and if the frame type is the same as the preset type, writes the acquisition instruction into the class shared memory.
Further, the shared memory comprises a second read-write control area and a second data area, the second read-write control area is used for preventing the FreeRTOS operating system and the Linux operating system from writing data at the same time, and the data area is used for data interaction between the FreeRTOS operating system and the Linux operating system.
Further, before the step a, the method further includes the following steps: after the Linux operating system is started, creating a communication_ Thread, camera _ Thread, rpmsgCtrl _thread sub-Thread:
Creating a special Communication protocol stack based on a gigabit network jumbo frame in the sub-Thread communication_thread for communicating with a user;
in the sub-Thread camera_thread, establishing communication connection with a USB3.0 industrial Camera, and configuring a working mode of the USB3.0 industrial Camera;
in the sub-Thread RPMSGCTRL _thread, an inter-core communication mechanism based on class shared memory+ipi interrupt is created for data communication with the FreeRTOS operating system and the collection instruction is transferred to the class shared memory.
The invention is also realized by the following technical scheme:
the image processing device based on the heterogeneous multi-core image processing method according to any one of the above claims, the image processing device comprises a main control core circuit, an acquisition control circuit and a communication transmission circuit, wherein the main control core circuit comprises a Zynq UltraScale +MPSOC chip, a DDR4 memory chip, a QSPI FLASH memory chip and an EMMC memory chip which are respectively connected with the Zynq UltraScale +MPSOC chip, the communication transmission circuit and the acquisition control circuit are respectively connected with the Zynq UltraScale +MPSOC chip, a user is connected with the communication transmission circuit, the acquisition control circuit is connected with the input end of the USB 3.0 industrial camera, the output end of the USB 3.0 industrial camera is connected with the communication transmission circuit, a Linux operating system, a FreeRTOS operating system and an image processor are operated on the Zynq UltraScale +MPSOC chip, the acquisition instruction input by the user is transmitted to the Linux operating system through the communication transmission circuit, the Linux operating system shares the acquisition instruction to the FreeRTOS operating system, the FreeRTOS operating system controls the USB 3.0 industrial camera to acquire images according to the acquisition instruction, the acquired images are transmitted to the Linux operating system through the communication transmission circuit, the Linux operating system shares the images to the image processor, the image processor processes the images, the images and the processed results are transmitted to the Linux operating system, and the Linux operating system shares the processing results to the user operating system.
Further, the communication circuit comprises a USB ULPI transceiver unit connected with the output end of the USB 3.0 industrial camera and a gigabit network PHY unit connected with a user.
Further, the acquisition control circuit comprises an encoder signal detection unit, an input IO circuit and an output IO circuit.
Further, the Zynq UltraScale +MPSoC chip model is XCZU EG-1SFVC784I of Zynq UltraScale + MPSoCs EG series, the DDR4 memory chip model is MT40A512M16GE, the QSPI FLASH memory chip model is MT25QU256ABA1EW9, and the EMMC memory chip model is MTFC8GAKAJCN-4M.
The invention has the following beneficial effects:
1. According to the invention, a dual-operating-system architecture of FreeRTOS operating systems and Linux operating systems is adopted, the Linux operating systems are operated on four A53 hard cores of Zynq UltraScale +MPSOC chips, the FreeRTOS operating systems are operated on two R5 hard cores of Zynq UltraScale +MPSOC chips, the A53 hard cores are high-performance hard cores, and the R5 hard cores are real-time hard cores, so that effective complementation of instantaneity and operability is realized, and the problem of insufficient hardware instantaneity of a single Linux system in the prior art is solved; after receiving an acquisition instruction of a user, the Linux operating system writes the instruction into a class shared memory, triggers an IPI interrupt to the FreeRTOS operating system, responds to the interrupt, reads the acquisition instruction from the class shared memory, outputs an image acquired by USB3.0 to the Linux operating system, places the image into an image cache area by the Linux operating system, reads the image in the image cache area according to a read address, processes the image according to a frame, carries a processing result to a result cache area after the processing is completed, and sends the result to the user by the Linux operating system, and the data can be shared in real time by the class shared memory+IP interrupt notification and the setting of the annular cache of the image cache area, wherein the image processor reads and processes the image in the concurrent execution process, so that the acquisition and the processing of the image can be completed at high speed; the FreeRTOS operating system can directly control the USB3.0 industrial camera to collect images according to the collection instruction, and the collected images can be transmitted to the Linux operating system, so that the USB3.0 industrial camera can be directly connected to use, a user does not need to independently develop data collection and processing software of the FPGA, and the research cost and the technical threshold of the high-speed image collection and processing system are greatly reduced.
Drawings
The invention is described in further detail below with reference to the accompanying drawings.
Fig. 1 is a schematic block diagram of the apparatus of the present invention.
Fig. 2 is a flow chart of the method of the present invention.
Wherein, 11, zynq UltraScale +MPSoC chip; 12. QSPI FLASH memory chips; 13. EMMC memory chip; 14. DDR4 memory chip; 21. an encoder signal detection unit; 22. an input IO circuit; 23. an output IO circuit; 31. USB ULPI receiving and transmitting unit; 32. a gigabit network PHY unit; 4. USB 3.0 industrial camera.
Detailed Description
As shown in fig. 1, the heterogeneous multi-core image processing device is based on Zynq UltraScale +mpsoc chip, and comprises a main control core circuit, an acquisition control circuit and a communication transmission circuit, wherein the main control core circuit comprises Zynq UltraScale +mpsoc chip 11, DDR4 memory chips 14 and QSPI FLASH memory chips 12 and EMMC memory chip 13 which are respectively connected with Zynq UltraScale +mpsoc chip 11 in a bidirectional manner, the communication transmission circuit and the acquisition control circuit are respectively connected with Zynq UltraScale +mpsoc chip 11, a user is connected with the communication transmission circuit, the acquisition control circuit is connected with the input end of USB3.0 industrial camera 4, the output end of USB3.0 industrial camera 4 is connected with the communication transmission circuit, a Linux operating system, freeRTOS operating system and an image processor are operated on Zynq UltraScale +mpsoc chip 11, the acquisition instruction input by the user is transmitted to the Linux operating system through the communication transmission circuit, the Linux operating system shares the acquisition instruction to FreeRTOS operating system, the FreeRTOS operating system controls the USB3.0 industrial camera 4 to acquire images according to the acquisition instruction, the communication transmission circuit transmits the acquired images to the Linux operating system, the image processor shares the images to the image processor, and the Linux operating system shares the image processor to the image processor and processes the image processor results to the Linux operating system.
More specifically, zynq UltraScale +mpsoc chip 11 includes a PS unit including 4a 53 hard cores (i.e., cortex-a53 hard cores) and 2R 5 cores (i.e., cortex-R5 hard cores), and a PL unit which is a programmable logic unit (FPGA), in which the Linux operating system runs on the four a53 hard cores, the FreeRTOS operating system runs on the R5 hard cores, and the image processor runs on the PL unit. The communication circuit comprises a USB ULPI transceiver unit 31 and a gigabit network PHY unit 32, wherein the USB ULPI transceiver unit 31 is in bidirectional connection with the output end of the USB3.0 industrial camera 4, the gigabit network PHY unit 32 is in bidirectional connection with a user, and the USB ULPI transceiver unit 31 and the gigabit network PHY unit 32 are also in bidirectional connection with the Zynq UltraScale +MPSOC chip 11 respectively. The acquisition control circuit comprises an encoder signal detection unit 21 and an input IO circuit 22, wherein the output end of the encoder signal detection unit is connected with the Zynq UltraScale +MPSOC chip 11, the input end of the input IO circuit 23 is connected with the Zynq UltraScale +MPSOC chip 11, and the output end of the output IO circuit 23 is connected with the USB3.0 industrial camera 4. The encoder signal detection unit 21, the input IO circuit 22, and the output IO circuit 23 are all conventional.
In this embodiment, the Zynq UltraScale +mpsoc chip 11 is Zynq UltraScale + MPSoCs EG series of XCZU EG-1SFVC784I, the DDR4 memory chip 14 is MT40a512M16GE, the QSPI FLASH memory chip 12 is MT25QU256ABA1EW9, the EMMC memory chip 13 is MTFC8GAKAJCN-4M, the gigabit network PHY unit 32 includes a chip model DP83867, and the USB ULPI transceiver unit 31 includes a chip model USB3320C.
As shown in fig. 2, the heterogeneous multi-core image processing method includes the following steps:
A. Initializing: the method specifically comprises the following steps:
A1, initializing an EMMC chip after power-on, and initializing a PS end and a PL end of a Zynq UltraScale +MPSOC chip 11;
a2, guiding the start of a Linux operating system from the EMMC memory chip 13, initializing a class shared memory, IPI interrupt and mutual exclusion lock used for real-time data Communication after the start of the Linux operating system, and creating four sub-threads of communication_ Thread, camera _ Thread, rpmsgCtrl _ Thread, period _thread:
Creating a special Communication protocol stack based on a gigabit network jumbo frame in the sub-Thread communication_thread for communicating with a user (i.e., a PC host computer); the specific protocol stack creation process is the prior art;
In the sub-Thread camera_thread, establishing communication connection with the USB 3.0 industrial Camera 4, and configuring the working mode of the USB 3.0 industrial Camera 4;
In the sub-Thread RPMSGCTRL _thread, an inter-core communication mechanism based on class shared memory+IPI interrupt is created and used for data communication with the FreeRTOS operating system, and an acquisition instruction is transmitted to the class shared memory;
Creating a Period management Thread in the sub-Thread period_thread, wherein the Period management Thread is used for managing parameter data, log information and the like;
A3, starting FreeRTOS an operating system by a Linux operating system, and creating TRIGGERTASK, RPMSGRECEIVETASK, RPMSGSENDTASK, PERIODTASK four sub-tasks after starting FreeRTOS the operating system:
In the subtask TRIGGERTASK, according to the acquisition instruction transmitted by the Linux operating system, the acquisition behavior of the USB 3.0 industrial camera 4 is controlled in response to the signal input to the IO circuit 22 or the signal of the encoder signal detection unit 21;
In the subtask RPMSGRECEIVETASK, an inter-core communication mechanism based on class shared memory+IPI interrupt is created, and the appointed data from the Linux operating system is received and processed;
in the subtask RPMSGSENDTASK, receiving message queues from TRIGGERTASK and PeriodTask and forwarding the message queues to a Linux operating system through an inter-core communication mechanism;
in the subtask PeriodTask, a period management task is created for processing fault alarms;
B. setting an instruction type of the acquisition action of the USB3.0 industrial camera 4 through a PC upper computer, wherein the action instruction type comprises an IO signal or an acquisition instruction sent by the upper computer, if the set action instruction type is the IO signal, starting the acquisition action of the USB3.0 industrial camera 4 is controlled by an external signal of an input IO circuit, the input IO circuit is in charge of a FreeRTOS operating system in real time detection, if the set action instruction type is the acquisition instruction, after a communication_thread of the Linux operating system receives the acquisition instruction, checking the acquisition instruction by adopting a CRC16 to determine whether the acquisition instruction is wrong in a transmission process, if the acquisition instruction is correct, analyzing a frame type corresponding to the acquisition instruction, if the frame type is the same as a preset type (the setting of the type is carried out through the PC), writing the acquisition instruction into a shared memory, and triggering IPI interruption (i.e. inter-core interruption) to the FreeRTOS operating system, otherwise, not processing;
C. B, the FreeRTOS operating system responds to the interrupt in the step B, and reads the acquisition instruction from the class shared memory, so that real-time transmission of each phase parameter contained in the acquisition instruction between the Linux operating system and the FreeRTOS operating system is realized; and controlling the USB 3.0 industrial camera 4 to collect images according to the collection instruction;
The class shared memory consists of a DDR4 memory area which is not used by a Linux operating system and a FreeRTOS operating system; the shared memory comprises a second read-write control area and a second data area, wherein the second read-write control area is used for preventing the FreeRTOS operating system and the Linux operating system from writing data at the same time, and the data area is used for data interaction between the FreeRTOS operating system and the Linux operating system;
D. The image acquired by the USB 3.0 industrial camera 4 is output to a Linux operating system, the Linux operating system puts the image into an image buffer area according to the current writing address by frames, and recalculates a new current writing address, wherein the image buffer area is a ring buffer, namely when the new current writing address exceeds a preset boundary, the initial address of the image buffer area is used as the new current writing address; the preset boundary is set according to the setting of the specific acquisition parameters of the USB 3.0 industrial camera 4;
In this embodiment, the image buffer area includes a first level buffer area and a second level buffer area, where the first level buffer area and the second level buffer area both include a first read-write control area and a first data area, where the first read-write control area of the first level buffer area stores a current write address, a current read address, a mutual exclusive lock, etc., the first data area of the first level buffer area stores a frame number, image information, and image data of an image, the first read-write control area of the second level buffer area stores a current write address, a current read address, and a processing state, the first data area of the second level buffer area stores image data, the Linux operating system places an image into the first level buffer area according to the current write address in frames, and copies the image data of the current read address of the first level buffer area to the current write address of the second level buffer area when the current write address data has been processed (i.e. has been read by the image processor), assigns a read address of the image processor, and opens a read enable signal; wherein, the image information refers to the information such as the image row size, the image column size, the image size and the like, and the image data refers to the data actually contained in the image;
E. the image processor reads the image in the secondary buffer area according to the read enabling signal sent by the Linux operating system, reads the image with the data width of 8 bytes per beat, processes the image according to the frame, and puts the processing result into the FIFO stack, wherein the reading and processing process of the image processor are concurrent execution processes, namely, the image is still subjected to the reading action when being processed, so that the timeliness of the data processing can be ensured; the specific process of image processing is the prior art;
F. The image processor carries the processing result in the FIFO stack to a result buffer area according to a write enabling signal sent by the Linux operating system, after carrying is completed, sends a write completion interrupt signal to the Linux operating system, and after the Linux operating system receives the interrupt signal, sends the data of the result buffer area to a communication_thread Thread, the communication_thread starts TCP huge frame transmission, and transmits the data to a user, wherein the data transmission process is as follows: and taking the data with the set frame number and continuous frame number as one group of data, and after the data of the group is transmitted, if the communication result replied by the user is successful, continuing to transmit the next group of data, otherwise, retransmitting the data of the group, so that the transmission efficiency and the reliability of the data can be ensured. The image buffer area and the result buffer area are also composed of DDR4 memory areas.
The foregoing description is only illustrative of the preferred embodiments of the present invention and is not to be construed as limiting the scope of the invention, i.e., the invention is not to be limited to the details of the claims and the description, but rather is to cover all modifications which are within the scope of the invention.
Claims (7)
1. The heterogeneous multi-core image processing method is based on Zynq UltraScale +MPSoC chip and is used for controlling a USB3.0 industrial camera to collect images and processing the collected images, and is characterized in that: the method comprises the following steps:
A. after receiving an acquisition instruction of a user, the Linux operating system writes the acquisition instruction into a class shared memory and triggers an IPI to interrupt to the FreeRTOS operating system, and the Linux operating system and the FreeRTOS operating system respectively run on four A53 hard cores and two R5 hard cores of a ZynqUltraScale +MPSOC chip;
B. C, the FreeRTOS operating system responds to the interrupt in the step A, reads an acquisition instruction from the class shared memory, and controls the USB3.0 industrial camera to acquire images according to the acquisition instruction;
C. Outputting the image acquired by the USB3.0 industrial camera to a Linux operating system, wherein the Linux operating system puts the image into an image buffer area according to a current writing address by frames, recalculates a new current writing address, and takes an initial address of the image buffer area as the new current writing address when the new current writing address exceeds a preset boundary;
D. The image processor reads the image in the image buffer area according to the read address according to the read enabling signal sent by the Linux operating system, processes the image according to the frame, puts the processed result into the FIFO stack, and the reading and processing processes of the image processor are concurrent execution processes, and the image processor operates on the PL unit of the ZynqUltraScale +MPSOC chip;
E. The image processor carries the processing result in the FIFO stack to a result buffer area according to a write enabling signal sent by a Linux operating system, the Linux operating system sends the data of the result buffer area to a user, and the data transmission process is as follows: taking the data with the set frame number and continuous frame number as a group of data, after the group of data is transmitted, if the communication result replied by the user is successful, continuing to transmit the next group of data, otherwise, retransmitting the group of data;
the class shared memory is composed of a DDR4 memory area which is not used by a Linux operating system and a FreeRTOS operating system;
The class shared memory comprises a second read-write control area and a second data area, the second read-write control area is used for preventing FreeRTOS operating systems and Linux operating systems from writing data at the same time, and the data area is used for data interaction between FreeRTOS operating systems and Linux operating systems;
Before the step A is performed, the method further comprises the following steps: after the Linux operating system is started, creating a communication_ Thread, camera _ Thread, rpmsgCtrl _thread sub-Thread:
Creating a special Communication protocol stack based on a gigabit network jumbo frame in the sub-Thread communication_thread for communicating with a user;
in the sub-Thread camera_thread, establishing communication connection with a USB3.0 industrial Camera, and configuring a working mode of the USB3.0 industrial Camera;
in the sub-Thread RPMSGCTRL _thread, an inter-core communication mechanism based on class shared memory+ipi interrupt is created for data communication with the FreeRTOS operating system and the collection instruction is transferred to the class shared memory.
2. The heterogeneous multi-core image processing method according to claim 1, wherein: the image buffer area in the step C comprises a first-level buffer area and a second-level buffer area, the first-level buffer area and the second-level buffer area both comprise a first read-write control area and a first data area, the first data area of the first-level buffer area stores frame numbers, image information and image data of images, the first read-write control area of the second-level buffer area stores processing states, the first data area stores image data, the Linux operating system places images into the first-level buffer area according to a current writing address, and when the current writing address data of the second-level buffer area is processed, copies the image data of the current reading address of the first-level buffer area to the current writing address of the second-level buffer area, and the image processor reads the image data in the second-level buffer area according to the reading address.
3. The heterogeneous multi-core image processing method according to claim 1 or 2, characterized in that: in the step a, after receiving the acquisition instruction of the user, the Linux operating system checks the acquisition instruction by using the CRC16, if the check is correct, analyzes the frame type corresponding to the acquisition instruction, and if the frame type is the same as the preset type, writes the acquisition instruction into the class shared memory.
4. An image processing apparatus based on the heterogeneous multi-core image processing method according to any one of claims 1 to 3, characterized in that: the system comprises a main control core circuit, an acquisition control circuit and a communication transmission circuit, wherein the main control core circuit comprises a ZynqUltraScale +MPSOC chip, a DDR4 memory chip, a QSPI FLASH memory chip and an EMMC memory chip which are respectively connected with the ZynqUltraScale +MPSOC chip, the communication transmission circuit and the acquisition control circuit are respectively connected with the ZynqUltraScale +MPSOC chip, a user is connected with the communication transmission circuit, the acquisition control circuit is connected with the input end of a USB 3.0 industrial camera, the output end of the USB 3.0 industrial camera is connected with the communication transmission circuit, a Linux operating system, a FreeRTOS operating system and an image processor are operated on the ZynqUltraScale +MPSOC chip, an acquisition instruction input by the user is transmitted to the Linux operating system through the communication transmission circuit, the Linux operating system shares the acquisition instruction to the FreeRTOS operating system, the FreeRTOS operating system controls the USB 3.0 industrial camera to acquire images according to the acquisition instruction, the Linux operating system shares the images to the image processor, the image processor processes the images, and shares the processing results to the Linux operating system, and the Linux operating system sends the processing results to the user.
5. The image processing apparatus according to claim 4, wherein: the communication transmission circuit comprises a USB ULPI transceiver unit connected with the output end of the USB 3.0 industrial camera and a gigabit network PHY unit connected with a user.
6. The image processing apparatus according to claim 4, wherein: the acquisition control circuit comprises an encoder signal detection unit, an input IO circuit and an output IO circuit.
7. The image processing apparatus according to claim 4 or 5 or 6, wherein: the ZynqUltraScale +MPSoC chip model is ZynqUltraScale + MPSoCs EG series XCZU EG-1SFVC784I, the DDR4 memory chip model is MT40A512M16GE, the QSPI FLASH memory chip model is MT25QU256ABA1EW9, and the EMMC memory chip model is MTFC8GAKAJCN-4M.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210053900.7A CN114463163B (en) | 2022-01-18 | 2022-01-18 | Heterogeneous multi-core image processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210053900.7A CN114463163B (en) | 2022-01-18 | 2022-01-18 | Heterogeneous multi-core image processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114463163A CN114463163A (en) | 2022-05-10 |
| CN114463163B true CN114463163B (en) | 2024-08-13 |
Family
ID=81409948
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210053900.7A Active CN114463163B (en) | 2022-01-18 | 2022-01-18 | Heterogeneous multi-core image processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114463163B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118402250A (en) * | 2022-11-24 | 2024-07-26 | 芯原微电子(上海)股份有限公司 | Image signal processing driving method, control module, processing module and medium |
| CN115934631B (en) * | 2022-12-30 | 2023-10-27 | 武汉麓谷科技有限公司 | Intelligent storage platform based on MPSoC |
| CN116055653B (en) * | 2023-03-27 | 2023-06-27 | 南京芯驰半导体科技有限公司 | Image data processing method, device, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107493419A (en) * | 2017-09-27 | 2017-12-19 | 中国科学院长春光学精密机械与物理研究所 | A kind of integral miniaturized high-speed realtime graphic acquisition device |
| CN108566357A (en) * | 2017-12-21 | 2018-09-21 | 中国科学院西安光学精密机械研究所 | Image transmission and control system and method based on ZYNQ-7000 and FreeRTOS |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3742324B1 (en) * | 2015-09-15 | 2024-11-27 | Gatekeeper Ltd. | System and method for securely connecting to a peripheral device |
| CN109558174A (en) * | 2018-11-28 | 2019-04-02 | 江苏艾萨克机器人股份有限公司 | RTOS-GPOS dual operating systems robot controller based on ZYNQ dual core processor |
| CN112527722B (en) * | 2020-12-09 | 2024-08-06 | 深圳市显控科技股份有限公司 | Heterogeneous multi-core embedded system and data interaction method of the system |
-
2022
- 2022-01-18 CN CN202210053900.7A patent/CN114463163B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107493419A (en) * | 2017-09-27 | 2017-12-19 | 中国科学院长春光学精密机械与物理研究所 | A kind of integral miniaturized high-speed realtime graphic acquisition device |
| CN108566357A (en) * | 2017-12-21 | 2018-09-21 | 中国科学院西安光学精密机械研究所 | Image transmission and control system and method based on ZYNQ-7000 and FreeRTOS |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114463163A (en) | 2022-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114463163B (en) | Heterogeneous multi-core image processing method and device | |
| US9639447B2 (en) | Trace data export to remote memory using remotely generated reads | |
| CN107527317B (en) | Data Transmission System Based on Image Processing | |
| US20140068134A1 (en) | Data transmission apparatus, system, and method | |
| WO2015047312A1 (en) | Techniques for tracing wakelock usage | |
| CN116541227A (en) | Fault diagnosis method and device, storage medium, electronic device and BMC chip | |
| US20150127993A1 (en) | Trace Data Export to Remote Memory Using Memory Mapped Write Transactions | |
| CN112616024A (en) | USB camera data acquisition method and device, electronic equipment and storage medium | |
| US20190271740A1 (en) | Non-intrusive on-chip debugger with remote protocol support | |
| WO2022032990A1 (en) | Command information transmission method, system, and apparatus, and readable storage medium | |
| CN119202976A (en) | Data processing method and system | |
| CN104317747B (en) | A kind of data buffer storage of grid receiver and dispensing device and method | |
| US12547560B2 (en) | Data interaction method, apparatus and system, and electronic device and storage medium | |
| CN115454881B (en) | Debugging system and debugging method of RISC-V architecture | |
| CN106330327B (en) | Method and apparatus for managing enable state of optical module | |
| US20240427717A1 (en) | Tracing for High Bandwidth Masters in SoC | |
| US8788644B2 (en) | Tracking data processing in an application carried out on a distributed computing system | |
| CN115904046A (en) | Reset control method of embedded system and embedded system | |
| CN116185499B (en) | Register data transmission method, register cache module, intelligent device and medium | |
| WO2025185285A1 (en) | Apparatus, method, system and device for acquiring protocol data, and medium and server | |
| CN112084099A (en) | Method, device and equipment for obtaining alarm state value based on host and storage medium | |
| CN106650006B (en) | Debugging method and system of programmable logic device and electronic design automation terminal | |
| CN109871281B (en) | A data interaction method and device based on inSE security chip | |
| US20060184708A1 (en) | Host controller device and method | |
| CN114968870A (en) | Navigation information processor and method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |