CN112099950B - Image preprocessing optimization method based on edge image processing system - Google Patents
Image preprocessing optimization method based on edge image processing system Download PDFInfo
- Publication number
- CN112099950B CN112099950B CN202010956879.2A CN202010956879A CN112099950B CN 112099950 B CN112099950 B CN 112099950B CN 202010956879 A CN202010956879 A CN 202010956879A CN 112099950 B CN112099950 B CN 112099950B
- Authority
- CN
- China
- Prior art keywords
- image
- edge computing
- edge
- computing device
- preprocessing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to an image preprocessing optimization method based on an edge image processing system, belonging to the fields of image processing and edge calculation. The method comprises the following steps: s1: acquiring average CPU utilization rate and action average response time of all edge computing devices in a database for image preprocessing in each period; s2: calculating the state of the next period through a load prediction model; s3: calculating each action response time of image preprocessing in the next period by using a response prediction model; s4: scheduling an image preprocessing action of a next period of each edge computing device; s5: the edge computing device performs image preprocessing according to the instruction of the main control node and feeds back the image preprocessing; s6: and integrating all the images preprocessed by the edge computing device. The invention realizes the edge calculation of the image preprocessing and the optimal configuration of the edge calculation device, improves the utilization rate of calculation resources, saves the transmission time and the cost of data and reduces the data delay.
Description
Technical Field
The invention relates to an image preprocessing optimization method based on an edge image processing system, and belongs to the fields of image processing and edge calculation.
Background
With the rapid development of 5G and industrial internet in recent years, the demand for edge computation by emerging services is urgent, and in many emerging services in vertical industries, the demand for edge computation is mainly reflected in three aspects of time delay, bandwidth and security. As is well known, edge computing reduces system response delay, saves network bandwidth and protects data security by pushing data processing from the cloud to edges closer to data and applications, so that the requirements of vertical industry application scenes including intelligent manufacturing, smart cities, live games, internet of vehicles and the like can be completely met. Meanwhile, with the gradual perfection of security and control projects of national key construction such as safe cities, space network projects and bright as snow projects, the number of monitoring probes for security and control is more than 5 hundred million by 2021, a large number of security and control probes generate a large number of image processing demands, and the cost, speed and precision requirements of image recognition in security and control scenes on image processing are higher and higher.
In the field of edge calculation, the current terminal product based on ARM architecture only can carry out simple algorithm analysis of a single image, and complex analysis of continuous images cannot be carried out at the end or an operation result cannot be obtained in real time; and the cloud server calculates three problems of time delay, bandwidth and safety. In practical applications, there is usually a large amount of unused computing resources, so that it is necessary to perform part of the image processing work, i.e. image preprocessing, at the edge.
Disclosure of Invention
In order to fully exert the value of front-end acquired data, improve the utilization rate of computing resources, greatly relieve the transmission pressure of a network and reduce data delay, the invention provides an image preprocessing optimization method based on an edge image processing system, which comprises the following steps:
s1: the master control node obtains the average CPU utilization rate and the action average response time of all edge computing devices in the database for image preprocessing in each period;
s2: the master control node calculates the state of the next period of each edge calculating device through a load prediction model;
S3: using a response prediction model, the main control node calculates each action response time of the edge computing device for image preprocessing in the next period;
S4: the main control node schedules the image preprocessing action of the next period of each edge computing device;
s5: the edge computing device performs image preprocessing according to the instruction of the main control node and feeds back the response time of each action and the CPU utilization rate;
S6: the main control node integrates the images preprocessed by all the edge computing devices.
Further, the preprocessing actions are as follows: (1) first, judging whether an image is out of focus: respectively differentiating the transverse direction and the longitudinal direction of the image, accumulating the differences, judging that the image is out of focus when the total value of the differences exceeds a threshold value, and deleting the image; (2) then, image quality evaluation is performed: evaluating the image quality of all images in a period, and reserving images with better quality; (3) finally, performing image recognition: and calculating the directional gradient characteristics of each area surface through the sliding window, comparing the directional gradient characteristics with the characteristics of the object or the person, classifying whether the person exists or not, and if the person exists, further extracting the eyes and nose characteristics of the person.
Still further, the image quality evaluation method may be: according to the methods of Tenengrad gradient, laplacian gradient, SMD (gray variance), SMD2 (gray variance product), and the like.
Further, the load prediction model is a first-order Markov prediction model which is established corresponding to the average CPU utilization rate in each period of the image preprocessing of each edge computing device; the predicted states are divided into: overload O, common N, low load U; generating the following state transition probability matrix:
Wherein P UU represents the probability of transition from the low-load state in the previous cycle to the low-load state in the next cycle, and the other symbols are the same.
Further, the response prediction model may be a response time prediction model established for the action average response time of each image preprocessing period of the whole edge image processing system, or may be a response time prediction model established for the action average response time of each image preprocessing period of each edge computing device.
Still further, the response time prediction model may be a Kalman filter, a particle filter, an ARMAX model (Autoregressive moving-average model with exogenous inputs), or the like.
Further, the step S4 specifically includes: (1) The main control node transfers all the preprocessing actions of the overload edge computing device in the next period to the idle edge computing device for processing according to the load prediction model result; (2) The main control node transfers the preprocessing action of the response timeout of the edge computing device in the next period to the edge computing device with surplus response time of the edge computing device according to the response prediction model; (3) And on the premise that the internal migration of the edge device ensures that overload and timeout are not caused, migrating the pretreatment actions which are not performed by all edge computing devices to the main control node for processing.
The edge image processing system is formed by connecting a main control node and at least one edge computing device through one or more of a network, bluetooth or USB, and further comprises a conventional peripheral interface (USB interface), a network module, a power supply module, a Bluetooth module and the like; the master control node can be a cloud server or an edge computing device node; the edge computing device is provided with an image processor and is connected with the image input device.
Further, the master node includes:
1. And the resource management module is used for collecting node state information uploaded by the edge computing device management module in the distributed edge computing, wherein the node state information comprises node memory state, CPU state, GPU state, running task information, response time and the like.
2. And the task management module is used for issuing the image preprocessing task to the edge computing device and completing scheduling.
3. And the data preprocessing module is used for preprocessing and segmenting the execution task data.
4. And the result gathering module is used for collecting the analysis results of the operation modules of the edge computing devices and sorting the result sets according to the sequence numbers.
5. And the output module is used for outputting the analysis result set to the application layer.
Further, the edge computing device includes:
1. The node management module is used for collecting state information of the edge computing device, including memory state, CPU state, GPU state and running task information; and receiving the operation task, starting the operation module and transmitting the specific operation parameters to the operation module.
2. And the operation module is used for specifically executing an image preprocessing algorithm.
The invention has the beneficial effects that: the invention builds an edge image processing system, extracts the whole simple task of image preprocessing from the whole task of image processing to the edge computing device end for processing, and realizes the edge computing of the image preprocessing and the optimal configuration of the edge computing device by establishing the load prediction model and the load prediction model, thereby improving the utilization rate of computing resources, saving the transmission time and the cost of data and reducing the data delay.
Drawings
FIG. 1 is a flow chart of an image preprocessing optimization method based on an edge image processing system of the present invention;
Fig. 2 is a block diagram of an edge image processing system according to an embodiment of the present invention, in which: the method comprises the steps of 1,2,3 and 7, 4 and 8, 5 and 6, and 9, wherein 1 is a cloud server, 2 is a cloud server database, 3 and 7 are edge computing devices, 4 and 8 are local databases, 5 and 6 are monitoring equipment, and 9 is a control end computer.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and examples in order to make the objects and technical solutions of the present invention more clear.
Examples
For real-time face recognition monitoring aiming at monitoring with an edge computing device in a cell, in order to further analyze whether the cell is a home owner, the face recognition preprocessing needs to be performed at the edge end because the data size of a video image of the cell is large every day and the cloud server configuration is low, and the implementation provides a face recognition preprocessing optimization method based on an edge image processing system.
The face recognition preprocessing optimization method based on the edge image processing system is realized by an edge image processing system, and is combined with fig. 2, wherein the edge image processing system is formed by connecting a main control node of a cloud server 1 and edge computing devices 3 and 7 which are in one-to-one correspondence and are connected with two monitoring devices 5 and 6 through a network, and the face recognition preprocessing optimization method also comprises a conventional peripheral interface, a network module, a power supply module and the like. The cloud server 1 is provided with a cloud server database 2 for storing images subjected to face recognition preprocessing; the edge computing devices 3 and 7 are respectively provided with local databases 4 and 8 for storing images shot by the monitoring equipment; the control end computer 9 can facilitate the user to input operation instructions, acquire related information and realize man-machine interaction.
Further, the cloud server 1 includes:
1. And the resource management module is used for collecting node state information uploaded by the edge computing device management module in the distributed edge computing, wherein the node state information comprises node memory state, CPU state, GPU state, running task information, response time and the like.
2. And the task management module is used for issuing the image preprocessing task to the edge computing device and completing scheduling.
3. And the data preprocessing module is used for preprocessing and segmenting the execution task data.
4. And the result gathering module is used for collecting the analysis results of the operation modules of the edge computing devices and sorting the result sets according to the sequence numbers.
5. And the output module is used for outputting the analysis result set to the application layer.
Further, the edge calculating means 3 and 7 include:
1. The node management module is used for collecting state information of the edge computing device, including memory state, CPU state, GPU state and running task information; and receiving the operation task, starting the operation module and transmitting the specific operation parameters to the operation module.
2. And the operation module is used for specifically executing an image preprocessing algorithm.
Referring to fig. 1, a face recognition preprocessing optimization method based on an edge image processing system includes the following steps:
step one: the cloud server 1 acquires average CPU utilization rate and action average response time of image preprocessing in each period in the local databases 4 and 8 of the edge computing devices 3 and 7;
Step two: establishing a load prediction model, determining parameters through training, and calculating the states of the edge computing devices 3 and 7 in the next period through the load prediction model by the cloud server 1;
The load prediction model is a first-order Markov prediction model which is established corresponding to the average CPU utilization rate in each period of the image preprocessing of each edge computing device; the predicted states are divided into: overload O, common N, low load U; generating the following state transition probability matrix:
Wherein P UU represents the probability of transition from the low-load state in the previous cycle to the low-load state in the next cycle, and the other symbols are the same.
Step three: establishing a response prediction model, determining parameters through training, and calculating each action response time of image preprocessing of the edge computing devices 3 and 7 in the next period by using the response prediction model by the cloud server 1;
the response prediction model can be used for respectively establishing a response time prediction model for the action average response time of each image preprocessing period of the whole edge image processing system, or respectively establishing a response time prediction model for the action average response time of each image preprocessing period of each edge computing device.
For better implementation of the prediction, the proposed response time prediction model is the ARMAX model (Autoregressive moving-average model with exogenous inputs) and its modifications.
Step four: the cloud server 1 schedules the image preprocessing action of the next period of the edge computing devices 3 and 7;
The method comprises the following steps: (1) The cloud server 1 transfers all preprocessing actions of the overload edge computing device in the next period to the idle edge computing device for processing according to the load prediction model result; (2) The cloud server 1 transfers the preprocessing action of the response timeout of the edge computing device in the next period to the edge computing device with surplus response time of the edge computing device according to the response prediction model; (3) On the premise that the internal migration of the edge device ensures that overload and timeout are not avoided, all edge computing devices cannot be used for preprocessing actions to be migrated to the cloud server 1 for processing.
Step five: the edge computing devices 3 and 7 execute image preprocessing according to the instruction of the cloud server 1 and feed back response time of each action and CPU utilization rate;
step six: the cloud server 1 integrates the images preprocessed by the edge computing devices 3 and 7 and stores the images in the cloud server database 2 so as to perform the next face recognition.
The invention is not limited to the embodiments described above, but any modifications or alterations to the above embodiments of the invention will be apparent to a person skilled in the art without departing from the scope of protection of the embodiments of the invention shown by way of example only and the appended claims, the described embodiments being intended only to facilitate an understanding of the invention and not to limit it in any way.
Claims (3)
1. The image preprocessing optimization method based on the edge image processing system is characterized by comprising the following steps of:
s1: the master control node obtains the average CPU utilization rate and the action average response time of all edge computing devices in the database for image preprocessing in each period;
s2: the master control node calculates the state of the next period of each edge calculating device through a load prediction model;
S3: using a response prediction model, the main control node calculates each action response time of the edge computing device for image preprocessing in the next period;
S4: the main control node schedules the image preprocessing action of the next period of each edge computing device;
s5: the edge computing device performs image preprocessing according to the instruction of the main control node and feeds back the response time of each action and the CPU utilization rate;
S6: the main control node integrates the images preprocessed by all the edge computing devices;
The load prediction model is a first-order Markov prediction model which is established corresponding to the average CPU utilization rate in each period of the image preprocessing of each edge computing device; the predicted states are divided into: overload O, common N, low load U; generating the following state transition probability matrix:
Wherein, P UU represents the probability of the low-load state transition to the low-load state in the next period in the previous period, and other symbols are the same;
the response prediction model is used for respectively establishing a response time prediction model for the action average response time of each image preprocessing period of the whole edge image processing system or respectively establishing a response time prediction model for the action average response time of each image preprocessing period of each edge computing device;
The step S4 specifically comprises the following steps: (1) The main control node transfers all the preprocessing actions of the overload edge computing device in the next period to the idle edge computing device for processing according to the load prediction model result; (2) The main control node transfers the preprocessing action of the response timeout of the edge computing device in the next period to the edge computing device with surplus response time of the edge computing device according to the response prediction model; (3) And on the premise that the internal migration of the edge device ensures that overload and timeout are not caused, migrating the pretreatment actions which are not performed by all edge computing devices to the main control node for processing.
2. The image preprocessing optimization method based on the edge image processing system according to claim 1, wherein the preprocessing actions are as follows in sequence: (1) first, judging whether an image is out of focus: respectively differentiating the transverse direction and the longitudinal direction of the image, accumulating the differences, judging that the image is out of focus when the total value of the differences exceeds a threshold value, and deleting the image; (2) then, image quality evaluation is performed: evaluating the image quality of all images in a period, and reserving images with better quality; (3) finally, performing image recognition: and calculating the directional gradient characteristics of each area surface through the sliding window, comparing the directional gradient characteristics with the characteristics of the object or the person, classifying whether the person exists or not, and if the person exists, further extracting the eyes and nose characteristics of the person.
3. An edge image processing system applied to the image preprocessing optimization method based on the edge image processing system as claimed in any one of claims 1-2, characterized in that the edge image processing system is formed by connecting a main control node and at least one edge computing device through a network, bluetooth or USB; the main control node is a cloud server or an edge computing device node; the edge computing device is provided with an image processor and is connected with the image input device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010956879.2A CN112099950B (en) | 2020-09-15 | 2020-09-15 | Image preprocessing optimization method based on edge image processing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010956879.2A CN112099950B (en) | 2020-09-15 | 2020-09-15 | Image preprocessing optimization method based on edge image processing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112099950A CN112099950A (en) | 2020-12-18 |
| CN112099950B true CN112099950B (en) | 2024-09-24 |
Family
ID=73752068
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010956879.2A Active CN112099950B (en) | 2020-09-15 | 2020-09-15 | Image preprocessing optimization method based on edge image processing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112099950B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN120408327B (en) * | 2025-07-01 | 2025-10-21 | 陕西省动物研究所 | Edge calculation driven wild animal image compression and encryption transmission method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111191584A (en) * | 2019-12-30 | 2020-05-22 | 电信科学技术第十研究所有限公司 | Face recognition method and device |
| CN111459665A (en) * | 2020-03-27 | 2020-07-28 | 重庆电政信息科技有限公司 | Distributed edge computing system and distributed edge computing method |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8645529B2 (en) * | 2010-10-06 | 2014-02-04 | Infosys Limited | Automated service level management of applications in cloud computing environment |
| CN105260235A (en) * | 2015-09-23 | 2016-01-20 | 浪潮集团有限公司 | Method and device for scheduling resources on basis of application scenarios in cloud platform |
| JP6681377B2 (en) * | 2017-10-30 | 2020-04-15 | 株式会社日立製作所 | System and method for optimizing resource allocation |
| CN108319501B (en) * | 2017-12-26 | 2022-01-14 | 中山大学 | Elastic resource supply method and system based on micro-service gateway |
| CN110865881A (en) * | 2018-08-27 | 2020-03-06 | 中移(苏州)软件技术有限公司 | Method and device for resource scheduling |
-
2020
- 2020-09-15 CN CN202010956879.2A patent/CN112099950B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111191584A (en) * | 2019-12-30 | 2020-05-22 | 电信科学技术第十研究所有限公司 | Face recognition method and device |
| CN111459665A (en) * | 2020-03-27 | 2020-07-28 | 重庆电政信息科技有限公司 | Distributed edge computing system and distributed edge computing method |
Non-Patent Citations (1)
| Title |
|---|
| 一种基于深度学习的云平台弹性伸缩算法;曹宇等;《计算机与现代化》;20190415;第17-24页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112099950A (en) | 2020-12-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wu et al. | Helmet detection based on improved YOLO V3 deep model | |
| Chen et al. | Embedded system real-time vehicle detection based on improved YOLO network | |
| CN108563739B (en) | Weather data acquisition method and device, computer device and readable storage medium | |
| CN112949673A (en) | Feature fusion target detection and identification method based on global attention | |
| CN116560341B (en) | Industrial robot fault diagnosis model and fault diagnosis method | |
| CN120510650A (en) | Gesture recognition method and device based on deep learning | |
| CN112099950B (en) | Image preprocessing optimization method based on edge image processing system | |
| CN117931426A (en) | Generation countermeasure method and system for cloud workload prediction | |
| CN115510077B (en) | Graph data updating method, device, equipment and medium based on message passing | |
| CN118819684B (en) | A terminal device unloading method and system based on edge orchestration technology | |
| US20250267072A1 (en) | Service flow perception method and apparatus for edge computing, and electronic device | |
| Chen et al. | Cloud-edge collaborative data processing architecture for state assessment of transmission equipments | |
| CN119274044A (en) | An image processing method for realizing enhanced image recognition | |
| CN119322670A (en) | Data center intelligent task scheduling and load balancing regulation and control method and system | |
| Zhang et al. | Channel compression optimization oriented bus passenger object detection | |
| CN111340648A (en) | Energy management and control method and system based on Internet of things | |
| CN110427920B (en) | A Real-time Pedestrian Analysis Method for Surveillance Environment | |
| CN113962880A (en) | License plate deblurring method and device, electronic equipment and storage medium | |
| CN120104316B (en) | A data processing method and device for high-performance parallel simulation | |
| CN120105683B (en) | Data processing method and device based on multi-resolution simulation object | |
| CN119762327B (en) | Vehicle video data learning method and device based on YTS engine AI algorithm | |
| Huangfu | Research on real-time data processing system combining edge computing and artificial intelligence | |
| CN110929118A (en) | Network data processing method, equipment, device and medium | |
| Li et al. | End-Side License Plate Detection Model Based on Improved YOLOv8s with Depth-Separable CNNs | |
| CN119322722A (en) | Computer performance evaluation system and method based on big data and AI |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |