CN120707642B - Cloud-edge cooperative visual slam method and system based on feature point prediction - Google Patents

Cloud-edge cooperative visual slam method and system based on feature point prediction

Info

Publication number
CN120707642B
CN120707642B CN202511194614.2A CN202511194614A CN120707642B CN 120707642 B CN120707642 B CN 120707642B CN 202511194614 A CN202511194614 A CN 202511194614A CN 120707642 B CN120707642 B CN 120707642B
Authority
CN
China
Prior art keywords
node
feature point
image
area
quadtree
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202511194614.2A
Other languages
Chinese (zh)
Other versions
CN120707642A (en
Inventor
卢旭
余富健
刘军
黄翰媛
刘海浚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Polytechnic Normal University
Original Assignee
Guangdong Polytechnic Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Polytechnic Normal University filed Critical Guangdong Polytechnic Normal University
Priority to CN202511194614.2A priority Critical patent/CN120707642B/en
Publication of CN120707642A publication Critical patent/CN120707642A/en
Application granted granted Critical
Publication of CN120707642B publication Critical patent/CN120707642B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G01C21/3833—Creation or updating of map data characterised by the source of data
    • G01C21/3837—Data obtained from a single source
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/84—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using probabilistic graphical models from image or video features, e.g. Markov models or Bayesian networks
    • G06V10/85—Markov-related models; Markov random fields

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Automation & Control Theory (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种基于特征点预测的云边协同视觉slam方法与系统。该方法包括将初始连续若干帧的图像加入滑动窗口中；对全部像素进行特征点提取；建立四叉树作为对应帧图像的先验信息，并合并建立全局先验信息；利用全局先验信息采用多步马尔可夫概率模型对下一帧图像的特征点分布进行预测；调整所述新的四叉树中的分支；将下一帧的图像加入滑动窗口，并去除最早一帧图像及对应先验信息，利用滑动窗口内的图像及对应先验信息预测后续帧图像的特征点分布。本发明考虑到相机在空间中运动具有连续性，前后帧间的特征点具有空间联系，通过构建多步马尔可夫概率模型，在保证定位与建图效果的前提下减少了计算量，极大节省了CPU的计算资源消耗。

This invention discloses a cloud-edge collaborative visual SLAM method and system based on feature point prediction. The method includes: adding several consecutive frames of images into a sliding window; extracting feature points from all pixels; establishing a quadtree as prior information for the corresponding frame images and merging them to establish global prior information; using the global prior information to predict the feature point distribution of the next frame image using a multi-step Markov probability model; adjusting the branches in the new quadtree; adding the next frame image into the sliding window, removing the earliest frame image and its corresponding prior information, and using the images within the sliding window and their corresponding prior information to predict the feature point distribution of subsequent frames. This invention considers the continuity of camera motion in space and the spatial relationship between feature points in consecutive frames. By constructing a multi-step Markov probability model, it reduces computational load while ensuring effective localization and mapping, significantly saving CPU computing resources.

Description

Cloud-edge cooperative visual slam method and system based on feature point prediction
Technical Field
The invention belongs to the technical field of robot vision, and particularly relates to a cloud edge cooperative vision slam method and system based on feature point prediction.
Background
With the popularization of intelligent terminal equipment and the rapid development of technologies such as mobile robots, unmanned aerial vehicles, augmented reality and the like, synchronous positioning and mapping (slam) technologies face new challenges and opportunities. Conventional slam systems typically rely on a single device to complete all computing tasks, which places extremely high demands on resource-constrained mobile devices. In order to realize stable and efficient slam functions under the conditions of limited computing power, storage space and battery endurance, a cloud edge cooperative slam architecture is generated, and the method becomes one of hot spot directions of current research.
Cloud edge cooperation slam can be used for unloading computation-intensive work to the cloud end through reasonable distribution of computation tasks, and meanwhile necessary real-time processing capacity is reserved on edge equipment, so that balance between system performance and resource consumption is achieved. The architecture is particularly suitable for miniaturized devices such as smart phones, unmanned aerial vehicles, service robots and the like which are provided with low-power-consumption processors. Although the computing capacity of the devices is limited, the limit of local hardware can be broken through by cloud edge cooperation, and the slam experience approaching to a high-performance computing platform is obtained.
In practical applications, low-power devices put various demands on cloud-edge collaborative slam architecture, including reducing the amount of computation and CPU burden as much as possible.
Disclosure of Invention
The invention aims to overcome the defects and shortcomings of the prior art, and provides a cloud edge collaborative visual slam method and a cloud edge collaborative visual slam system based on characteristic point prediction, which extract characteristic points based on quadtree and Markov properties, so that the calculated amount is reduced on the premise of ensuring the positioning and mapping effects.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the invention provides a cloud edge collaborative visual slam method based on feature point prediction, which comprises the following steps of:
adding a plurality of initial continuous frame images into a sliding window;
extracting feature points of all pixels of a plurality of initial continuous frames of images;
establishing a quadtree for the feature points extracted from each frame;
The quadtree is saved as prior information of the corresponding frame image;
Combining prior information corresponding to a plurality of initial continuous frame images, and establishing a new quadtree as global prior information;
predicting the feature point distribution of the next frame of image by using the global priori information and adopting a multi-step Markov probability model;
the branches in the new quadtree are adjusted according to the actual characteristic point distribution of the next frame of image;
And adding the next frame image into the sliding window, removing the earliest frame image and the corresponding prior information, and predicting the characteristic point distribution of the next frame image by utilizing the image in the sliding window and the corresponding prior information.
As a preferred technical solution, the building a quadtree for the feature points extracted from each frame includes the following steps:
creating a root node representing the entire image area;
Starting from a root node, checking whether the current node is divided, and if the current node is not divided and the spatial density (N/Area) of feature points in the node is more than ρ1, dividing the current region into four sub-regions and distributing the feature points to corresponding sub-regions, wherein N is the number of the feature points of the current node, ρ1 is a set spatial density threshold, area= (x max-xmin)×(ymax-ymin) is the Area of the current node, and x max、xmin、ymax and y min are the maximum and minimum values of the abscissa and the ordinate of the current region;
dividing by the node center (x c,yc):
;
Stopping dividing when the area reaches the minimum size limit s=W/(2 n')×H/(2n'), n' E [6,8], wherein W and H are the width and height of the image frame, and Q 1-Q4 is four sub-areas obtained by dividing;
The process is recursively performed until feature points are assigned to the appropriate leaf nodes.
As a preferred technical scheme, the method establishes a new quadtree as global prior information, wherein the number of node characteristic pointsWherein w 1,w2,…,wn is the weight of the images of different time frames, the subscript represents the sequence number, the weight value is determined by the time sequence of the image frames, the new image frames have larger weight, and vice versa, V 1,V2,…,Vn is the number of the characteristic points of the images of different time frames, and the subscript represents the sequence number;
and removing leaf nodes of the space density (N/Area) < ρ2 of the feature points, wherein N is the number of the feature points of the current node, ρ2 is a preset space density threshold, area= (x max-xmin)×(ymax-ymin) is the Area of the current node, and x max、xmin、ymax and y min are the maximum and minimum values of the abscissa and the ordinate of the current region.
As an optimal technical scheme, the method for predicting the characteristic point distribution of the next frame of image by using the global prior information and adopting a multi-step Markov probability model specifically comprises the following steps:
Defining a multi-step Markov probability model P(Xn+1|Xn,Xn-1,...,X0)=P(Xn+1|Xn,Xn-1,...,Xn-k+1),P(Xn+1|Xn,Xn-1,...,X0) to represent the feature point distribution state X n+1 of the current image, related to all distribution states X n,Xn-1,...,X0 preceding X n+1, P (X n+1|Xn,Xn-1,...,Xn-k+1) to limit X n+1 to only a limited k states X n,Xn-1,...,Xn-k+1 preceding X n+1;
predicting the distribution state of image characteristic points of n+1 through X n,Xn-1,...,Xn-k+1 :
;
Wherein E ∙ is the maximum likelihood.
As a preferred technical scheme, X n+1 is related to the limited k=3 states X n,Xn-1,...,Xn-2 before X n+1 to obtain a three-order Markov probability model P(Xn+1|Xn,Xn-1,...,X0)=P(Xn+1|Xn,Xn-1,...,Xn-2);
The state transition of the characteristic point distribution between the front image frame and the rear image frame in a short time satisfies the linear relation that X n+1=aXn+bXn-1+cXn-2+...+X1 +epsilon, epsilon is Gaussian noise, and then:
;
wherein a, b and c are linear coefficients.
As a preferable technical solution, the following actual feature point distribution of the next frame image adjusts branches in the new quadtree, and specifically includes the following steps:
extracting feature points, namely extracting feature points from the corresponding images of the leaf nodes, counting the number of the feature points, and updating the leaf node data;
Dividing a current region into four sub-regions, distributing the feature points to the corresponding sub-regions, and performing recursion until the feature points are distributed to proper leaf nodes, wherein N is the number of the feature points of the current node, ρ1 is a set space density threshold, area= (x max-xmin)×(ymax-ymin) is the Area of the current node, x max、xmin、ymax and y min are the maximum and minimum values of the abscissa and the ordinate of the current region, and when the region reaches the minimum size limit s=W/(2 n')×H/(2n'), N' epsilon [6,8], dividing is stopped, and W and H are the width and height of an image frame;
If one father node P has only leaf nodes and the space density (N '/Area') < ρ3 of each leaf node characteristic point, then completing four leaf nodes of the father node P, wherein N 'is the number of leaf node characteristic points, area' is the Area of the leaf node, ρ3 is the set space density threshold of the leaf node;
Extracting characteristic points from the complemented leaf node area, if the number of the characteristic points of each leaf node of the complemented father node P does not reach a set threshold T2, enabling the father node P to be a leaf node and reconstructing the leaf node locally upwards;
The upward local reconstruction specifically comprises the step of judging whether a father node P1 of the P has only leaf nodes, if so, executing the local reconstruction on the P1.
As a preferable technical solution, the number of layers of the quadtree is set at the beginning of the step of adjusting branches in the new quadtree, and the corresponding leaf node whose number of leaf node characteristic points is smaller than the set threshold T2 is assigned to-1, which indicates the abandoned region, and the abandoned nodes do not play a role in merging.
The invention also provides a cloud edge collaborative visual slam system based on feature point prediction, which comprises an edge end, an edge server and a cloud server which are communicated with each other through a communication module;
the edge end adopts the cloud edge collaborative visual slam method based on the feature point prediction to obtain image feature point distribution, and constructs and stores a sub-map;
The edge server is used for restoring the data uploaded by the edge end according to the slam visual odometer to obtain a real-time path and roadside points, and carrying out optimization processing on the real-time path and roadside points through the slam rear end to obtain an edge map;
the cloud server is used for merging the edge maps uploaded by the edge servers as global maps, and carrying out optimization processing and storage on the global maps.
The invention further provides cloud edge cooperative visual slam equipment based on the characteristic point prediction, which comprises a memory and at least one processor, wherein the memory is stored with instructions, the memory is connected with the at least one processor through a line, and the at least one processor calls the instructions in the memory so that the cloud edge cooperative visual slam equipment based on the characteristic point prediction executes the cloud edge cooperative visual slam method based on the characteristic point prediction.
In another aspect of the present invention, a storage medium is further provided, where a program is stored, where the program, when executed by a processor, implements a cloud edge collaborative visual slam method based on feature point prediction as described above.
Compared with the prior art, the invention has the following advantages and beneficial effects:
(1) According to the invention, the continuity of the movement of the camera in space is considered, the space relation is provided for the characteristic points between the front frame and the back frame, and the calculation resource consumption of the CPU is greatly saved by constructing a multi-step Markov probability model and then an algorithm.
(2) The invention adapts to the requirements of low-power equipment in a cloud-edge cooperative mode, extracts and manages the characteristic points of the visual odometer on the premise of ensuring the robustness of the system, and solves the problem that the extraction of the characteristic points (exemplified by orb characteristic points) of the whole image in the prior art consumes resources and time very much.
Drawings
FIG. 1 is a flow chart of a cloud edge collaborative visual slam method based on feature point prediction according to an embodiment of the present invention;
FIG. 2 is a flow chart of the set-up of a quadtree according to an embodiment of the invention;
FIG. 3 is a schematic diagram of a corresponding relationship between a quadtree and an image according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of quadtree weighted merge of an embodiment of the present invention;
FIG. 5 is a schematic diagram of a priori quadtree dynamic adjustment of an embodiment of the present invention;
FIG. 6 is a schematic view of window sliding in accordance with an embodiment of the present invention;
Fig. 7 is a schematic structural diagram of a cloud edge cooperative visual slam system based on feature point prediction according to an embodiment of the present invention.
Detailed Description
In order to enable those skilled in the art to better understand the present application, the following description will make clear and complete descriptions of the technical solutions according to the embodiments of the present application with reference to the accompanying drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Example 1:
as shown in fig. 1, the embodiment provides a cloud edge cooperative visual slam method based on feature point prediction, which includes the following steps:
S1, inputting images, namely adding a plurality of initial continuous frame images into a sliding window.
S2, extracting global feature points of the image, namely extracting feature points of all pixels of a plurality of initial continuous frames of images.
And S3, building a quadtree, namely building quadtree management image feature points for the feature points extracted from each frame. The flow chart is shown in fig. 2, and the correspondence between the image and the quadtree is shown in fig. 3.
Quadtrees are a tree-like data structure for efficiently managing two-dimensional space. It recursively divides the region into four equal-sized sub-regions, with each node having a maximum of four sub-nodes. The quadtree is widely applied to image processing and can rapidly process space inquiry and collision detection. The hierarchical structure can optimize the storage and retrieval efficiency, and is particularly suitable for processing data with uneven distribution.
Further, step S3 includes the steps of:
s301, initializing a root node, namely creating the root node representing the whole image area.
S302, inserting feature points, namely checking whether the current node is divided from a root node, and dividing the current region into four sub-regions and distributing the feature points to corresponding sub-regions if the current node is not divided and the space density (unit Area point number) (N/Area) > ρ1 of the feature points in the node is not divided, wherein N is the number of the feature points of the current node, ρ1 is a set space density threshold value, area= (x max-xmin)×(ymax-ymin) is the Area of the current node, and x max、xmin、ymax and y min are the maximum and minimum values of the abscissa and the ordinate of the current region.
Division rules (division bounded by node center (x c,yc):
;
The region reaches a minimum size limit s=w/(2 n')×H/(2n'), the division is stopped when n' e [6,8], W and H are the width and height of the image frame, and Q 1-Q4 is the four sub-regions obtained by the division.
S303, recursion processing is performed until the feature points are allocated to the proper leaf nodes.
S4, storing the prior information, namely storing the image frames in the sliding window corresponding to the quadtree as the prior information.
S5, establishing a quadtree by using prior information, namely establishing a new quadtree corresponding to the quadtree by combining images in the sliding window as global prior information, wherein the node characteristic point number is calculated by the new quadtreeWherein w 1,w2,…,wn is the weight of the images of different time frames, the subscript indicates the sequence number, the weight value is determined by the time sequence of the image frames, the new image frames have larger weight, and vice versa, V 1,V2,…,Vn is the number of the characteristic points of the images of different time frames, and the subscript indicates the sequence number. Removing leaf nodes of the space density (N/Area) < ρ2 of the feature points, wherein ρ2 is a preset space density threshold value, and is smaller than or equal to ρ1, and the specific value is obtained by adjusting according to scenes and experience. The merging process of the quadtree a and B into C is as shown in fig. 4, and assuming that the spatial density (N/Area) < ρ2 of the characteristic points of the leaf node a 2 of the quadtree a, the nodes of the same position of the quadtree a and the quadtree B are weighted and merged to the corresponding position of the quadtree C, and the node with the zero number of characteristic points does not play a role in the merging process, for example, the leaf node a l、a2、a3、a4 of the quadtree B does not exist, so the number of the corresponding characteristic points is 0, the weight w.v =0, and only the leaf node a 1、a3、a4 of the quadtree a, the leaf node B 1 of the quadtree B and the leaf node B 2 of the quadtree B remain in the quadtree C in the third layer.
S6, predicting the feature point distribution of the next frame of image by using the global priori information and adopting a multi-step Markov probability model.
Markov properties (Markov properties) refer to the Property that a random process, given a current state, relies on the current state alone for future states, and not on past states. Mathematical expression:
for the discrete-time random process { X n }, if:
P(Xn+1=x|Xn=xn,Xn-1=xn-1,...,X0=x0)=P(Xn+1=x|Xn=xn);
the process is said to have markov properties.
When the state at a certain instant depends on the state at the first k instants (not just the current instant), it is called a k-order markov process.
The mathematical expression is as follows:
P(Xn+1|Xn,Xn-1,...,X0)=P(Xn+1|Xn,Xn-1,...,Xn-k+1);
I.e. the conditional probability of the future state depends on the last k historical states.
Defining a multi-step Markov probability model P(Xn+1|Xn,Xn-1,...,X0)=P(Xn+1|Xn,Xn-1,...,Xn-k+1),P(Xn+1|Xn,Xn-1,...,X0) to represent the feature point distribution state X n+1 of the current image, related to all distribution states X n,Xn-1,...,X0 preceding X n+1, P (X n+1|Xn,Xn-1,...,Xn-k+1) to limit X n+1 to only a limited k states X n,Xn-1,...,Xn-k+1 preceding X n+1;
predicting the distribution state of image characteristic points of n+1 through X n,Xn-1,...,Xn-k+1 :
;
Wherein E ∙ is the maximum likelihood.
Further, in this embodiment, let X n+1 relate only to the limited k=3 states X n,Xn-1,...,Xn-2 before X n+1, and obtain a third order markov probability model P(Xn+1|Xn,Xn-1,...,X0)=P(Xn+1|Xn,Xn-1,...,Xn-2);
The state transition of the characteristic point distribution between the front image frame and the rear image frame in a short time satisfies the linear relation that X n+1=aXn+bXn-1+cXn-2+...+X1 +epsilon, epsilon is Gaussian noise, and then:
;
wherein a, b and c are linear coefficients.
More specifically, a, b, c can be expressed as:
;
;
。
S7, dynamically adjusting the prior quadtree, namely adjusting branches in the new quadtree in the step S5 by the actual distribution of the image characteristic points of the next frame of image. The flow chart is shown in fig. 5.
Further, step S7 includes the steps of:
And S701, extracting characteristic points, namely extracting the characteristic points from the corresponding images of the leaf nodes, counting the number of the characteristic points, and updating the leaf node data.
And S702, splitting the nodes, namely dividing the current region into four sub-regions if the space density (number of unit Area) of the characteristic points in the nodes (N/Area) > ρ1, distributing the characteristic points to the corresponding sub-regions, and carrying out recursion until the characteristic points are distributed to proper leaf nodes.
The region reaches a minimum size limit s=w/(2 n')×H/(2n'), and the division is stopped when n' ∈ [6,8 ].
S703, locally reconstructing, namely if one father node P has only leaf nodes, and the spatial density (N '/Area') of each leaf node characteristic point is less than ρ3, then completing four leaf nodes of the father node P, wherein N 'is the number of leaf node characteristic points, area' is the Area of the leaf node, ρ3 is a set leaf node spatial density threshold value which is less than or equal to ρ1, and the specific value is obtained by adjusting according to scenes and experience. ;
Extracting characteristic points from the complemented leaf node area, if the quantity of the characteristic points of each leaf node of the father node P after the complementation does not reach a set threshold T2, enabling the father node P to be a leaf node and reconstructing the local part upwards;
The upward local reconstruction specifically includes determining whether the parent node P1 of P has only leaf nodes, if so, executing step S703 on P1.
Further, since the increase and decrease branches of the quadtree are bulky and complicated in actual operation, in this embodiment, by setting the number of quadtree layers at the beginning of the step of adjusting the branches in the new quadtree, the corresponding leaf node whose leaf node feature point number is smaller than the set threshold T2 is assigned to-1, which indicates the discarded area, and the discarded node does not play a role in merging.
S8, sliding the window, namely removing the image frame at the oldest moment in the sliding window, and adding a new image frame, as shown in FIG. 6.
The method processes the sequence data by dynamically maintaining a local data sliding window with a fixed size, and is characterized in that the sliding window discards old data and brings new data into the same way as a sliding viewport along with the advancement of the sequence, thereby realizing real-time analysis and calculation of continuous data. The method skillfully balances time efficiency and space efficiency, avoids the cost of processing the whole data set, can capture the local characteristics of the data, and is particularly suitable for scenes such as time sequence analysis, real-time signal processing, stream data mining and the like.
Example 2:
As shown in fig. 7, in this embodiment, a cloud-edge collaborative visual slam system based on feature point prediction is provided, where the system includes an edge end, an edge server and a cloud server that communicate with each other through a communication module;
The edge end applies the cloud edge collaborative visual slam method based on the feature point prediction to obtain image feature point distribution, and builds and stores a sub-map;
The edge server is used for restoring the data uploaded by the edge end according to the slam visual odometer to obtain a real-time path and roadside points, and carrying out optimization processing on the real-time path and roadside points through the slam rear end to obtain an edge map;
the cloud server is used for merging the edge maps uploaded by the edge servers as global maps, and carrying out optimization processing and storage on the global maps.
It should be noted that, the system provided in the foregoing embodiment is only exemplified by the division of the foregoing functional modules, in practical application, the foregoing functional allocation may be performed by different functional modules according to needs, that is, the internal structure is divided into different functional modules to complete all or part of the functions described above, and the system is a cloud edge collaborative visual slam method based on feature point prediction applied to the foregoing embodiment.
Example 3:
In the embodiment, the cloud edge cooperative visual slam device based on the feature point prediction comprises a memory and at least one processor, wherein instructions are stored in the memory, the memory is connected with the at least one processor through a line, and the at least one processor calls the instructions in the memory so that the cloud edge cooperative visual slam device based on the feature point prediction can execute the cloud edge cooperative visual slam method based on the feature point prediction.
Example 4:
In this embodiment, a storage medium is provided, where a program is stored, and when the program is executed by a processor, a cloud edge cooperative visual slam method based on feature point prediction is implemented, specifically:
adding images of a plurality of initial continuous frames into a sliding window;
extracting feature points from all pixels of the image of a plurality of frames which are initially continuous;
establishing a quadtree for the feature points extracted from each frame;
The quadtree is saved as prior information of the corresponding frame image;
Combining prior information corresponding to the images of a plurality of initial continuous frames, and establishing a new quadtree as global prior information;
predicting the feature point distribution of the next frame of image by using the global priori information and adopting a multi-step Markov probability model;
the branches in the new quadtree are adjusted according to the actual characteristic point distribution of the next frame of image;
And adding the image of the next frame into the sliding window, removing the image of the earliest frame and the corresponding prior information, and predicting the characteristic point distribution of the image of the next frame by utilizing the image in the sliding window and the corresponding prior information.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of techniques known in the art, discrete logic circuits with logic gates for implementing logic functions on data signals, application specific integrated circuits with appropriate combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
The above examples are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above examples, and any other changes, modifications, substitutions, combinations, and simplifications that do not depart from the spirit and principle of the present invention should be made in the equivalent manner, and the embodiments are included in the protection scope of the present invention.

Claims (9)

1. The cloud edge cooperative visual slam method based on feature point prediction is characterized by comprising the following steps of:
adding a plurality of initial continuous frame images into a sliding window;
extracting feature points of all pixels of a plurality of initial continuous frames of images;
establishing a quadtree for the feature points extracted from each frame;
The quadtree is saved as prior information of the corresponding frame image;
Combining prior information corresponding to a plurality of initial continuous frame images, and establishing a new quadtree as global prior information;
predicting the feature point distribution of the next frame of image by using the global priori information and adopting a multi-step Markov probability model;
the branches in the new quadtree are adjusted with the actual feature point distribution of the next frame image, the method specifically comprises the following steps:
extracting feature points, namely extracting feature points from the corresponding images of the leaf nodes, counting the number of the feature points, and updating the leaf node data;
Dividing a current region into four sub-regions, distributing the feature points to the corresponding sub-regions, and performing recursion until the feature points are distributed to proper leaf nodes, wherein N is the number of the feature points of the current node, ρ1 is a set space density threshold, area= (x max-xmin)×(ymax-ymin) is the Area of the current node, x max、xmin、ymax and y min are the maximum and minimum values of the abscissa and the ordinate of the current region, and when the region reaches the minimum size limit s=W/(2 n')×H/(2n'), N' epsilon [6,8], dividing is stopped, and W and H are the width and height of an image frame;
If one father node P has only leaf nodes and the space density (N '/Area') < ρ3 of each leaf node characteristic point, then completing four leaf nodes of the father node P, wherein N 'is the number of leaf node characteristic points, area' is the Area of the leaf node, ρ3 is the set space density threshold of the leaf node;
Extracting characteristic points from the complemented leaf node area, if the number of the characteristic points of each leaf node of the complemented father node P does not reach a set threshold T2, enabling the father node P to be a leaf node and reconstructing the leaf node locally upwards;
Judging whether the father node P1 of the P has only leaf nodes, if so, executing the partial reconstruction on the P1;
And adding the next frame image into the sliding window, removing the earliest frame image and the corresponding prior information, and predicting the characteristic point distribution of the next frame image by utilizing the image in the sliding window and the corresponding prior information.
2. The cloud edge collaborative visual slam method based on feature point prediction according to claim 1, wherein the feature point extracted from each frame is set up into a quadtree, and the method comprises the following steps:
creating a root node representing the entire image area;
Starting from a root node, checking whether the current node is divided, and if the current node is not divided and the spatial density (N/Area) of feature points in the node is more than ρ1, dividing the current region into four sub-regions and distributing the feature points to corresponding sub-regions, wherein N is the number of the feature points of the current node, ρ1 is a set spatial density threshold, area= (x max-xmin)×(ymax-ymin) is the Area of the current node, and x max、xmin、ymax and y min are the maximum and minimum values of the abscissa and the ordinate of the current region;
Dividing by the node center (xc, yc):
;
Stopping dividing when the area reaches the minimum size limit s=W/(2 n')×H/(2n'), n' E [6,8], wherein W and H are the width and height of the image frame, and Q 1-Q4 is four sub-areas obtained by dividing;
The process is recursively performed until feature points are assigned to the appropriate leaf nodes.
3. The cloud edge collaborative visual slam method based on feature point prediction according to claim 1, wherein the new quadtree is built as global prior information, wherein the number of node feature pointsWherein w 1,w2,…,wn is the weight of the images of different time frames, the subscript represents the sequence number, the weight value is determined by the time sequence of the image frames, the new image frames have larger weight, and vice versa, V 1,V2,…,Vn is the number of the characteristic points of the images of different time frames, and the subscript represents the sequence number;
and removing leaf nodes of the space density (N/Area) < ρ2 of the feature points, wherein N is the number of the feature points of the current node, ρ2 is a preset space density threshold, area= (x max-xmin)×(ymax-ymin) is the Area of the current node, and x max、xmin、ymax and y min are the maximum and minimum values of the abscissa and the ordinate of the current region.
4. The cloud edge collaborative visual slam method based on feature point prediction according to claim 1, wherein the feature point distribution of the next frame of image is predicted by using a multi-step markov probability model by using global prior information specifically comprises:
Defining a multi-step Markov probability model P(Xn+1|Xn,Xn-1,...,X0)=P(Xn+1|Xn,Xn-1,...,Xn-k+1),P(Xn+1|Xn,Xn-1,...,X0) to represent the feature point distribution state X n+1 of the current image, related to all distribution states X n,Xn-1,...,X0 preceding X n+1, P (X n+1|Xn,Xn-1,...,Xn-k+1) to limit X n+1 to only a limited k states X n,Xn-1,...,Xn-k+1 preceding X n+1;
predicting the distribution state of image characteristic points of n+1 through X n,Xn-1,...,Xn-k+1 :
;
Wherein E ∙ is the maximum likelihood.
5. The cloud edge collaborative visual slam method based on feature point prediction according to claim 4, wherein X n+1 is related to the finite k=3 states X n,Xn-1,...,Xn-2 before X n+1 to obtain a third order Markov probability model P(Xn+1|Xn,Xn-1,...,X0)=P(Xn+1| Xn,Xn-1,...,Xn-2);
The state transition of the characteristic point distribution between the front image frame and the rear image frame in a short time satisfies the linear relation that X n+1=aXn +bXn-1+cXn-2+...+ X1 +epsilon, epsilon is Gaussian noise, and then:
;
wherein a, b and c are linear coefficients.
6. The cloud edge collaborative visual slam method based on feature point prediction according to claim 1, wherein the number of quadtree layers is set at the beginning of the step of adjusting branches in the new quadtree, and corresponding leaf nodes with leaf node feature point numbers smaller than a set threshold T2 are assigned to-1, indicating discarded regions, and discarded nodes do not play a role in merging.
7. The cloud edge collaborative visual slam system based on the feature point prediction is characterized by comprising an edge end, an edge server and a cloud server which are communicated with each other through a communication module;
The edge end applies the cloud edge collaborative visual slam method based on the characteristic point prediction according to any one of claims 1-6 to obtain image characteristic point distribution, and builds and stores a sub-map;
The edge server is used for restoring the data uploaded by the edge end according to the slam visual odometer to obtain a real-time path and roadside points, and carrying out optimization processing on the real-time path and roadside points through the slam rear end to obtain an edge map;
the cloud server is used for merging the edge maps uploaded by the edge servers as global maps, and carrying out optimization processing and storage on the global maps.
8. The cloud edge cooperative visual slam device based on the feature point prediction is characterized by comprising a memory and at least one processor, wherein the memory is stored with instructions, the memory is connected with the at least one processor through a line, and the at least one processor calls the instructions in the memory to enable the cloud edge cooperative visual slam device based on the feature point prediction to execute the cloud edge cooperative visual slam method based on the feature point prediction according to any one of claims 1-6.
9. A storage medium storing a program, wherein the program when executed by a processor implements a cloud edge collaborative visual slam method based on feature point prediction as set forth in any one of claims 1-6.
CN202511194614.2A 2025-08-26 2025-08-26 Cloud-edge cooperative visual slam method and system based on feature point prediction Active CN120707642B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202511194614.2A CN120707642B (en) 2025-08-26 2025-08-26 Cloud-edge cooperative visual slam method and system based on feature point prediction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202511194614.2A CN120707642B (en) 2025-08-26 2025-08-26 Cloud-edge cooperative visual slam method and system based on feature point prediction

Publications (2)

Publication Number Publication Date
CN120707642A CN120707642A (en) 2025-09-26
CN120707642B true CN120707642B (en) 2025-11-14

Family

ID=97122264

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202511194614.2A Active CN120707642B (en) 2025-08-26 2025-08-26 Cloud-edge cooperative visual slam method and system based on feature point prediction

Country Status (1)

Country Link
CN (1) CN120707642B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022011898A1 (en) * 2020-07-17 2022-01-20 歌尔股份有限公司 Three-dimensional reconstruction method, apparatus, and device, and computer readable storage medium
CN119206106A (en) * 2024-09-04 2024-12-27 西安电子科技大学 A three-dimensional mapping method and system

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6845130B1 (en) * 2000-10-12 2005-01-18 Lucent Technologies Inc. Motion estimation and compensation for video compression
CN119124138A (en) * 2024-10-18 2024-12-13 南京工程学院 A SLAM method based on static feature point extraction

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022011898A1 (en) * 2020-07-17 2022-01-20 歌尔股份有限公司 Three-dimensional reconstruction method, apparatus, and device, and computer readable storage medium
CN119206106A (en) * 2024-09-04 2024-12-27 西安电子科技大学 A three-dimensional mapping method and system

Also Published As

Publication number Publication date
CN120707642A (en) 2025-09-26

Similar Documents

Publication Publication Date Title
US9454851B2 (en) Efficient approach to estimate disparity map
CN115782867A (en) Track collision risk assessment method and device, electronic equipment and storage medium
KR102340988B1 (en) Method and Apparatus for Detecting Objects from High Resolution Image
CN114771572A (en) Automatic driving track prediction method, device, equipment and storage medium
CN112287803B (en) Edge collaborative target detection method and device based on RoI coding
CN113780578B (en) Model training method, device, electronic equipment and readable storage medium
CN116168132B (en) Street scene reconstruction model acquisition and street scene reconstruction methods, devices, equipment and media
CN115272407A (en) A method and device for determining the motion trajectory of a target object
CN110008789A (en) Multiclass object detection and knowledge method for distinguishing, equipment and computer readable storage medium
CN118470581A (en) Unmanned aerial vehicle cross-modal semantic communication method and system for target detection task
CN113496138B (en) Method, device, computer equipment and storage medium for generating dense point cloud data
WO2024160219A1 (en) Model quantization method and apparatus
Peng et al. Dynamic visual SLAM and MEC technologies for B5G: a comprehensive review
CN115866229A (en) Viewpoint conversion method, device, device and medium for multi-viewpoint images
CN120707642B (en) Cloud-edge cooperative visual slam method and system based on feature point prediction
CN116309316A (en) Method and device for detecting vehicle passable area
CN116309634A (en) Image processing method, device, terminal equipment, storage medium and program product
Nie et al. A deep reinforcement learning assisted task offloading and resource allocation approach towards self-driving object detection
Choi et al. A survey on mobile edge computing for Deep learning
CN120783318A (en) Target detection method, device, equipment, medium and product based on aerial view
WO2021237727A1 (en) Method and apparatus of image processing
WO2023272662A1 (en) Adaptive object detection
CN115063467B (en) Method and device for estimating high-resolution image parallax in underground coal mine
CN117636307A (en) Target detection methods, equipment and autonomous vehicles based on semantic information
CN117057840A (en) Cloud resource cost optimization method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant