CN118172507B - Digital twinning-based three-dimensional reconstruction method and system for fusion of transformer substation scenes - Google Patents

Digital twinning-based three-dimensional reconstruction method and system for fusion of transformer substation scenes Download PDF

Info

Publication number
CN118172507B
CN118172507B CN202410587044.2A CN202410587044A CN118172507B CN 118172507 B CN118172507 B CN 118172507B CN 202410587044 A CN202410587044 A CN 202410587044A CN 118172507 B CN118172507 B CN 118172507B
Authority
CN
China
Prior art keywords
scene
substation
image
model
fusion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410587044.2A
Other languages
Chinese (zh)
Other versions
CN118172507A (en
Inventor
赵相涛
梁同然
颜潇
王晓婷
王磊
王子薇
赵淳
宋杰
施光明
赵慧惠
吴磊
东海
杨智文
李晓龙
苏文
王艳春
王钰泽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jining Power Supply Co of State Grid Shandong Electric Power Co Ltd
Original Assignee
Jining Power Supply Co of State Grid Shandong Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jining Power Supply Co of State Grid Shandong Electric Power Co Ltd filed Critical Jining Power Supply Co of State Grid Shandong Electric Power Co Ltd
Priority to CN202410587044.2A priority Critical patent/CN118172507B/en
Publication of CN118172507A publication Critical patent/CN118172507A/en
Application granted granted Critical
Publication of CN118172507B publication Critical patent/CN118172507B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00—Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Landscapes

  • Image Processing (AREA)

Abstract

The disclosure provides a digital twinning-based substation scene fusion three-dimensional reconstruction method and system, which relate to the technical field of digital twinning three-dimensional reconstruction and comprise the following steps: acquiring multi-view live-action aerial images of a target transformer substation, and preprocessing; extracting scene characteristics in the multi-view live-action aerial image, and constructing an initial scene three-dimensional model; taking an image containing the entity equipment as a candidate image, inputting the candidate image into an image position and posture prediction model, and outputting a position and posture result; determining the spatial range of the entity equipment according to the position and posture result, and carrying out superposition fusion on the entity equipment and the scene three-dimensional model based on projection transformation to construct a final visual substation scene fusion three-dimensional model; dynamic changes in the transformer substation are obtained in real time, real-time reconstruction is carried out through the BIM-NeRF model, and real-time reconstruction results are transmitted to the visual transformer substation scene fusion three-dimensional model, so that real-time update of the transformer substation entity equipment is realized.

Description

Digital twinning-based three-dimensional reconstruction method and system for fusion of transformer substation scenes
Technical Field
The disclosure relates to the technical field of digital twin three-dimensional reconstruction, in particular to a substation scene fusion three-dimensional reconstruction method and system based on digital twin.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
In recent years, with the rapid development of power industry and industrial technology, large-scale power equipment such as transformers, wind driven generators and the like in a transformer substation tend to be complicated and intelligent, and the probability of equipment failure and performance degradation is greatly increased. The transformer substation is used as an important link of various energy transmission and consumption of an electric power supply network, the topological structure of the transformer substation is dynamically changed, the transformer substation is evolved into a huge maintenance system with complex structure, various devices and complex technology, the transformer substation has typical nonlinear random characteristics and multi-scale dynamic characteristics, and the conventional operation and maintenance management mode is difficult to meet the requirements of planning, design, monitoring, analysis and operation optimization of the transformer substation.
Therefore, on-line monitoring of the operation state of the power equipment, fault diagnosis, degradation, life prediction, and the like are hot spots of current research. Meanwhile, the development of the Internet of things technology and the sensing technology multiplies the data volume monitored by the power equipment, and the system has the characteristics of multi-source isomerism, high complexity, large information volume and the like, and provides higher requirements for the fine management of the power industry.
The development of a Digital Twin (DT) technology brings convenience to intelligent management of a power system, the technologies of deep learning, three-dimensional reconstruction, dynamic rendering, cloud edge cooperation and the like under a digital twin frame are mature, the digital twin technology is deeply combined in each link of transformer substation production and operation and maintenance, the state accurate perception, the fault intelligent analysis, the trend accurate prediction and the operation and maintenance scientific decision of the transformer substation are pushed, and a new operation and maintenance management mode of the digital intelligent transformer substation is opened up, so that higher requirements are also provided for a method for fusion reconstruction of a digital twin three-dimensional scene of the transformer substation.
In reconstruction of a digital twin three-dimensional scene of a transformer substation at present, data feature extraction or data fusion is mainly focused on the features of entity equipment in an acquired image, including textures, edges or other features, and the position and the gesture of the entity equipment are lack of attention when the image is acquired; most data acquisition is aerial, but because the aerial process has dead angles, the influence characteristic matching has errors and the like, a three-dimensional model reconstructed by oblique photography can possibly express a flat surface as a curved surface, and a scene fusion method is mostly carried out on the premise of knowing relevant parameters such as image positions, postures and the like, only later superposition is realized, so that the accuracy of acquiring the data characteristics in the earlier stage is lower, the constructed three-dimensional reconstruction model is influenced, and the robust requirement cannot be met.
Disclosure of Invention
In order to solve the problems, the present disclosure provides a three-dimensional reconstruction method and system for fusing a transformer substation scene based on digital twinning, which are implemented by predicting the position and the gesture of an aerial image, obtaining the position and the gesture parameters of the fused image, mapping the position and the gesture parameters into the actual position of an entity, and fusing and superposing the actual position and the actual three-dimensional scene to realize the three-dimensional reconstruction of the transformer substation scene fusion.
According to some embodiments, the present disclosure employs the following technical solutions:
A transformer substation scene fusion three-dimensional reconstruction method based on digital twinning comprises the following steps:
Acquiring multi-view live-action aerial images of a target transformer substation, and preprocessing;
extracting scene characteristics in the multi-view live-action aerial image, and constructing an initial scene three-dimensional model;
taking an image containing the entity equipment as a candidate image, inputting the candidate image into an image position and posture prediction model, and outputting a position and posture result;
Determining the spatial range of the entity equipment according to the position and posture result, and carrying out superposition fusion on the entity equipment and the scene three-dimensional model based on projection transformation to construct a final visual substation scene fusion three-dimensional model;
dynamic changes in the transformer substation are obtained in real time, real-time reconstruction is carried out through the BIM model, and real-time reconstruction results are transmitted to the visual transformer substation scene fusion three-dimensional model, so that real-time update of the transformer substation entity equipment is realized.
According to some embodiments, the present disclosure employs the following technical solutions:
Substation scene fusion three-dimensional reconstruction system based on digital twinning comprises:
The data acquisition module is used for acquiring multi-view live-action aerial images of the target transformer substation and preprocessing the multi-view live-action aerial images;
the initial scene model construction module is used for extracting scene characteristics in the multi-view live-action images and constructing an initial scene three-dimensional model;
the positioning module is used for taking the image containing the entity equipment as a candidate image, inputting the candidate image into the image position and posture prediction model and outputting a position and posture result;
The scene fusion module is used for determining the spatial range of the entity equipment according to the position and posture result, carrying out superposition fusion on the entity equipment and the scene three-dimensional model based on projection transformation, and constructing a final visual substation scene fusion three-dimensional model;
The updating module is used for acquiring dynamic changes in the transformer substation in real time, carrying out real-time reconstruction through the BIM model, and transmitting a real-time reconstruction result to the visual transformer substation scene fusion three-dimensional model to realize real-time updating of the transformer substation entity equipment.
According to some embodiments, the present disclosure employs the following technical solutions:
a non-transitory computer readable storage medium for storing computer instructions which, when executed by a processor, implement the digital twinning-based substation scene fusion three-dimensional reconstruction method.
According to some embodiments, the present disclosure employs the following technical solutions:
an electronic device, comprising: a processor, a memory, and a computer program; the processor is connected with the memory, the computer program is stored in the memory, and when the electronic equipment runs, the processor executes the computer program stored in the memory so that the electronic equipment executes the three-dimensional reconstruction method for realizing the digital twin-based transformer substation scene fusion.
Compared with the prior art, the beneficial effects of the present disclosure are:
According to the digital twin-based substation scene fusion three-dimensional reconstruction method, an image position and posture prediction model is constructed, end-to-end image pose estimation is carried out by utilizing an original image of unmanned aerial vehicle oblique photography, the model is optimized and improved from three angles of a network structure, a loss function and a regressor of the model, and the position and the posture of any image in a target range can be obtained.
According to the digital twinning-based substation scene fusion three-dimensional reconstruction method, the texture and the structure of the model can be clearer and clearer through fusion of the live-action three-dimensional scene and the original image. In addition, the mode of overlapping the images compensates for the problem of reduced reality of the scene model.
The digital twinning-based substation scene fusion three-dimensional reconstruction method combines BIM modeling, can realize high-precision scene reconstruction and real-time update of an electric power equipment body, auxiliary operation and maintenance equipment and abnormal scenes (such as illegal intrusion, oil leakage and the like) in actual operation and maintenance, and provides a new thought for a future power grid scene digital twinning real-time modeling method.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure, illustrate and explain the exemplary embodiments of the disclosure and together with the description serve to explain the disclosure, and do not constitute an undue limitation on the disclosure.
FIG. 1 is a schematic flow diagram of a method in accordance with an embodiment of the present disclosure;
Fig. 2 is a model block diagram of an embodiment of the present disclosure.
Detailed Description
The disclosure is further described below with reference to the drawings and examples.
It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments in accordance with the present disclosure. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.
Example 1
In one embodiment of the present disclosure, a method for three-dimensional reconstruction of a substation scene fusion based on digital twinning is provided, including the following steps:
Step one: acquiring multi-view live-action aerial images of a target transformer substation, and preprocessing;
step two: extracting scene characteristics in the multi-view live-action aerial image, and constructing an initial scene three-dimensional model;
step three: taking the image containing the entity equipment as a candidate image, inputting the candidate image into an image position and posture prediction model, and outputting a position and posture result of the entity equipment;
step four: determining a spatial range of the entity equipment according to the position and posture result of the entity equipment, and carrying out superposition fusion on the entity equipment and the scene three-dimensional model based on projection transformation to construct a final visual substation scene fusion three-dimensional model;
Step five: dynamic changes in the transformer substation are obtained in real time, real-time reconstruction is carried out through the BIM-NeRF model, and real-time reconstruction results are transmitted to the visual transformer substation scene fusion three-dimensional model, so that real-time update of the transformer substation entity equipment is realized.
As an embodiment, the digital twinning-based substation scene fusion three-dimensional reconstruction method disclosed by the invention predicts the spatial position and the gesture of information in an image through image data, realizes the superposition of a 2D image and a scene and an entity in a 3D space, and realizes the fusion visualization of any image data in a target area, and the specific implementation process is as follows:
1) Acquiring multi-view live-action aerial images of target transformer substation
Specifically, utilize unmanned aerial vehicle to carry single-view high-resolution camera to take a photograph to the transformer substation area to carry out image acquisition in the form of single image, obtain live-action aerial image, and cut out and cut apart preprocessing operation to the aerial image that obtains, include:
When the unmanned aerial vehicle acquires the image of the target area, a single-view high-resolution camera is firstly carried, shooting is carried out in the time with good illumination conditions, the acquired image has the width and the high resolution of 5456 multiplied by 3632, and the horizontal resolution and the vertical resolution of the image are both 350dpi. The requirements of high resolution and high definition of the image are met on the whole so as to ensure the resolution and definition of the image; and secondly, planning a route according to the ground feature distribution condition of the target area, and shooting and acquiring images.
The obtained image data comprises a partial image of Zhang Baohan whole entity equipment and an integral image of a transformer substation target area, each image shooting area comprises an orthographic image and a plurality of inclined images with different angles, and the overlapping degree of the images can meet the reconstruction requirement.
Further, preprocessing operation is carried out on the acquired image data, each image is cut and segmented, useless boundaries are eliminated, and useless backgrounds are reduced; and the gray level conversion of the image is carried out, and the size and the dimension of the image are unified.
Step 2: constructing an initial scene three-dimensional model
Specifically, the whole image of the substation target area is utilized to extract the actual scene of the substation, the substation actual scene data is utilized to construct an initial scene three-dimensional model by using an open-source three-dimensional GIS visualization platform, and a Cesium rendering frame is selected to realize three-dimensional scene visualization.
Further, extracting the actual scene of the transformer substation includes:
Analyzing the whole image of the target area of the transformer substation, detecting the size of the area occupied by the entity equipment in the transformer substation according to the gray level characteristic of the whole image of the target area of the transformer substation, setting the window length of one-dimensional median filtering, and extracting the surrounding actual scene of the transformer substation by using a bidirectional one-dimensional median filtering method, specifically:
Calculating and sequencing the pixel sum of each column of the whole image to obtain the maximum front N columns of the pixel sum; taking the pixel point (x 0,y0) with the largest pixel value in the previous N columns as a center, intercepting a rectangular window local area with the size of m multiplied by m in the whole image, wherein the rectangular window local area is shown in the following formula:
im_temp(i ,j)＝f(h ,b)，(h ,b)∈W(i ,j)
where im_temp (i, j) represents a rectangular window partial region, f (h, b) represents a gray value of the input whole image at the (h, b) position, and W (i, j) represents a rectangular window centered at (x 0, y 0).
Calculating the mean value mu and standard deviation sigma of all pixels in a rectangular window, setting a threshold value TH=mu+a×sigma, and carrying out binarization to obtain a binarized partial image, wherein a and m are coefficient constants, and the binarization expression is shown as the following formula:
where im_bw (i, j) represents the gray value of the binarized image at (i, j); performing morphological open operation on the binarized partial image by using a structural element se:
Where im_open represents an image after an open operation, and ° represents an open operator in morphology.
Wherein the structural element is a multidimensional matrix:
And carrying out connected domain processing on the partial images after the open operation, wherein the length or width of the largest Size in the connected domain of the partial areas is Size n, N is more than or equal to 1 and less than or equal to N, and the largest Size in all N partial areas is Size max of the solid equipment area in the whole image:
Wherein W k、Lk represents the width and length of the kth connected domain in the region, represents the horizontal and vertical coordinate sets of the kth connected domain in the region, size n represents the Size of the entity device in the whole local region, { Wn }, { Ln } represents the width set and length set of all connected domains in the whole region, size max is the estimated maximum Size of the entity device in the whole image, and { Size } represents the Size set of the entity device; the window length of one-dimensional median filtering is set by amplifying the maximum Size max of the estimated entity equipment as a whole: l=b×size max +1, b being the magnification coefficient.
Further, extracting an image background by using a median filtering method, setting the window length of one-dimensional median filtering in the horizontal direction as L, expanding (L-1)/2 pixel positions outwards at the left edge position and the right edge position of the integral image im_org, wherein the expanded pixel value is obtained by translating the corresponding position of the integral image im_org, and the expanded image is marked as im_ extend1:
Where im_ extend 1 (i, j) represents the gray value of the extension image at (i, j), X, Y represents the number of pixels of the rows and columns of the overall image, respectively, and im_org (x, y) represents the gray value of the overall image at (x, y).
The method comprises the steps of obtaining a background image of a transformer substation through the method, then carrying out model construction and rendering on the background image in an open-source three-dimensional GIS visual platform, triggering a requestAnimationFrame function of cyclic rendering through setting useDefaultRenderLoop attributes to be true, and organizing rendering tasks through a render function of scenes, so that loading of all elements such as topography, models, vector primitives and the like in continuous scenes is achieved, and an initial scene three-dimensional model is obtained.
Step 3: physical device position and posture result output
And after the initial scene three-dimensional model is obtained, spatial positioning is carried out on the entity equipment in the whole image, superposition rendering with the scene model is realized, and the entity equipment and the scene model are fused into a final substation visual three-dimensional model. The method specifically comprises the following steps:
screening the image containing the entity equipment, taking the image containing the entity equipment as a candidate image, inputting the candidate image into an image position and posture prediction model, and outputting a position and posture result;
Further, the image position and posture prediction model is an improved GoogLeNet model, the learned features are migrated into a regression network, an affine regression is used for replacing a softmax classifier of the GoogLeNet model, and a full connection layer is added in front of an output layer to form a positioning feature vector layer. The improved GoogLeNet network propagates with a stack of multiple Inception modules, including multiple 1 x1 convolution kernels and a max pooling layer, and extracts low level features at the time two auxiliary regressors are set.
The image position and posture prediction model is based on CNN large data quantity classification network training, the learned characteristics are migrated into a regression network, single RGB image regression pose output is realized, the specific structure is shown in figure 2, a solid line frame network layer in the figure is a structure of an original GoogLeNet model, and a dotted line frame network layer is an improvement of a GoogLeNet model. The model input is an image of 224 x 224 pixels and the output is 7 features. The softmax classifier of GoogLeNet models is replaced by an affine regressor, and a full-connection layer is added to an output layer to form a positioning feature vector for realizing feature generalization. The whole model has 23 layers in total, and is propagated by using the stacking of 9 Inception modules, and 2 auxiliary regressors are arranged during training to give consideration to the characteristics of the lower layers, so as to help the convergence of the network. The Inception module of the GoogLeNet model adopts a 1 multiplied by 1 convolution kernel to realize dimension reduction, so that the calculated amount is reduced, and meanwhile, the information transmission connection is sparse. And a maximum pooling layer is arranged, and the characteristic images fused with different receptive fields can be obtained through a Inception module.
The 7 features output by the modified GoogLeNet model contain three position features (x, y, z coordinate values) and 4 pose features (quaternions of three rotation angles) of the image. The output features are denoted by c, p and q are the position and attitude features, respectively, and the obtained output sequence is c= [ p, q ], the model initially employs a loss function based on euclidean norms, and a super parameter β is used to balance the errors of the position and attitude features, which are highly coupled because they are regressed from the same model weights. This results in the super parameter β being a very flexible parameter that requires constant de-experimental adjustment according to the training effect of the model, so the present disclosure improves the loss function, automatically learns the super parameter based on covariance, as follows:
In the method, in the process of the invention, Representing the overall error of the device,AndThe position and attitude errors of the L1 norm respectively,、Parameters of the pose and position errors, respectively, that need to be learned.
And carrying out the model prediction on each candidate image containing the entity equipment, and outputting the position and posture parameters of each entity equipment.
Step 4: constructing final visual substation scene fusion three-dimensional model
After the position and posture coordinate parameters of the entity equipment are obtained, the position and posture coordinate parameters are superimposed into an initial scene model, the space range of the entity equipment is determined according to the position and posture results, the entity equipment and the scene three-dimensional model are superimposed and fused based on projection transformation, and a final visual transformer substation scene fusion three-dimensional model is constructed, specifically:
the idea of realizing perspective projection based on OpenGL is to firstly realize perspective transformation, transform a view Jing Ti of a quadrangular frustum into a cuboid, and then combine orthographic projection to realize mapping, so as to complete the perspective projection process. The perspective transformation is completed based on similar triangles, the coordinates of the acquired position space points of the entity equipment are (x, y, z), the coordinates of the position space points on the projection surface are (x p,yp -g), and the similar triangles can be used for obtaining the formula:
When the image superposition based on perspective projection is realized, not only translation and scaling are restored, but also the relationship of similar triangles of the perspective transformation is restored. Translation may be restored by viewpoint position, scaling may be restored by the size of the image and camera focal length, and a similar triangle needs to be determined according to the distance of the near-far plane from the viewpoint. Comprising the following steps: and restoring perspective projection, and determining the position of the viewpoint, the size of the far-near plane and the position of the central point of the far-near plane, namely constructing the view cone of the perspective projection, wherein the view cone can be realized in an existing mode.
The construction parameters include perspective view cone, view cone origin coordinates, rotation quaternion. The view cone origin coordinates, i.e. the viewpoint position, the rotation quaternion is used to define the camera coordinate system of the view cone, i.e. the directions of the up, direct, right axes of the camera. The perspective view object defines a perspective projection view of the camera, including fov (angle of view), aspectRatio (aspect ratio), near (near planar distance), far (far planar distance).
After the view cone is constructed in Cesium, the coordinates of four corner points of the far plane can be calculated by using the parameters of the view cone. And constructing a geometric object based on the calculated corner coordinates and endowing the geometric object with a picture material, so that superposition of entity equipment and a scene in the perspective projection image can be realized.
Further, as a three-dimensional map engine Cesium is equivalent to creating a large geometry library, cesium, in addition to normally loading two-dimensional data and creating two-dimensional geometry, also assigns a ground-contacting attribute to a part of the geometry, encapsulating the ground-contacting geometry (GroundPrimitive). These interfaces create convenient conditions for the fusion of two-dimensional images with three-dimensional scenes.
Step 5: dynamic changes in the transformer substation are obtained in real time, real-time reconstruction is carried out through the BIM model, and real-time reconstruction results are transmitted to the visual transformer substation scene fusion three-dimensional model, so that real-time update of the transformer substation entity equipment is realized.
Further, the BIM model can realize rapid real-time reconstruction of small equipment, the BIM model comprises the steps of carrying out BIM (building information model) modeling, accelerating scene reconstruction time, and finally combining with BIM modeling results, and transmitting updated results back to the visual transformer substation scene fusion three-dimensional model platform.
The method can realize high-precision quick reconstruction and real-time update of the electric power equipment body, auxiliary operation and maintenance equipment and abnormal scenes (such as illegal intrusion, oil leakage and the like) in actual operation and maintenance, and can realize quick reconstruction aiming at small parts, does not need to reconstruct a three-dimensional model of the whole transformer substation, and can realize fusion real-time update.
Example 2
In one embodiment of the present disclosure, a digital twinning-based three-dimensional reconstruction system for fusion of transformer substation scenes is provided, including:
The data acquisition module is used for acquiring multi-view live-action aerial images of the target transformer substation and preprocessing the multi-view live-action aerial images;
the initial scene model construction module is used for extracting scene characteristics in the multi-view live-action images and constructing an initial scene three-dimensional model;
the positioning module is used for taking the image containing the entity equipment as a candidate image, inputting the candidate image into the image position and posture prediction model and outputting a position and posture result;
The scene fusion module is used for determining the spatial range of the entity equipment according to the position and posture result, carrying out superposition fusion on the entity equipment and the scene three-dimensional model based on projection transformation, and constructing a final visual substation scene fusion three-dimensional model;
The updating module is used for acquiring dynamic changes in the transformer substation in real time, carrying out real-time reconstruction through the BIM model, and transmitting a real-time reconstruction result to the visual transformer substation scene fusion three-dimensional model to realize real-time updating of the transformer substation entity equipment.
Example 3
In one embodiment of the disclosure, a non-transitory computer readable storage medium is provided, where the non-transitory computer readable storage medium is configured to store computer instructions, and when the computer instructions are executed by a processor, the digital twinning-based substation scene fusion three-dimensional reconstruction method is implemented.
Example 4
In one embodiment of the present disclosure, there is provided an electronic device including: a processor, a memory, and a computer program; the processor is connected with the memory, the computer program is stored in the memory, and when the electronic equipment runs, the processor executes the computer program stored in the memory so that the electronic equipment executes the three-dimensional reconstruction method for realizing the digital twin-based transformer substation scene fusion.
The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the specific embodiments of the present disclosure have been described above with reference to the drawings, it should be understood that the present disclosure is not limited to the embodiments, and that various modifications and changes can be made by one skilled in the art without inventive effort on the basis of the technical solutions of the present disclosure while remaining within the scope of the present disclosure.

Claims (8)

1.基于数字孪生的变电站场景融合三维重建方法，其特征在于，包括：1. A method for 3D reconstruction of substation scenes based on digital twins, characterized by comprising: 获取目标变电站的多视角实景航拍影像，并进行预处理；Acquire multi-view real-life aerial images of the target substation and perform preprocessing; 提取多视角实景航拍影像中的场景特征，构建初始场景三维模型；Extract scene features from multi-view real-scene aerial images and construct an initial scene 3D model; 将包含实体设备的图像作为候选图像，输入至图像位置姿态预测模型中，输出位置姿态结果；图像位置姿态预测模型为改进的GoogLeNet网络，将学习的特征迁移到回归网络中，使用affine回归器代替GoogLeNet模型的softmax分类器，并在输出层前增加全连接层，形成定位特征向量层；The image containing the physical device is used as a candidate image and input into the image position and posture prediction model to output the position and posture result. The image position and posture prediction model is an improved GoogLeNet network. The learned features are transferred to the regression network, and the affine regressor is used to replace the softmax classifier of the GoogLeNet model. A fully connected layer is added before the output layer to form a positioning feature vector layer. 改进的GoogLeNet网络利用多个Inception模块的堆叠进行传播，并在设置两个辅助回归器进行提取低层特征，所述Inception模块包括多个1×1卷积核以及最大池化层；The improved GoogLeNet network uses the stacking of multiple Inception modules for propagation, and sets two auxiliary regressors to extract low-level features. The Inception module includes multiple 1×1 convolution kernels and a maximum pooling layer. 根据位置姿态结果确定实体设备空间范围，基于投影变换将实体设备与场景三维模型进行叠加融合，构建最终的可视化变电站场景融合三维模型；Determine the spatial range of the physical equipment based on the position and posture results, superimpose and fuse the physical equipment with the scene 3D model based on projection transformation, and construct the final visualized substation scene fusion 3D model; 实时获取变电站中动态变化，通过BIM模型进行实时重建，将实时重建结果传输至可视化变电站场景融合三维模型中，实现变电站实体设备的实时更新。Acquire dynamic changes in the substation in real time, reconstruct in real time through the BIM model, and transmit the real-time reconstruction results to the visualized substation scene fusion 3D model to achieve real-time update of the substation physical equipment. 2.如权利要求1所述的基于数字孪生的变电站场景融合三维重建方法，其特征在于，获取目标变电站的多视角实景航拍影像，包括：利用无人机搭载单目高分辨率相机对变电站区域进行倾斜摄影，以单张图像、多个视角的形式进行影像采集，获取正射与倾斜两类视角实景航拍影像，并对获取的航拍影像进行裁剪以及分割预处理操作。2. The method for fusion three-dimensional reconstruction of substation scenes based on digital twins as described in claim 1 is characterized in that multi-perspective real-life aerial images of the target substation are obtained, including: using an unmanned aerial vehicle equipped with a monocular high-resolution camera to perform oblique photography of the substation area, collecting images in the form of a single image and multiple perspectives, obtaining real-life aerial images of both orthogonal and oblique perspectives, and performing cropping and segmentation preprocessing operations on the obtained aerial images. 3.如权利要求1所述的基于数字孪生的变电站场景融合三维重建方法，其特征在于，筛选出包含实体设备的图像，并将其作为候选图像，输入至图像位置姿态预测模型中，输出候选图像的7个特征，分别包含三个位置特征，即x、y、z的坐标值以及四个姿态特征，即三个旋转角的四元数值。3. The method for fusion three-dimensional reconstruction of substation scenes based on digital twins as described in claim 1 is characterized in that images containing physical equipment are screened out and used as candidate images, input into the image position and posture prediction model, and 7 features of the candidate images are output, which respectively include three position features, namely the coordinate values of x, y, and z, and four posture features, namely the quaternion values of three rotation angles. 4.如权利要求1所述的基于数字孪生的变电站场景融合三维重建方法，其特征在于，根据位置姿态结果确定实体设备空间范围，空间位置通过图像的元数据以及图像内容包含的环境特征来获取，空间范围通过透视投影变换的关系来确定并与场景叠加，基于OpenGL 实现透视投影，先进行透视变换，将四棱台的视景体变换成长方体，再结合正投影实现映射，完成透视投影过程。4. The method for fusion three-dimensional reconstruction of substation scenes based on digital twins as described in claim 1 is characterized in that the spatial range of the physical equipment is determined according to the position and posture results, the spatial position is obtained through the metadata of the image and the environmental characteristics contained in the image content, the spatial range is determined by the relationship of perspective projection transformation and superimposed on the scene, and perspective projection is implemented based on OpenGL. First, perspective transformation is performed to transform the viewing volume of the tetrahedron into a cuboid, and then mapping is implemented in combination with orthographic projection to complete the perspective projection process. 5.如权利要求4所述的基于数字孪生的变电站场景融合三维重建方法，其特征在于，实现基于透视投影的图像叠加包括：还原平移与缩放以及恢复透视变换的相似三角形的关系，还原透视投影，先确定视点的位置、远近平面的尺寸以及远近平面的中心点位置，构建出该次透视投影的视锥体，利用视锥体的参数计算出其远平面四个角点的坐标，基于计算出的角点坐标构建几何体对象并赋予图片材质，实现透视投影图像与场景的叠加。5. The method for fusion three-dimensional reconstruction of substation scenes based on digital twins as described in claim 4 is characterized in that realizing image superposition based on perspective projection includes: restoring the relationship between translation and scaling and restoring the relationship between similar triangles of perspective transformation, restoring perspective projection, first determining the position of the viewpoint, the size of the near and far planes, and the center point position of the near and far planes, constructing the viewing cone of the perspective projection, using the parameters of the viewing cone to calculate the coordinates of the four corner points of its far plane, constructing a geometric object based on the calculated corner point coordinates and assigning picture material to realize the superposition of the perspective projection image and the scene. 6.基于数字孪生的变电站场景融合三维重建系统，其特征在于，包括：6. A substation scene fusion 3D reconstruction system based on digital twins, characterized by including: 数据获取模块，用于获取目标变电站的多视角实景航拍影像，并进行预处理；The data acquisition module is used to acquire multi-view real-life aerial images of the target substation and perform preprocessing; 初始场景模型构建模块，用于提取多视角实景航拍影像中的场景特征，构建初始场景三维模型；The initial scene model building module is used to extract scene features from multi-view real-scene aerial images and build an initial scene 3D model; 定位模块，用于将包含实体设备的图像作为候选图像，输入至图像位置姿态预测模型中，输出位置姿态结果；图像位置姿态预测模型为改进的GoogLeNet网络，将学习的特征迁移到回归网络中，使用affine回归器代替GoogLeNet模型的softmax分类器，并在输出层前增加全连接层，形成定位特征向量层；The positioning module is used to input the image containing the physical device as a candidate image into the image position and posture prediction model, and output the position and posture results; the image position and posture prediction model is an improved GoogLeNet network, which migrates the learned features to the regression network, uses the affine regressor to replace the softmax classifier of the GoogLeNet model, and adds a fully connected layer before the output layer to form a positioning feature vector layer; 改进的GoogLeNet网络利用多个Inception模块的堆叠进行传播，并在设置两个辅助回归器进行提取低层特征，所述Inception模块包括多个1×1卷积核以及最大池化层；The improved GoogLeNet network uses the stacking of multiple Inception modules for propagation, and sets two auxiliary regressors to extract low-level features. The Inception module includes multiple 1×1 convolution kernels and a maximum pooling layer. 场景融合模块，用于根据位置姿态结果确定实体设备空间范围，基于投影变换将实体设备与场景三维模型进行叠加融合，构建最终的可视化变电站场景融合三维模型；The scene fusion module is used to determine the spatial range of the physical equipment according to the position and posture results, and to superimpose and fuse the physical equipment with the scene 3D model based on the projection transformation to construct the final visualized substation scene fusion 3D model; 更新模块，用于实时获取变电站中动态变化，通过BIM模型进行实时重建，将实时重建结果传输至可视化变电站场景融合三维模型中，实现变电站实体设备的实时更新。The update module is used to obtain dynamic changes in the substation in real time, perform real-time reconstruction through the BIM model, and transmit the real-time reconstruction results to the visualized substation scene fusion 3D model to achieve real-time update of the substation physical equipment. 7.一种非暂态计算机可读存储介质，其特征在于，所述非暂态计算机可读存储介质用于存储计算机指令，所述计算机指令被处理器执行时，实现如权利要求1-5任一项所述的基于数字孪生的变电站场景融合三维重建方法。7. A non-transitory computer-readable storage medium, characterized in that the non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the substation scene fusion three-dimensional reconstruction method based on digital twins is implemented as described in any one of claims 1 to 5. 8.一种电子设备，其特征在于，包括：处理器、存储器以及计算机程序；其中，处理器与存储器连接，计算机程序被存储在存储器中，当电子设备运行时，所述处理器执行所述存储器存储的计算机程序，以使电子设备执行实现如权利要求1-5任一项所述的基于数字孪生的变电站场景融合三维重建方法。8. An electronic device, characterized in that it comprises: a processor, a memory and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the substation scene fusion three-dimensional reconstruction method based on digital twins as described in any one of claims 1-5.
CN202410587044.2A 2024-05-13 2024-05-13 Digital twinning-based three-dimensional reconstruction method and system for fusion of transformer substation scenes Active CN118172507B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410587044.2A CN118172507B (en) 2024-05-13 2024-05-13 Digital twinning-based three-dimensional reconstruction method and system for fusion of transformer substation scenes

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410587044.2A CN118172507B (en) 2024-05-13 2024-05-13 Digital twinning-based three-dimensional reconstruction method and system for fusion of transformer substation scenes

Publications (2)

Publication Number Publication Date
CN118172507A CN118172507A (en) 2024-06-11
CN118172507B true CN118172507B (en) 2024-08-02

Family

ID=91354903

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410587044.2A Active CN118172507B (en) 2024-05-13 2024-05-13 Digital twinning-based three-dimensional reconstruction method and system for fusion of transformer substation scenes

Country Status (1)

Country Link
CN (1) CN118172507B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119273931B (en) * 2024-09-23 2025-04-22 江门职业技术学院 A method and related device for extracting three-dimensional features of intangible cultural heritage physical products
CN119135849B (en) * 2024-11-12 2025-02-18 电子科技大学 Digital twin method, device, equipment and medium for airport terminal video
CN119814993B (en) * 2024-12-26 2025-10-31 杨智 Naked-eye 3D screen and 3D image display method
CN119378085B (en) * 2024-12-27 2025-07-04 山东大学 Method and system for constructing digital twin model of large steel structure integrating multi-source data
CN119850847B (en) * 2025-03-13 2025-06-13 浙江大学 A method for generating harmonious objects in 3D scenes based on physical rendering and uncertainty estimation

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115935637A (en) * 2022-12-01 2023-04-07 厦门亿力吉奥信息科技有限公司 A visual operation and maintenance system of substation based on GIS digital twin
CN116258817A (en) * 2023-02-16 2023-06-13 浙江大学 A method and system for constructing an autonomous driving digital twin scene based on multi-view 3D reconstruction

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111369678A (en) * 2018-12-25 2020-07-03 浙江舜宇智能光学技术有限公司 Three-dimensional scene reconstruction method and system
US12118715B2 (en) * 2019-02-01 2024-10-15 Owkin, Inc. Systems and methods for image classification
EP4128159A1 (en) * 2020-03-24 2023-02-08 Nokia Solutions and Networks Oy Camera relocalization methods for real-time ar-supported network service visualization
CN112367514B (en) * 2020-10-30 2022-12-09 京东方科技集团股份有限公司 Three-dimensional scene construction method, device and system and storage medium
CN112560137B (en) * 2020-12-04 2024-11-01 武汉光谷信息技术股份有限公司 Multi-model fusion method and system based on smart city
CN113223162B (en) * 2021-04-13 2023-06-09 交通运输部科学研究院 Construction method and device of inland waterway digital twin scene
CN114417616B (en) * 2022-01-20 2024-11-05 青岛理工大学 A digital twin modeling method and system for assembly robot teleoperation environment
CN115761133A (en) * 2022-11-23 2023-03-07 广东电网有限责任公司 Training and three-dimensional reconstruction method and device of three-dimensional reconstruction model and storage medium
CN116129112A (en) * 2022-12-28 2023-05-16 深圳市人工智能与机器人研究院 Oral three-dimensional point cloud segmentation method and robot for nucleic acid detection robot
CN116612240A (en) * 2023-05-22 2023-08-18 南京工业大学 A Construction Method of Building Digital Twin Model Oriented to Structural Safety Monitoring
CN116740547A (en) * 2023-06-29 2023-09-12 中国电力科学研究院有限公司 Substation target detection methods, systems, equipment and media based on digital twins

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115935637A (en) * 2022-12-01 2023-04-07 厦门亿力吉奥信息科技有限公司 A visual operation and maintenance system of substation based on GIS digital twin
CN116258817A (en) * 2023-02-16 2023-06-13 浙江大学 A method and system for constructing an autonomous driving digital twin scene based on multi-view 3D reconstruction

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于无监督学习的相机位姿估计算法研究;吴炎桐;《中国优秀硕士学位论文全文数据库 信息科技辑》;20190815(第8期);第I138-927页 *

Also Published As

Publication number Publication date
CN118172507A (en) 2024-06-11

Similar Documents

Publication Publication Date Title
CN118172507B (en) Digital twinning-based three-dimensional reconstruction method and system for fusion of transformer substation scenes
CN115423978A (en) Image laser data fusion method based on deep learning and used for building reconstruction
US11544898B2 (en) Method, computer device and storage medium for real-time urban scene reconstruction
CN113537180B (en) Tree barrier identification method, device, computer equipment and storage medium
CN115082254A (en) Lean control digital twin system of transformer substation
CN111951368A (en) A deep learning method for point cloud, voxel and multi-view fusion
CN119068114A (en) A method for 3D reconstruction of building automation
CN116612235A (en) Multi-view geometric unmanned aerial vehicle image three-dimensional reconstruction method and storage medium
CN114067060A (en) Dense point cloud rapid generation method for three-dimensional modeling
CN118736107A (en) A three-dimensional reconstruction method, system, device and medium based on dilated convolution and self-attention mechanism
Zou et al. A 3D reconstruction method for heterogeneous data of power equipment based on improved neural radiation fields
CN117745934A (en) Point cloud reconstruction method and device based on multi-mode mask strategy
CN121304795A (en) Construction method, device and terminal of three-dimensional electronic fence for crane operation
CN119741449B (en) Large scene reconstruction method, system, electronic equipment and computer readable storage medium based on 3D Gaussian algorithm
CN120823560A (en) A real-time monitoring method for urban flood depth based on intelligent three-dimensional vision
CN121330056A (en) A stereo vision-based method for model pose measurement
CN114565917A (en) Building group modeling method and device
CN114299230A (en) A data generation method, device, electronic device and storage medium
CN119478292A (en) Substation 3D modeling method based on laser point cloud and visible light fusion
Li et al. Self-supervised indoor scene point cloud completion from a single panorama
CN119963766A (en) Method, device, equipment, medium and program product for three-dimensional reconstruction
CN118864740A (en) A method and system for generating three-dimensional urban scenes based on video images
CN114241025B (en) Point cloud data processing method and device for high voltage tower in power grid
Zhang et al. Multi-Source and Multi-Dimensional Data Fusion of Magnetic Levitation Track Transportation Based on Digital Twin
CN119672212B (en) Scene reconstruction method, device and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant