CN116977183A - Intelligent construction site holographic image display splicing method and system using same - Google Patents

Abstract

This application relates to a smart construction site holographic image display splicing method and a system applying the same. The method includes: obtaining holographic images of multiple groups of tower crane scenes; taking any two holographic images of the holographic images of each group of tower crane scenes as the first spliced image to be displayed and the second spliced image to be displayed, and using the first The spliced image to be displayed and the second spliced image to be displayed are stored in the first spatial light modulator and the second spatial light modulator respectively; the first spatial light modulator is used to perform a diffraction operation on all pixels in the first spliced image to be displayed, Use the second spatial light modulator to perform a diffraction operation on all pixels in the second spliced image to be displayed, and display the first three-dimensional reconstructed image corresponding to the first spliced image to be displayed, and the first three-dimensional reconstructed image corresponding to the second spliced image to be displayed. Two- and three-dimensional reconstructed images; display and splice the first three-dimensional reconstructed image and the second three-dimensional reconstructed image. This application can achieve accurate and clear three-dimensional splicing effects, making it easy to identify abnormalities in the construction site.

Description

A smart construction site holographic image display splicing method and a system applying it

Technical field

This application relates to the technical fields of smart construction sites and image processing. More specifically, this application relates to a smart construction site holographic image display splicing method and a system applying the same.

Background technique

In recent years, smart construction site technology has become more and more widely used, and its degree of intelligence has indeed saved costs for construction operations. However, smart construction sites equipped with multiple smart tower cranes have many scenarios involving many entities, such as tower crane operation records, goods loaded on transport vehicles, people entering the construction site, etc. However, it is often impossible to achieve both global monitoring and local fine-grained monitoring of smart construction sites in the existing technology. In particular, the construction scenes are constantly changing, and it is difficult to accurately detect construction abnormalities from the collected images. Even individual abnormal information comes from citizens or passers-by around the smart construction site. report.

Contents of the invention

Based on the above technical problems, this application aims to provide a smart construction site holographic image display splicing method to solve the problem of difficulty in accurately detecting construction abnormalities in smart construction sites.

The first aspect of this application provides a smart construction site holographic image display splicing method. The method includes:

Deploy holographic cameras in multiple areas of the target smart construction site, and adjust the distance between adjacent holographic cameras deployed on the same horizontal plane and the same vertical plane in each area according to the different scenes corresponding to each area;

Obtain an initial holographic image set of the target smart construction site by photographing the target smart construction site from the first moment to the second moment by all the deployed holographic cameras;

Classify all images in the initial holographic image collection of the target smart construction site to obtain multiple sets of holographic images of tower crane scenes, where each set of holographic images of tower crane scenes includes at least two holographic images at different times, each of which The holographic images taken at different times include at least one of the boom, the slewing mechanism, the trolley, the hook, and the winch;

For the holographic images of each group of tower crane scenes, any two holographic images in the holographic images of each group of tower crane scenes are taken as the first spliced image to be displayed and the second spliced image to be displayed, and the first spliced image is The spliced image to be displayed and the second spliced image to be displayed are stored in the first spatial light modulator and the second spatial light modulator respectively;

Use the first spatial light modulator to perform diffraction operations on all pixels in the first spliced image to be displayed, and use the second spatial light modulator to perform diffraction operations on all pixels in the second spliced image to be displayed. Operation, display the first three-dimensional reconstructed image corresponding to the first spliced image to be displayed, and the second three-dimensional reconstructed image corresponding to the second spliced image to be displayed;

The first three-dimensional reconstructed image and the second three-dimensional reconstructed image are displayed and spliced.

In some embodiments of the present application, the first spatial light modulator is used to perform a diffraction operation on all pixels in the first spliced image to be displayed, and the second spatial light modulator is used to perform a diffraction operation on the third All pixels in the second spliced image to be displayed undergo a diffraction operation to display a first three-dimensional reconstructed image corresponding to the first spliced image to be displayed, and a second three-dimensional reconstructed image corresponding to the second spliced image to be displayed, including:

Use the first spatial light modulator to perform a diffraction operation on all pixels in the first spliced image to be displayed to obtain and display a first virtual image;

Use the second spatial light modulator to perform a diffraction operation on all pixels in the second spliced image to be displayed to obtain and display a second virtual image;

Use the first virtual image as a first three-dimensional reconstructed image corresponding to the first spliced image to be displayed;

The second virtual image is used as a second three-dimensional reconstructed image corresponding to the second spliced image to be displayed.

In some embodiments of the present application, using the first spatial light modulator to perform a diffraction operation on all pixels in the first spliced image to be displayed to obtain and display a first virtual image includes:

Calculate the diffraction parameters of the first spatial light modulator;

Constructing a diffraction field using a preset observation window and the first spatial light modulator after calculating and modulating the diffraction parameters;

Setting a three-dimensional field of view area in the diffraction field;

A diffraction operation is performed in the diffraction field after setting a three-dimensional field of view area, and a first virtual image is displayed in the three-dimensional field of view area.

In some embodiments of the present application, displaying and splicing the first three-dimensional reconstructed image and the second three-dimensional reconstructed image includes:

Adjusting the diffraction field by adjusting the positions of the first spatial light modulator and the second spatial light modulator;

Using a half mirror to increase the area of the three-dimensional field of view to accommodate the splicing result of the first three-dimensional reconstructed image and the second three-dimensional reconstructed image;

The first three-dimensional reconstructed image and the second three-dimensional reconstructed image are displayed and spliced through the adjusted diffraction field, and the splicing result is displayed in the enlarged three-dimensional field of view area.

In some embodiments of the present application, using a half-mirror to increase the area of the three-dimensional field of view includes:

Place a half mirror between the first spatial light modulator and the second spatial light modulator, such that the angle between the half mirror and the first spatial light modulator is , and the angles between the half mirror and the second spatial light modulator are equal and are preset angles.

In some embodiments of the present application, the first three-dimensional reconstructed image and the second three-dimensional reconstructed image are displayed through splicing through the adjusted diffraction field, within the enlarged three-dimensional field of view area. Display the stitching results, including:

Configure the preset carrier frequency and preset incident light illumination for the adjusted diffraction field, and perform the diffraction operation after configuration;

Calculate the holographic image corresponding to the first three-dimensional reconstructed image using a preset encoding algorithm, and load it through the first spatial light modulator;

Calculate the holographic image corresponding to the second three-dimensional reconstructed image using a preset encoding algorithm, and load it through the second spatial light modulator;

Splicing the first three-dimensional reconstructed image and the second three-dimensional reconstructed image respectively corresponding to the first spatial light modulator and the second spatial light modulator in the horizontal direction;

When both the first spatial light modulator and the second spatial light modulator are loaded, the first three-dimensional reconstructed image and the second three-dimensional image are displayed in the enlarged three-dimensional field of view area. The stitching result of the reconstructed image.

In some embodiments of the present application, after obtaining the initial holographic image set of the target smart construction site, it further includes: performing holographic processing on all images in the initial holographic image set of the target smart construction site, including:

The transformation step is to convert the pre-acquired reference image into a high-resolution reference image through Fourier transform;

Obtaining a spatial spectrum corresponding to any image in the initial holographic image set of the target smart construction site from the high-resolution reference image;

An update step, updating the spatial spectrum to the high-resolution reference image to obtain a high-resolution hologram;

The transformation step, the acquisition step, and the update step are executed cyclically until all images in the initial holographic image set of the target smart construction site are holographically processed.

In some embodiments of the present application, all images in the initial holographic image collection of the target smart construction site are classified to obtain multiple sets of holographic images of tower crane scenes, including:

The SVM classification algorithm is used to classify the scene based on the scene implicit feature distribution information of the target smart construction site, and obtain holographic images of multiple groups of tower crane scenes.

The second aspect of this application provides a smart construction site holographic image splicing system. The system includes:

The acquisition module is used to obtain the initial holographic image collection of the target smart construction site;

A classification module, used to classify all images in the initial holographic image set of the target smart construction site to obtain multiple sets of holographic images of tower crane scenes, where each set of holographic images of tower crane scenes includes at least two holographic images at different times. Holographic image, each holographic image taken at different times contains at least one of the big arm, slewing mechanism, trolley, hook, and winch;

A storage module configured to take any two holographic images of the holographic images of each group of tower crane scenes as the first spliced image to be displayed and the second spliced image to be displayed, respectively, for the holographic images of each group of tower crane scenes, and Store the first spliced image to be displayed and the second spliced image to be displayed in the first spatial light modulator and the second spatial light modulator respectively;

A diffraction module, configured to use the first spatial light modulator to perform diffraction operations on all pixels in the first spliced image to be displayed, and to use the second spatial light modulator to perform diffraction operations on all pixels in the second spliced image to be displayed. Perform a diffraction operation on all pixels to display a first three-dimensional reconstructed image corresponding to the first spliced image to be displayed, and a second three-dimensional reconstructed image corresponding to the second spliced image to be displayed;

A display splicing module is used to display and splice the first three-dimensional reconstructed image and the second three-dimensional reconstructed image.

The third aspect of this application provides an electronic device, including a memory and a processor. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, they cause the processor to execute the smart construction site described in each embodiment. Holographic image display stitching method.

The technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

In this application, for the holographic images of each group of tower crane scenes, any two holographic images in the holographic images of each group of tower crane scenes are taken as the first spliced image to be displayed and the second spliced image to be displayed, and the first spliced image to be displayed is The spliced image and the second spliced image to be displayed are respectively stored in the first spatial light modulator and the second spatial light modulator. The first spatial light modulator is used to perform a diffraction operation on all pixels in the first spliced image to be displayed. The two spatial light modulators perform a diffraction operation on all pixels in the second spliced image to be displayed, and display a first three-dimensional reconstructed image corresponding to the first spliced image to be displayed, and a second three-dimensional reconstructed image corresponding to the second spliced image to be displayed. Reconstructed image, the first three-dimensional reconstructed image and the second three-dimensional reconstructed image are displayed and spliced, so as to achieve an accurate and clear three-dimensional splicing effect. The spliced image can be observed and displayed from the three-dimensional field of view through the preset observation window. Three-dimensional effect to identify anomalies in the target smart construction site. Not only that, this application can achieve both global monitoring and local fine-grained monitoring of the target smart construction site. Even if the construction scene changes rapidly, it can accurately locate abnormal targets in the construction scene.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present application.

Description of the drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for the purpose of illustrating preferred embodiments only and are not to be construed as limiting the application. Also throughout the drawings, the same reference characters are used to designate the same components. In the attached picture:

Figure 1 shows a schematic diagram of the steps of a smart construction site holographic image display splicing method in an exemplary embodiment of the present application;

Figure 2 shows a schematic diagram of initial holographic images collected at different times on the same day in an exemplary embodiment of the present application;

Figure 3 shows a schematic diagram of the positional relationship between a half mirror, a first spatial light modulator and a second spatial light modulator in an exemplary embodiment of the present application;

Figure 4 shows a partial schematic diagram of a splicing result in an exemplary embodiment of the present application;

Figure 5 shows a schematic structural diagram of a smart construction site holographic image display splicing system in an exemplary embodiment of the present application;

FIG. 6 shows a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application.

Detailed ways

The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments depicted here are only used to explain the relevant invention, but not to limit the invention. It should also be noted that, for convenience of description, only the parts related to the invention are shown in the drawings.

First, a brief explanation of the smart construction site is given. The application of smart construction site technology is becoming more and more widespread, and its degree of intelligence has indeed saved costs for construction operations. Smart construction sites have many scenarios involving many entities, but the most important entity in smart construction sites is tower cranes. Smart construction sites have multiple tower cranes. A tower crane usually consists of a base, a tower body, a large arm (also called a long arm or cantilever), and a rotating platform on the top. The boom is an important part of the tower crane. It is an arm-like structure that connects the tower body and the hook and is used to lift and carry goods. The length of the boom can vary according to the design and purpose of the tower crane to adapt to different working scenarios and needs. Tower cranes can carry out precise cargo handling operations in both vertical and horizontal directions through the combination of a rotating platform and a large arm. Collecting images of smart construction sites needs to be both comprehensive and detailed. It not only needs to cover the entire smart construction site, but also needs to accurately locate abnormal points in the smart construction site, which requires greater difficulty.

Therefore, in some embodiments of the present application, a smart construction site holographic image display splicing method is provided, which displays a global fresh three-dimensional effect by displaying spliced holograms, as shown in Figure 1. The method includes:

S1. Deploy holographic cameras in multiple areas of the target smart construction site, and adjust the distance between adjacent holographic cameras deployed on the same horizontal plane and the same vertical plane in each area according to the different scenes corresponding to each area;

S2. Obtain an initial holographic image set of the target smart construction site by photographing the target smart construction site from the first moment to the second moment through all the deployed holographic cameras;

S3. Classify all images in the initial holographic image set of the target smart construction site to obtain multiple sets of holographic images of tower crane scenes, where the holographic images of each set of tower crane scenes include at least two holographic images at different times. The holographic image taken at each different moment contains at least one of the boom, slewing mechanism, trolley, hook, and winch;

S4. For the holographic images of each group of tower crane scenes, take any two holographic images of the holographic images of each group of tower crane scenes as the first spliced image to be displayed and the second spliced image to be displayed, and use the The first spliced image to be displayed and the second spliced image to be displayed are stored in the first spatial light modulator and the second spatial light modulator respectively;

S5. Use the first spatial light modulator to perform a diffraction operation on all pixels in the first spliced image to be displayed, and use the second spatial light modulator to perform a diffraction operation on all pixels in the second spliced image to be displayed. Perform a diffraction operation to display a first three-dimensional reconstructed image corresponding to the first spliced image to be displayed, and a second three-dimensional reconstructed image corresponding to the second spliced image to be displayed;

S6. Display and splice the first three-dimensional reconstructed image and the second three-dimensional reconstructed image.

Deploy holographic cameras in multiple areas of the target smart construction site. According to the different scenes corresponding to each area, adjust the distance between adjacent holographic cameras deployed on the same horizontal plane and the same vertical plane in each area, so that the initial holographic image collected by the camera is It will fully cover the entire target smart construction site. Of course, the target smart construction site is also equipped with radar and various sensors, such as tilt sensors, wind speed sensors, load sensors, etc. Radar is mainly used to detect moving objects such as the speed and direction of cars running in the target smart construction site, the distance of material lifting, etc. Various sensors are used to sense various physical quantities or information in the environment, such as sensing the movement of different areas in the target smart construction site. Temperature, humidity, pressure, light, sound, etc. Figure 2 shows the initial holographic images collected at different times on the same day. As shown in Figure 2, it is the initial holographic image at four times on August 29, 2023. Preferably, the information collected by radar and various sensors and the initial holographic image are fused. Optionally, the information and images collected by the radar and various sensors are processed separately, and then fused together, for example, by matching the distance information of the radar with the visual information of the image to determine the position and movement status of the obstacle. . Feature-level fusion can also be used, for example, features are extracted from information and images collected by radar and various sensors, and different feature vectors are connected and fused together, and finally the fused initial holographic images are classified into the set.

In a preferred implementation, all images in the initial holographic image collection of the target smart construction site are classified, and when holographic images of multiple groups of tower crane scenes are obtained, the SVM classification algorithm is used based on the scene implicit features of the target smart construction site. The distribution information is used to classify the scenes and obtain holographic images of multiple groups of tower crane scenes. The full name is Support VectorMachine, which is a common supervised learning algorithm used for binary or multi-classification tasks. Its main idea is to find an optimal hyperplane that separates samples from different categories. Because the holographic image is divided into multiple regions, it can be used to subsequently mine the hidden feature distribution information of entities, making the entity classification more detailed. Finally, abnormal entities are identified based on the hidden feature distribution information of the entities, which can achieve both global monitoring and control of the target smart construction site. Local fine monitoring can detect abnormal targets in the construction scene promptly and accurately even if the construction scene is ever-changing. Before classification, the ResNet-50 model needs to be used to extract features of various scenes in the target smart construction site. The ResNet-50 model is obtained by adding a classification layer after the convolutional layer of the resnet network. The ResNet-50 model is a migration model with an accuracy of more than 95%. Preferably, the input accuracy is 64*64*256, where 256 represents the number of channels, and the vector is output after several convolutions. The ResNet-50 model also includes Relu. The full name of Relu is Linear rectification function, which represents a linear rectification function, also known as a modified linear unit. It is a commonly used activation function in artificial neural networks. The ResNet-50 model does not change the corresponding position of the pixels in the image (i.e., the regional holographic image), and only changes the number of channels. Optionally, before mining the entity implicit feature distribution information in the regional holographic image, attention mechanism processing can also be performed on each regional holographic image. Increasing the accuracy of the initial holographic image through a weighted sum operation includes: extracting each regional holographic image. the key features and key feature vectors; obtain the attention coefficient corresponding to the key feature vector through the key features; perform a weighted sum calculation of the key feature vector and the attention coefficient. Among them, the attention transformation formula is:

Among them, e is the feature before transformation, is the transformed feature, and W is the learning weight. The ResNet-50 model needs to be trained before use. It adopts a supervised training method and stops training when the number of iterations exceeds the preset number and the model reaches convergence. The training method adopts common technical means in the existing technology and will not be described in detail here.

In specific implementation, after obtaining the initial holographic image set of the target smart construction site, it also includes: holographic processing of all images in the initial holographic image set of the target smart construction site, including: a transformation step, using Fourier transform to convert the pre-acquired reference The image is converted into a high-resolution reference image; the acquisition step is to obtain the spatial spectrum corresponding to any image in the initial holographic image collection of the target smart construction site from the high-resolution reference image; the update step is to update the spatial spectrum to the high-resolution reference image image to obtain a high-resolution hologram; the transformation steps, acquisition steps, and update steps are executed in a loop until all images in the initial holographic image set of the target smart construction site are holographically processed.

In a preferred implementation, a first spatial light modulator is used to perform a diffraction operation on all pixels in the first spliced image to be displayed, and a second spatial light modulator is used to perform a diffraction operation on all pixels in the second spliced image to be displayed. The diffraction operation displays a first three-dimensional reconstructed image corresponding to the first spliced image to be displayed, and a second three-dimensional reconstructed image corresponding to the second spliced image to be displayed, including: using a first spatial light modulator to Display all pixels in the spliced image to perform a diffraction operation to obtain and display a first virtual image; use the second spatial light modulator to perform a diffraction operation on all pixels in the second spliced image to be displayed to obtain and display a second virtual image; convert the first The virtual image is used as the first three-dimensional reconstructed image corresponding to the first spliced image to be displayed; the second virtual image is used as the second three-dimensional reconstructed image corresponding to the second spliced image to be displayed. Further, using the first spatial light modulator to perform a diffraction operation on all pixels in the first spliced image to be displayed to obtain and display the first virtual image includes: calculating the diffraction parameters of the first spatial light modulator; using preset observation The window and the first spatial light modulator after calculating and modulating the diffraction parameters construct a diffraction field; setting a three-dimensional field of view area in the diffraction field; performing a diffraction operation in the diffraction field after setting the three-dimensional field of view area, and displaying it in the three-dimensional field of view area The first virtual image.

The above-mentioned spatial light modulator is called Spatial Light Modulator, or SLM for short. It is a device that can modulate the phase, amplitude or polarization state of light waves based on external input signals. It has a wide range of applications in the fields of optics and optical communications, such as optical information processing, beam shaping, holographic imaging, etc. In a spatial light modulator, by modulating the phase distribution of light waves, diffraction can be controlled to achieve various optical functions. The diffraction parameters of the spatial light modulator include spatial resolution, phase modulation range, diffraction efficiency, etc. In the above diffraction field, the preset observation window, three-dimensional field of view area and the position of the spatial light modulator need to be arranged as necessary. The three-dimensional field of view area is located between the spatial light modulator and the preset observation window to display the final three-dimensional stitching result. From the splicing results, we can accurately identify abnormalities in the boom, slewing mechanism, trolley, hook, and winch.

In a preferred implementation, splicing the first three-dimensional reconstructed image and the second three-dimensional reconstructed image display includes: adjusting the diffraction field by adjusting the positions of the first spatial light modulator and the second spatial light modulator; using The half mirror increases the area of the three-dimensional field of view to accommodate the splicing results of the first three-dimensional reconstructed image and the second three-dimensional reconstructed image; the first three-dimensional reconstructed image and the second three-dimensional reconstructed image are combined through the adjusted diffraction field. The reconstructed image displays the stitching, and the stitching result is displayed within the enlarged three-dimensional field of view area. Using a half mirror to increase the area of the three-dimensional field of view, as shown in Figure 3, includes: placing the half mirror 3 between the first spatial light modulator 1 and the second spatial light modulator 2, Make the angle between the half mirror and the first spatial light modulator and the angle between the half mirror and the second spatial light modulator equal and both be preset angles, preferably Ground, the default angle is 45 degrees.

In some embodiments of the present application, the first three-dimensional reconstructed image and the second three-dimensional reconstructed image are displayed and spliced through the adjusted diffraction field, and the splicing result is displayed in the enlarged three-dimensional field of view area, including: adjusting The final diffraction field is configured with a preset carrier frequency and a preset incident light illumination, and the diffraction operation is performed after configuration; a preset encoding algorithm is used to calculate a holographic image corresponding to the first three-dimensional reconstructed image, and is loaded through the first spatial light modulator ; Use a preset encoding algorithm to calculate the holographic image corresponding to the second three-dimensional reconstructed image, and load it through the second spatial light modulator; splice the corresponding images of the first spatial light modulator and the second spatial light modulator in the horizontal direction The first three-dimensional reconstructed image and the second three-dimensional reconstructed image; when the first spatial light modulator and the second spatial light modulator are both loaded, the first three-dimensional reconstructed image is displayed in the enlarged three-dimensional field of view area The stitching result with the second three-dimensional reconstructed image. Figure 4 is a partial schematic diagram of a splicing result. As shown in Figure 4, the display screen is enlarged, the visual effect is broader and clearer, and the screen identifies the data collected by the radar and various sensors corresponding to the image. The above-mentioned preset encoding algorithm can adopt a digital holographic encoding algorithm and use computer-generated digital simulation to generate holographic images. The digital holographic encoding algorithm is based on the principle of diffraction and computer graphics technology and can achieve high-quality holographic images. The first three-dimensional reconstructed image and the second three-dimensional reconstructed image are essentially virtual images, so the display splicing in the embodiment is essentially a splicing of virtual images and virtual images, which is a display effect rather than a physical effect. Splicing: After first splicing two holographic images, the splicing result can be reused as a holographic image, and then spliced with a third holographic image, thereby achieving display splicing of multiple holograms. The splicing results are displayed through the enlarged three-dimensional field of view area, making the display screen clearer, showing both the overall situation and the local part, and anomalies in the tower crane's boom, slewing mechanism, trolley, hook, and winch can be accurately identified , thereby achieving more powerful monitoring of the target smart construction site.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present application.

In some embodiments of the present application, a smart construction site holographic image splicing system is also provided, applying the smart construction site holographic image splicing method described in each embodiment. As shown in Figure 5, the system includes:

The acquisition module 601 is used to acquire the initial holographic image collection of the target smart construction site;

The classification module 602 is used to classify all images in the initial holographic image set of the target smart construction site to obtain multiple sets of holographic images of tower crane scenes, wherein the holographic images of each set of tower crane scenes include at least two different moments. The holographic image, each holographic image taken at different times contains at least one of the big arm, the slewing mechanism, the trolley, the hook, and the winch;

The storage module 603 is used for taking any two holographic images of the holographic images of each group of tower crane scenes as the first spliced image to be displayed and the second spliced image to be displayed, respectively, for the holographic images of each group of tower crane scenes, and storing the first spliced image to be displayed and the second spliced image to be displayed in the first spatial light modulator and the second spatial light modulator respectively;

Diffraction module 604, configured to use the first spatial light modulator to perform diffraction operations on all pixels in the first spliced image to be displayed, and to use the second spatial light modulator to perform diffraction operations on the second spliced image to be displayed. All pixels in the diffraction operation are performed to display the first three-dimensional reconstructed image corresponding to the first spliced image to be displayed, and the second three-dimensional reconstructed image corresponding to the second spliced image to be displayed;

The display splicing module 605 is used to display and splice the first three-dimensional reconstructed image and the second three-dimensional reconstructed image.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present application.

Please refer to FIG. 6 below, which shows a schematic diagram of an electronic device provided by some embodiments of the present application. As shown in Figure 6, the electronic device 2 includes: a processor 200, a memory 201, a bus 202 and a communication interface 203. The processor 200, the communication interface 203 and the memory 201 are connected through the bus 202; the memory 201 stores There is a computer program that can be run on the processor 200. When the processor 200 runs the computer program, it executes the smart construction site holographic image display splicing method provided by any of the foregoing embodiments of this application.

Among them, the memory 201 may include high-speed random access memory (RAM: Random Access Memory), and may also include non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 203 (which can be wired or wireless), and the Internet, wide area network, local network, metropolitan area network, etc. can be used.

The bus 202 may be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. The memory 201 is used to store a program, and the processor 200 executes the program after receiving the execution instruction. The smart construction site holographic image display splicing method disclosed in any of the embodiments of the present application can be applied to processing in the processor 200, or implemented by the processor 200.

The processor 200 may be an integrated circuit chip with signal processing capabilities. During the implementation process, each step of the above method can be completed by instructions in the form of hardware integrated logic circuits or software in the processor 200 . The above-mentioned processor 200 can be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step and logical block diagram disclosed in the embodiment of this application can be implemented or executed. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly implemented by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other mature storage media in this field. The storage medium is located in the memory 201. The processor 200 reads the information in the memory 201 and completes the steps of the above method in combination with its hardware.

The embodiment of the present application also provides a computer-readable storage medium corresponding to the smart construction site holographic image display splicing method provided in the previous embodiment, with a computer program stored thereon, and the computer program will execute when run by the processor. The smart construction site holographic image display splicing method provided in any of the aforementioned embodiments. In addition, examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), and other types of random access memory. (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be described one by one here.

An embodiment of the present application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the smart construction site holographic image display splicing method provided in any of the foregoing embodiments.

It should be noted that the algorithms and displays provided here are not inherently related to any particular computer, virtual appliance, or other device. Various general-purpose devices can also be used with teaching based on this. The structure required to construct such a device will be apparent from the above description. Furthermore, this application is not specific to any specific programming language. It should be understood that the subject matter described herein may be implemented using a variety of programming languages, and that the above descriptions of specific languages are for the purpose of disclosing the best mode for carrying out the subject matter. In the instructions provided here, a number of specific details are described. However, it is understood that embodiments of the present application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description.

Those skilled in the art will understand that various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some or all components in the device for creating a virtual machine according to embodiments of the present application.

The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any person familiar with the technical field can easily think of changes or modifications within the technical scope disclosed in the present application. Replacements shall be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims (10)

1. An intelligent construction site holographic image display splicing method is characterized by comprising the following steps:

disposing holographic cameras in a plurality of areas of the target intelligent construction site, and adjusting the distance between adjacent holographic cameras disposed in the same horizontal plane and the same vertical plane of each area according to different scenes corresponding to each area;

shooting the target intelligent building site from a first moment to a second moment through all deployed holographic cameras to obtain an initial holographic image set of the target intelligent building site;

classifying all images in an initial holographic image set of the target intelligent building site to obtain holographic images of a plurality of groups of tower crane scenes, wherein the holographic images of each group of tower crane scenes comprise at least two holographic images at different moments, and at least one of a big arm, a slewing mechanism, a trolley, a lifting hook and a winch is shot on the holographic images at each different moment;

taking any two holographic images in the holographic images of each group of tower crane scenes as a first spliced image to be displayed and a second spliced image to be displayed respectively, and storing the first spliced image to be displayed and the second spliced image to be displayed in a first spatial light modulator and a second spatial light modulator respectively;

performing diffraction operation on all pixels in the first spliced image to be displayed by using the first spatial light modulator, performing diffraction operation on all pixels in the second spliced image to be displayed by using the second spatial light modulator, and displaying a first three-dimensional reconstruction image corresponding to the first spliced image to be displayed and a second three-dimensional reconstruction image corresponding to the second spliced image to be displayed;

and displaying and splicing the first three-dimensional reconstruction image and the second three-dimensional reconstruction image.

2. The smart worksite holographic image display stitching method of claim 1, wherein the performing a diffraction operation on all pixels in the first stitched image to be displayed using the first spatial light modulator, performing a diffraction operation on all pixels in the second stitched image to be displayed using the second spatial light modulator, displaying a first three-dimensional reconstructed image corresponding to the first stitched image to be displayed, and a second three-dimensional reconstructed image corresponding to the second stitched image to be displayed, comprises:

performing diffraction operation on all pixels in the first spliced image to be displayed by using the first spatial light modulator to obtain and display a first virtual image;

performing diffraction operation on all pixels in the second spliced image to be displayed by using the second spatial light modulator to obtain and display a second virtual image;

taking the first virtual image as a first three-dimensional reconstruction image corresponding to a first spliced image to be displayed;

and taking the second virtual image as a second three-dimensional reconstruction image corresponding to the second spliced image to be displayed.

3. The smart worksite holographic image display stitching method of claim 2, wherein the performing a diffraction operation on all pixels in the first stitched image to be displayed using the first spatial light modulator to obtain and display a first virtual image comprises:

calculating diffraction parameters of the first spatial light modulator;

constructing a diffraction field by using a preset observation window and the first spatial light modulator after calculating and modulating diffraction parameters;

setting a three-dimensional field of view region in the diffraction field;

and performing a diffraction operation in the diffraction field after the three-dimensional field-of-view region is set, and displaying a first virtual image in the three-dimensional field-of-view region.

4. The smart worksite holographic image display stitching method of claim 3, in which the stitching the first three-dimensional reconstructed image with the second three-dimensional reconstructed image display comprises:

adjusting the diffraction field by adjusting the positions of the first spatial light modulator and the second spatial light modulator;

the area of the three-dimensional view field area is increased by utilizing a half-mirror to accommodate the splicing result of the first three-dimensional reconstruction image and the second three-dimensional reconstruction image;

and displaying and splicing the first three-dimensional reconstruction image and the second three-dimensional reconstruction image through the adjusted diffraction field, and displaying a splicing result in the increased three-dimensional view field area.

5. The smart worksite holographic image display stitching method of claim 4, in which the increasing the area of the three-dimensional field of view region with a half mirror comprises:

the half-mirror is arranged between the first spatial light modulator and the second spatial light modulator, so that the included angle between the half-mirror and the first spatial light modulator, and the included angle between the half-mirror and the second spatial light modulator are equal and are all preset angles.

6. The smart worksite holographic image display stitching method of claim 4, in which the stitching the first three-dimensional reconstructed image with the second three-dimensional reconstructed image display by the adjusted diffraction field, displaying the stitching result in the increased three-dimensional field of view region, comprising:

configuring preset carrier frequency and preset incident light illumination for the adjusted diffraction field, and performing diffraction operation after configuration;

calculating a holographic image corresponding to the first three-dimensional reconstruction image by adopting a preset coding algorithm, and loading the holographic image through the first spatial light modulator;

calculating a holographic image corresponding to the second three-dimensional reconstruction image by adopting a preset coding algorithm, and loading the holographic image through the second spatial light modulator;

splicing the first three-dimensional reconstruction image and the second three-dimensional reconstruction image which are respectively corresponding to the first spatial light modulator and the second spatial light modulator in the horizontal direction;

and when the loading of the first spatial light modulator and the second spatial light modulator is completed, displaying the splicing result of the first three-dimensional reconstruction image and the second three-dimensional reconstruction image in the increased three-dimensional view field area.

7. The smart worksite holographic image display stitching method of claim 1, further comprising, after the obtaining the initial set of holographic images for the target smart worksite: carrying out holographic processing on all images in the initial holographic image set of the target intelligent building site, wherein the holographic processing comprises the following steps:

a conversion step of converting a reference map acquired in advance into a high-resolution reference map by fourier transform;

an acquisition step of acquiring a spatial frequency spectrum corresponding to any one image in an initial holographic image set of the target intelligent building site from the high-resolution reference image;

updating, namely updating the spatial frequency spectrum to the high-resolution reference picture to obtain a high-resolution hologram;

and circularly executing the transformation step, the acquisition step and the updating step until all images in the initial holographic image set of the target intelligent building site are subjected to holographic processing.

8. The method for displaying and stitching the holographic images on the intelligent construction site according to claim 1, wherein the step of classifying all images in the initial holographic image set of the target intelligent construction site to obtain holographic images of a plurality of groups of tower crane scenes comprises the steps of:

and classifying the scenes based on the scene implicit characteristic distribution information of the target intelligent building site by using an SVM classification algorithm to obtain holographic images of a plurality of groups of tower crane scenes.

9. An intelligent worksite holographic image stitching system, the system comprising:

the acquisition module is used for acquiring an initial holographic image set of the target intelligent construction site;

the classification module is used for classifying all images in the initial holographic image set of the target intelligent construction site to obtain holographic images of a plurality of groups of tower crane scenes, wherein the holographic images of each group of tower crane scenes comprise at least two holographic images at different moments, and at least one of a big arm, a slewing mechanism, a trolley, a lifting hook and a winch is shot on the holographic image at each different moment;

the storage module is used for taking any two holographic images in the holographic images of each group of tower crane scenes as a first spliced image to be displayed and a second spliced image to be displayed respectively, and storing the first spliced image to be displayed and the second spliced image to be displayed in the first spatial light modulator and the second spatial light modulator respectively;

the diffraction module is used for performing diffraction operation on all pixels in the first spliced image to be displayed by using the first spatial light modulator, performing diffraction operation on all pixels in the second spliced image to be displayed by using the second spatial light modulator, and displaying a first three-dimensional reconstruction image corresponding to the first spliced image to be displayed and a second three-dimensional reconstruction image corresponding to the second spliced image to be displayed;

and the display splicing module is used for splicing the first three-dimensional reconstruction image and the second three-dimensional reconstruction image.

10. An electronic device comprising a memory and a processor, wherein the memory has stored therein computer readable instructions that, when executed by the processor, cause the processor to perform the method of any of claims 1-8.