CN116719176B - Intelligent display system of intelligent exhibition hall - Google Patents
Intelligent display system of intelligent exhibition hall Download PDFInfo
- Publication number
- CN116719176B CN116719176B CN202311004565.2A CN202311004565A CN116719176B CN 116719176 B CN116719176 B CN 116719176B CN 202311004565 A CN202311004565 A CN 202311004565A CN 116719176 B CN116719176 B CN 116719176B
- Authority
- CN
- China
- Prior art keywords
- observer
- module
- unit
- data
- human body
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Processing Or Creating Images (AREA)
Abstract
本发明涉及投影技术领域，特别涉及一种智能展厅的智能显示系统。针对现有展厅显示系统的交互方式单一以及显示内容有限的问题，所采用的方案为：系统包括全息投影模块、裸眼3D显示模块、场景化还原模块、互动投影模块、体感交互模块，全息投影模块用于向观展者提供展品的三维影像；裸眼3D显示模块用于向观展者提供展品的三维影像；场景化还原模块用于向观展者展示与展品相关的还原场景；互动投影模块用于供观展者与投影内容互动；体感交互模块用于供观展者通过肢体动作实现与展品的互动。通过前述方案，观展者在展厅观展时，能获得多维度的显示效果，不再局限于二维显示和单一的观看角度，能获得更多的三维效果；同时交互方式也多样化。
The invention relates to the field of projection technology, and in particular to an intelligent display system for an intelligent exhibition hall. In order to solve the problems of single interaction mode and limited display content of the existing exhibition hall display system, the solution adopted is: the system includes a holographic projection module, a naked-eye 3D display module, a scene restoration module, an interactive projection module, a somatosensory interactive module, and a holographic projection module. It is used to provide visitors with three-dimensional images of exhibits; the naked-eye 3D display module is used to provide visitors with three-dimensional images of exhibits; the scene restoration module is used to show visitors restored scenes related to exhibits; the interactive projection module is used It is used for visitors to interact with the projected content; the somatosensory interaction module is used for visitors to interact with the exhibits through body movements. Through the aforementioned solution, visitors can obtain multi-dimensional display effects when viewing exhibitions in the exhibition hall. They are no longer limited to two-dimensional display and a single viewing angle, but can obtain more three-dimensional effects; at the same time, the interaction methods are also diversified.
Description
Technical Field
The invention relates to the technical field of projection, in particular to an intelligent display system of an intelligent exhibition hall.
Background
The intelligent exhibition hall is an exhibition space for digitizing exhibition contents by utilizing scientific and technological means. The devices used in the current mainstream exhibition hall display system include an LED screen, a touch screen, a projector, an intelligent television and a multimedia terminal, and the projector is combined with an interaction technology, so that an image can be projected on a stereoscopic space, and interaction can be performed through gestures or touch.
However, the interactive mode of the exhibition hall display system is single, and the display device has the corresponding defects, for example, the LED screen can only display and watch in two dimensions, the viewing angle is limited, the touch screen and the multimedia terminal have only a single interactive mode, the display content is limited, the projector can only display in two dimensions, and the observer cannot obtain more exhibition details.
Therefore, improvements to existing exhibition hall display systems are still needed to solve the problems of single interaction mode and limited display content of the existing exhibition hall display systems.
Disclosure of Invention
The invention mainly aims to provide an intelligent display system of an intelligent exhibition hall, which aims to solve the problems of single interaction mode and limited display content of the existing exhibition hall display system.
In order to achieve the above object, the present invention provides an intelligent display system for an intelligent exhibition hall, comprising:
the holographic projection module is used for providing a three-dimensional image of the exhibit for the observer; wherein the three-dimensional image is a floating image;
the naked eye 3D display module is used for providing three-dimensional images of the exhibits for observers; wherein the three-dimensional image is a screen image;
the scene reduction module is used for displaying reduction scenes related to the exhibits to observers;
the interactive projection module is used for enabling a viewer to interact with the projection content, wherein the projection content is related to the exhibited article;
and the somatosensory interaction module is used for enabling observers to interact with the exhibits through limb actions.
Optionally, the intelligent display system of the intelligent exhibition hall further comprises a content recommendation module; the content recommendation module is used for recommending the exhibit information to the observers.
Optionally, the content recommendation module includes:
the first acquisition unit is used for acquiring behavior data of a spectator in a real exhibition hall and/or a virtual exhibition hall;
the first preprocessing unit is used for preprocessing the behavior data to obtain first data corresponding to the behavior data;
the first generation unit is used for generating behavioral characteristics of the observer according to the first data;
the content recommending unit is used for inputting the behavior characteristics into a preset predicting model to obtain a predicting result and recommending the exhibit information corresponding to the predicting result to the observers; the prediction result is the type of the exhibit of interest to the observer.
Optionally, the intelligent display system of the intelligent exhibition hall further comprises an augmented reality module and/or a virtual reality module for providing immersive interaction to the observer.
Optionally, the scene change reduction module includes:
a scene creation unit for creating a scene in the virtual exhibition hall;
the second acquisition unit is used for acquiring real data of a real exhibition hall; the real data comprise scene data and exhibit data;
the second preprocessing unit is used for preprocessing the real data and obtaining second data corresponding to the real data;
a second generation unit for generating a three-dimensional model from the second data; wherein the three-dimensional model corresponds to a scene or an exhibit;
the rendering unit is used for rendering the three-dimensional model according to the real data;
an adding unit for adding picture control logic and a viewing path for a viewer in the virtual exhibition hall;
and the display unit is used for displaying the virtual exhibition hall.
Optionally, the interactive projection module includes:
the exhibit identification module is used for identifying the exhibits and obtaining an exhibit identification result;
the gesture recognition module is used for recognizing gestures of the observers and obtaining gesture recognition results;
the track generation module is used for generating a motion track of the observer in a preset period;
the human body posture estimation module is used for carrying out human body posture estimation on the observers to obtain a human body posture estimation result.
Optionally, the exhibit identification module includes:
the third acquisition unit is used for acquiring auxiliary identification data; the auxiliary identification data comprise third data and/or fourth data, wherein the third data are three-dimensional coordinate information of the exhibited item, the third data are acquired through a depth camera, and the fourth data are distance information of the exhibited item, and the fourth data are acquired through an ultrasonic sensor;
and the exhibit identification unit is used for carrying out exhibit identification based on the optimized Yolo v5 algorithm and auxiliary identification data to obtain an exhibit identification result.
Optionally, the gesture recognition module includes:
the fourth acquisition unit is used for acquiring gesture pictures of observers; wherein, the collection is carried out by a depth camera;
the first extraction unit is used for extracting hand 3D key points in the gesture picture; wherein, extracting by a HandPointNet algorithm;
and the first recognition unit is used for carrying out gesture recognition of the observer according to the hand 3D key points to obtain a gesture recognition result.
Optionally, the track generation module,
or comprises:
the fifth acquisition unit is used for acquiring the posture and motion information of the observer in a preset period; wherein, the acquisition is carried out by a gyroscope or an accelerometer;
a first determining unit for determining a position of the observer within a preset period according to the posture and motion information of the observer;
the third generation unit is used for generating a motion track of the observer in a preset period according to the position of the observer in the preset period;
or comprises:
a sixth acquisition unit for acquiring an image of the observer within a preset period;
a second determining unit for determining a position of the observer within a preset period according to the image of the observer;
a fourth generation unit, configured to generate a motion trail of the observer in a preset period according to the position of the observer in the preset period;
or comprises:
the seventh acquisition unit is used for acquiring wifi signal intensity information of the mobile terminal held by the observer in a preset period;
the third determining unit is used for determining the position of the observer in a preset period according to the wifi signal intensity information; the position of the observer in the preset period is the position of the mobile terminal held by the observer;
and the fifth generation unit is used for generating a motion track of the observer in the preset period according to the position of the observer in the preset period.
Optionally, the human body posture estimation module,
or comprises:
an eighth acquisition unit for acquiring depth images of the observers; wherein, the collection is carried out by a depth camera;
the second extraction unit is used for extracting 2D human body key point coordinates in the depth image; extracting by a 2D CNN algorithm;
the first conversion unit is used for carrying out coordinate conversion according to the depth information corresponding to the depth image and the 2D human body key point coordinates to obtain 3D human body key point coordinates;
the first posture estimation unit is used for estimating the human body posture of the observer based on a 3D posture regression algorithm and 3D human body key point coordinates to obtain a human body posture estimation result;
or comprises:
a ninth acquisition unit for acquiring three-dimensional coordinate information of the observer; wherein, the acquisition is carried out by a laser radar;
a sixth generation unit for generating a depth image according to the three-dimensional coordinate information of the observer;
the third extraction unit is used for extracting 2D human body key point coordinates in the depth image; extracting by a 2D CNN algorithm;
the second conversion unit is used for carrying out coordinate conversion according to the depth information corresponding to the depth image and the 2D human body key point coordinates to obtain 3D human body key point coordinates;
the second posture estimation unit is used for estimating the human body posture of the observer based on the 3D posture regression algorithm and the 3D human body key point coordinates to obtain a human body posture estimation result.
The intelligent display system provided by the invention comprises a holographic projection module, a naked eye 3D display module, a scene reduction module, an interactive projection module and a somatosensory interaction module, and has the following specific effects: (1) The holographic projection module presents various exhibits in a holographic projection mode, so that observers can more intuitively know the culture, history or three-dimensional form of the exhibits; (2) The naked eye 3D display module provides another three-dimensional morphological technology for presenting the exhibits, and the technology is applied to the exhibition hall, so that the exhibits and scenes related to the exhibits can be presented, and observers can more intuitively know the history and culture of the exhibits; (3) Aiming at some special exhibits, a specific implementation process or a specific display mode is required to be analyzed for an observer, at the moment, the scene reduction module can well meet the requirements, and the scene reduction enables the observer to feel contents in the aspects of history, culture, nature and the like in an immersive manner, and also can help to protect the real exhibits and reduce damage or abrasion of the exhibits; (4) The interactive projection module combines the projection technology and the interactive technology, so that more real and immersed observation experience can be provided, different exhibition and interaction modes are needed according to exhibition and interaction requirements of different exhibits, and more natural observation interaction experience can be provided for observers through the interactive projection module; (5) Through the somatosensory interaction module, observers can realize interaction with the exhibits through limb actions, and experience is strong. In general, through the system provided by the invention, a viewer can obtain better viewing experience through more advanced technology, a multi-dimensional display effect can be obtained during exhibition in an exhibition hall, the system is not limited to two-dimensional display and a single viewing angle, and more three-dimensional effects can be obtained; meanwhile, the interaction modes are also diversified.
Drawings
Fig. 1 is a structural diagram of an intelligent display system of an intelligent exhibition hall according to an embodiment of the present invention;
fig. 2 is a structural diagram of an intelligent display system of an intelligent exhibition hall according to a second embodiment of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Example 1
The embodiment of the invention provides an intelligent display system of an intelligent exhibition hall, as shown in fig. 1, comprising:
a holographic projection module 10 for providing a three-dimensional image of an exhibit to an observer; wherein the three-dimensional image is a floating image;
the naked eye 3D display module 11 is used for providing a three-dimensional image of the exhibit to the observer; wherein the three-dimensional image is a screen image;
a scene reduction module 12 for displaying a reduction scene related to the exhibits to the observers;
an interactive projection module 13 for interaction between the viewer and the projected content, wherein the projected content is related to the exhibit;
the somatosensory interaction module 14 is used for enabling observers to interact with the exhibits through limb actions.
The intelligent display system provided by the embodiment comprises a holographic projection module 10, a naked eye 3D display module 11, a scene reduction module 12, an interactive projection module 13 and a somatosensory interaction module 14, and the specific effects include: (1) The holographic projection module 10 presents various exhibits in a holographic projection mode, so that observers can more intuitively know the culture, history or three-dimensional form of the exhibits; (2) The naked eye 3D display module 11 provides another three-dimensional morphological technology for presenting the exhibits, and the technology is applied to the exhibition hall, so that the exhibits and scenes related to the exhibits can be presented, and observers can more intuitively know the history and culture of the exhibits; (3) Aiming at some special exhibits, a specific implementation process or a display mode needs to be analyzed for an observer, at this time, the scene reduction module 12 can well meet the requirements, and the scene reduction enables the observer to feel contents in the aspects of history, culture, nature and the like in an immersive manner, and also can help to protect the real exhibits and reduce damage or abrasion of the exhibits; (4) The interactive projection module 13 combines the projection technology and the interactive technology, so that more real and immersed observation experience can be provided, different exhibition and interaction modes are required according to exhibition and interaction requirements of different exhibits, and more natural observation interaction experience can be provided for observers through the interactive projection module 13; (5) Through the somatosensory interaction module 14, observers can realize interaction with the exhibited items through limb actions, and experience is strong.
In general, through the system provided by the embodiment, a viewer can obtain better viewing experience through a more advanced technology, and a multi-dimensional display effect can be obtained during exhibition in an exhibition hall, so that the system is not limited to two-dimensional display and a single viewing angle, and more three-dimensional effects can be obtained; meanwhile, the interaction modes are also diversified.
Optionally, the present embodiment further includes:
(1) An LED screen;
(2) Touch screen, a display device that can be operated by finger touch. The interactive display can be realized, so that a viewer can acquire display information, navigate, perform interactive games and the like by touching the screen;
(3) A projector, in an exhibition hall, the projector can show videos for introducing exhibits, demonstration PPT, etc.;
(4) The intelligent television can acquire various media resources such as video, audio, images and the like through the Internet, and can play videos for introducing exhibits in an exhibition hall;
(5) The multimedia terminal can play videos for introducing exhibits, demonstrate PPT, allow observers to interact with the exhibits, and the like.
Optionally, the somatosensory interaction module 14 integrates 3D somatosensory photography, somatosensory interaction software and three-dimensional digital content, captures real-time video stream or depth image of the observer, emits infrared rays to carry out three-dimensional positioning on the whole exhibition hall, meanwhile, the camera recognizes motion of the observer by means of the infrared rays, the motion sensor tracks motion of the whole body of the observer, and a skeleton of the observer can be built according to data, so that multiple parts of the observer can be tracked in real time, and the gesture of the observer is calculated by utilizing a recognition processing algorithm, and limb motion of the observer is recognized, so that the observer can directly control the limb motion of the observer, and the screen image is controlled in a high altitude.
Optionally, the embodiment may further add a voice recognition module for the observer to interact, so as to watch different exhibited contents.
Optionally, the embodiment may further incorporate an intelligent management module to optimize operation and maintenance of the exhibition hall. Specifically, the intelligent management module utilizes the internet of things technology to realize remote monitoring and fault early warning of other various modules, shortens maintenance time, reduces maintenance cost, and can optimize the layout of an exhibition hall and the content of an exhibited item through data analysis and an intelligent algorithm, so that the satisfaction degree of an observer is improved.
Example two
The embodiment of the invention also provides an intelligent display system of an intelligent exhibition hall, which is the same as the first embodiment and is not repeated, as shown in fig. 2, and the difference is that: the intelligent display system of the intelligent exhibition hall also comprises a content recommendation module 15; the content recommendation module 15 is used for recommending exhibit information to observers. By adding the content recommendation module 15, the propaganda effect of the exhibition hall can be optimized, so that the relationship between the exhibition hall and the observers is tighter.
Example III
The embodiment of the present invention also provides an intelligent display system of an intelligent exhibition hall, which is the same as the second embodiment, and is not described in detail, and the difference is that the content recommendation module 15 includes:
the first acquisition unit is used for acquiring behavior data of a spectator in a real exhibition hall and/or a virtual exhibition hall;
the first preprocessing unit is used for preprocessing the behavior data to obtain first data corresponding to the behavior data;
the first generation unit is used for generating behavioral characteristics of the observer according to the first data;
the content recommending unit is used for inputting the behavior characteristics into a preset predicting model to obtain a predicting result and recommending the exhibit information corresponding to the predicting result to the observers; the prediction result is the type of the exhibit of interest to the observer.
In the intelligent display system provided by the embodiment, the first acquisition unit acquires the behavior data, the first preprocessing unit preprocesses the behavior data to obtain first data corresponding to the behavior data, the first generation unit analyzes and excavates the first data to obtain the behavior characteristics of the observer, the content recommendation unit judges the correlation degree between the current behavior characteristics and the type of the exhibited item based on the real-time behavior characteristic information, automatically recommends the exhibited item information which is most suitable for the observer, and helps the observer to find and acquire the interested exhibited item information. In general, by the system provided by the embodiment, the exhibit information can be recommended to the observers in a targeted manner, and the interest of the observers in continuous observation is cultivated.
Optionally, (1) behavioral data includes, but is not limited to, historical behavior, hobbies, click-through rates of exhibits, residence time for viewing exhibits, search records, frequency of repeated viewing; (2) with respect to training a predictive model, the specific process includes: collecting behavior data of a viewer in an exhibition hall, preprocessing the behavior data to obtain first data, generating behavior characteristics of the viewer according to the first data, performing characteristic engineering and vectorization processing on the behavior characteristics to obtain fifth data corresponding to the behavior characteristics, and modeling based on a machine learning algorithm and the fifth data to obtain a prediction model; further starting a prediction model, and recommending exhibit information to observers according to the prediction result; (3) After the prediction model is started, model evaluation optimization is performed by using a cross-validation method, and the optimal model parameters and characteristic combinations are found, so that the accuracy and generalization capability of the model are improved, and the satisfaction of observers is improved; (4) The first acquisition unit can be a part installed in the exhibition hall, such as a sensor and a monitoring camera, or can be an exhibition hall website; (5) Preprocessing content of the first preprocessing unit includes, but is not limited to, removing duplicate data, processing missing values; (6) The behavior features generated by the first generating unit include, but are not limited to, the number of clicks, residence time, search keywords, and the number of times video is viewed, and the behavior features may be processed and combined to generate new behavior features, such as average residence time, and the popularity of the search keywords.
Example IV
The embodiment of the invention also provides an intelligent display system of the intelligent exhibition hall, which is the same as the first embodiment and is not repeated, and the difference is that the intelligent display system of the intelligent exhibition hall also comprises an augmented reality module and/or a virtual reality module for providing immersive interaction for observers. The augmented reality module and the virtual reality module can both present the exhibits in a digital form for the observers to play the interactive game, and through providing immersive interaction for the observers, the participation of the observers can be improved, so that the observers obtain personalized observation experience, and the satisfaction of the observers is improved.
The specific effects include:
(1) The virtual reality module can provide a guiding system, and by creating a virtual environment around the real exhibits, observers can walk around by wearing helmets of the virtual reality module, and can freely move in the virtual environment, so that the observers can observe the exhibits better, and acquire more exhibits information and multimedia contents; meanwhile, the navigation system can provide better interactivity, for example, a viewer can interact with the exhibited article by using gestures or a controller of the virtual reality module. Regarding the use example of the virtual reality module, a 3D exhibit can be created and displayed in the virtual environment, so that the observer can enjoy the exhibit freely, and then a virtual environment is created, so that the observer can use the controller to control the exhibit, and know the background and design process of the exhibit, etc.
(2) And the augmented reality module can create a viewing experience with stronger immersion and interactivity by combining the augmented reality with a real exhibit. For example, using augmented reality technology to create a virtual environment on a viewer's mobile terminal (e.g., a cell phone and tablet computer) and combine the virtual image with a real exhibit, this can provide more exhibit information and multimedia content to the viewer.
Example five
The embodiment of the present invention further provides an intelligent display system of an intelligent exhibition hall, which is the same as the first embodiment, and is not described in detail, and the difference is that the scene reduction module 12 includes:
a scene creation unit for creating a scene in the virtual exhibition hall;
the second acquisition unit is used for acquiring real data of a real exhibition hall; the real data comprise scene data and exhibit data;
the second preprocessing unit is used for preprocessing the real data and obtaining second data corresponding to the real data;
a second generation unit for generating a three-dimensional model from the second data; wherein the three-dimensional model corresponds to a scene or an exhibit;
the rendering unit is used for rendering the three-dimensional model according to the real data;
an adding unit for adding picture control logic and a viewing path for a viewer in the virtual exhibition hall;
and the display unit is used for displaying the virtual exhibition hall.
Optionally, (1) the scene creation unit creates a scene by using Unity 3D software, and adds elements such as a light source, a shadow, a ground, etc. in the scene; (2) The second acquisition unit is at least one of a laser radar, a camera and a scanner; (3) The second preprocessing unit processes the acquired real data by utilizing three-dimensional reconstruction software; (4) The second generating unit generates a three-dimensional model by utilizing three-dimensional reconstruction software; (5) The rendering unit performs 3D rendering by using Unity 3D software, designs textures, materials and the like on the three-dimensional model, simulates light and environment, and renders a real 3D scene; (6) The picture control logic added by the adding unit is scripts and animations, so that the display can be rotated, zoomed, moved and the like in the display process to display different angles and characteristics of the display, the display path is fixed, a mobile camera is used for shooting along the display path, the display picture is continuously played in a film fragment mode, and then the lens of the mobile camera is controlled to randomly switch each scene; (7) The display unit displays the virtual exhibition hall in the form of roaming animation.
Example six
The embodiment of the invention also provides an intelligent display system of the intelligent exhibition hall, which is the same as the first embodiment and is not repeated, and the difference is that: the interactive projection module 13 includes:
the exhibit identification module is used for identifying the exhibits and obtaining an exhibit identification result;
the gesture recognition module is used for recognizing gestures of the observers and obtaining gesture recognition results;
the track generation module is used for generating a motion track of the observer in a preset period;
the human body posture estimation module is used for carrying out human body posture estimation on the observers to obtain a human body posture estimation result.
Specifically, (1) the exhibit identification module utilizes ultrasonic detection technology or image identification technology to identify the exhibit, and after the exhibit identification result is obtained, the exhibit identification module can be used for triggering corresponding actions; (2) The gesture recognition module recognizes the gesture of the observer by utilizing an image recognition technology, and can be used for triggering corresponding actions after a gesture recognition result is obtained; (3) The track generation module generates a motion track of an observer in a preset period by using a sensor detection technology or an image recognition technology, and can be used for triggering corresponding actions after the motion track is obtained; (4) The human body posture estimation module performs human body posture estimation on the observer by using an image recognition technology or a radar detection technology, and can be used for triggering corresponding actions after a human body posture estimation result is obtained.
Therefore, the interactive projection module 13 integrates multiple monitoring modes such as gestures, body states, positions and the like, and monitors the states of observers from multiple ways, so that the observers can select different interaction modes according to the display and interaction requirements of different exhibits, and thus, more natural, more real and more immersive interaction experience is provided.
Example seven
The embodiment of the invention also provides an intelligent display system of an intelligent exhibition hall, which is the same as the sixth embodiment and is not repeated, and the difference is that the exhibit identification module comprises:
the third acquisition unit is used for acquiring auxiliary identification data; the auxiliary identification data comprise third data and/or fourth data, wherein the third data are three-dimensional coordinate information of the exhibited item, the third data are acquired through a depth camera, and the fourth data are distance information of the exhibited item, and the fourth data are acquired through an ultrasonic sensor;
and the exhibit identification unit is used for carrying out exhibit identification based on the optimized Yolo v5 algorithm and auxiliary identification data to obtain an exhibit identification result.
Specifically, (1) the third acquisition unit acquires auxiliary identification data by utilizing an ultrasonic technology and/or an image identification technology so as to be used for subsequent exhibit identification; (2) The exhibit identification unit adopts an optimized Yolo v5 algorithm, the algorithm has high precision, good real-time performance, light weight and strong flexibility, auxiliary identification data are combined during identification, accuracy of exhibit identification can be improved, and after the exhibit identification result is obtained, the algorithm can be used for triggering corresponding actions.
Example eight
The embodiment of the invention also provides an intelligent display system of an intelligent exhibition hall, which is the same as the sixth embodiment and is not repeated, and the difference is that the gesture recognition module comprises:
the fourth acquisition unit is used for acquiring gesture pictures of observers; wherein, the collection is carried out by a depth camera;
the first extraction unit is used for extracting hand 3D key points in the gesture picture; wherein, extracting by a HandPointNet algorithm;
and the first recognition unit is used for carrying out gesture recognition of the observer according to the hand 3D key points to obtain a gesture recognition result.
Specifically, (1) the fourth acquisition unit acquires gesture pictures of the observers by using an image technology, so that the subsequent acquisition of 3D key points of the hands is facilitated; (2) The HandPointNet algorithm adopted by the first extraction unit has good real-time performance, robustness and accuracy; (3) The first recognition unit performs gesture recognition of the observer according to the hand 3D key points, and after a gesture recognition result is obtained, the first recognition unit can be used for triggering corresponding actions.
Example nine
The embodiment of the invention also provides an intelligent display system of an intelligent exhibition hall, which is the same as the sixth embodiment and is not repeated, and the difference is that a track generating module,
or comprises:
the fifth acquisition unit is used for acquiring the posture and motion information of the observer in a preset period; wherein, the acquisition is carried out by a gyroscope or an accelerometer;
a first determining unit for determining a position of the observer within a preset period according to the posture and motion information of the observer;
the third generation unit is used for generating a motion track of the observer in a preset period according to the position of the observer in the preset period;
or comprises:
a sixth acquisition unit for acquiring an image of the observer within a preset period;
a second determining unit for determining a position of the observer within a preset period according to the image of the observer;
a fourth generation unit, configured to generate a motion trail of the observer in a preset period according to the position of the observer in the preset period;
or comprises:
the seventh acquisition unit is used for acquiring wifi signal intensity information of the mobile terminal held by the observer in a preset period;
the third determining unit is used for determining the position of the observer in a preset period according to the wifi signal intensity information; the position of the observer in the preset period is the position of the mobile terminal held by the observer;
and the fifth generation unit is used for generating a motion track of the observer in the preset period according to the position of the observer in the preset period.
In particular, the method comprises the steps of,
when the track generation module comprises a fifth acquisition unit, a first determination unit and a third generation unit, (1) the fifth acquisition unit acquires the posture and motion information of the observer by using a Micro-Electro-Mechanical Systems system for subsequent determination of the position of the observer; (2) The first determining unit determines the position of the observer in a preset period according to the posture and the motion information of the observer, and is used for generating a motion trail subsequently; (3) The third generating unit generates a motion trail of the observer in a preset period according to the position of the observer in the preset period, and after the motion trail is obtained, different actions can be triggered according to different motion trail. The module is suitable for being carried in mobile terminals, such as mobile phones and watches, has the advantages of small size, low power consumption and quick response, and can accurately capture motion trajectories.
When the track generating module comprises a sixth acquisition unit, a second determining unit and a fourth generating unit, (1) the sixth acquisition unit acquires an image of the observer by using an image technology, so as to be used for determining the position of the observer subsequently; (2) The second determining unit determines the position of the observer in a preset period according to the image of the observer, and is used for generating a motion trail subsequently; (3) The fourth generation unit generates a motion trail of the observer in a preset period according to the position of the observer in the preset period, and after the motion trail is obtained, different actions can be triggered according to different motion trail. The foregoing modules are particularly effective in indoor environments, and can capture more detailed observer status information.
When the track generation module comprises a seventh acquisition unit, a third determination unit and a fifth generation unit, (1) the seventh acquisition unit acquires wifi signal strength information by using a network transmission technology, so as to be used for subsequently determining the position of the observer; (2) The third determining unit determines the position of the observer in a preset period according to wifi signal intensity information, and the position is used for generating a motion track subsequently; (3) The fifth generation unit generates a motion trail of the observer in a preset period according to the position of the observer in the preset period, and after the motion trail is obtained, different actions can be triggered according to different motion trail. The module has wide coverage and is suitable for indoor and outdoor environments.
Examples ten
The embodiment of the invention also provides an intelligent display system of an intelligent exhibition hall, which is the same as the sixth embodiment and is not repeated, and the difference is that a human body posture estimation module,
or comprises:
an eighth acquisition unit for acquiring depth images of the observers; wherein, the collection is carried out by a depth camera;
the second extraction unit is used for extracting 2D human body key point coordinates in the depth image; extracting by a 2D CNN algorithm;
the first conversion unit is used for carrying out coordinate conversion according to the depth information corresponding to the depth image and the 2D human body key point coordinates to obtain 3D human body key point coordinates;
the first posture estimation unit is used for estimating the human body posture of the observer based on a 3D posture regression algorithm and 3D human body key point coordinates to obtain a human body posture estimation result;
or comprises:
a ninth acquisition unit for acquiring three-dimensional coordinate information of the observer; wherein, the acquisition is carried out by a laser radar;
a sixth generation unit for generating a depth image according to the three-dimensional coordinate information of the observer;
the third extraction unit is used for extracting 2D human body key point coordinates in the depth image; extracting by a 2D CNN algorithm;
the second conversion unit is used for carrying out coordinate conversion according to the depth information corresponding to the depth image and the 2D human body key point coordinates to obtain 3D human body key point coordinates;
the second posture estimation unit is used for estimating the human body posture of the observer based on the 3D posture regression algorithm and the 3D human body key point coordinates to obtain a human body posture estimation result.
In particular, the method comprises the steps of,
when the human body posture estimation module comprises an eighth acquisition unit, a second extraction unit, a first conversion unit and a first posture estimation unit, (1) the eighth acquisition unit acquires a depth image of the observer by using an image technology, so as to be used for subsequent key point coordinate extraction; (2) The 2D CNN algorithm adopted by the second extraction unit is very efficient in image processing, and the calculation can be processed in parallel, so that the method is suitable for parallel computing equipment such as a Graphic Processing Unit (GPU); (3) The first conversion unit performs coordinate conversion according to the depth information corresponding to the depth image and the 2D human body key point coordinates to obtain 3D human body key point coordinates for subsequent posture estimation; (4) The 3D gesture regression algorithm adopted by the first gesture estimation unit can directly predict the three-dimensional gesture of the observer, and has better robustness in processing occlusion and visual angle change. The module provides abundant depth information, the depth camera can directly acquire the depth information of different parts of the body of the observer, provides abundant three-dimensional space information and is beneficial to more accurately estimating the human body posture; meanwhile, the depth image is not influenced by shielding and visual angle change, and the stability is good;
when the human body posture estimation module comprises a ninth acquisition unit, a sixth generation unit, a third extraction unit, a second conversion unit and a second posture estimation unit, (1) the ninth acquisition unit acquires three-dimensional coordinate information of an observer by utilizing a radar technology, so as to be used for generating a depth image subsequently; (2) A sixth generation unit generates a depth image according to the three-dimensional coordinate information of the observer, so as to be used for subsequent coordinate extraction of the key points; (3) The third extraction unit and the second extraction unit have the same function; (4) The second conversion unit and the first conversion unit have the same function; (5) The second posture estimation unit and the first posture estimation unit function identically. The three-dimensional coordinate information acquired by the module is accurate and stable, and accurate human body posture estimation results are obtained.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, apparatus, article or method that comprises the element.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the invention, and all equivalent structures or equivalent processes using the descriptions and drawings of the present invention or direct or indirect application in other related technical fields are included in the scope of the present invention.
Claims (2)
1. An intelligent display system for an intelligent exhibition hall, comprising:
the holographic projection module is used for providing a three-dimensional image of the exhibit for the observer; wherein the three-dimensional image is a floating image;
the naked eye 3D display module is used for providing three-dimensional images of the exhibits for observers; wherein the three-dimensional image is a screen image;
the scene reduction module is used for displaying reduction scenes related to the exhibits to observers;
the interactive projection module is used for enabling a viewer to interact with the projection content, wherein the projection content is related to the exhibited article;
the somatosensory interaction module is used for enabling observers to interact with the exhibits through limb actions;
the intelligent display system of the intelligent exhibition hall further comprises a content recommendation module; the content recommendation module is used for recommending exhibit information to observers;
the content recommendation module includes:
the first acquisition unit is used for acquiring behavior data of a spectator in a real exhibition hall and/or a virtual exhibition hall;
the first preprocessing unit is used for preprocessing the behavior data to obtain first data corresponding to the behavior data;
the first generation unit is used for generating behavioral characteristics of the observer according to the first data;
the content recommending unit is used for inputting the behavior characteristics into a preset predicting model to obtain a predicting result and recommending the exhibit information corresponding to the predicting result to the observers; the prediction result is the type of the exhibit of interest to the observer;
the scene reduction module comprises:
a scene creation unit for creating a scene in the virtual exhibition hall;
the second acquisition unit is used for acquiring real data of a real exhibition hall; wherein the real data comprises scene data or exhibit data;
the second preprocessing unit is used for preprocessing the real data and obtaining second data corresponding to the real data;
a second generation unit for generating a three-dimensional model from the second data; wherein the three-dimensional model corresponds to a scene or an exhibit;
the rendering unit is used for rendering the three-dimensional model according to the real data;
an adding unit for adding picture control logic and a viewing path for a viewer in the virtual exhibition hall;
the display unit is used for displaying the virtual exhibition hall;
the interactive projection module comprises:
the exhibit identification module is used for identifying the exhibits and obtaining an exhibit identification result;
the gesture recognition module is used for recognizing gestures of the observers and obtaining gesture recognition results;
the track generation module is used for generating a motion track of the observer in a preset period;
the human body posture estimation module is used for carrying out human body posture estimation on the observers to obtain a human body posture estimation result;
the exhibit identification module comprises:
the third acquisition unit is used for acquiring auxiliary identification data; the auxiliary identification data comprise third data and/or fourth data, wherein the third data are three-dimensional coordinate information of the exhibited item, the third data are acquired through a depth camera, and the fourth data are distance information of the exhibited item, and the fourth data are acquired through an ultrasonic sensor;
the exhibit identification unit is used for carrying out exhibit identification based on the optimized Yolo v5 algorithm and auxiliary identification data to obtain an exhibit identification result;
the gesture recognition module comprises:
the fourth acquisition unit is used for acquiring gesture pictures of observers; wherein, the collection is carried out by a depth camera;
the first extraction unit is used for extracting hand 3D key points in the gesture picture; wherein, extracting by a HandPointNet algorithm;
the first recognition unit is used for carrying out gesture recognition of the observer according to the hand 3D key points to obtain a gesture recognition result;
the track generation module comprises:
the fifth acquisition unit is used for acquiring the posture and motion information of the observer in a preset period; wherein, the acquisition is carried out by a gyroscope or an accelerometer;
a first determining unit for determining a position of the observer within a preset period according to the posture and motion information of the observer;
the third generation unit is used for generating a motion track of the observer in a preset period according to the position of the observer in the preset period;
alternatively, the track generation module includes:
a sixth acquisition unit for acquiring an image of the observer within a preset period;
a second determining unit for determining a position of the observer within a preset period according to the image of the observer;
a fourth generation unit, configured to generate a motion trail of the observer in a preset period according to the position of the observer in the preset period;
alternatively, the track generation module includes:
the seventh acquisition unit is used for acquiring wifi signal intensity information of the mobile terminal held by the observer in a preset period;
the third determining unit is used for determining the position of the observer in a preset period according to the wifi signal intensity information; the position of the observer in the preset period is the position of the mobile terminal held by the observer;
a fifth generation unit, configured to generate a motion trail of the observer in a preset period according to the position of the observer in the preset period;
the human body posture estimation module includes:
an eighth acquisition unit for acquiring depth images of the observers; wherein, the collection is carried out by a depth camera;
the second extraction unit is used for extracting 2D human body key point coordinates in the depth image; extracting by a 2D CNN algorithm;
the first conversion unit is used for carrying out coordinate conversion according to the depth information corresponding to the depth image and the 2D human body key point coordinates to obtain 3D human body key point coordinates;
the first posture estimation unit is used for estimating the human body posture of the observer based on a 3D posture regression algorithm and 3D human body key point coordinates to obtain a human body posture estimation result;
alternatively, the human body posture estimation module includes:
a ninth acquisition unit for acquiring three-dimensional coordinate information of the observer; wherein, the acquisition is carried out by a laser radar;
a sixth generation unit for generating a depth image according to the three-dimensional coordinate information of the observer;
the third extraction unit is used for extracting 2D human body key point coordinates in the depth image; extracting by a 2D CNN algorithm;
the second conversion unit is used for carrying out coordinate conversion according to the depth information corresponding to the depth image and the 2D human body key point coordinates to obtain 3D human body key point coordinates;
the second posture estimation unit is used for estimating the human body posture of the observer based on the 3D posture regression algorithm and the 3D human body key point coordinates to obtain a human body posture estimation result.
2. The intelligent display system of an intelligent exhibition hall according to claim 1, further comprising an augmented reality module and/or a virtual reality module that provides immersive interactions to the spectators.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311004565.2A CN116719176B (en) | 2023-08-10 | 2023-08-10 | Intelligent display system of intelligent exhibition hall |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311004565.2A CN116719176B (en) | 2023-08-10 | 2023-08-10 | Intelligent display system of intelligent exhibition hall |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116719176A CN116719176A (en) | 2023-09-08 |
| CN116719176B true CN116719176B (en) | 2023-09-29 |
Family
ID=87870211
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311004565.2A Active CN116719176B (en) | 2023-08-10 | 2023-08-10 | Intelligent display system of intelligent exhibition hall |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116719176B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117519480B (en) * | 2023-11-20 | 2024-07-30 | 广东鸿威国际会展集团有限公司 | Exhibition information interaction and processing method and system |
| CN117312477B (en) * | 2023-11-28 | 2024-02-20 | 北京三月雨文化传播有限责任公司 | AR technology-based indoor intelligent exhibition positioning method, device, equipment and medium |
| CN117970823B (en) * | 2024-01-08 | 2024-12-20 | 中凯智慧物联科技(广东)有限公司 | Central control system of digital multimedia exhibition hall |
| CN117971041B (en) * | 2024-01-10 | 2025-06-06 | 广州市新芯信息科技股份有限公司 | Interactive control method and system of intelligent experimental cabin based on image recognition |
| CN117826557A (en) * | 2024-01-15 | 2024-04-05 | 广州美术学院 | A mobile multi-dimensional holographic projection display system |
| CN119200845B (en) * | 2024-09-12 | 2025-03-18 | 北京三月雨文化传播有限责任公司 | Interactive projection optimization method for exhibition halls |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107957782A (en) * | 2017-12-18 | 2018-04-24 | 广东互动电子网络媒体有限公司 | A kind of museum's Interactive Experience system |
| CN114035688A (en) * | 2021-11-12 | 2022-02-11 | 苏州和氏设计营造股份有限公司 | Exhibition hall multimedia exhibition item design method and device based on VR technology |
| CN114357344A (en) * | 2021-12-10 | 2022-04-15 | 德马吉国际展览有限公司 | Interactive visual cloud exhibition system and method based on Internet technology |
| CN114967925A (en) * | 2022-05-27 | 2022-08-30 | 杭州师范大学 | Use digital museum of VR panorama technique |
-
2023
- 2023-08-10 CN CN202311004565.2A patent/CN116719176B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107957782A (en) * | 2017-12-18 | 2018-04-24 | 广东互动电子网络媒体有限公司 | A kind of museum's Interactive Experience system |
| CN114035688A (en) * | 2021-11-12 | 2022-02-11 | 苏州和氏设计营造股份有限公司 | Exhibition hall multimedia exhibition item design method and device based on VR technology |
| CN114357344A (en) * | 2021-12-10 | 2022-04-15 | 德马吉国际展览有限公司 | Interactive visual cloud exhibition system and method based on Internet technology |
| CN114967925A (en) * | 2022-05-27 | 2022-08-30 | 杭州师范大学 | Use digital museum of VR panorama technique |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116719176A (en) | 2023-09-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116719176B (en) | Intelligent display system of intelligent exhibition hall | |
| Rabbi et al. | A survey on augmented reality challenges and tracking | |
| CN106484115B (en) | Systems and methods for augmented and virtual reality | |
| US20120200667A1 (en) | Systems and methods to facilitate interactions with virtual content | |
| EP3106963B1 (en) | Mediated reality | |
| CN111862348B (en) | Video display method, video generation method, device, equipment and storage medium | |
| CN117333645A (en) | Annular holographic interaction system and equipment thereof | |
| US20180239514A1 (en) | Interactive 3d map with vibrant street view | |
| Wang et al. | Transitioning360: Content-aware nfov virtual camera paths for 360 video playback | |
| Chen et al. | Context-aware mixed reality: A framework for ubiquitous interaction | |
| US12277655B2 (en) | Traveling in time and space continuum | |
| US11961190B2 (en) | Content distribution system, content distribution method, and content distribution program | |
| Rematas | Watching sports in augmented reality | |
| CN118118643B (en) | A video data processing method and related device | |
| Tukur et al. | Panoramic imaging in immersive extended reality: a scoping review of technologies, applications, perceptual studies, and user experience challenges | |
| Kim et al. | A view direction-driven approach for automatic room mapping in mixed reality | |
| Kim et al. | 3-d virtual studio for natural inter-“acting” | |
| US20240394992A1 (en) | Interactive virtual reality environment based on biomimicry | |
| CN114302234A (en) | Air skill rapid packaging method | |
| Billinghurst | Foundations and TrendsR• in Human-Computer Interaction Vol. 8, No. 2-3 (2014) 73–272 c• 2015 M. Billinghurst, A. Clark, and G. Lee | |
| CN116934959A (en) | Particle image generation method and device based on gesture recognition, electronic equipment and medium | |
| Khan et al. | TransUser's: a transformer based salient object detection for users experience generation in 360 videos | |
| Beimler et al. | Smurvebox: A smart multi-user real-time virtual environment for generating character animations | |
| Hamadouche | Augmented reality X-ray vision on optical see-through head mounted displays | |
| Teixeira et al. | Analysis of XR research in brazil from 21 years of SVR publications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |