CN107767460A - The methods of exhibiting and device of augmented reality - Google Patents

Abstract

The present invention relates to a display method of augmented reality. The method includes: calling a camera to identify items in a display area, and obtaining feature information of the items; querying matching preset pictures according to the feature information; and calculating in real time Recognition duration, judging whether the preset picture is queried within the preset duration; if yes, then obtain the visual element corresponding to the preset picture; otherwise, call the radio frequency identification function and the radio frequency identification tag in the display area performing a connection to obtain the visual element corresponding to the radio frequency identification tag; displaying the visual element in the display area to form an augmented reality display effect. The above augmented reality display method can improve the accuracy and speed of obtaining visual elements corresponding to items in the display area. In addition, an augmented reality display device is correspondingly provided.

Description

Augmented reality display method and device

technical field

The present invention relates to the technical field of terminal applications, in particular to an augmented reality display method and device.

Background technique

Augmented Reality (Augmented Reality, AR) is a technology that calculates the position and angle of camera images in real time and adds corresponding images, videos, and 3D models. The goal of this technology is to put the virtual world on the screen. world and interact with it.

The traditional method is to scan the scanned object, then obtain a video corresponding to the scanned object, and play the video. However, the traditional technology has very strict requirements on the application environment, and is easily limited by factors such as light, so it cannot accurately identify the scanned object, resulting in low recognition efficiency.

Contents of the invention

Based on this, it is necessary to provide an augmented reality display method and device capable of improving the recognition efficiency of scanned items for the above technical problems.

A method for displaying augmented reality, said method comprising:

Call the camera to identify the items in the display area, and obtain the feature information of the items;

Searching for matching preset pictures according to the feature information;

Calculate the recognition time in real time, and judge whether the preset picture is queried within the preset time;

If so, obtain the visual element corresponding to the preset picture;

Otherwise, call the radio frequency identification function to connect with the radio frequency identification tag in the display area, and obtain the visual element corresponding to the radio frequency identification tag;

The visual elements are displayed in the display area to form an augmented reality display effect.

In one of the embodiments, when the queried preset pictures include multiple ones, the step of acquiring the visual elements corresponding to the preset pictures includes:

Detect whether the visual elements corresponding to each preset picture are the same, and if not, obtain the visual elements corresponding to each preset picture;

The step of displaying the visual elements in the display area includes:

Display the visual element in a position that matches the item.

In one of the embodiments, when there are multiple radio frequency identification tags in the display area, the step of obtaining the visual elements corresponding to the radio frequency identification tags includes:

Detect whether the visual elements corresponding to each radio frequency identification tag are the same, if not, obtain the visual elements corresponding to different items;

The step of displaying the visual elements in the display area includes:

The visual elements are evenly displayed in split screens according to the quantity of the different items.

In one of the embodiments, after the step of displaying the visual elements in the display area, it includes:

When it is detected that the item is not in the display area, stop displaying the visual element in the display area.

In one of the embodiments, when there are multiple visual elements corresponding to the preset picture, the step of displaying the visual elements in the display area includes:

Select one of the plurality of visual elements corresponding to the preset picture, and display the selected visual element in the display area in a loop; or

The multiple visual elements corresponding to the preset picture are displayed in a preset display order.

A display device for augmented reality, said device comprising:

The feature information acquisition module is used to call the camera to identify the items in the display area, and acquire the feature information of the items;

A preset picture matching module, configured to search for a matched preset picture according to the feature information;

A judging module, configured to calculate the recognition time in real time, and judge whether the preset picture is queried within the preset time;

A visual element acquisition module, configured to acquire visual elements corresponding to the preset picture;

The radio frequency identification module is used to call the radio frequency identification function to connect with the radio frequency identification tags in the display area when the preset picture is not queried within the preset time period;

The visual element obtaining module is also used to obtain the visual element corresponding to the radio frequency identification tag;

The display module is configured to display the visual elements in the display area to form an augmented reality display effect.

In one of the embodiments, the device also includes:

The first detection module is used to detect the number of preset pictures found;

The visual element acquisition module is also used to detect whether the visual elements corresponding to each preset picture are the same when the queried preset pictures include multiple ones, and if not, obtain the corresponding visual elements of each preset picture. view element;

The display module is also used to display the visual element in a position that matches the item.

In one of the embodiments, the device also includes:

The second detection module is used to detect the number of radio frequency identification tags in the display area;

The visual element acquisition module is also used to detect whether the visual elements corresponding to each radio frequency identification tag are the same when there are multiple radio frequency identification tags in the display area. view element;

The display module is further configured to display the visual elements evenly in split screens according to the quantity of the different items.

In one of the embodiments, the display module is further configured to stop displaying the visual element in the display area when it is detected that the item is not in the display area.

In one of the embodiments, the display module is further configured to select one of the plurality of visual elements corresponding to the preset picture when the preset picture corresponds to multiple visual elements, and display the selected The visual elements of the preset picture are displayed circularly in the display area; or the multiple visual elements corresponding to the preset picture are displayed in a preset display order.

The above augmented reality display method and device set the recognition time length so that when the matching picture corresponding to the item in the display area is found within the recognition time length, the visual element is obtained according to the found matching picture. When the matching picture can be obtained within the preset time period, call the radio frequency identification function to connect with the radio frequency identification tag on the item to obtain the corresponding visual elements, and send the obtained visual elements to the In the above display area, an augmented reality display effect is formed, thereby improving the accuracy and acquisition speed of obtaining visual elements corresponding to items in the display area.

Description of drawings

Fig. 1 is the application environment diagram of the display method of augmented reality in an embodiment;

FIG. 2 is a flow chart of an augmented reality display method in an embodiment;

FIG. 3 is a flow chart of an augmented reality display method in another embodiment;

FIG. 4 is a structural block diagram of an augmented reality display device in an embodiment;

Fig. 5 is a structural block diagram of an augmented reality display device in another embodiment;

Fig. 6 is a structural block diagram of an augmented reality display device in yet another embodiment.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

The augmented reality display method provided in the embodiment of the present invention can be applied to the application environment shown in FIG. 1 . The terminal 102 is connected to the server 104 through a network. The terminal 102 is pre-installed with application software capable of realizing augmented reality and functional devices of radio frequency identification, and the server 104 stores corresponding visual elements. The terminal 102 can call the camera to identify the items in the display area, and obtain the feature information of the items; then query the matching preset pictures according to the feature information; and calculate the recognition time in real time, and judge whether the preset image is found within the preset time. picture; if so, then obtain the visual element corresponding to the preset picture stored on the server 104; otherwise, call the radio frequency identification function to connect with the radio frequency identification tag in the display area, and obtain the radio frequency identification tag stored on the server 104 Corresponding visual elements; display the visual elements in the display area to form an augmented reality display effect. By combining the functions of image recognition and radio frequency identification, the accuracy and speed of obtaining visual elements corresponding to items in the display area can be improved.

In one embodiment, as shown in FIG. 2 , a method for displaying augmented reality is provided, and the method includes the following steps 202 - 212 .

Step 202, call the camera to identify the items in the display area, and obtain the feature information of the items.

In this embodiment, a corresponding display interface may be provided on the display interface of the terminal, and the display interface may be a display interface of pre-installed application software capable of performing augmented reality. The display interface includes an activation button for augmented reality. When a trigger operation of the activation button is detected, a recognition instruction for performing augmented reality may be generated. Or when it is detected that the user clicks on a corresponding augmented reality application icon to start, an augmented reality recognition instruction is directly generated.

The terminal can invoke the camera to scan the display area according to the recognition instruction. The area scanned by the camera and presented on the display interface of the terminal is the display area.

In one embodiment, after the terminal forms the recognition instruction, it can detect whether it has the authority to open the terminal camera. If not, a corresponding permission acquisition request can be generated, and after the permission to turn on the camera is obtained, the camera of the terminal is turned on, and the display area is identified based on the camera.

The terminal can extract feature information of items in the display area. The item can be any item, such as a gift box, a picture, or a teacup. Feature information may be used to reflect characteristics of a corresponding item. There may be multiple items in the display area, and the terminal may determine the number of items in the display area and extract the The characteristic information of the item.

Step 204, searching for matching preset pictures according to the feature information.

In this embodiment, the preset picture can be located in the database of the terminal itself, or in the database of the server. When the preset picture is stored in the terminal's own database, the terminal can compare the extracted feature credit with feature information of the preset picture in the database to find out a matching preset picture.

Specifically, a matching degree threshold may be preset, and the matching degree threshold may be an arbitrarily set matching degree, such as 80% or 90%. As long as it is determined that the matching degree between the captured image and a pre-stored image reaches or exceeds the matching degree threshold corresponding to the pre-stored image, it can be determined that the acquired captured image matches the pre-stored image.

When the preset picture is stored in the database of the terminal itself, after a matching preset picture is found, a message indicating that the preset picture is found may be returned.

When the preset picture is located in the server, the terminal can establish a communication connection with the server according to the preset communication method, upload the extracted feature information to the server, and send a matching query request to the server, so that the server can The feature information queries the preset picture that matches it. After the preset picture is found, a message indicating that the preset picture is found is sent to the terminal.

In one embodiment, when the matched preset pictures that reach or exceed the matching threshold include multiple preset pictures, the one with the highest matching degree is selected from the multiple preset pictures as the final picture that matches the feature information. Default picture.

When the matching degree between the extracted feature information and the feature information of the preset picture reaches the matching degree threshold, the matched preset picture may be selected.

Step 206, calculate the recognition time in real time, and judge whether the preset picture is queried within the preset time. If yes, execute step 208 , otherwise, execute step 210 .

In this embodiment, the terminal also sets a preset duration, which may be any duration, such as 1 second, 2 seconds, or 3 seconds. When it is detected that the terminal has turned on the camera to identify the display area, real-time calculation of the terminal's identification time can be started.

The terminal can call a relevant timing function device to record the preset duration. If it is detected that the preset picture has been queried within the preset time period, step 208 is executed. When it is detected that the recognition duration has reached the preset duration and the corresponding preset picture has not been queried, step 210 is performed. Specifically, when it is detected that a message indicating that a preset picture has been queried is obtained within a preset time period, it means that a matching preset picture has been obtained, and step 208 is performed; otherwise, step 210 is performed.

Step 208, obtaining visual elements corresponding to preset pictures.

In this embodiment, each preset picture has an association relationship with one or more corresponding visual elements. Wherein the visual element may be text, picture, animation video and so on. Multiple preset images can correspond to the same visual element. When there are multiple visual elements corresponding to the preset picture, one or more of them may be randomly selected from the multiple visual elements.

Wherein, a large number of visual elements are stored in the server, and the visual elements can be arbitrary text, pictures, animation videos, etc., such as advertisement information of various products. Each visual element has a corresponding relationship with one or more preset pictures. Similarly, different preset pictures may also correspond to one or more visual elements. That is, the correspondence between preset pictures and visual elements may be one-to-one, one-to-many or many-to-many.

For example, the scanned item is a packaging box of a product, and the pattern information on each side of the packaging box is different. Wherein, the pattern on each side corresponds to a style picture, and each style picture corresponds to the same visual element, and the visual element is a 3D advertisement of the corresponding product. Then when the camera of the terminal scans the pattern of which display surface of the packaging box, it can obtain the style picture corresponding to the display surface, and then according to the correspondence between the style picture and the corresponding visual elements, obtain the 3D advertisement of the product.

In one embodiment, step 208 includes: acquiring the storage address of the visual element corresponding to the preset picture, and reading the visual element in the storage address.

In this embodiment, after a matching preset picture is found, the returned message indicating that the preset picture has been found further includes the storage address of the visual element corresponding to the preset picture. The terminal can obtain the corresponding storage address from the message, and read the visual element stored in the storage address. When there are multiple corresponding visual elements, the message contains the storage address of each corresponding visual element, and the terminal can randomly select one or more from all the storage addresses included, and obtain the corresponding visual element .

Step 210, call the radio frequency identification function to connect with the radio frequency identification tags in the display area, and obtain the visual elements corresponding to the radio frequency identification tags.

In this embodiment, when the preset picture cannot be found within the preset time period, it can be judged that the identification of the pattern information in the display area has failed. At this time, the radio frequency identification function can be used to obtain the corresponding visual elements.

Specifically, the radio frequency identification function is integrated in the terminal, and the scanned items in the display area also carry radio frequency identification tags. The terminal can call the bluetooth identification function to connect with the radio frequency identification tag, and read the coded information in the radio frequency identification tag. Each of these radio frequency identification tags has a unique code. The corresponding relationship between the code and one or more visual elements can be pre-established in the server, so that the corresponding visual element can be obtained according to the corresponding relationship after the corresponding code.

After the terminal reads the coded information in the radio frequency identification tag, it can send the coded information to the server, so that the server can find out the corresponding visual elements according to the coded information, and send the found visual elements to the terminal. The terminal can obtain the visual elements sent by the server.

Step 212, displaying visual elements in the display area to form an augmented reality display effect.

After the visual element is obtained, the playback software corresponding to the visual element can be called to display it, and it can be displayed together with the real scene in the display area to form a display effect combining the virtual scene and the real scene, that is Augmented reality display.

In this embodiment, the recognition duration is set so that when the matching picture corresponding to the item in the display area is queried within the recognition duration, the visual element is obtained according to the found matching picture. When the matching picture is obtained within the time period, the radio frequency identification function is called to connect with the radio frequency identification tag on the item to obtain the corresponding visual elements, and the obtained visual elements are placed in the display area to form a The display effect of augmented reality improves the accuracy and acquisition speed of obtaining the visual elements corresponding to the items in the display area.

In one embodiment, when the queried preset pictures include multiple preset pictures, the step of obtaining the visual elements corresponding to the preset pictures includes: detecting whether the visual elements corresponding to each preset picture are the same. If not, get the visual element corresponding to each preset picture.

In this embodiment, when the queried preset pictures include multiple, it means that there may be multiple items in the display area. The terminal can detect whether the visual elements corresponding to each preset picture are the same. Specifically, it may be detected whether the storage addresses in the messages corresponding to each preset picture are the same, and if they are the same, it means that the multiple detected items are the same item. Then the display of visual elements can be performed in the same way as only one item exists. If not, it means that there are multiple different items in the display area, and the terminal can acquire the visual elements corresponding to each preset picture, that is, acquire the visual elements related to each item.

The step of displaying the visual elements in the display area includes: displaying the visual elements in positions matching the items.

When there are multiple items corresponding to different visual elements, the position of each item in the display area can be obtained, and the display interface of the terminal can be divided into a corresponding number of blocks according to the position information and the number of items , the size and position of each block corresponds to the corresponding item, and visual elements are displayed at the position of the corresponding block.

In this embodiment, when there are multiple items with corresponding visual elements in the display area, a virtual reality display is performed for each item in the display area, which can increase the diversity of display.

In one embodiment, when there are multiple radio frequency identification tags in the display area, the step of obtaining the visual elements corresponding to the radio frequency identification tags includes: detecting whether the visual elements corresponding to each preset picture are the same, if not , then get the corresponding different visual elements.

In this embodiment, since each item that can be scanned carries a radio frequency identification tag, when multiple radio frequency identification tags are detected, it means that there are the same number of items. The terminal can detect whether the visual elements corresponding to each radio frequency identification tag are the same. Specifically, it may be detected whether the visual elements corresponding to the coded information in each radio frequency identification tag are the same, and if they are the same, it means that the multiple detected items are the same item. Then the display of visual elements can be performed in the same way as only one item exists. If they are different, it means that there are multiple different items in the display area, and the terminal can acquire the visual elements corresponding to each different item.

The step of displaying the visual elements in the display area includes: displaying the visual elements evenly in split screens according to the number of radio frequency identification tags.

In this embodiment, the terminal may divide the display interface of the terminal into a corresponding number of blocks according to the quantity of different items, each block has the same size, and display corresponding visual elements in the corresponding blocks.

In one embodiment, as shown in FIG. 3 , another augmented reality display method is provided, and the method also includes:

Step 214, stop displaying visual elements in the display area when it is detected that the item is not in the display area.

In this embodiment, the terminal can acquire the location of the scanned item in the display area in real time, and correspondingly change the location of the displayed visual element according to the location. When it is detected that the item does not exist in the display area, the display of the visual element is stopped, so that when the item in the display area disappears, the corresponding displayed visual element also disappears, thereby further improving the diversity of display.

In one embodiment, when there are multiple visual elements corresponding to the preset picture, the step of displaying the visual elements in the display area includes: selecting one of the multiple visual elements corresponding to the preset picture, and The selected visual elements are displayed circularly in the display area.

In this embodiment, if there are multiple corresponding visual elements, one of the multiple visual elements is selected, and displayed together with the existing real scene on the display interface of the terminal, forming a combination of the virtual scene and the real scene display effect.

Specifically, one of the corresponding visual elements may be selected for cyclic display according to factors such as ratings, playback volume, and popularity of the corresponding visual elements.

In one embodiment, when there are multiple visual elements corresponding to the preset picture, the step of displaying the visual elements in the display area includes: displaying the multiple visual elements corresponding to the preset picture in a preset display order to show.

In this embodiment, when there are multiple corresponding visual elements, all the corresponding visual elements can also be obtained, and combined with the real scene in the display area according to the preset play sequence, the display can be played one by one, thereby also Can increase the variety of display.

In one embodiment, after the step of displaying visual elements in the display area to form an augmented reality display effect, it further includes: receiving a shooting instruction for the display area; Show content for filming.

In this embodiment, the photographing instruction includes any one of a photographing instruction and a recording instruction. The terminal receives the user's operation instruction, and when the operation instruction is a shooting instruction, it can simultaneously shoot the display scene existing in the display interface and the superimposed visual elements, thereby improving the diversity of shooting.

In one embodiment, as shown in FIG. 4 , an augmented reality display device is provided, the device comprising:

The feature information acquisition module 402 is used to call the camera to identify the items in the display area and acquire feature information of the items.

The preset picture matching module 404 is configured to search for matching preset pictures according to feature information.

The judging module 406 is used to calculate the recognition time in real time, and judge whether the preset picture is queried within the preset time.

The visual element acquiring module 408 is configured to acquire the visual element corresponding to the preset picture.

The radio frequency identification module 410 is configured to call the radio frequency identification function to connect with the radio frequency identification tags in the display area when the preset picture is not found within the preset time period.

The visual element acquiring module 408 is also used to acquire the visual element corresponding to the radio frequency identification tag.

The display module 412 is configured to display visual elements in the display area to form an augmented reality display effect.

In one embodiment, as shown in FIG. 5 , another augmented reality display device is provided, which further includes:

The first detection module 405 is configured to detect the number of preset pictures found.

The visual element acquisition module 408 is also used to detect whether the visual elements corresponding to each preset picture are the same when the queried preset pictures include multiple ones, and if not, obtain the visual element corresponding to each preset picture. element.

The display module 412 is also used for displaying visual elements in positions that match the items.

In one embodiment, as shown in FIG. 6 , another augmented reality display device is provided, which further includes:

The second detection module 411 is configured to detect the number of radio frequency identification tags in the display area.

The visual element acquisition module 408 is also used to detect whether the visual elements corresponding to each radio frequency identification tag are the same when there are multiple radio frequency identification tags in the display area, and if not, obtain the visual elements corresponding to different items.

The display module 410 is also configured to display visual elements evenly in split screens according to the quantity of different items.

In one embodiment, the display module 410 is further configured to stop displaying visual elements in the display area when it is detected that the item is not in the display area.

In one embodiment, the display module 410 is further configured to select one of the multiple visual elements corresponding to the preset picture when there are multiple visual elements corresponding to the preset picture, and display the selected visual element Circular display in the region; or display multiple visual elements corresponding to the preset image in a preset display order.

The technical features of the above-mentioned embodiments can be combined arbitrarily. To make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, should be considered as within the scope of this specification.

The above-mentioned embodiments only express several implementation modes of the present invention, and the descriptions thereof are relatively specific and detailed, but should not be construed as limiting the patent scope of the invention. It should be pointed out that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention should be based on the appended claims.

Claims (10)

1. A method for displaying augmented reality, characterized in that said method comprises: Call the camera to identify the items in the display area, and obtain the feature information of the items; Searching for matching preset pictures according to the feature information; Calculate the recognition time in real time, and judge whether the preset picture is queried within the preset time; If so, obtain the visual element corresponding to the preset picture; Otherwise, call the radio frequency identification function to connect with the radio frequency identification tag in the display area, and obtain the visual element corresponding to the radio frequency identification tag; The visual elements are displayed in the display area to form an augmented reality display effect. 2. The method according to claim 1, wherein when the queried preset pictures include multiple ones, the step of acquiring the visual elements corresponding to the preset pictures comprises: Detect whether the visual elements corresponding to each preset picture are the same, and if not, obtain the visual elements corresponding to each preset picture; The step of displaying the visual elements in the display area includes: Display the visual element in a position that matches the item. 3. The method according to claim 1, wherein when there are multiple radio frequency identification tags in the display area, the step of acquiring the visual element corresponding to the radio frequency identification tag comprises: Detect whether the visual elements corresponding to each radio frequency identification tag are the same, if not, obtain the visual elements corresponding to different items; The step of displaying the visual elements in the display area includes: The visual elements are evenly displayed in split screens according to the quantity of the different items. 4. The method according to claim 1, characterized in that, after the step of displaying the visual elements in the display area, comprising: When it is detected that the item is not in the display area, stop displaying the visual element in the display area. 5. The method according to claim 1, wherein when the preset picture corresponds to multiple visual elements, the step of displaying the visual elements in the display area includes : Select one of the plurality of visual elements corresponding to the preset picture, and display the selected visual element in the display area in a loop; or The multiple visual elements corresponding to the preset picture are displayed in a preset display order. 6. A display device for augmented reality, characterized in that the device comprises: The feature information acquisition module is used to call the camera to identify the items in the display area, and acquire the feature information of the items; A preset picture matching module, configured to search for a matched preset picture according to the characteristic information; A judging module, configured to calculate the recognition time in real time, and judge whether the preset picture is queried within the preset time; A visual element acquisition module, configured to acquire visual elements corresponding to the preset picture; The radio frequency identification module is used to call the radio frequency identification function to connect with the radio frequency identification tags in the display area when the preset picture is not queried within the preset time period; The visual element obtaining module is also used to obtain the visual element corresponding to the radio frequency identification tag; The display module is configured to display the visual elements in the display area to form an augmented reality display effect. 7. The device according to claim 6, further comprising: The first detection module is used to detect the number of preset pictures found; The visual element acquisition module is also used to detect whether the visual elements corresponding to each preset picture are the same when the queried preset pictures include multiple ones, and if not, obtain the corresponding visual elements of each preset picture. view element; The display module is also used to display the visual element in a position that matches the item. 8. The device according to claim 6, further comprising: The second detection module is used to detect the number of radio frequency identification tags in the display area; The visual element acquisition module is also used to detect whether the visual elements corresponding to each radio frequency identification tag are the same when there are multiple radio frequency identification tags in the display area. view element; The display module is further configured to display the visual elements evenly in split screens according to the quantity of the different items. 9. The device according to claim 6, wherein the display module is further configured to stop displaying the visual element in the display area when it is detected that the item is not in the display area. 10. The device according to claim 6, wherein the display module is further configured to, when the preset picture corresponds to a plurality of visual elements, from the plurality of visual elements corresponding to the preset picture Select one of the elements, and display the selected visual element in a loop in the display area; or display multiple visual elements corresponding to the preset picture in a preset display order.