CN106101576B - A method, device and mobile terminal for taking augmented reality photos - Google Patents

Abstract

The embodiment of the invention discloses a method and a device for shooting an augmented reality photo and a mobile terminal. The method comprises the following steps: when the mobile terminal starts an augmented reality shooting mode, acquiring a target augmented image and a target augmented sound effect; and when a photographing instruction is received, generating an augmented reality photo according to the image captured by the camera, the target augmented image and the target augmented sound effect. By adopting the technical scheme, when the augmented reality photo is shot, the image and the sound effect can be added into the augmented reality photo as the augmented content, so that the expression form of the augmented reality photo is enriched, and the augmented effect is improved.

Description

A method, device and mobile terminal for taking augmented reality photos

technical field

Embodiments of the present invention relate to the technical field of augmented reality, and in particular, to a method, an apparatus, and a mobile terminal for taking an augmented reality photo.

Background technique

Augmented Reality (AR) technology is a technology that increases the user's perception of the real world through the information provided by the computer system, applies virtual information to the real world, and superimposes computer-generated virtual objects, scenes or system prompt information on the real world. scene, so as to realize the augmentation of reality.

At present, the camera function has become a standard configuration of mobile terminals, and mobile terminal users can easily record the scenery and wonderful moments around them in their daily life. The photos taken by the ordinary camera mode are basically the same as what the human eye sees. Many scenes, phenomena or things cannot be captured only by adjusting the camera parameters. Not to the moon and so on. More and more users are not satisfied with the shooting of real scenes, but want to incorporate more virtual content into their photos when taking pictures to make the photos more beautiful or eye-catching. Augmented reality technology can meet this demand of users, and can superimpose virtual content as augmented content with images obtained by cameras to obtain augmented reality photos.

However, current augmented reality photos are limited to adding virtual images to real photos, and the achieved enhancement effects are limited and cannot meet user needs.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present invention is to provide a method, device and mobile terminal for taking an augmented reality photo, so as to enrich the representation form of the augmented reality photo.

In a first aspect, an embodiment of the present invention provides a method for capturing an augmented reality photo, including:

When the mobile terminal starts the augmented reality shooting mode, acquire the target-enhanced image and the target-enhanced sound effect;

When a photographing instruction is received, an augmented reality photo is generated according to the image captured by the camera, the target-enhanced image and the target-enhanced sound effect.

In a second aspect, an embodiment of the present invention provides a device for capturing an augmented reality photo, including:

The sound effect acquisition module is used to acquire the target enhanced image and the target enhanced sound effect when the mobile terminal starts the augmented reality shooting mode;

The photo generating module is configured to generate an augmented reality photo according to the image captured by the camera, the target-enhanced image and the target-enhanced sound effect when receiving a photographing instruction.

In a third aspect, an embodiment of the present invention provides a mobile terminal, where the mobile terminal integrates the device for taking an augmented reality photo in the embodiment of the present invention.

In the augmented reality photo shooting solution provided in the embodiment of the present invention, the target enhanced image and the target enhanced sound effect are obtained first, and when a photographing instruction is received, the augmented reality photo is generated according to the image captured by the camera, the target enhanced image and the target enhanced sound effect. By adopting the above technical solution, when taking an augmented reality photo, images and sound effects can be added to the augmented reality photo together as enhanced content, so as to enrich the representation of the augmented reality photo and enhance the enhancement effect.

Description of drawings

1 is a schematic flowchart of a method for taking an augmented reality photo according to Embodiment 1 of the present invention;

2 is a schematic diagram of a shooting preview interface provided by Embodiment 1 of the present invention;

3 is a schematic flowchart of a method for taking an augmented reality photo according to Embodiment 2 of the present invention;

4 is a schematic diagram of a shooting preview provided in Embodiment 2 of the present invention;

5 is a schematic flowchart of a method for taking an augmented reality photo according to Embodiment 3 of the present invention;

FIG. 6 is a schematic diagram of browsing an augmented reality photo according to Embodiment 3 of the present invention;

FIG. 7 is a structural block diagram of a device for taking an augmented reality photo according to Embodiment 4 of the present invention.

Detailed ways

The technical solutions of the present invention are further described below with reference to the accompanying drawings and through specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for the convenience of description, the drawings only show some but not all structures related to the present invention.

Before discussing the exemplary embodiments in greater detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowchart depicts the steps as a sequential process, many of the steps may be performed in parallel, concurrently, or concurrently. Furthermore, the order of the steps can be rearranged. The process may be terminated when its operation is complete, but may also have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, subroutines, and the like.

Example 1

FIG. 1 is a schematic flowchart of a method for taking an augmented reality photo according to Embodiment 1 of the present invention. The method can be executed by a device for taking an augmented reality photo, wherein the device can be implemented by software and/or hardware, and can generally be integrated in mobile devices. in the terminal. As shown in Figure 1, the method includes:

Step 101: When the mobile terminal starts the augmented reality shooting mode, acquire the target-enhanced image and the target-enhanced sound effect.

Exemplarily, the mobile terminal in this embodiment may specifically be a terminal equipped with a camera, such as a mobile phone, a tablet computer, and a digital camera.

Exemplarily, an enhanced photo shooting mode may be configured in the mobile terminal. When the mobile terminal activates this mode, the screen of the mobile terminal displays the preview image obtained by the camera in real time, and obtains the target enhanced image and the target enhanced sound effect.

Exemplarily, the enhanced images may include real images (such as images of real objects such as trees, moons, beaches, and mascots), special effects images (such as smoke effects, steam effects, and motion trail effects), and natural phenomenon images (such as rain, virtual content images such as images of snow, rainbows, and sun circles). The target enhanced image may specifically be a virtual content image associated with the scene contained in the preview image.

Exemplarily, before acquiring the target enhanced image, the target enhanced image may be determined first. There are many ways to determine the target enhanced image, which is not specifically limited in this embodiment, and several specific implementation ways are given below as a schematic description. The target enhanced image can be determined according to the user's input information. For example, the user can let the mobile terminal identify the target enhanced image required by the user from the text or voice through text or voice input; the content of the preview image obtained by the camera can be identified. , and obtain candidate enhanced images for user selection. For example, FIG. 2 is a schematic diagram of a shooting preview interface provided by Embodiment 1 of the present invention. As shown in FIG. 2 , the preview image obtained by the camera includes the sea, and it can be concluded that the candidate enhanced images include waves, ships, and seagulls. One or more items, the target enhanced image can be determined according to the user's selection. After the target enhanced image is determined, the determined target enhanced image can be acquired locally from the mobile terminal, from a remote server, or from the Internet, and used for subsequent operations such as previewing and taking pictures.

Exemplarily, the target enhanced sound effect may specifically be a sound effect associated with the scene included in the preview image. Whether there is a correlation between the target enhanced sound effect and the target enhanced image is not limited in this embodiment.

In this step, before acquiring the target enhanced sound effect, the target enhanced sound effect to be acquired may be determined first. There may also be many manners for determining the target to enhance the sound effect, which is not specifically limited in this embodiment, and several specific implementation manners are given below as schematic descriptions. The target enhanced sound effect can be determined according to the user's input information. For example, the user can let the mobile terminal identify the target enhanced sound effect required by the user from the text or voice through text or voice input; the content of the preview image obtained by the camera can be identified. , to obtain candidate enhanced sound effects for the user to choose. For example, if the preview image contains rivers and trees, it can be concluded that the candidate enhanced sound effects include the sound of flowing water and birdsong. Determine the target enhanced sound effect. In addition, in order to help the user quickly determine the enhanced sound effect he is satisfied with, the candidate enhanced sound effect can also be played for the user to listen to.

Exemplarily, after determining the target enhanced sound effect, the database containing the enhanced sound effect in the local or remote server of the mobile terminal can be queried to obtain the corresponding enhanced sound effect; it is also possible to search the Internet according to the determined target enhanced sound effect, and download the corresponding enhanced sound effect. sound effects.

Preferably, in order to allow the user to listen to the target enhanced sound effect before taking a photo, the target enhanced sound effect may be played during the shooting preview process.

Further, in order to allow the user to preview the superimposed effect of the target enhanced image and the real-time image when taking a photo, the target enhanced image can also be displayed on the shooting preview interface during the shooting preview process.

Step 102: When receiving the photographing instruction, generate an augmented reality photo according to the image captured by the camera, the target enhanced image and the target enhanced sound effect.

Exemplarily, the user can adjust the shooting content in the shooting preview interface by changing the position of the mobile terminal. When the user thinks that the combined effect of the shooting content, the target-enhanced image and the target-enhancing sound effect meets his own requirements, he can click the camera button to carry out. When taking a photo, the mobile terminal will receive the photo-taking instruction, and synthesize the image captured by the camera with the target-enhanced image and the target-enhanced sound effect to generate an augmented reality photo to meet the user's needs for taking an augmented reality photo.

Exemplarily, an augmented reality image can be obtained by superimposing the image captured by the camera and the target augmented image, and then the target augmented sound effect can be embedded in the augmented reality image to generate an augmented reality photo.

The method for taking an augmented reality photo provided by the first embodiment of the present invention first obtains a target enhanced image and a target enhanced sound effect, and when receiving a photographing instruction, generates an augmented reality photo according to the image captured by the camera, the target enhanced image and the target enhanced sound effect. By adopting the above technical solution, when taking an augmented reality photo, images and sound effects can be added to the augmented reality photo together as enhanced content, so as to enrich the representation of the augmented reality photo and enhance the enhancement effect.

Embodiment 2

FIG. 3 is a schematic flowchart of a method for taking an augmented reality photo according to Embodiment 2 of the present invention. This embodiment is optimized on the basis of the above-mentioned embodiment, and the enhanced sound effect and the enhanced image added to the augmented reality photo are related. , which can further enhance the enhancement effect.

In this embodiment, the step of "obtaining the target enhanced image and the target enhanced sound effect" is optimized as follows: determine the target enhanced image, and obtain the determined target enhanced image; and obtain the corresponding target enhanced sound effect according to the target enhanced image.

Correspondingly, the method of this embodiment includes the following steps:

Step 301 , when the mobile terminal starts the augmented reality shooting mode, determine the target enhanced image, and acquire the determined target enhanced image.

Step 302: Acquire a corresponding target enhanced sound effect according to the target enhanced image.

When there is a corresponding relationship between the target-enhanced sound effect and the target-enhanced image, the visual effect and the auditory effect of the augmented reality photo can be comprehensively enhanced, thereby enhancing the enhancement effect of the enhanced image.

As an example in the above embodiment, it is assumed that the user selects ships and seagulls as the target enhanced images among the multiple candidate enhanced images, the target enhanced sound effect corresponding to the ship may be the whistle of the ship, and the target enhanced sound effect corresponding to the seagull may be Seagulls cry.

Preferably, there may be many kinds of enhanced sound effects corresponding to the same target enhanced image, and a trial playback of multiple enhanced sound effects may be performed for the user to select.

Exemplarily, in order to make the sound more realistic, the target enhanced sound effect may be a sound effect obtained based on various sound effect processing technologies. For example, BBE High Definition Sound System (BBE High Definition Sound System) technology, EQ equalizer (Equalizer) technology and Sound Retrieval System (Sound Retrieval System, SRS) technology and so on. Preferably, the target enhanced sound effects include enhanced sound effects based on the SRS technology. SRS is the most representative 3D stereo technology developed by the SRS Institute. It is a general term for a series of patented audio processing technologies based on psychoacoustic principles. zone, creating a wider sound field and positioning different instruments very accurately in the audio reverberation, and finally creating an immersive three-dimensional sound field through only two speakers, so this technology is also called SRS 3D technology. By using SRS sound effects, the enhanced sound effects in the augmented reality photos can be made more realistic, and the user's audio-visual experience can be improved.

Step 303: Play the target enhanced sound effect during the shooting preview process, and simultaneously display the target enhanced image on the shooting preview interface.

Exemplarily, during the shooting preview process, the target enhanced image is superimposed with the real-time image obtained by the camera included in the shooting preview interface, so that the user can preview the image enhancement effect, and at the same time play the target enhanced sound effect, so that the user can predict the display of the augmented reality photo. Effect.

Preferably, the attribute information of the target enhanced image can be adjusted according to the received preset adjustment instruction. The attribute information may include at least one of display position, display size, color and transparency on the shooting preview interface. Exemplarily, the preset adjustment instruction may be a user's gesture operation on the screen of the mobile terminal, such as dragging, zooming, etc. on the target enhanced image; it may also be that the user clicks on the target enhanced image to trigger the pop-up of the attribute list, Then select in the attribute list or input attribute content and other corresponding adjustment instructions for a series of operations.

FIG. 4 is a schematic diagram of a shooting preview provided by Embodiment 2 of the present invention. As shown in FIG. 4 , the shooting preview interface displays an image including the sea acquired by a camera, and also displays the acquired images of ships and seagulls as target enhanced images. image. At the same time, the mobile terminal plays the ship whistle and the seagull call as the target enhanced sound effect.

Step 304 , when receiving the photographing instruction, generate an augmented reality photo according to the image captured by the camera, the target enhanced image and the target enhanced sound effect.

Exemplarily, this step may include: generating an augmented reality image according to the image captured by the camera and the target-enhanced image; embedding the target-enhanced sound effect into the augmented reality image, and generating an augmented reality photo. The image is generally stored in the form of binary data. Optionally, the target enhanced sound effect is converted into binary data, and embedded in the binary data of the augmented reality image to generate an augmented reality photo.

As the above example, the images of ships and seagulls and the image of the sea are combined into the image part of the augmented reality photo, and then the sound of the ship whistle and the sound of seagulls are embedded in the image part to generate the augmented reality photo.

In the method for shooting an augmented reality photo provided by the second embodiment of the present invention, there is a corresponding relationship between the target-enhanced sound effect and the target-enhanced image. The shooting needs of users can improve the shooting experience of users.

Embodiment 3

5 is a schematic flowchart of a method for taking an augmented reality photo according to Embodiment 3 of the present invention. This embodiment is optimized on the basis of the above-mentioned embodiment, and the method includes the following steps:

Step 501 , when the mobile terminal starts the augmented reality shooting mode, determine the target enhanced image, and acquire the determined target enhanced image.

Step 502: Acquire a corresponding target enhanced sound effect according to the target enhanced image.

Step 503: Play the target enhanced sound effect during the shooting preview process, and simultaneously display the target enhanced image on the shooting preview interface.

Step 504 , when receiving the photographing instruction, generate an augmented reality photo according to the image captured by the camera, the target-enhanced image and the target-enhanced sound effect.

Step 505: When receiving the browsing instruction for the augmented reality photo, display the image included in the augmented reality photo and play the enhanced sound effect included in the augmented reality photo.

The advantage of this step is that the user can get an immersive feeling when browsing photos, which can enhance the interest of browsing the augmented reality photos, and bring a brand new photo browsing experience to the user.

FIG. 6 is a schematic diagram of browsing an augmented reality photo according to Embodiment 3 of the present invention. It is assumed that the augmented reality photo is displayed on the screen of a mobile terminal when a browsing instruction for the augmented reality photo generated in the example described in the above embodiment is received. Included in the image are images of the sea, ships, and seagulls, along with ship whistles and seagulls.

On the basis of the above-mentioned embodiments, the third embodiment of the present invention increases the browsing operation of augmented reality photos, so that users can get an immersive feeling when browsing photos, which can enhance the fun of browsing augmented reality photos, and bring users A brand new photo browsing experience.

Embodiment 4

7 is a structural block diagram of a device for taking an augmented reality photo according to Embodiment 4 of the present invention. The device can be implemented by software and/or hardware, and is generally integrated in a mobile terminal, and can be captured by executing a method for taking an augmented reality photo Augmented reality photos. As shown in FIG. 7 , the apparatus includes an acquisition module 701 and a photo generation module 702 .

Wherein, the acquisition module 701 is used to acquire the target enhanced image and the target enhanced sound effect when the mobile terminal starts the augmented reality shooting mode; the photo generation module 702 is used to, when receiving the photographing instruction, according to the image captured by the camera, the target The augmented image and the target augmented sound effect generate an augmented reality photo.

The device for taking an augmented reality photo provided by the third embodiment of the present invention first obtains the target enhanced image and the target enhanced sound effect, and when receiving a photographing instruction, generates an augmented reality photo according to the image captured by the camera, the target enhanced image and the target enhanced sound effect. By adopting the above technical solution, when taking an augmented reality photo, images and sound effects can be added to the augmented reality photo together as enhanced content, so as to enrich the representation of the augmented reality photo and enhance the enhancement effect.

On the basis of the above-mentioned embodiment, the apparatus may further include a sound effect playing module, configured to play the target enhanced sound effect during the shooting preview process after acquiring the target enhanced image and the target enhanced sound effect.

Based on the above embodiments, the acquisition module may include an image acquisition unit and a sound effect acquisition unit. Wherein, the image acquisition unit is used to determine the target enhanced image when the mobile terminal starts the augmented reality shooting mode, and acquire the determined target enhanced image; the sound effect acquisition unit is used to acquire the corresponding target enhanced sound effect according to the target enhanced image .

On the basis of the above-mentioned embodiment, the apparatus may further include a display module, configured to display the target enhanced image on the shooting preview interface after acquiring the target enhanced image and the target enhanced sound effect.

On the basis of the above embodiments, the photo generation module includes an image generation unit and a photo generation unit. Wherein, the image generation unit is used for generating an augmented reality image according to the image captured by the camera and the target enhanced image; the photo generation unit is used for embedding the target enhanced sound effect into the augmented reality image and generating an augmented reality photo.

On the basis of the above-mentioned embodiment, the apparatus may further include: a browsing module, configured to, after generating the augmented reality photo according to the image captured by the camera, the target-enhanced image and the target-enhanced sound effect, after receiving the When the browsing instruction of the augmented reality photo is executed, the image contained in the augmented reality photo is displayed and the enhanced sound effect contained in the augmented reality photo is played.

On the basis of the above embodiment, the target enhanced sound effects include enhanced sound effects based on the sound restoration system SRS technology.

Embodiment 5

The fifth embodiment provides a mobile terminal, where the mobile terminal includes the device for taking an augmented reality photo described in the embodiment of the present invention, and can take an augmented reality photo by executing a method for taking an augmented reality photo.

Exemplarily, the mobile terminal in this embodiment may specifically be a terminal equipped with a camera, such as a mobile phone, a tablet computer, and a digital camera, and is preferably a smart phone.

When the user uses the mobile terminal in this embodiment, when the mobile terminal starts the augmented reality shooting mode, the target-enhanced image and the target-enhanced sound effect are obtained first, and when a photographing instruction is received, according to the image captured by the camera, the target-enhanced image and the target Augmented sound effects generate augmented reality photos. By adopting the above technical solution, when taking an augmented reality photo, images and sound effects can be added to the augmented reality photo together as enhanced content, so as to enrich the representation of the augmented reality photo and enhance the enhancement effect.

The augmented reality photo shooting device and mobile terminal provided in the above embodiments can execute the augmented reality photo shooting method provided by any embodiment of the present invention, and have corresponding functional modules and beneficial effects for executing the method. For technical details not described in detail in the foregoing embodiments, reference may be made to the method for capturing an augmented reality photo provided by any embodiment of the present invention.

Note that the above are only preferred embodiments of the present invention and applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present invention. The scope is determined by the scope of the appended claims.

Claims (13)

1. a kind of image pickup method of augmented reality photo characterized by comprising

When mobile terminal starts augmented reality screening-mode, the preview graph of the screen real-time display camera acquisition of mobile terminal Picture, and obtain targets improvement image and targets improvement audio, wherein the targets improvement image be with included in preview image The associated virtual content image of scenery；

When receiving photographing instruction, the image, the targets improvement image and the targets improvement sound that are captured according to camera Effect generates augmented reality photo；

Wherein, the acquisition targets improvement image and targets improvement audio include:

It determines targets improvement image, and obtains identified targets improvement image, wherein same targets improvement image correspondence is more A candidate enhancing audio；

Successively the multiple candidate enhancing audio pilot putting, for selection by the user；

The candidate enhancing audio that user is selected is as the corresponding targets improvement audio of the targets improvement image.

2. the method according to claim 1, wherein in the acquisition targets improvement image and targets improvement audio Later, further includes:

The targets improvement audio is played during shooting preview.

3. the method according to claim 1, wherein in the acquisition targets improvement image and targets improvement audio Later, further includes:

The targets improvement image is shown in shooting preview interface.

4. the method according to claim 1, wherein the image captured according to camera, the target increase Strong image and the targets improvement audio generate augmented reality photo, comprising:

The image and the targets improvement image captured according to camera generates augmented reality image；

The targets improvement audio is embedded in the augmented reality image, and generates augmented reality photo.

5. the method according to claim 1, wherein in the image captured according to camera, the target Enhancing image and the targets improvement audio generate after augmented reality photo, further includes:

When receiving the browsing instructions for the augmented reality photo, the image for including in the augmented reality photo is shown And play the enhancing audio for including in the augmented reality photo.

6. the method according to claim 1, wherein the targets improvement audio includes being based on sound recovery system The enhancing audio of SRS technology.

7. a kind of filming apparatus of augmented reality photo characterized by comprising

Module is obtained, for when mobile terminal starts augmented reality screening-mode, the screen real-time display of mobile terminal to be imaged The preview image that head obtains, and obtains targets improvement image and targets improvement audio, wherein the targets improvement image be with it is pre- The associated virtual content image of scenery included in image of looking at；

Photo generation module, image, the targets improvement image for being captured according to camera when receiving photographing instruction Augmented reality photo is generated with the targets improvement audio；

Wherein, the acquisition module includes:

Image acquisition unit, for determining targets improvement image, and obtain when mobile terminal starts augmented reality screening-mode Identified targets improvement image, wherein the same targets improvement image corresponds to multiple candidate enhancing audios；

It pilots unit, for successively pilot putting to the multiple candidate enhancing audio, for selection by the user；

Audio acquiring unit, the candidate enhancing audio for selecting user increase as the corresponding target of the targets improvement image Forte effect.

8. device according to claim 7, which is characterized in that further include:

Audio playing module, for being broadcast during shooting preview after obtaining targets improvement image and targets improvement audio Put the targets improvement audio.

9. device according to claim 7, which is characterized in that further include:

Display module, for being shown in shooting preview interface after the acquisition targets improvement image and targets improvement audio The targets improvement image.

10. device according to claim 7, which is characterized in that the photo generation module includes:

Image generation unit, image and the targets improvement image for being captured according to camera generate augmented reality image；

Photo generation unit for the targets improvement audio to be embedded in the augmented reality image, and generates augmented reality Photo.

11. device according to claim 7, which is characterized in that further include:

Browsing module, in image, the targets improvement image and the targets improvement sound captured according to camera After effect generates augmented reality photo, when receiving the browsing instructions for the augmented reality photo, the enhancing is shown The image that includes in real photo simultaneously plays the enhancing audio for including in the augmented reality photo.

12. device according to claim 7, which is characterized in that the targets improvement audio includes being based on sound restorer The enhancing audio for SRS technology of uniting.

13. a kind of mobile terminal, which is characterized in that the mobile terminal is integrated with the enhancing as described in claim 7-12 is any The filming apparatus of real photo.