CN120812234A

CN120812234A - Real-time digital image defocus processing method, device, equipment and medium

Info

Publication number: CN120812234A
Application number: CN202510924265.9A
Authority: CN
Inventors: 石振宇; 姚俊; 赵阳; 瞿佳; 沈文睿; 刘明
Original assignee: Shanghai Ruishi Health Technology Co ltd
Current assignee: Shanghai Ruishi Health Technology Co ltd
Priority date: 2025-07-04
Filing date: 2025-07-04
Publication date: 2025-10-17

Abstract

The embodiment of the disclosure discloses a real-time digital image defocus processing method, device, equipment and medium. The method comprises the steps of responding to received image data to be processed, splitting the image data to be processed according to a preset number of color channels to obtain color channel images, determining image processing parameters of each color channel image in the color channel images, wherein the image processing parameters comprise relative defocus amounts, carrying out preset processing on the color channel images according to the image processing parameters corresponding to the color channel images for each color channel image in the color channel images to obtain first processed images, carrying out preset processing comprising defocus processing, and superposing the obtained first processed images to obtain second processed images. According to the embodiment, the digital images can be split according to the color channels and respectively subjected to defocusing treatment and then superimposed, so that the generation of chromatic aberration is reduced. Thus, the display effect of the digital defocused image can be improved.

Description

Real-time digital image defocus processing method, device, equipment and medium

Technical Field

The embodiment of the disclosure relates to the technical field of digital defocus, in particular to a real-time digital image defocus processing method, device, equipment and medium.

Background

With the development of digital defocus technology (DigitalDefocus), it is attracting attention in the application fields of myopia prevention and control. Conventional digital defocus technology generally directly performs defocus processing on an original image.

However, when the above digital defocus technique is adopted, there are often the following technical problems:

when the original image is directly defocused, all color channels are uniformly blurred, so that excessive color mixing is easy to cause chromatic aberration (such as color blurring at the edge), and the display effect of the defocused image is poor.

The above information disclosed in this background section is only for enhancement of understanding of the background of the disclosed concept and, therefore, may contain information that does not form the prior art that is already known to those of ordinary skill in the art.

Disclosure of Invention

The disclosure is in part intended to introduce concepts in a simplified form that are further described below in the detailed description. The disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

Some embodiments of the present disclosure propose a real-time digital image defocus processing method, a real-time digital image defocus processing apparatus, an electronic device, and a computer readable medium to solve one or more of the technical problems mentioned in the background section above.

In a first aspect, some embodiments of the present disclosure provide a real-time digital image defocus processing method, where the real-time digital image defocus processing method includes splitting image data to be processed according to a preset number of color channels in response to receiving the image data to be processed to obtain respective color channel images, determining an image processing parameter of each of the respective color channel images, where the image processing parameter includes a relative defocus amount, and performing, for each of the respective color channel images, a predetermined process on the color channel images according to the image processing parameter corresponding to the color channel image to obtain a first processed image, where the predetermined process includes defocus processing, and superposing the obtained respective first processed images to obtain a second processed image.

Optionally, splitting the image data to be processed according to a preset number of color channels to obtain each color channel image, including dividing the image data to be processed into control areas with a preset number of areas to obtain each control area image, and splitting the control area image according to a preset number of color channels for each control area image in each control area image to obtain each color channel image corresponding to the control area image.

Optionally, the preset processing is performed on the color channel image according to the image processing parameters corresponding to the color channel image to obtain a first processed image, wherein the preset processing comprises performing defocusing processing on the color channel image to obtain a defocused image, and fusing the defocused image with a preset image corresponding to the color channel image to obtain the first processed image.

The method comprises the steps of acquiring a motion image sequence of a user, identifying motion information of the user from the motion image sequence, and dividing each control area with the preset area number from the image data to be processed according to the motion information to serve as each control area image.

Optionally, the dividing the image data to be processed into each control area with the preset area number to obtain each control area image includes detecting hot spot content of the image data to be processed to obtain hot spot information, and dividing each control area with the preset area number from the image data to be processed as each control area image according to the hot spot information.

Optionally, the method further comprises the steps of responding to the received two images to be processed, respectively performing defocusing treatment on the two images to be processed by adopting the first image processing parameters and the second image processing parameters to obtain two second processed images, and performing binocular different display on the two second processed images.

Optionally, the determining the image processing parameters of each color channel image comprises generating a relative defocus amount based on reference information, wherein the reference information comprises at least one of vision condition, display screen characteristics, viewing distance and pupil size of a user, generating the image processing parameters based on the relative defocus amount, and determining the image processing parameters as the image processing parameters of each color channel image.

Optionally, the determining the image processing parameter of each color channel image in the respective color channel images includes generating a relative defocus amount based on a preset time period in response to the image data to be processed not meeting a preset image type condition, generating a relative defocus amount through a preset regulation and control process in response to the image data to be processed meeting the preset image type condition, generating an image processing parameter based on the generated relative defocus amount, and determining the image processing parameter as the image processing parameter of each color channel image in the respective color channel images.

Optionally, the method further comprises the step of superposing the processing images to obtain a second processing image in response to the image data to be processed including the processing images after the predetermined processing.

Optionally, each color channel of the color channels is associated with a shader set, each shader set comprises a contrast shader, a sharpening shader and a color intensity shader, and before the out-of-focus image and a preset image corresponding to the color channel image are fused to obtain a first processed image, the method further comprises the substeps of performing feature extraction on the color channel image to obtain feature information, calling the contrast shader according to the feature information, processing the color channel image through the contrast shader to obtain a contrast image, calling the sharpening shader according to the feature information, processing the color channel image through the contrast shader to obtain a sharpening image, calling the color intensity shader according to the feature information, processing the color channel image to obtain a color intensity, processing the color channel image through the color intensity shader according to the feature information, obtaining a color intensity image through the color intensity shader, and fusing the preset images.

In a second aspect, some embodiments of the present disclosure provide a real-time digital image defocus processing apparatus, which includes a splitting unit configured to split image data to be processed according to a preset number of respective color channels in response to receiving the image data, a determining unit configured to determine an image processing parameter of each of the respective color channel images, where the image processing parameter includes a relative defocus amount, and a processing unit configured to perform, for each of the respective color channel images, a predetermined process on the color channel images according to the image processing parameter corresponding to the color channel image, to obtain a first processed image, where the predetermined process includes a defocus process, and a superimposing unit configured to superimpose the obtained respective first processed images, to obtain a second processed image.

In a third aspect, some embodiments of the present disclosure provide an electronic device comprising one or more processors, a storage device having one or more programs stored thereon, which when executed by the one or more processors, cause the one or more processors to implement a method as described in any of the implementations of the first aspect.

In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements a method as described in any of the implementations of the first aspect.

Some embodiments of the present disclosure provide a real-time digital image defocus processing method that can improve the display effect of a defocus image. In particular, the reason why most of the out-of-focus images are displayed poorly is that the current digital out-of-focus technology generally performs out-of-focus processing directly on the original image. However, when the original image is directly defocused, all color channels are uniformly blurred, so that color mixing is excessive, chromatic aberration is generated (such as color blurring at the edge), and the display effect of the defocused image is poor. Based on the above, some embodiments of the present disclosure provide a real-time digital image defocus processing method, which includes splitting image data to be processed according to a preset number of color channels in response to receiving the image data to be processed to obtain respective color channel images, determining image processing parameters of each of the respective color channel images, where the image processing parameters include a relative defocus amount, and performing predetermined processing on each of the respective color channel images according to the image processing parameters corresponding to the color channel images to obtain a first processed image, where the predetermined processing includes defocus processing, and superposing the obtained respective first processed images to obtain a second processed image. According to the real-time digital image defocusing processing method, the original image can be split into a plurality of images according to the color channels, defocusing processing is carried out on the images corresponding to the color channels respectively, and then the images corresponding to the color channels after the defocusing processing are overlapped, so that defocusing is not carried out on the original image directly, and the possibility of color blurring at the edge is reduced. Thus, the display effect of the out-of-focus image can be improved.

Drawings

The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.

FIG. 1 is an architecture diagram of an exemplary system in which some embodiments of the present disclosure may be applied;

FIG. 2 is a flow chart of some embodiments of a real-time digital image defocus processing method according to the present disclosure;

FIG. 3 is a schematic block diagram of some embodiments of a real-time digital image defocus processing apparatus according to the present disclosure;

Fig. 4 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.

It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings. Embodiments of the present disclosure and features of embodiments may be combined with each other without conflict.

It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.

It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.

The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

Fig. 1 illustrates an exemplary system architecture 100 in which a real-time digital image defocus processing method or real-time digital image defocus processing apparatus of some embodiments of the present disclosure may be applied.

As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as a web browser application, an AI intelligent question and answer application, a search application, an instant messaging tool, a mailbox client, social platform software, etc., may be installed on the terminal devices 101, 102, 103.

The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices having a display screen and supporting information display, including but not limited to smartphones, tablet computers, electronic book readers, laptop and desktop computers, and the like. When the terminal devices 101, 102, 103 are software, they can be installed in the above-listed electronic devices. It may be implemented as a plurality of software or software modules, for example, for providing distributed services, or as a single software or software module. The present invention is not particularly limited herein.

The server 105 may be a server providing various services, such as a background server providing support for information displayed on the terminal devices 101, 102, 103. The background server can analyze and process the received data such as the request and the like, and feed back the processing result to the terminal equipment.

It should be noted that, the real-time digital image defocus processing method provided by the embodiment of the present disclosure may be performed by the terminal devices 101, 102, 103 or may be performed by the server 105. Accordingly, the real-time digital image defocus processing apparatus may be provided in the terminal devices 101, 102, 103 or in the server 105. The present invention is not particularly limited herein.

It should be noted that, the server may be hardware, or may be software. When the server is hardware, the server may be implemented as a distributed server cluster formed by a plurality of servers, or may be implemented as a single server. When the server is software, it may be implemented as a plurality of software or software modules, for example, for providing distributed services, or as a single software or software module. The present invention is not particularly limited herein.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

With continued reference to fig. 2, a flow 200 of some embodiments of a real-time digital image defocus processing method according to the present disclosure is shown. The real-time digital image defocus processing method comprises the following steps:

in step 201, in response to receiving the image data to be processed, splitting the image data to be processed according to a preset number of color channels, and obtaining images of the color channels.

In some embodiments, an execution subject of the real-time digital image defocus processing method (e.g., the server shown in fig. 1) may split the image data to be processed by a preset number of respective color channels in response to receiving the image data to be processed, resulting in respective color channel images. The image data to be processed may include pictures (e.g., JPG format) or video (e.g., MP4 format). When the image data to be processed is a video, each frame of image in the video or a specific frame of image preset by a user is processed. The sources of the image data to be processed may include, but are not limited to, windows devices, android devices, or IOS devices. The color channels described above may be included in the digital image as separate channels for storing different color information. Each channel records intensity information of a specific color, and a rich and colorful image can be presented through the combination of a plurality of channels. Taking RGB color mode as an example, a color image is composed of three color channels of red (R), green (G), and blue (B). The color channel image may include an image obtained by splitting the image to be processed, for example, in an RGB color image, each channel is a two-dimensional matrix, and each element in the matrix represents a color intensity value of the channel at a corresponding pixel position, where the value range is 0-255. When the color image is split into three primary colors, the three channels are actually separated to obtain three images containing only single color information.

In practice, the execution body may split the image data to be processed according to a preset number of color channels by using a cv2.Split function of an OpenCV library. The preset number may be three, for example, the color channels may include red, green, and blue primary color channels. Finer division may be performed on the red, green, and blue channels, for example, based on RGB intensity values, red 0-127 corresponds to one channel, red 128-255 corresponds to another channel, and green and blue channel divisions may be similar. The specific color channel division can be determined according to actual requirements, so long as each divided color channel is ensured to be capable of independently performing defocusing processing on the image.

In some optional implementations of some embodiments, the executing body may split the image data to be processed according to a preset number of color channels to obtain the color channel images by:

The first step, dividing the image data to be processed into control areas with the number of preset areas, and obtaining images of the control areas. The control areas may include areas divided in a preset manner on the image data to be processed. For example, a picture may be divided into an upper half and a lower half. The purpose of dividing the region includes applying defocus processing parameters (such as weak defocus in the middle region and strong defocus in the peripheral region) which can be divided into regions in the subsequent defocus processing, so that the defocus processing is more flexible and the display effect is better. The respective control area images may include respective partial images of the respective control areas corresponding to the image data to be processed. The number of the preset areas may be 1, that is, the image data to be processed is not partitioned, and it may be understood that the whole image data to be processed is a partition. The specific value of the number of preset regions may be changed according to the division manner.

Two ways of dividing the control area are given below:

the first is an action dividing mode taking action information of a user as a parameter, and the second is a hot spot dividing mode taking hot spot content of the image data to be processed as a parameter.

The steps for obtaining the control area image by adopting the action dividing mode are as follows:

Step one, a sequence of action images of a user is collected. The motion image sequence may be an image set in which pupil movement of the user is recorded. For example, two images taken by a fixed camera that displace the pupil of the user over a period of time. In practice, the executing body may call the camera to collect pupil images of the user once every 2 ms, and determine two adjacent pictures as a motion image sequence.

And step two, identifying action information of the user in the action image sequence. In practice, the execution body can identify the outline of the pupil on each picture by performing edge detection on the image sequence, and record the position of the pupil by using the central coordinate of the pupil. And comparing the central coordinates of the pupils in the two images to obtain the displacement of the pupils. The displacement is stored in the form of a vector and is determined as motion information.

And thirdly, dividing each control area with the preset area number from the image data to be processed according to the action information to serve as each control area image. The respective control areas may include a central visual field (circular or elliptical), a central peripheral visual field (four circular or elliptical sectors located above, below, and to the left and right of the central visual field), and a peripheral visual field (other total area), for a total of 6 divisions. The positions of the 6 partitions can be relatively fixed, namely one area moves, and other areas correspondingly follow the movement. The number of the preset areas may be 6, or may be divided more finely, which is not particularly limited herein. In practice, taking VR glasses as an example, since the displayed image is closer to the eyes of the user, the displacement of the pupil may be approximated as the displacement of the gaze point on the image displayed by the VR glasses. The execution subject can obtain the offset of the gaze point through the action information. And determining the position of the deflected gaze point as the center of the central visual field, and dividing other control areas based on the central visual field. The division may be such that the control area varies with the user's gaze point.

The step of obtaining the control area image by adopting the hot spot dividing mode is as follows:

Step one, hot content detection is carried out on image data to be processed, and hot information is obtained. The hot content may include one or several areas that are most likely to be focused on by a user among the image data to be processed. For example, characters, vehicles, animals in the foreground in a broad background, or moon in the night sky, or eyes, nose and lips in close-up of a close-up face, or a game character body operated by a user in a game scene. Since the user is watching these images with a high probability looking at these hot content. The hot spot content area can be treated as a central field of view area with a low digital defocus and the near hot spot area as a central near field of view area with a moderate digital defocus applied, the other areas as peripheral fields of view plus a larger digital defocus. The hot spot information may include a probability parameter that each pixel point in the image is hot spot content, and may be a numerical value. In practice, the execution subject may input the image data to be processed into a hot content detection model, and the hot content detection model outputs hot information (between 0 and 1) of each pixel point. Wherein, the hot content detection model is predefined that 0.7-1.0 is a high-heat region, 0.4-0.7 is a medium-heat region and the other is a low-heat region in the training process. The hot content detection model may be a neural network model that takes image data (such as JPG) as input and takes the hot probability (i.e., hot information) of each pixel as output. For example, a ViT (Vision Transformer) model using LayerNorm and residual connections may be used. The model may include a 12-tier network structure, 8 attention headers.

And step two, dividing each control area with the preset area number from the image data to be processed according to the hot spot information to serve as each control area image. In practice, the execution body may divide the image data to be processed according to a numerical range to which the hot spot information corresponding to each pixel point belongs. The number of preset areas may be the number of numerical ranges set in advance. For example, when the hot spot information of the pixel is within the numerical range of 0.7-1.0, the pixel can be divided into a high-heat region, when the hot spot information of the pixel is within the numerical range of 0.4-0.7, the pixel can be divided into a medium-heat region, and when the hot spot information of the pixel is within the numerical range of 0-0.4, the pixel can be divided into a low-heat region.

Secondly, splitting the control area images according to the preset number of color channels for each control area image in the control area images to obtain the color channel images corresponding to the control area images. The splitting of each color channel according to the preset number is the same as the splitting of the image data to be processed according to each color channel according to the preset number in the step 201, so that the image principles of each color channel are the same, and are not described herein.

Step 202, determining image processing parameters of each of the color channel images.

In some embodiments, the execution subject may determine the image processing parameters for each of the respective color channel images. Wherein the image processing parameters may include a relative defocus amount. The relative defocus amount may include parameters when performing defocus processing on an image, and since defocus effects may be achieved in different manners, corresponding parameters may be different. For example, when the image is defocus-processed with gaussian blur, the relative defocus amount can be expressed by the radius of the gaussian blur kernel. In addition, the relative defocus amount may be a value in diopters. For example, -0.5D may represent what one would see to defocus an image to an eye diopter of-0.5D. The specific form of the relative defocus amount is not limited herein. In practice, the execution subject may manually input data by a user, and then determine the manually input data by the user as the image processing parameter.

In some optional implementations of some embodiments, the executing entity may determine the image processing parameters of each of the respective color channel images by:

First, a relative defocus amount is generated based on reference information. Wherein the reference information includes at least one of a user's vision condition, a display screen characteristic, a viewing distance, and a pupil size. The above-mentioned user's vision conditions may include parameters (e.g., degree of myopia) that characterize the various vision conditions (myopia, hyperopia, astigmatism) of the user. The display screen characteristics may include some specific parameters of the display screen (e.g., screen size). The viewing distance may include a distance between the user and the display screen. The pupil size may include a pupil size variation value (e.g., a diameter variation value) of the user's pupil during viewing of the defocus processing image. The viewing distance may be detected in real time by the executing body invoking the infrared sensor.

In practice, when the reference information is the vision condition of the user, the execution subject may generate the relative defocus amount based on the reference information by:

Step one, responding to the vision condition (such as myopia degree) input manually by a user, and matching the relative defocus amount corresponding to the myopia degree from a mapping relation table of the preset myopia degree and the relative defocus amount. For example, at-3.00 to-2.75D myopia, the corresponding relative defocus amount may be-0.5D. It should be noted that D represents diopters and may describe the severity of myopia or hyperopia, for example, -3.00D represents myopia and +2.50D represents hyperopia.

And step two, determining the matched relative defocus amount as the relative defocus amount to be generated.

Further, the user may experience a requirement for on-screen viewing when viewing out-of-focus images by the method described in connection with those embodiments corresponding to fig. 2. The screen may not change relative to the defocus amount due to the change of the size of the display screen, resulting in poor out-of-focus image display effect. It is therefore necessary to generate a relative defocus amount based on the display screen characteristics.

When the reference information is a display screen feature, the execution subject may generate the relative defocus amount based on the reference information by:

step one, in response to detecting that a user starts a screen throwing function, the display screen characteristics of a screen throwing picture receiving end are read.

And step two, comparing the read display screen characteristics with the display screen characteristics of the screen throwing picture transmitting end to obtain adjustment parameters. The adjustment parameter may include a difference between display screen sizes of two terminals transmitting and receiving the screen-casting screen, that is, adjustment parameter=display screen size of receiving terminal-display screen size of transmitting terminal. For example, the display screen size of the screen shot transmitting end is set to 6.0 inches, and the display screen size of the screen shot receiving end is set to 8.0 inches. The adjustment parameter at this time is 8.0-6.0=2.0.

And thirdly, generating a relative defocus amount based on the adjustment parameters. Wherein the generated relative defocus amount is calculated by the following formula:

Wherein T represents the generated relative defocus amount, T ₀ represents the original relative defocus amount used at the screen-projection screen transmitting end, Representing the adjustment parameters. For example, when the display screen size of the screen shot transmitting end is 6.0 inches, the display screen size of the screen shot receiving end is 8.0 inches, and the original relative defocus amount is 0.5D, t=0.5+2×0.5×50% =1.0D.

When the reference information is a viewing distance, the execution subject may generate the relative defocus amount based on the reference information by:

In response to detecting that the distance between the user and the display screen is changed through the infrared sensor, generating a distance change multiple. The distance change multiple may be a value obtained by dividing the current viewing distance by the original viewing distance of the user.

And step two, generating a relative defocus amount based on the distance change multiple. The relative defocus amount is a value obtained by multiplying the relative defocus amount at the original distance by the distance change multiple. For example, if the viewing distance of the user is enlarged from 1 meter to 1.5 meters and the relative defocus amount in the 1 meter state is 0.5D, the generated relative defocus amount is 0.75D when the viewing distance is 1.5 meters.

When the reference information is pupil size, the execution subject may generate the relative defocus amount based on the reference information by:

Step one, the camera collects photos of the pupils of the user twice according to a preset frequency (for example, each time of 0.1 second), and a photo set is obtained. Wherein the photo album comprises two photos.

And step two, comparing the two photos included in the photo set with the pupil diameter change value (such as pupil outline obtained by edge detection) in the previous photo (according to the front-back distinction of the acquisition time point), and obtaining the pupil diameter change value, wherein the pupil diameter change value = pupil diameter in the next photo-pupil diameter in the previous photo.

And thirdly, generating a relative defocus amount according to the pupil diameter variation value. The relative defocus amount is obtained by adjusting the original relative defocus amount (manually input by a user), and the adjustment mode comprises that when the pupil diameter is increased or decreased by a preset value, the relative defocus amount is increased or decreased by a preset size on the original basis. For example, when the pupil diameter variation value is +1 and the original relative defocus amount is 0.5D, the generated relative defocus amount is 0.5+0.125=0.625D, when the pupil diameter variation value is-1 and the original relative defocus amount is 0.5D, the generated relative defocus amount is 0.5 to 0.125=0.375D, and when the pupil diameter variation value is 0, the magnitude of the relative defocus amount is not changed. When the pupil diameter is varied by less than 1mm or more, the amount of variation in the relative defocus amount may be reduced or increased in the same proportion. For example, when the pupil diameter is reduced by 0.5mm (50% of 1 mm), the relative defocus amount may be reduced by 0.125×50% =0.0625d.

And a second step of generating image processing parameters based on the relative defocus amount. In practice, the execution subject may determine the relative defocus amount as an image processing parameter.

Third, the image processing parameters are determined as the image processing parameters of each of the color channel images.

First, in response to the image data to be processed not meeting a preset image type condition, a relative defocus amount is generated based on a preset time period. The preset image type condition may be whether the image data (such as video) to be processed has at least one pair of adjacent two-frame images with a pixel change rate greater than a preset threshold. When the pixel change rate of two adjacent frames of pictures in the video is larger than a preset threshold (such as 60%), determining that the video meets a preset image type condition. The above pixel change rate may be obtained by dividing the number of pixels changed in the next frame image compared to the previous frame image by the total number of pixels in the one frame image. In practice, the execution body may call the following function of the relative defocus amount to obtain the relative defocus amount:

F(t)=0.4×sin(2πt)-0.4

Wherein F (t) represents the relative defocus amount (unit: diopter D), t represents the current time point (unit: second, t. Gtoreq.0), and the playing timing is started from the video. For example, if the relative defocus amount of the video played to 20 seconds is to be determined, t=20 may be substituted into the formula. It is noted that in general, a user will have better compliance only when the user is able to see the image for a portion of the time. If the user sees the blurred image after the defocus treatment for a long time, psychological or physiological discomfort of the user may be caused, and even visual fatigue may be caused. Therefore, the relative defocus amount is generated based on the preset time period, and the picture blurring degree is adjusted in real time, so that the look and feel of a user can be improved.

And secondly, generating relative defocus through preset regulation and control processing in response to the fact that the image data to be processed meet the preset image type conditions. In practice, the execution subject may determine 0 as the relative defocus amount of the two-frame image satisfying the preset image type condition.

Third, based on the generated relative defocus amount, image processing parameters are generated. In practice, the execution subject may select the corresponding relative defocus amount determination mode according to whether the image data to be processed satisfies the preset image type condition. And when the image data to be processed meets the preset image type condition, determining the relative defocus amount generated in the second step as an image processing parameter. And when the image data to be processed does not meet the preset image type condition, determining the relative defocus amount generated in the first step as an image processing parameter.

Fourth, the image processing parameters are determined as the image processing parameters of each of the color channel images. In practice, the execution subject may determine the image processing parameters generated in the third step as the image processing parameters of each of the respective color channel images.

Step 203, for each color channel image in each color channel image, performing predetermined processing on the color channel image according to the image processing parameters corresponding to the color channel image, so as to obtain a first processed image.

In some embodiments, the executing body may perform, for each color channel image in the respective color channel images, a predetermined process on the color channel image according to an image processing parameter corresponding to the color channel image, to obtain a first processed image. Wherein the predetermined process may include a defocus process. The above-described defocus processing may include processing capable of causing a blurring effect to occur in the color channel image. In practice, the execution body may perform convolution operation on the color channel image by using the convolution kernel as an image processing parameter, so that the color channel image is blurred, thereby achieving a defocus effect and obtaining a first processed image.

In some optional implementations of some embodiments, the executing body may perform predetermined processing on the color channel image according to image processing parameters corresponding to the color channel image to obtain a first processed image by:

And firstly, performing defocusing treatment on the color channel image to obtain a defocused image. Wherein the above-mentioned defocus processing can be realized by a defocus shader. The defocus shader described above may be a program for performing defocus processing on an image. The out-of-focus shader can change the definition and the focusing degree of an image by performing specific operation on pixels of the image (such as performing Gaussian blur processing on the image), and simulate the blurring effect generated by out-of-focus in the real world. The above-described defocus shaders may include Point Spread Function (PSF) defocus shaders and variable defocus amount shaders. The point spread function defocus shader described above achieves defocus effects by a predefined parameterized point spread function (e.g., gaussian function). The user can control the defocus amount by adjusting PSF parameters (such as blur radius), and the method is suitable for scenes with less partitions. When high frequency switching of defocus is required (e.g., pixel level partitioning, where a pixel is a partition), dynamically updating the PSF parameters may be limited by real-time computational load. The variable defocus amount shader described above can be used. The variable defocus amount shader described above integrates a lightweight neural network model. The model is designed to have small parameter scale and low calculation complexity so as to meet the real-time requirement in the shader environment. The model may be a neural network model with a feature vector containing the relative defocus amount (range [ -5,5] floating point type), astigmatism (range [0,2] floating point type) as input, and a two-dimensional PSF matrix (floating point type, normalized to [0,1] range) as output. The model may include a depth separable convolutional layer with a 3 x 3 convolutional kernel, 16 channels (activated by ReLU 6), and a depth separable convolutional layer with a 3 x 3 convolutional kernel, 32 channels (activated by HARDSWISH), as well as a convolutional layer with a 1 x1 point-by-point convolutional dimension reduction to 8 channels, and a fully connected layer that ultimately generates a 7x7 two-dimensional PSF matrix.

In practice, the execution body may choose to call the point spread function defocus shader or the variable defocus shader to perform defocus processing according to whether the number of actual partitions is greater than a preset threshold, so as to obtain a defocused image. For example, the variable defocus shader is called when the number of partitions exceeds 6, and the point spread function defocus shader is called when the number of partitions is less than 6. The threshold value of the number of partitions is not particularly limited here, and may be determined by the user according to the load capacity of the actual device.

And secondly, fusing the defocused image with a preset image of the corresponding color channel image to obtain a first processed image. In practice, the execution subject may use an Alpha transparency channel to fuse the out-of-focus image and the preset image based on transparency to obtain the first processed image. The preset image can be obtained through the following steps:

step one, for each color channel image in the respective color channel images, the following sub-steps are performed:

And step one, extracting the characteristics of the color channel image to obtain characteristic information. The characteristic information may include standard deviation of gray values of pixels of the color channel image, the number of edges of the color channel image, and average brightness values of all pixels in the color channel image. In practice, the execution subject may obtain the feature information through the following steps:

firstly, extracting standard deviation of pixel gray values of a color channel image to obtain the standard deviation.

And secondly, detecting edges in the color channel image by using an edge detection algorithm (such as a Sobel operator and a Canny operator), and determining the number of all coordinate points obtained by edge detection as the number of edges.

And thirdly, generating an average brightness value based on the brightness values of all the pixel points in the color channel image.

And fourth, determining the standard deviation, the edge number and the average brightness value as characteristic information.

And secondly, calling a contrast shader in response to the characteristic information characterization to adjust the contrast. The contrast shader may be a program including the cv2.normal () function of OpenCV. In practice, the executing body may determine whether the feature information is characterized as requiring adjustment of contrast by comparing the standard deviation in the feature information with a preset threshold. If the standard deviation is lower than a preset threshold (such as 50), a floating point value 0 is output, if the standard deviation is higher than the preset threshold (such as 100), a floating point value 2 is output, and if the standard deviation is 50-100, a floating point value 1 is output. When the output floating point value is not '1', namely, the characteristic information is determined to be characterized as that the contrast is required to be adjusted, and the contrast shader is called.

And step three, processing the color channel image through a contrast shader to obtain a contrast image. In practice, the execution body may input the floating point value output in the second sub-step to the contrast shader, and the contrast shader may perform contrast adjustment on the color channel image through a cv2.Normal () function of OpenCV to obtain a contrast image. Wherein, when the floating point value is 0, the contrast is improved by 25% on the original basis, and when the floating point value is 2, the contrast is reduced by 25% on the original basis.

And step four, calling a sharpening shader in response to the characteristic information characterization needs to adjust sharpening. The sharpening shader may be a program for adjusting the sharpness of an image by gaussian filtering. In practice, the executing body may determine whether the feature information is characterized as needing to be sharpened by comparing the number of edges in the feature information with a preset threshold. When the number of edges is lower than a preset threshold (such as 400 coordinate points), determining that the characteristic information characterization needs to be sharpened, and calling the sharpening shader.

And fifthly, processing the color channel image through a sharpening shader to obtain a sharpened image. In practice, the executing body may increase the sharpening degree of the color channel image by adjusting the window size of the gaussian filter included in the sharpening shader, to obtain the sharpened image. Wherein, the window size of 3×3 is adopted when the number of edges detected in the sub-step four is 0-200, and the window size of 5×5 is adopted when the number of edges is 200-400.

And step six, calling a color intensity shader in response to the characteristic information characterization to adjust the color intensity. The color intensity shader may be a program that converts RGB pixels of an input image into HSV format, adjusts saturation components, and converts the RGB pixels back into an RGB image. In practice, the execution body can determine whether the characteristic information represents that color intensity needs to be adjusted or not by calculating average brightness values of all pixel points in the color channel image, when the average brightness values are smaller than 80-180, the image is dark, the color intensity needs to be enhanced, a floating point value 0 is output, if the average brightness values are higher than 80-180, the image is bright, the color intensity needs to be reduced, a floating point value 2 is output, if the characteristic information belongs to 80-180, the image brightness is normal, the color intensity does not need to be adjusted, and a floating point value 1 is output. The condition that the output floating point number is not '1' is determined as the condition that the characteristic information characterizes the color intensity needs to be adjusted.

And seventhly, processing the color channel image through a color intensity shader to obtain a color intensity image. In practice, the execution body may input the floating point value output in the sixth sub-step into the color intensity shader, and the color intensity shader converts the color channel image into HSV format, adjusts the saturation component, and converts the saturation component back into RGB image to obtain the color intensity image. When the floating point value of the color intensity shader is input to be 0, the saturation value of the color channel image is increased by 20 percent, and when the floating point value of the color intensity shader is input to be 2, the saturation value of the color channel image is reduced by 20 percent.

And step two, fusing the obtained images to obtain a first processed image. Wherein, because some color channel images do not reach the condition of calling the shader to process, the corresponding processed images are not obtained. The first processed image may be an image obtained by superimposing one or more of a contrast image, a sharpened image, and a color intensity image. In practice, the execution subject may fuse the respective images obtained through the sub-steps based on transparency using an Alpha transparency channel to obtain the first processed image.

And 204, superposing the obtained first processing images to obtain a second processing image.

In some embodiments, the executing body may superimpose the obtained first processed images to obtain the second processed image. In practice, the execution body may directly add the color values of the corresponding pixels of the image after the predetermined processing for each color channel, to obtain the second processed image. For example, in Python using OpenCV library, assuming that three single-channel images are obtained by performing defocus processing on red (R), green (G), and blue (B) channels of the image, respectively, the function of pixel value addition can be realized by codes. In order to avoid the problem of color value overflow and distortion of the image color, the pixel values can be normalized after addition, and the pixel values are limited to be within the effective range of 0-255.

Alternatively, the execution body may further superimpose each of the processed images to obtain the second processed image in response to the image data to be processed including each of the processed images after the predetermined processing. Wherein each of the processed images after the predetermined processing described above may include monochromatic color channel images having different degrees of defocus. For example, each of the processed images after the predetermined processing may include monochrome images of three color channels of the same image, and each of the monochrome images of the color channels includes 10 monochrome images having different degrees of defocus. When the image data to be processed includes each processed image after the predetermined processing, the processes of splitting the color channel and respectively performing defocus processing on the single-color channel image can be omitted, and the method can be applied to equipment with weaker computing power. In practice, the execution subject may select 1 monochrome image having a preset defocus level from among the monochrome images of each of the three color channels, to obtain 3 monochrome images. The 3 single-color images obtained are fused (refer to step 204) to obtain a second processed image. For example, when the user needs a second processed image having a-0.5D defocus effect, a relative defocus amount of-0.5D can be manually input. The execution body may select 1 single-color image with defocus amount of-0.5 from the single-color images of each of the three color channels, respectively, to obtain 3 single-color images with defocus amount of-0.5. And then overlapping the 3 monochromatic images with the defocus amount of-0.5 to obtain a second processed image with the defocus effect of-0.5.

Optionally, the above execution body may further execute the following steps:

In the first step, in response to receiving two images to be processed, respectively performing defocusing processing on the two images to be processed by adopting a first image processing parameter and a second image processing parameter to obtain two second processed images. It should be noted that, the receiving of the actual application scenario corresponding to the two images to be processed includes that the executing body is an apparatus such as AR glasses, which can display the two images respectively. Taking AR glasses as an example, when the directions of astigmatism and the degrees of myopia and hyperopia of two eyes of a user are not completely the same, in order to adapt to the vision of each eye of the user, corresponding defocus processing is required to be performed on images seen by the left and right eyes of the user according to the vision of the left and right eyes of the user. The first image processing parameter and the second image processing parameter may each include a relative defocus amount, and the relative defocus amounts included in the two parameters may be different to accommodate different vision conditions of the left and right eyes of the user. The first image processing parameter and the second image processing parameter may be preset according to the vision condition of the user. In practice, the execution subject may use the first image processing parameter as a processing parameter for displaying the left-eye image of the user, and use the second image processing parameter as a processing parameter for displaying the right-eye image of the user. And respectively performing defocusing treatment on the two images to be treated to obtain two second treated images. The defocus processing manner may refer to the steps 201 to 204, and will not be described herein.

And secondly, performing binocular different display on the two second processed images. In practice, the execution subject may perform binocular disparity display on the two second processed images by a 3D display technique of binocular disparity (dual image channels).

Optionally, the above execution body may further execute the following steps:

and firstly, extracting a depth map of the second processed image to obtain an initial depth map. In practice, the executing body may calculate the disparity map using the OpenCV StereoSGBM algorithm, then convert the disparity map into a depth map, and determine the depth map as an initial depth map.

And secondly, carrying out normalization processing on the initial depth map to obtain a normalized depth map. In practice, the execution body may unify the depth value range of the initial depth map using the normalize function of OpenCV, where a large span of reduction factor values results in a subsequent threshold segmentation or algorithm computation anomaly. For example, the initial depth map depth value range is set to be [100,500], and the initial depth map depth value range is normalized to be [0,1], so that unified thresholds can be set during threshold segmentation.

And thirdly, threshold segmentation is carried out on the normalized depth map, and a clear area mask and a defocused area mask are obtained. Wherein the mask may comprise a binary image and the regions of sharpness and defocus are distinguished by white and black. In practice, the executing body may use the threshold function of OpenCV to distinguish the clear region (usually foreground, with lower depth value) and the defocus region (usually background, with higher depth value) according to the preset depth value threshold, so as to obtain the clear region mask and the defocus region mask. And determining that the depth value is smaller than a preset depth value threshold value as a clear region, and the rest regions are defocused regions. For example, assuming that the preset depth value threshold is 0.3, pixels with depth values smaller than 0.3 in the normalized depth map may be divided into clear area masks (white), and the rest is out-of-focus area masks (black).

And fourthly, performing morphological expansion operation on the clear region mask to obtain the transition region mask. Wherein the morphological dilation operation may include enlarging a boundary of a target region (e.g., a clear region mask) in the binary image using structural elements (e.g., rectangles, ellipses). Therefore, a transition zone can be generated at the periphery of the clear area, and faults after noise reduction caused by hard segmentation of the boundary between the clear area and the defocused area are avoided. In practice, the execution body may use the dilate function of OpenCV to extend the edge of the clear region mask outward by 2 pixels using 3×3 rectangular structural elements to form a transition zone (gray region), so as to obtain a transition region mask.

And fifthly, performing morphological corrosion operation on the mask in the defocused area to obtain the mask in the core defocused area. Wherein the morphological erosion operation may include narrowing the boundary of the out-of-focus region mask in the binary image using the structural element. The operation can remove the transition pixels with blurred edges from the defocused area, and the pure defocused core area is reserved, so that the targeted noise reduction is facilitated. In practice, the execution body may erode the defocus region mask by using 3×3 rectangular structural elements through the erode function of OpenCV, resulting in a core defocus region mask that is shrunk inward by 2 pixels compared to the edge of the defocus region mask.

And sixthly, generating a first noise reduction parameter based on the pixel covered by the clear area mask. The first noise reduction parameter may be a parameter (such as a kernel size of mean filtering) that controls the noise reduction of the clear region mask in the subsequent step. Suitable first noise reduction parameters may protect pixels at the edges of the clear area mask while denoising. In practice, the executing body may read the gray value of each pixel covered by the clear area mask, calculate the standard deviation of the gray value, and determine the first noise reduction parameter by comparing the standard deviation with a preset threshold (e.g. 25). And when the standard deviation is larger than a preset threshold value, determining the kernel size of 3 multiplied by 3 as a first noise reduction parameter. And when the standard deviation is smaller than or equal to a preset threshold value, determining 0 as a first noise reduction parameter, namely, subsequently, not reducing noise of the clear area mask.

And seventh, generating a second noise reduction parameter based on the pixel covered by the mask of the core defocused area. The second noise reduction parameter may be a parameter (such as a kernel size of mean filtering) for controlling the noise reduction of the mask in the core defocus region in the subsequent step. In practice, the executing body may read the gray value of each pixel covered by the mask in the core defocus region, and calculate the variance of the gray value. And when the variance is less than or equal to a preset value of 5, determining the kernel size of 7×7 as a second noise reduction parameter. And when the variance is larger than a preset value of 5, determining 0 as a second noise reduction parameter, namely, subsequently, not carrying out noise reduction on the mask of the core defocused area.

And eighth, performing edge protection noise reduction on the clear region mask by adopting the first noise reduction parameters to obtain a first noise reduction subgraph. The edge protection noise reduction may include, but is not limited to, mean filtering or bilateral filtering on the clear region mask. In practice, the executing body may use the first noise reduction parameter (for example, the kernel is 3×3) generated in the sixth step to perform mean filtering on the clear region mask, so as to obtain a first noise reduction subgraph.

And ninth, carrying out noise reduction on the mask of the core defocused area by adopting second noise reduction parameters to obtain a second noise reduction subgraph. In practice, the executing body may use the second noise reduction parameter (for example, the kernel is 7×7) generated in the seventh step to perform mean filtering on the mask of the core defocus region to obtain a second noise reduction subgraph.

And tenth, performing linear interpolation noise reduction on the transition region mask to obtain a third noise reduction subgraph. The linear interpolation noise reduction may include generating a noise reduction parameter between the first noise reduction parameter and the second noise reduction parameter, and performing noise reduction processing on the transition region mask by using the generated noise reduction parameter as a kernel size of the mean filtering. In practice, the execution body may determine the noise reduction parameters of the transition region mask through the following formula:

h=σ×first noise reduction parameter + (1- σ) ×second noise reduction parameter

Where H represents the noise reduction parameters of the transition region mask and σ represents the weight coefficients (preset by the user). The weight coefficient has a value range of [0,1]. For example, let the kernel of the first noise reduction parameter be 3×3, the kernel of the second noise reduction parameter be 7×7, and when the weight coefficient takes 0.5, the kernel size of the noise reduction parameter of the transition region mask is 5×5 (approximately rounded) by taking the above data into the formula. And then denoising the transition region mask (such as mean value filtering) by using the denoising parameters obtained by the formula to obtain a third denoising sub-graph.

And eleventh step, performing pixel level fusion on the first noise reduction subgraph, the second noise reduction subgraph and the third noise reduction subgraph to obtain a preliminary noise reduction image. In practice, the execution subject may use the bit operation combination mask to combine the three subgraphs (the first noise reduction subgraph, the second noise reduction subgraph and the third noise reduction subgraph) according to the mask, so as to obtain the preliminary noise reduction image.

And twelfth, performing edge sharpening compensation on the preliminary noise reduction image to obtain a noise reduction image. The edge sharpening compensation can recover edge details possibly lost in the noise reduction process, and image definition is improved. In practice, the executing body may use the filter2D function of OpenCV, apply a sharpened convolution kernel (such as laplace operator) to enhance edge details, and obtain a noise-reduced image.

The first to twelfth steps are an invention point of the embodiments of the present disclosure, and solve the technical problem of "poor edge definition of the out-of-focus image". The factors causing poor edge definition of the defocused image are specifically that a global noise reduction algorithm is generally adopted for noise reduction of the defocused image at present, and the algorithm can excessively blur edge details of a clear area. If the above factors are solved, the problem of poor edge definition of the out-of-focus image can be improved. To achieve this effect, the present disclosure additionally provides a method of zonal noise reduction that distinguishes between a clear zone and a defocus zone by a depth map and establishes a transition zone region at the edge of the two zones. And adopting different noise reduction parameters to reduce noise in the three areas. Finally, sharpening compensation is added to the edges to help strengthen edge protection. The protection effect on the image edge is realized. Thus, the problem of poor edge definition of the out-of-focus image is improved.

Optionally, the above execution body may further execute the following steps:

The first step, collecting the ambient light according to the preset frequency to obtain the ambient light information. The ambient light information may include, among others, illumination intensity (unit: lux, lux) and ambient color temperature (unit: kelvin, K). In practice, the executing body can record the illumination intensity and the color temperature of the ambient light according to the frequency of 1 time per second through the digital ambient light sensor, and store the illumination intensity and the ambient color temperature in the form of texts to obtain the ambient light information.

And secondly, preprocessing the ambient light information to obtain illumination intensity data and color temperature data. The preprocessing may be sliding window filtering of the ambient light information to eliminate the influence of instantaneous light fluctuation (such as light flicker) on the ambient light to a certain extent. The illumination intensity data may be indicative of a current illumination intensity value, and the color temperature data may be indicative of a current color temperature intensity value. In practice, the execution subject may use 5-frame sliding window filtering, that is, respectively preserve the illumination intensity and the ambient color temperature acquired for the last 5 times, and respectively calculate the average value of the two sets of data as the illumination intensity data and the color temperature data. For example, assuming that the illumination intensity continuously collected by the digital ambient light sensor is [8000,7500,8200,7800,8100], the filtered illumination intensity can be obtained by the following equation (8000+7500+8200+7800+8100)/5=7920.

And thirdly, comparing the illumination intensity data with a preset interval to obtain illumination level information. The preset interval may be an interval divided according to different illumination intensities corresponding to different illumination levels. The above-described light level information may include three pieces of text information ("first level", "second level", "third level"), corresponding to three levels of light environments (strong light environment, weak light environment, and normal light environment), respectively. In practice, the preset intervals may be divided in such a manner that the light intensity is a strong light environment when the light intensity is equal to or greater than 5000Lux and the light intensity information when the light intensity is equal to or greater than 5000Lux is determined as a first level, the light intensity is a weak light environment when the light intensity is equal to or less than 500Lux and the light intensity information when the light intensity is equal to or less than 500Lux is determined as a second level, the light intensity is a normal light environment when the light intensity is greater than 500Lux and less than 5000Lux and the light intensity information when the light intensity is greater than 500Lux and less than 5000Lux is determined as a third level. The execution subject can compare the illumination intensity data with the preset interval through if conditional sentences to obtain illumination level information. For example, if the illumination intensity data is 6000Lux, and if the comparison of the conditional sentences shows that 6000 is greater than 5000 and belongs to a strong light environment, the text information "first level" is output as illumination level information.

And fourthly, comparing the illumination intensity data with a preset compression coefficient mapping table to obtain a compression coefficient in response to the illumination level information being the first level. The preset compression coefficient mapping table may be a mapping relationship table of different illumination intensities and compression coefficients. The compression coefficient may be a parameter for processing the second processed image in a subsequent step. In practice, after receiving the illumination level information with the content of "first level", the executing body may compare the illumination intensity data with the preset compression coefficient mapping table through if conditional statements, and output a corresponding compression coefficient. For example, assume that the illumination intensity data is 5500 and the corresponding compression coefficient is 25.

And fifthly, carrying out logarithmic transformation on the brightness of the second processed image according to the compression coefficient to obtain a compressed image serving as an image to be displayed. In practice, for each of the individual pixels of the second processed image, the above-described execution subject may logarithmically transform the luminance value by the following formula:

adjusted luminance value = compression coefficient x ln (1 + original luminance value)

Where "ln" represents a logarithmic operation based on a natural logarithmic base e. "ln (1+original luminance value)" means an operation of taking the natural logarithm e as a base and obtaining the logarithm of "(1+original luminance value)". For example, when the original luminance value of a pixel of the second processed image is 250 and the compression coefficient is 25, the adjusted luminance value is 137.5. When the logarithmic operation is performed in the above formula, if the result includes a multi-bit decimal, only the decimal point of the next bit is retained. And determining the second processed image with the brightness subjected to logarithmic transformation through each pixel point as a compressed image and taking the compressed image as an image to be displayed. The step can compress the brightness of the second processed image to the human eye comfort range through the compression coefficient. And the overexposure of the highlight area (such as sky) of the second processed image to white caused by the over-strong illumination is reduced.

And sixthly, comparing the illumination intensity data with a preset amplification factor mapping table to obtain an amplification factor in response to the illumination level information being the second level. The preset amplification factor mapping table may be a mapping relation table of different illumination intensities and amplification factors. The magnification factor may be a parameter for processing the second processed image in a subsequent step. In practice, after receiving the illumination level information with the content of "second level", the executing body may compare the illumination intensity data with the preset amplification factor mapping table through if conditional statements, and output a corresponding amplification factor. For example, assume that the illumination intensity data is 100Lux, and the corresponding magnification factor is 2.0.

And seventhly, carrying out brightening treatment on the second treatment image according to the amplification factor to obtain a brightening image serving as an image to be displayed. The brightening process may include increasing a brightness value of a pixel point in the second processed image. In practice, the execution subject may multiply the luminance value of each pixel of the second processed image by the magnification factor, generate a brightening image, and determine the brightening image as the image to be displayed. In order to avoid overexposure, it is necessary to make a saturation cut-off of the luminance value multiplied by the amplification factor, that is, to set a luminance value exceeding 255 to 255. For example, when the luminance value of a certain pixel is 200 and the amplification factor is 1.5, the luminance value of the pixel should be 300 on the above-mentioned brightness-enhanced image, but the luminance value of the pixel is 255 due to the saturation clipping mechanism. The step can improve the overall brightness of the image in the low-light scene, so that dark details (such as characters and objects in shadows) can be seen.

And eighth, comparing the color temperature data with a preset scene mapping table to obtain scene information in response to the illumination level information being the third level. The preset scene mapping table may be a mapping relationship table between different color temperature data and a scene. The scenes may include warm light scenes (color temperature data < 4000K), cool light scenes (color temperature data > 6500K), and natural light scenes (4000-6500K). The scene information may be information (e.g., text information) indicating which scene the current color temperature belongs to. In practice, after receiving the illumination level information with the content of "third level", the executing body may compare the color temperature data with the preset scene mapping table through if conditional statements, and output corresponding scene information. The corresponding scene information may be text information with the content of "first scene" when the above color temperature data <4000K, and text information with the content of "second scene" when the above color temperature data > 6500K.

And ninth, responding to the scene information to represent the first scene, and carrying out brightening treatment on the blue channel image of the second treatment image to obtain a first scene image as an image to be displayed. In practice, the executing body may split the second processed image into red, green and blue images after receiving the scene information with the content of "first scene" (refer to step 201), and increase the brightness of each pixel point in the blue channel image by 10%. And then, overlapping the blue channel image after the brightening process with the red and green channel images to generate a first scene image (refer to step 204), and determining the generated first scene image as an image to be displayed. This step can reduce the yellow-warm color shift on the second processed image, and make the color more natural.

And tenth, responding to the scene information to represent a second scene, and carrying out brightening treatment on the red channel image of the second treatment image to obtain a second scene image as an image to be displayed. In practice, the executing body may split the second processed image into red, green and blue images after receiving the scene information with the content of "second scene" (refer to step 201), and increase the brightness of each pixel point in the red channel image by 10%. And then, overlapping the red channel image and the blue and green channel images after the brightening treatment to generate a second scene image (refer to step 204), and determining the generated first scene image as an image to be displayed. The step can reduce the cold blue color cast on the second processed image, so that the color is more natural.

And eleventh, displaying the determined image to be displayed.

The first to eleventh steps are an invention point of the embodiments of the present disclosure, and solve the technical problem of "poor image display effect after defocus processing". The factors causing poor image display effect after defocusing treatment are specifically that the fixed parameters of the traditional defocusing algorithm cannot adapt to the light intensity change, and the color shift under different light sources is obvious. If the above factors are solved, the display effect of the image after the defocus processing can be improved. In order to achieve the effect, the disclosure further provides a light intensity and color temperature linked defocused image processing method, which adjusts the second processed image and the monochromatic channel image obtained by splitting the second processed image by collecting the illumination intensity value and the ambient color temperature value of the ambient light. The brightness of the image is regulated and controlled, and the influence of color cast on the image is reduced. Thereby, the display effect of the image after the defocus processing is improved.

With further reference to fig. 3, as an implementation of the method shown in fig. 2, the present disclosure provides embodiments of a real-time digital image defocus processing apparatus that correspond to those shown in fig. 2, which may find particular application in a variety of electronic devices.

As shown in fig. 3, the real-time digital image defocus processing apparatus 300 of some embodiments includes a splitting unit 301, a determining unit 302, a processing unit 303, and a superimposing unit 304. The splitting unit is configured to split the image data to be processed according to a preset number of color channels to obtain color channel images in response to receiving the image data to be processed, the determining unit is configured to determine image processing parameters of each color channel image in the color channel images, wherein the image processing parameters comprise relative defocus amounts, the processing unit is configured to perform preset processing on the color channel images according to the image processing parameters corresponding to the color channel images to obtain first processed images, the preset processing comprises defocus processing, and the superimposing unit is configured to superimpose the obtained first processed images to obtain second processed images.

It will be appreciated that the elements described in the apparatus 300 correspond to the various steps in the method described with reference to fig. 2. Thus, the operations, features and resulting benefits described above with respect to the method are equally applicable to the apparatus 300 and the units contained therein, and are not described in detail herein.

Referring now to fig. 4, a schematic diagram of an electronic device 400 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic device shown in fig. 4 is merely an example and should not impose any limitations on the functionality and scope of use of embodiments of the present disclosure.

As shown in fig. 4, the electronic device 400 may include a processing means 401 (e.g., a central processing unit, a graphics processor, etc.) that may perform various suitable actions and processes in accordance with a program stored in a Read Only Memory (ROM) 402 or a program loaded from a storage means 408 into a Random Access Memory (RAM) 403. In the RAM403, various programs and data necessary for the operation of the electronic device 400 are also stored. The processing device 401, the ROM402, and the RAM403 are connected to each other by a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.

In general, devices may be connected to I/O interface 405 including input devices 406 such as a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc., output devices 407 including a Liquid Crystal Display (LCD), speaker, vibrator, etc., storage devices 408 including magnetic tape, hard disk, etc., and communications devices 409. The communication means 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. While fig. 4 shows an electronic device 400 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead. Each block shown in fig. 4 may represent one device or a plurality of devices as needed.

In particular, according to some embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via communications device 409, or from storage 408, or from ROM 402. The above-described functions defined in the methods of some embodiments of the present disclosure are performed when the computer program is executed by the processing device 401.

It should be noted that, the computer readable medium described in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, the computer-readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to electrical wiring, fiber optic cable, RF (radio frequency), and the like, or any suitable combination of the foregoing.

In some embodiments, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperTextTransferProtocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

The computer readable medium may be included in the electronic device or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to split the image data to be processed according to a preset number of respective color channels in response to receiving the image data to be processed to obtain respective color channel images, determine image processing parameters of each of the respective color channel images, wherein the image processing parameters include a relative defocus amount, and perform predetermined processing on each of the respective color channel images according to the image processing parameters corresponding to the color channel images to obtain a first processed image, wherein the predetermined processing includes defocus processing, and superimpose the obtained respective first processed images to obtain a second processed image.

Computer program code for carrying out operations for some embodiments of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in some embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The described units may also be arranged in a processor, for example as a processor comprising a splitting unit, a determining unit, a processing unit, an overlaying unit. The names of these units do not limit the unit itself in some cases, and for example, the splitting unit may also be described as "a unit that splits image data to be processed by a preset number of individual color channels to obtain individual color channel images".

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic that may be used include Field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems-on-a-chip (SOCs), complex Programmable Logic Devices (CPLDs), and the like.

The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above technical features, but encompasses other technical features formed by any combination of the above technical features or their equivalents without departing from the spirit of the invention. Such as the above-described features, are mutually substituted with (but not limited to) the features having similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A real-time digital image defocus processing method comprising:

Responding to received image data to be processed, splitting the image data to be processed according to each color channel of a preset number to obtain each color channel image;

Determining an image processing parameter of each color channel image in the respective color channel images, wherein the image processing parameter comprises a relative defocus amount;

For each color channel image in the color channel images, carrying out preset processing on the color channel images according to image processing parameters corresponding to the color channel images to obtain first processed images, wherein the preset processing comprises defocusing processing;

And superposing the obtained first processing images to obtain a second processing image.

2. The method according to claim 1, wherein the splitting the image data to be processed according to a preset number of respective color channels to obtain respective color channel images includes:

dividing the image data to be processed into control areas with the number of preset areas to obtain images of the control areas;

and splitting each control area image in the control area images according to a preset number of color channels to obtain color channel images corresponding to the control area images.

3. The method according to claim 1, wherein the performing, according to the image processing parameters corresponding to the color channel image, a predetermined process on the color channel image to obtain a first processed image includes:

Performing defocusing treatment on the color channel image to obtain a defocused image;

and fusing the defocused image with a preset image corresponding to the color channel image to obtain a first processed image.

4. The method according to claim 2, wherein the dividing the image data to be processed into the respective control areas of the preset area number, to obtain the respective control area images, includes:

collecting a motion image sequence of a user;

identifying motion information of the user from the sequence of motion images;

and according to the action information, dividing each control area with the preset area number from the image data to be processed as each control area image.

5. The method according to claim 2, wherein the dividing the image data to be processed into the respective control areas of the preset area number, to obtain the respective control area images, includes:

Performing hot content detection on the image data to be processed to obtain hot information;

and according to the hot spot information, dividing each control area with the preset area number from the image data to be processed as each control area image.

6. The method of claim 1, wherein the method further comprises:

in response to receiving two images to be processed, respectively performing defocusing treatment on the two images to be processed by adopting a first image processing parameter and a second image processing parameter to obtain two second processed images;

and performing binocular heterogenous display on the two second processed images.

7. The method of claim 1, wherein the determining the image processing parameters for each of the respective color channel images comprises:

generating a relative defocus amount based on reference information, wherein the reference information includes at least one of a user's vision condition, display screen characteristics, viewing distance, and pupil size;

generating image processing parameters based on the relative defocus amount;

and determining the image processing parameters as the image processing parameters of each color channel image in the respective color channel images.

8. The method of claim 1, wherein the determining the image processing parameters for each of the respective color channel images comprises:

generating a relative defocus amount based on a preset time period in response to the image data to be processed not meeting a preset image type condition;

generating a relative defocus amount through preset regulation and control processing in response to the image data to be processed meeting the preset image type condition;

generating image processing parameters based on the generated relative defocus amount;

9. The method of claim 1, wherein the method further comprises:

and responding to the image data to be processed, wherein the image data to be processed comprises all the processed images after the predetermined processing, and superposing the processed images to obtain a second processed image.

10. A method according to claim 3, wherein one shader set is associated with each of the individual color channels, each shader set comprising a contrast shader, a sharpening shader, a color intensity shader, and wherein prior to said fusing the out-of-focus image with the pre-set image corresponding to the color channel image, the method further comprises:

for each of the individual color channel images, the following sub-steps are performed:

extracting the characteristics of the color channel image to obtain characteristic information;

invoking the contrast shader in response to the characterization of the feature information requiring adjustment of contrast;

processing the color channel image through the contrast shader to obtain a contrast image;

invoking the sharpening shader in response to the characterization information characterizing a need to adjust sharpening;

processing the color channel image through the sharpening shader to obtain a sharpened image;

invoking the color intensity shader in response to the characterization information characterizing a need to adjust color intensity;

Processing the color channel image by the color intensity shader to obtain a color intensity image;

And fusing the obtained images to obtain a preset image.

11. A real-time digital image defocus processing apparatus comprising:

The splitting unit is configured to respond to the received image data to be processed, split the image data to be processed according to each color channel of a preset number, and obtain each color channel image;

a determining unit configured to determine an image processing parameter of each of the respective color channel images, wherein the image processing parameter includes a relative defocus amount;

A processing unit configured to perform predetermined processing on each color channel image in the respective color channel images according to image processing parameters corresponding to the color channel images to obtain a first processed image, wherein the predetermined processing includes defocus processing;

And a superimposing unit configured to superimpose the obtained first processed images to obtain second processed images.

12. An electronic device, comprising:

One or more processors;

a storage device having one or more programs stored thereon;

When executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1 to 10.

13. A computer readable medium having stored thereon a computer program, wherein the program when executed by a processor implements the method of any of claims 1 to 10.