WO2021031781A1 - Method and device for calibrating projection image and projection device - Google Patents

Abstract

Disclosed in embodiments of the present application are a method and device for calibrating a projection image, and a projection device. The method comprises: acquiring spatial depth information collected by three or more ranging sensors located in an exit plane about a projection image to be calibrated; determining projection plane information on the basis of the spatial depth information and the relative positional relationship between the ranging sensors; on the basis of exit plane information and the projection plane information, obtaining a rotation matrix of the projection plane relative to the exit plane; according to the rotation matrix, carrying out perspective transformation on a projection picture of the projection image to be calibrated in order to adjust the projection picture to be a target shape. The present method obtains the rotation matrix of the projection plane relative to the exit plane on the basis of the exit plane information of the exit plane where the ranging sensors are located and the projection plane information. Then, when the rotation matrix is obtained, the perspective transformation of the projection picture of the projection image to be calibrated is performed according to the rotation matrix, so that automatic correction of the projection image may be completed quickly, enhancing the user experience.

Description

Projection image calibration method, device and projection equipment Technical field

The present application relates to the field of image processing technology, and more specifically, to a projection image calibration method, device, projection equipment, and storage medium.

Background technique

With the development of display technology, the application of projection equipment has become more and more extensive, including educational projectors, home projectors and engineering projectors. Projection technology has brought great changes to people's lives, studies and work. In order to ensure the display effect of the projection screen of the projector, it is necessary to calibrate the distortion of the projection image of the projector after each relocation, which seriously affects the user's viewing experience.

Summary of the invention

In view of the above problems, this application proposes a projection image calibration method, device, projection equipment, and storage medium to improve the above problems.

In the first aspect, an embodiment of the present application provides a projection image calibration method, the method includes: acquiring the spatial depth information of the projection image to be calibrated collected by three or more ranging sensors located on the exit plane, and the spatial depth The information is the distance information from the exit plane to the projection plane formed by the ranging sensor; the projection plane information is determined based on the spatial depth information and the relative position relationship between the ranging sensors; the projection plane relative to the exit plane is obtained based on the exit plane information and the projection plane information According to the rotation matrix, the projection screen of the projected image to be calibrated is transformed into perspective to adjust the projection screen to the target shape.

In the second aspect, an embodiment of the present application provides a projection image calibration device that runs on a projection device. The device includes: a data acquisition unit for acquiring data collected by three or more ranging sensors located on the exit plane. The spatial depth information of the projected image to be calibrated. The spatial depth information is the distance information from the projection plane to the projection plane formed by the ranging sensor; the data processing unit is used to determine the projection plane based on the spatial depth information and the relative position relationship between the ranging sensors The data processing unit is also used to obtain the rotation matrix of the projection plane relative to the exit plane based on the exit plane information and the projection plane information; the projection unit is used to perform perspective transformation on the projection screen of the projected image to be calibrated according to the rotation matrix to project The picture is adjusted to the target shape.

In a third aspect, an embodiment of the present application provides a projection device, including a data acquisition module, a projection module, one or more processors, and a memory; one or more programs are stored in the memory and configured to be The one or more processors execute, and the one or more programs are configured to execute the method described in the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program code stored in the computer-readable storage medium, wherein the method described in the first aspect is executed when the program code is running.

The projection image calibration method, device, projection equipment, and storage medium provided by the present application obtain the spatial depth information of the projection image to be calibrated collected by three or more range-finding sensors located on the exit plane, and the spatial depth information is Distance information from the exit plane to the projection plane formed by the ranging sensor; then determine the projection plane information based on the spatial depth information and the relative position relationship between the ranging sensors; then obtain the projection plane relative to the exit plane based on the exit plane information and the projection plane information According to the rotation matrix, the projection screen of the projection image to be calibrated is transformed into perspective to adjust the projection screen to the target shape. In this way, the rotation matrix of the projection plane relative to the projection plane is obtained based on the exit plane information and the projection plane information of the exit plane where the ranging sensor is located, and then when the rotation matrix is acquired, the projection image is to be calibrated according to the rotation matrix. The projection image of the camera is transformed into perspective, so that the automatic correction of the projected image can be completed quickly and the user experience is enhanced.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

Fig. 1 shows a method flowchart of a projection image calibration method proposed in an embodiment of the present application.

Fig. 2 shows a method flowchart of a projection image calibration method proposed by another embodiment of the present application.

FIG. 3 shows a diagram of an example projection of the projection device provided by the embodiment of the present application.

FIG. 4 shows an example diagram of a projection mode of the projection device provided by the embodiment of the present application.

Fig. 5 shows an example diagram of another projection mode of the projection device provided by the embodiment of the present application.

FIG. 6 shows a method flowchart of the method of step S260 in FIG. 2.

FIG. 7 shows a method flowchart of the method of step S263 in FIG. 3.

FIG. 8 shows a schematic diagram of the rotation angle between the emission plane of the projection light of the projection device provided by the embodiment of the present application and the plane where the projection area is located.

Fig. 9 shows a structural block diagram of a projection image calibration device proposed in an embodiment of the present application.

Fig. 10 shows a structural block diagram of a projection device of the present application for executing a projection image calibration method according to an embodiment of the present application.

Fig. 11 shows a storage unit for storing or carrying program codes for implementing a projection image calibration method according to an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of this application.

With the development of display technology, the application of projection equipment has become more and more extensive, including educational projectors, home projectors and engineering projectors. Projection technology has brought great changes to people's lives, studies and work. However, the inventor found in the research that it is difficult for the projector to be placed completely parallel to the plane of the projection screen and present a perfect rectangular projection screen. Therefore, in order to ensure the display effect of the projection screen of the projector, it is necessary to correct each time it is repositioned. The projection image of the projector undergoes distortion calibration. Among them, distortion calibration is mainly divided into two categories: manual calibration and automatic calibration.

As a way, the keystone correction of the vertical direction of the projected image can be achieved by completely manual adjustment, but this way has randomness and contingency, which affects the calibration efficiency of the projected image. Further, the tilt angle can be obtained through the gyroscope, and then the automatic keystone correction in the vertical direction of the projected image is performed. Compared with the manual calibration method, the user experience of this method is much improved, but the left and right horizontal calibration is still not possible. With the rapid development of computer image processing technology, as another way, four-point keystone correction can be used to calibrate the distortion of the projected image. This method can be corrected by manually adjusting the four vertices of the projected image to the correct position. All-round adjustment of up, down, left and right, but still requires a certain amount of manual assistance. Or there are some projector products that use large amounts of data and complex algorithms to automatically complete the keystone correction of the projected image with one key after being turned on. This intelligent automatic calibration method greatly reduces the difficulty of user operation, but due to a large number of The data and complex algorithms result in slow calibration and high cost. Currently, products that can implement this function are limited.

Therefore, in order to improve the above-mentioned problems, the inventor proposes that the present application can make it possible to determine the projection plane information based on the spatial depth information and the relative position relationship between the ranging sensors when the spatial depth information is acquired, and then based on the ranging The exit plane information of the exit plane where the sensor is located and the projection plane information obtain the rotation matrix of the projection plane relative to the exit plane. Then, when the rotation matrix is acquired, the projection screen of the projection image to be calibrated is transformed according to the rotation matrix, making it fast The projected image is automatically corrected to enhance the user experience.

The following first introduces the projection image calibration method provided by the embodiments of the present application and the projection image calibration system involved in the device.

The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Please refer to FIG. 1, which is a method flowchart of a projection image calibration method provided by an embodiment of the application. The method of this embodiment can be executed by a device for calibrating a projected image. The device can be implemented by hardware and/or software, and can generally be integrated in a projection device provided with at least 3 range-finding sensors on the same plane. The projection equipment can include devices with projection functions such as laser TVs, projectors, and micro-projectors. It can also be a computer system that connects to a device with projection function and uses the distance measuring sensor of the device, such as a personal computer connected to a projection device, Laptops, tablets, smart phones, etc. The execution of this method depends on a computer program, which can run on a computer system, and the computer system can be an operating system of the projection device. It should be noted that the projection direction of the projection device in the embodiment of the present application is not limited, and may be rear projection or front projection. The method includes:

Step S110: Obtain the spatial depth information of the projected image to be calibrated collected by three or more ranging sensors located on the exit plane.

Wherein, the spatial depth information in the embodiment of the present application can be understood as the spatial distance information of the projection area corresponding to the projection device distance along the projection direction of the projection device. Optionally, for example, the spatial distance information may include the distance, direction, and tilt angle difference between the projection device and the projection area. Optionally, the projection equipment may be a projector or a laser projector or other equipment with projection function, which is not limited here.

As a way, the spatial depth information may include the distance between the projection area for displaying the projection image to be calibrated and the projection device. Wherein, the projected image to be calibrated can be understood as an image with image distortion after being projected onto the projection area by the projection device.

In order to eliminate the distortion of the projected image to be calibrated, as a way, the projection device can obtain the change rule of the pixel points of the projected image. Optionally, for the image data that needs to be projected, the projection device can obtain the projection image corresponding to the image data, and then determine the position of each pixel row according to the projected image, and then obtain the horizontal line length sum of each pixel row in the projection image The horizontal distortion length can then calculate the corrected pixel amount of the projected image according to the horizontal distortion length and the horizontal line length, and then it can be determined whether there is pixel distortion in the projected image.

Wherein, the image data can be pre-stored (for example, the audio and video data that needs to be projected is copied to the projection device for storage) or instant storage (for example, the mobile hard disk storing the audio and video data that needs to be projected is inserted in the projection device Above), the specific storage method is not limited. It is understandable that if image distortion is detected, these projected images with image distortion can be used as the projected images to be calibrated.

Optionally, in the case where it is detected that the projection image is distorted, the projection device may acquire the spatial depth information of the projection image to be calibrated collected by three or more distance measuring sensors located on the exit plane. Wherein, the distance measuring sensor can be configured in the projection device (for example, installed on the light-emitting surface of the projection device, which can be understood as the projection plane of the projection light of the projection device) for measuring the projection area of the projected image to be calibrated and the projection device Various distance sensors for distance, such as ToF (Time of Flight, Time of Flight) laser distance sensor, infrared distance sensor, ultrasonic distance sensor, etc. Among them, the ToF laser ranging sensor can transmit and receive laser light with a wavelength of 940nm to measure the space flight time difference and obtain the target distance. It has outstanding resistance to ambient light interference and can be applied to bright environments. It should be noted that the embodiments of the present application do not limit the specific types of distance measurement sensors. For example, it may be a (laser) distance measurement sensor with depth information measurement function that can be realized by existing or future technologies, which can meet the requirements for collecting and calibrating. It is sufficient to project the spatial depth information of the image. Subsequent embodiments of this application will use a ToF laser ranging sensor as an example for description.

In one implementation, if the above-mentioned distance measuring sensor is a ToF laser distance measuring sensor, the ToF laser distance measuring sensor can be installed facing the direction of the projection device (ie, forward installation), so that the ToF laser distance measuring sensor can be used Collect the spatial depth information of the projected image to be calibrated.

By combining a distance measuring sensor with a depth (distance) information measurement function and a projection device, it is possible to realize distortion calibration of the projection image based on the distance information, and reduce the calibration cost.

Step S120: Determine projection plane information based on the spatial depth information and the relative position relationship between the ranging sensor.

Wherein, the projection plane refers to the plane where the image (or image) projected by the projection device is located. For example, if the projector in the conference room puts the PPT presentation on the projection screen, then the plane where the projection screen (that is, the projection area) is located at this time is the projection plane. It is understandable that, in order to facilitate the calibration of the pixel distortion of the projected image, after the spatial depth information of the projected image to be calibrated collected by the ranging sensor is obtained, it can be based on the spatial depth information and multiple (ie three or more) The relative position relationship between the ranging sensors obtains the projection plane information of the plane where the projection area is located.

As a way, the projection plane information can be expressed by a projection plane equation. Optionally, the installation position of the distance measuring sensor is fixed, then the respective position coordinates of multiple distance measuring sensors can be obtained, and then the projection plane equation representing the projection plane information can be obtained based on the spatial depth information and the position coordinates of the distance measuring sensor . It should be noted that the projection plane includes the projection area, and the projection area is usually fitted to the projection plane, that is, the distance between the projection area and the projection plane (it can be the vertical distance) can be ignored, so the obtained projection plane equation As the equation of the projection plane.

It should be noted that the projection plane equation is an equation that characterizes the plane of the projection area in the specified coordinate system. Optionally, the designated coordinate system represents the coordinate system of the projection device. It can be understood that, for a distance measuring sensor installed in a projection device, for example, for a ToF laser distance measuring sensor installed on the projection device in a forward direction, the plane where the ToF laser distance measuring sensor is installed on the projection device can be used as the starting coordinate plane. Optionally, the plane on which the projection device is installed with the ToF laser ranging sensor can be taken as the OXY plane, that is, Z=0, and the projection direction facing the projection device is taken as the positive direction of the Z axis. Then, since correcting the projection image actually removes the pixel distortion of the projected image to be calibrated, and the spatial depth information of the projected image to be calibrated is the projection light exit plane of the projection device (for example, the OXY plane with Z=0, it should be noted that, The coordinate system here is the reference coordinate system, which can be adjusted according to the specific implementation situation, and the specific value of Z can be adjusted according to the actual situation, such as Z=1, 2, 3...) to the projection area (for example The distance between the projection screen or other projection area), that is to say, the projection light emission plane of the projection device and the plane on which the ToF laser ranging sensor is installed on the projection device can be the same plane or different planes. Then it can be understood that the projection plane equation is an equation that characterizes the plane of the projection area in the specified coordinate system.

In one implementation, the projection can be determined according to the position where the projection device is installed with multiple ToF ranging sensors and the obtained projection light exit plane of 3 or more projection devices to the plane where the projection area is located (that is, the projection plane). Plane equation. By obtaining the projection plane equation, it is convenient to subsequently calculate the rotation matrix between the plane where the projection area is located (the projection plane) and the projection light exit plane of the projection device.

Step S130: Obtain a rotation matrix of the projection plane relative to the exit plane based on the exit plane information and the projection plane information.

Among them, the exit plane information includes the exit plane equation. Optionally, since the plane on which the ToF laser ranging sensor is installed on the projection device and the position of the ToF laser ranging sensor are known, and the coordinate system of the plane where the ToF laser ranging sensor is located is not fixed, that is, the ToF laser The coordinate system of the plane where the ranging sensor is located is the reference coordinate system.

Then, it can be understood that, because the plane where the ToF laser distance sensor is installed is parallel to the projection plane of the projection device, then as an implementation, the plane where the ToF laser distance sensor is installed (this time is different The ToF laser ranging sensor is located on the same plane) as the exit plane of the projection light of the projection device, then the exit plane equation can be obtained in advance according to the position of the ToF laser ranging sensor. As another implementation method, according to the position where the ToF laser ranging sensor is installed, the equation of the plane where the ToF laser ranging sensor is located (similarly, when different ToF laser ranging sensors are located on the same plane), Then, according to the equation and the vertical distance between the plane where the ToF laser distance sensor is installed and the exit plane of the projection light of the projection device, the exit plane equation of the exit plane of the projection light of the projection device is calculated.

It can be understood that, in the process of delivering the image from the projection device to the projection area, the projection device (or the placement position of the projection device) cannot be completely parallel to the plane where the projection area is located, resulting in pixel displacement of the projected image, resulting in projection The image has pixel distortion. Therefore, in order to eliminate the projection error caused by such pixel distortion and enhance the visual effect of the projected image, the rotation matrix of the projection plane relative to the exit plane can be obtained based on the exit plane information and the projection plane information, that is to say , The rotation matrix can be used to eliminate the pixel distortion caused by the pixel displacement of the projected image.

As a way, the rotation matrix of the projection plane relative to the exit plane can be calculated based on the exit plane equation and the projection plane equation, so that the automatic distortion calibration of the projected image to be calibrated can be performed based on the rotation matrix.

Step S140: Perform perspective transformation on the projection image of the projection image to be calibrated according to the rotation matrix, so as to adjust the projection image to a target shape.

As a way, the automatic distortion calibration can be realized by performing perspective transformation on the projection image of the projection image to be calibrated according to the above-mentioned rotation matrix. The perspective transformation can be understood as the correction of the pixel point displacement of the projection image of the projection image to be calibrated. Through the perspective transformation, the projection image of the projection image to be calibrated can be adjusted to the target shape. For example, adjust to a square rectangle. It should be noted that the specific shape of the target shape here is not limited, and can be any desired shape, for example, a square rectangle, a positive direction or a circle, etc., which can be set according to actual conditions.

The projection image calibration method provided by the present application obtains the spatial depth information of the projected image to be calibrated collected by three or more ranging sensors located on the exit plane, and then based on the spatial depth information and the distance between the ranging sensors The relative position relationship determines the projection plane information, and then obtains the rotation matrix of the projection plane relative to the projection plane based on the exit plane information and the projection plane information, and then performs perspective transformation on the projection screen of the projected image to be calibrated according to the rotation matrix to adjust the projection screen to the target shape. It is realized that the rotation matrix of the projection plane relative to the emission plane is obtained based on the emission plane information of the emission plane where the distance measuring sensor is located and the projection plane information. Then, when the rotation matrix is obtained, the projection screen of the projection image to be calibrated is performed according to the rotation matrix. Perspective transformation enables rapid automatic correction of projected images and enhances user experience.

Please refer to FIG. 2, which is a method flowchart of a projection image calibration method provided by another embodiment of this application. The method of this embodiment can be executed by a device for calibrating a projected image. The device can be implemented by hardware and/or software, and can generally be integrated into a projection device, which can include a laser TV, a projector, and a micro-projector. And other equipment with projection function. The execution of this method depends on a computer program, which can run on a computer system, and the computer system can be an operating system of the projection device. It should be noted that the projection direction of the projection device in the embodiment of the present application is not limited, and may be rear projection or front projection. The method includes:

Step S210: Obtain the spatial depth information of the projection image to be calibrated collected by three or more ranging sensors located on the exit plane.

Step S220: Obtain the position coordinates of the preset number of the distance measuring sensors provided on the projection device.

Wherein, the preset number in the embodiment of the present application may be 3 or more, and the positions where the preset number of distance measuring sensors are placed are not on the same straight line. Optionally, if 3 or more ToF laser range finding sensors are placed on the same plane in front of the projection device, then the exit plane of the projection light of multiple projection devices can be obtained to the plane where the projection area is located (projection plane) the distance. Specifically, in one implementation, a three-dimensional coordinate system can be constructed, and the three-dimensional coordinate system can be constructed with the projection point of the projection device as the origin; in another implementation, any of the planes on which the ToF laser ranging sensor can be placed can also be constructed. One point is the origin to construct a three-dimensional coordinate system, and then the position coordinates of each ToF laser ranging sensor in the constructed three-dimensional coordinate system are obtained.

For example, in a specific application scenario, as shown in FIG. 3, the ToF laser ranging sensor 103 is installed forwardly on the projection device 100 (the number of ToF laser ranging sensors is not limited here, and it can be 3 or more than 3). , Figure 3 shows 3) The installation position is known, then as a way, the plane on which the projection device is installed with the ToF laser ranging sensor can be taken as the OXY plane, that is, Z=0, and the direction facing the projection is taken as In the positive direction of the Z axis, the position coordinates of the ToF laser ranging sensor can be (x1, y1, 0), (x2, y2, 0), ... (xn, yn, 0).

It is worth noting that the above-mentioned OXY plane is not fixed, but a reference plane. Then, if the value of Z changes, for example, if Z=1, then the position coordinates of the ToF laser ranging sensor can be changed to (x1, y1, 1), (x2, y2, 1), ... (xn, yn, 1). Therefore, the selection of the OXY plane and the specific value of Z are not limited in this embodiment.

It should be noted that the placement positions of the projection devices in the embodiments of the present application may be different, that is, the relationship between the emission plane of the projection light of the projection device and the plane (projection plane) where the projection area is located may be different. For example, in an implementation manner, as shown in FIG. 4, the projection device 100 may project in a direction directly facing the plane (projection plane) where the projection area is located. In this case, the normal line of the lens of the projection device 100 and The plane 101 where the projection area is located is vertical. In another implementation manner, as shown in FIG. 5, the projection direction of the projection device 100 may also have a certain tilt angle to the plane 101 where the projection area is located. In this case, the normal line of the lens of the projection device 100 and the projection The plane 101 where the area is located may be non-vertical. For example, this projection mode may be applied to an ultra-short-range laser TV.

Specifically, in the projection mode shown in FIG. 5, compared with the projection mode shown in FIG. 4, the projection mode shown in FIG. 5 is different in constructing the reference coordinate system of the projection light emission plane of the projection device. In addition, the distance between the exit plane of the projection light of the projection device and the plane where the projection area is located, measured by the ToF laser distance measuring sensor, will also vary. Optionally, in the process of distance measurement by the ToF laser ranging sensor, because the placement position of the projection device is inclined, the vertical distance between the projection light of the projection device and the plane of the projection area can be taken as The distance between the exit plane of the projection light of the projection device and the plane where the projection area is located. Then for the projection mode shown in Figure 5, the distance from the vertical line of the projection plane of the projection light of the projection device to the plane where the projection area is located and the ToF laser ranging sensor (correspondingly, the number of ToF laser ranging sensors in this case The position coordinates of 3 or more and not on the same straight line) are used to obtain the equation of the plane where the projection area is located, that is, the projection plane equation, so as to facilitate the subsequent processing of the shadow image in the projection mode as shown in Figure 5. Distortion is automatically calibrated.

Step S230: Determine an exit plane corresponding to the projection device based on the position coordinates.

As a way, since 3 points can determine a plane, the exit plane corresponding to the projection device can be determined based on the position coordinates of the ToF laser ranging sensor, that is, the exit plane of the projection light of the projection device can be determined. For example, optionally, if the position coordinates of the ToF laser ranging sensor are obtained as (x1, y1, 0), (x2, y2, 0), (x3, y3, 0), then it can be determined that they correspond to the projection device The equation of the exit plane is ax+by=0. Optionally, if the position coordinates of the ToF laser ranging sensor are obtained as (x1, y1, 1), (x2, y2, 1), (x3, y3, 1), then the distance can be determined to install the ToF laser ranging The plane of the sensor is the plane where Z=1 is the exit plane of the projection light of the projection device, and the equation of the exit plane can be expressed as ax+by+1=0.

Step S240: Obtain projection plane information based on the exit plane and the spatial depth information.

As a way, the equation of the plane where the projection area is located (ie, the projection plane equation) can be calculated according to the distance from the exit plane of the projection light of the multiple projection devices to the plane where the projection area is located and the position coordinates of the multiple ToF laser ranging sensors.

For example, in a specific application scenario, assuming that the distances from the exit plane of the projection light of the projection device obtained by the ToF laser ranging sensor to the plane of the projection area are d1, d2, d3, ... dn, then you can determine In the same coordinate system, the z coordinates of the projection points on the plane of the projection area are d1, d2, d3,...dn, then the coordinates of the above projection points can be obtained as (x1, y1, z1), (x2, y2, z2), (x3, y3, z3), ... (xn, yn, zn).

Optionally, in order to facilitate obtaining a more accurate projection plane equation of the projection area, the least square method may be used in this embodiment to fit the projection plane equation of the plane where the projection area is located. Specifically, the equation of the projection area can be represented by Ax+By+Cz+1=0, then the coordinates of all projection points (x1, y1, z1), (x2, y2, z2), (x3, y3, z3),……(xn, yn, zn) into the equation can get:

可以得到： Multiply both sides:

You can get:

Further, after simplification, we can get:

Simplify to:

Then you can get:

Among them, the coefficients A, B, and C are the obtained fitting plane parameters. Therefore, the projection plane equation of the plane where the projection area is located in the coordinate system of the projection device can be obtained as:

Ax+By+Cz+1=0,

The normal vector can be determined as N(A, B, C).

Step S250: Obtain a rotation angle of the projection plane relative to the exit plane based on the exit plane information and the projection plane information.

Optionally, as can be seen from the foregoing description, the installation position of the ToF laser ranging sensor is known, and the plane on which the ToF laser ranging sensor is installed can be taken as the O-XYZ plane, where Z=0, the projection device faces the projection The direction of is taken as the positive direction of the Z axis, and its normal vector can be determined as n(0, 0, 1). In this case, the rotation angle of the projection plane relative to the exit plane can be obtained based on the exit plane equation and the projection plane equation, so that the rotation matrix of the projection plane relative to the exit plane can be obtained according to the rotation angle, and then the correction can be achieved. Automatic calibration of the projected image.

Step S260: Calculate the rotation matrix of the projection plane relative to the exit plane based on the rotation angle.

As shown in FIG. 6, as a manner, step S260 may include:

Step S261: Obtain the normal vector of the projection plane.

As a way, the normal vector of the projection plane is the normal vector of the plane where the projection area is located. Optionally, the rotation matrix between the two planes can be obtained by obtaining the rotation matrix between the normal vectors of the two planes, and thus the normal vector of the projection plane can be obtained. As an implementation manner, taking the above example as an example, the normal vector of the projection plane may be N(A, B, C).

Step S262: Obtain the normal vector of the exit plane.

Correspondingly, in combination with the above example, the normal vector of the exit plane may be n(0, 0, 1).

Step S263: Obtain the rotation angle of the normal vector of the exit plane transformed to the normal vector of the projection plane.

As shown in FIG. 7, as a manner, step S263 may include:

Step S2631: Obtain the angle between the projection vector of the normal vector of the projection plane on the two-dimensional plane and the positive direction of the corresponding coordinate axis.

Among them, the two-dimensional plane refers to the surface projected by the normal vector N (A, B, C), and the corresponding coordinate axis can be understood as the starting coordinate axis of the surface projected by N (A, B, C), for example, suppose The plane projected by the normal vector N (A, B, C) is the XOY plane, then the X axis can be used as the corresponding coordinate axis of the projection vector of the normal vector N (A, B, C) on the two-dimensional plane (XOY plane) . Then, it can be understood that the positive direction of the corresponding coordinate axis can be understood as the positive direction of the initial coordinate axis. For example, the positive direction of the X axis can be taken as the positive direction of the corresponding coordinate axis.

For example, in a specific application scenario, it is assumed that the normal vector N(A, B, C) is rotated by θx around the X axis, θy around the Y axis, and θz around the Z axis. Then, according to the normal vector N (A, B, C), the rotation angle around the three axes (ie X axis, Y axis, Z axis) can be calculated. Optionally, the rotation angle θz around the Z axis can be defined as the angle between the projection vector of the normal vector N(A, B, C) on the XOY plane and the positive direction of the X axis, and the rotation angle θy around the Y axis is the normal vector The angle between the projection vector of N(A, B, C) on the ZOX plane and the positive direction of the Z axis, and the rotation angle θx around the X axis is the normal vector N(A, B, C) on the YOZ plane. The included angle in the positive direction of the Y axis.

Step S2632: Transform the included angles corresponding to different coordinate axes based on the three-dimensional coordinate axes as the normal vector of the exit plane to the rotation angle of the normal vector of the projection plane.

Optionally, when the angles between the projection vectors of the normal vector N(A, B, C) on the different coordinate axes of the three-dimensional coordinate axis and the different coordinate axes are obtained, the different coordinate axes based on the three-dimensional coordinate axis can be The corresponding included angle is taken as the rotation angle of the normal vector of the exit plane transformed to the normal vector of the projection plane.

Specifically, as shown in Figure 8, if θz is the angle between the projection vector of the normal vector N(A, B, C) on the XOY plane and the positive direction of the X axis, then the normal vector N(A, B, C) is The projection vector on the XOY plane can be determined as Nz(A, B, 0). Optionally, the X-axis positive direction vector can be Vx(1, 0, 0), then:

Similarly, if θy is the angle between the projection vector of the normal vector N(A, B, C) on the ZOX plane and the positive direction of the Z axis, then the projection vector of the normal vector N(A, B, C) on the ZOX plane It can be determined as Ny(A,0,C) (as shown in Figure 8). Optionally, the positive direction vector of the Z axis can be Vz(0,0,1), then:

Similarly, if θx is the angle between the projection vector of the normal vector N (A, B, C) on the YOZ plane and the positive direction of the Y axis, then the projection vector of the normal vector N (A, B, C) on the YOZ plane It can be determined as Nx(0, B, C) (as shown in Figure 8). Optionally, the positive direction vector of the Y-axis can be Vy(0, 1, 0), then:

That is, the included angles θx, θy, and θz can be transformed into the rotation angle of the normal vector of the projection plane as the normal vector of the exit plane.

Step S264: Calculate the rotation matrix between the normal vector of the exit plane and the normal vector of the projection plane based on the rotation angle.

Optionally, as shown in Figure 8, the normal vector of the projection plane of the projection area is N(A, B, C). If its projection vector on the YOZ plane is Nx, then according to the angle between Nx and the positive direction of the Y axis It can be obtained as θx. Optionally, the normal vector N(A, B, C) can be rotated around the X axis by α=90°-θx, so that the normal vector N(A, B, C) can be rotated to the ZOX plane to obtain Nz, where α The angle is the deflection angle of the projection device in the vertical direction.

Correspondingly, if the projection vector of the normal vector N(A, B, C) on the ZOX plane is Nz, then the angle between Nz and the positive direction of the Z axis can be obtained as θy. Optionally, this angle can be regarded as the angle of the normal vector N (A, B, C) rotating around the Y axis. Through this rotation, the vector Nz can be rotated to coincide with the positive direction of the Z axis. The deflection angle in the horizontal direction. Similarly, the normal vector N(A, B, C) can be rotated around the X axis by α=90°-θx to adjust the deflection angle in the vertical direction. By rotating the normal vector N(A, B, C) around the Y axis by θy, the deflection angle in the horizontal direction can be adjusted to coincide with the normal vector n direction (positive direction of the Z axis) of the projection light projection plane of the projection device.

Specifically, in a specific application scenario, the rotation matrix between the normal vector of the exit plane and the normal vector of the projection plane can be expressed as:

Step S265: Use the rotation matrix as a rotation matrix of the projection plane relative to the exit plane.

It is understandable that the rotation matrix between the normal vector of the plane where the projection area is located and the normal vector of the exit plane of the projection light of the projection device obtained above can be taken as the difference between the plane where the projection area is located and the exit plane of the projection light of the projection device. Rotation matrix between in order to realize the distortion calibration of the projected image.

Step S270: Perform perspective transformation on the projection image of the projection image to be calibrated according to the rotation matrix to adjust the projection image to a target shape.

Through the above-mentioned rotation matrix, the image of the projection screen in the projection area can be coordinate transformed. Specifically, all pixels of the projection screen can be perspective transformed to realize automatic distortion calibration of the projection device to adjust the projection screen to a square Rectangle, or other arbitrary shapes, it should be noted that the embodiment itself does not limit the shape of the projection screen or the shape that may exist in the future.

Step S280: Perform bilinear interpolation processing on the projection image after perspective transformation.

As a way, in order to present a better display effect of the projection screen, post-processing operations such as bilinear interpolation can be performed on the distortion-calibrated projection screen (ie, the projection image) to optimize the projection effect of the projection device.

The projection image calibration method provided by the present application obtains the normal vector of the projection plane and the normal vector of the exit plane, and then obtains the rotation angle of the normal vector of the exit plane transformed to the normal vector of the projection plane, and then calculates the exit plane based on the rotation angle According to the rotation matrix between the normal vector of and the normal vector of the projection plane, the projection screen of the projected image to be calibrated is subjected to perspective transformation according to the rotation matrix, and then bilinear interpolation is performed on the projection screen after perspective transformation. It is realized that the rotation matrix of the projection plane relative to the emission plane is obtained based on the emission plane information of the emission plane where the distance measuring sensor is located and the projection plane information. Then, when the rotation matrix is obtained, the projection screen of the projection image to be calibrated is performed according to the rotation matrix. Perspective transformation enables rapid automatic correction of projected images and enhances user experience.

Please refer to FIG. 9, a projection image calibration device 300 provided by an embodiment of the present application runs on a projection device, and the device 300 includes:

The data acquisition unit 310 is used to acquire the spatial depth information of the projection image to be calibrated collected by three or more range-finding sensors located on the exit plane, where the spatial depth information is the projection plane to the projection image formed by the range-finding sensor The distance information of the plane.

The data processing unit 320 is configured to determine projection plane information based on the spatial depth information and the relative position relationship between the ranging sensor.

Specifically, as an implementation manner, the data processing unit 320 may be used to obtain the position coordinates of a preset number of range-finding sensors provided on the projection device; determine the exit plane corresponding to the projection device based on the position coordinates; and based on the exit plane and space The depth information acquires projection plane information. It should be noted that the preset number may include 3 or more, and the preset number of ranging sensors are not on the same straight line.

As a way, the data processing unit 320 is further configured to obtain a rotation matrix of the projection plane relative to the exit plane based on the exit plane information and the projection plane information.

In an implementation manner, the data processing unit 320 may be specifically configured to obtain the rotation angle of the projection plane relative to the emission plane based on the emission plane information and the projection plane information, and then calculate the rotation matrix of the projection plane relative to the emission plane based on the rotation angle. .

Wherein, as an embodiment, the step of calculating the rotation matrix of the projection plane relative to the exit plane based on the rotation angle may include: obtaining the normal vector of the projection plane, obtaining the normal vector of the exit plane; obtaining the normal vector of the exit plane and transforming it to the projection plane The rotation angle of the normal vector of, calculate the rotation matrix between the normal vector of the exit plane and the normal vector of the projection plane based on the rotation angle; take the rotation matrix as the rotation matrix of the projection plane relative to the exit plane.

Optionally, as a way, the step of obtaining the rotation angle of the normal vector of the exit plane transformed to the normal vector of the projection plane may include: obtaining the projection vector of the normal vector of the exit plane on the two-dimensional plane and the corresponding coordinate axis. The included angle in the positive direction; the included angle corresponding to the different coordinate axes based on the three-dimensional coordinate axis is used as the normal vector of the exit plane to transform the rotation angle of the normal vector of the projection plane.

The projection unit 330 is configured to perform perspective transformation on the projection image of the projection image to be calibrated according to the rotation matrix, so as to adjust the projection image to a target shape.

As a way, the projection unit 330 may be used to perform perspective transformation on the pixels of the projection screen of the projection image to be calibrated according to the rotation matrix.

Optionally, as a manner, the device 300 may further include a post-processing unit configured to perform bilinear interpolation processing on the projection image after the perspective transformation, so as to optimize the projection display effect.

It should be noted that the device embodiment in this application and the foregoing method embodiment correspond to each other. For specific principles in the device embodiment, please refer to the content in the foregoing method embodiment, which will not be repeated here.

Hereinafter, a projection device provided by the present application will be described with reference to FIG. 10.

Please refer to FIG. 10, based on the above-mentioned projection image calibration method, system, and device, an embodiment of the present application also provides another projection device 100 that can execute the above-mentioned projection image calibration method. The projection device 100 includes one or more (only one shown in the figure) processor 102, a memory 104, a data acquisition module 11, and a projection module 12 coupled to each other. Wherein, the memory 104 stores a program that can execute the content in the foregoing embodiment, and the processor 102 can execute the program stored in the memory 104, and the memory 104 includes the apparatus 300 described in the foregoing embodiment.

The processor 102 may include one or more processing cores. The processor 102 uses various interfaces and lines to connect various parts of the entire projection device 100, and executes by running or executing instructions, programs, code sets, or instruction sets stored in the memory 104, and calling data stored in the memory 104. Various functions and processing data of the projection device 100. Optionally, the processor 102 may use at least one of digital signal processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). A kind of hardware form to realize. The processor 102 may be integrated with one or a combination of a central processing unit (CPU), a video image processor (Graphics Processing Unit, GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is used for rendering and drawing of display content; the modem is used for processing wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 102, but may be implemented by a communication chip alone.

The memory 104 may include random access memory (RAM) or read-only memory (Read-Only Memory). The memory 104 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 104 may include a storage program area and a storage data area. The storage program area may store instructions for implementing the operating system and instructions for implementing at least one function (such as touch function, sound playback function, video image playback function, etc.) ), instructions for implementing the foregoing method embodiments, etc. The data storage area can also store data (for example, audio and video data) created by the projection device 100 during use.

The data collection module 11 is used to obtain the spatial depth information of the projection image to be calibrated. Optionally, the spatial depth information may include the distance between the projection area for displaying the projection image to be calibrated and the projection device.

The projection module 12 can be used to perform perspective transformation on the projection image of the projection image to be calibrated according to the rotation matrix, so as to adjust the projection image to a target shape, for example, to a square rectangle.

Please refer to FIG. 11, which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. The computer-readable medium 400 stores program code, and the program code can be invoked by a processor to execute the method described in the foregoing method embodiment.

The computer-readable storage medium 400 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 400 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 400 has storage space for the program code 410 for executing any method steps in the above-mentioned methods. These program codes can be read out from or written into one or more computer program products. The program code 410 may be compressed in a suitable form, for example.

In summary, the projection image calibration method, device, projection equipment, and storage medium provided by the present application acquire the spatial depth information of the projection image to be calibrated collected by three or more range-finding sensors located on the exit plane , The spatial depth information is the distance information from the exit plane to the projection plane formed by the ranging sensor; then the projection plane information is determined based on the spatial depth information and the relative position relationship between the ranging sensors; and then the projection is obtained based on the exit plane information and the projection plane information The rotation matrix of the plane relative to the exit plane; then, according to the rotation matrix, a perspective transformation is performed on the projection image of the projection image to be calibrated to adjust the projection image to the target shape. In this way, the rotation matrix of the projection plane relative to the projection plane is obtained based on the exit plane information and the projection plane information of the exit plane where the ranging sensor is located, and then when the rotation matrix is acquired, the projection image is to be calibrated according to the rotation matrix. The projection image of the camera is transformed into perspective, so that the automatic correction of the projected image can be completed quickly and the user experience is enhanced.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application, not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the foregoing embodiments are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims (11)

A projection image calibration method, characterized in that the method includes: Acquiring the spatial depth information of the projection image to be calibrated collected by three or more range-finding sensors located on the exit plane, where the spatial depth information is the distance information from the exit plane formed by the range-finding sensor to the projection plane; Determining projection plane information based on the spatial depth information and the relative position relationship between the ranging sensor; Acquiring a rotation matrix of the projection plane relative to the exit plane based on the exit plane information and the projection plane information; Perform perspective transformation on the projection image of the projection image to be calibrated according to the rotation matrix, so as to adjust the projection image to a target shape. The method according to claim 1, wherein the step of obtaining a rotation matrix of the projection plane relative to the exit plane based on the exit plane information and the projection plane information comprises: Acquiring a rotation angle of the projection plane relative to the exit plane based on the exit plane information and the projection plane information; A rotation matrix of the projection plane relative to the exit plane is calculated based on the rotation angle. The method according to claim 2, wherein the step of calculating a rotation matrix of the projection plane relative to the exit plane based on the rotation angle comprises: Obtaining the normal vector of the projection plane; Obtaining the normal vector of the exit plane; Obtaining the rotation angle of the normal vector of the exit plane transformed to the normal vector of the projection plane; Calculating a rotation matrix between the normal vector of the exit plane and the normal vector of the projection plane based on the rotation angle; The rotation matrix is taken as the rotation matrix of the projection plane relative to the exit plane. The method according to claim 3, wherein the step of obtaining the rotation angle of the normal vector of the exit plane transformed to the normal vector of the projection plane comprises: Acquiring the angle between the projection vector of the normal vector of the exit plane on the two-dimensional plane and the positive direction of the corresponding coordinate axis; The included angle corresponding to the different coordinate axes based on the three-dimensional coordinate axis is transformed into the rotation angle of the normal vector of the projection plane as the normal vector of the exit plane. The method according to claim 3, wherein the step of performing perspective transformation on the projection image of the projection image to be calibrated according to the rotation matrix comprises: Perform perspective transformation on the pixels of the projection screen of the projection image to be calibrated according to the rotation matrix. The method according to claim 1, wherein the step of determining projection plane information based on the relative position relationship between the spatial depth information and the distance measuring sensor comprises: Acquiring the position coordinates of the preset number of the distance measuring sensors set on the projection device; Determining an exit plane corresponding to the projection device based on the position coordinates; Obtain projection plane information based on the exit plane and the spatial depth information. The method according to claim 6, wherein the preset number includes 3 or more, and the preset number of ranging sensors are not on the same straight line. The method according to any one of claims 1-7, wherein the method further comprises: Perform bilinear interpolation processing on the projection image after the perspective transformation. A projection image calibration device, characterized in that it runs on a projection device, and the device includes: The data acquisition unit is used to acquire the spatial depth information of the projection image to be calibrated collected by three or more range-finding sensors located on the exit plane, where the spatial depth information is the exit plane to the projection plane formed by the range-finding sensor Distance information; A data processing unit, configured to determine projection plane information based on the spatial depth information and the relative position relationship between the ranging sensor; The data processing unit is further configured to obtain a rotation matrix of the projection plane relative to the exit plane based on the exit plane information and the projection plane information; The projection unit is configured to perform perspective transformation on the projection image of the projection image to be calibrated according to the rotation matrix to adjust the projection image to a target shape. A projection device, characterized in that it comprises a data acquisition module, a projection module, one or more processors and a memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method of any one of claims 1-8. A computer-readable storage medium, wherein a program code is stored in the computer-readable storage medium, wherein the method according to any one of claims 1-8 is executed when the program code is run by a processor.

Images