CN116599942A - Image processing method, device, server and storage medium for a virtual desktop - Google Patents

Abstract

The embodiment of the present application discloses an image processing method of a virtual desktop. The method includes: after receiving an operation instruction representing a change in the display size of the virtual desktop on the client, determining the current display size of the virtual desktop; The alignment step of the encoder and the current display size are used to expand the original image data of the virtual desktop to obtain the processed image data; the alignment step represents the number of bytes of data read by the hardware encoder each time; through the hardware encoder The processed image data is read, and the original image data in the processed image data is encoded to obtain desktop encoded data; and the desktop encoded data is sent to the client. The embodiment of the present application also discloses an image processing device, a server and a storage medium of a virtual desktop.

Description

Image processing method, device, server and storage medium for a virtual desktop

technical field

The present application relates to the field of communication technology, and in particular to a virtual desktop image processing method, device, server and storage medium.

Background technique

In the desktop cloud system, when the cloud desktops of the server and the client are transmitting data, they need to encode the picture to be displayed on the cloud desktop through the encoder in the Simple Protocol for Independent Computing Environments (SPICE) , to increase the transmission rate of the desktop cloud system. However, because the size of the client's cloud desktop can be scaled arbitrarily, this will cause the size of the encoded picture to change accordingly, and the encoder reads the information in the picture according to a fixed number of bytes; when the size of the cloud desktop changes Finally, the encoder cannot read the information in the screen according to the fixed number of bytes, but can only read word by word, which reduces the efficiency of the encoder to read information and increases the overall time consumption of encoding.

Contents of the invention

In order to solve the above technical problems, the embodiment of the present application expects to provide a virtual desktop image processing method, device, server and storage medium, which solves the problem in the related art that after the size of the cloud desktop changes, the encoder cannot The information in the screen read by the number of sections can only be read word by word, which reduces the efficiency of the encoder to read information and increases the overall time-consuming problem of encoding.

The technical scheme of the present application is realized like this:

An image processing method for a virtual desktop, the method comprising:

After receiving an operation instruction representing a change in the display size of the virtual desktop on the client, determine the current display size of the virtual desktop;

Based on the alignment step of the hardware encoder of the server and the current display size, the original image data of the virtual desktop is expanded to obtain processed image data; the alignment step represents the hardware encoding The number of bytes of data read by the device each time;

Reading the processed image data through the hardware encoder, and encoding the original image data in the processed image data to obtain desktop encoded data;

Send the desktop encoded data to the client.

In the above solution, the display size changes, including:

The virtual desktop changes from a full-screen state to a windowed state; or,

The virtual desktop changes from the first window state to the second window state; or,

The virtual desktop switches from the first display to the second display.

In the above solution, the determining the current display size of the virtual desktop includes:

determining the current display size according to the coordinates of the virtual desktop sent by the client and displayed on the display screen of the client and the display size of the virtual desktop before any change;

Correspondingly, based on the alignment step of the hardware encoder of the server and the current display size, the original image data of the virtual desktop is expanded to obtain processed image data, including:

Obtaining a virtual width and a virtual height of the processed image data based on the current display size and the alignment step;

The original image data is expanded according to the virtual width and the virtual height to obtain the processed image data.

In the above solution, the virtual width and virtual height of the processed image data are obtained based on the current display size and the alignment step size, including:

determining an alignment step size of the hardware encoder;

The actual width and the actual height of the virtual desktop are processed based on the alignment step to obtain the virtual height and the virtual height; the current display size includes the actual width and the actual height.

In the above solution, the actual width and actual height of the virtual desktop are processed based on the alignment step to obtain the virtual height and the virtual height, including:

determining a difference between the alignment step and the actual width and the actual height;

Expanding the actual width and the actual height according to the difference to obtain the imaginary width and the imaginary height; the imaginary width and the imaginary height are integer multiples of the alignment step.

In the above solution, the said original image data is expanded according to said virtual height and said virtual height to obtain processed image data, including:

expanding the original image data according to the virtual height and the virtual width to obtain expanded image data;

The expanded image data is filled according to the target character to obtain the processed image data.

In the above solution, reading the processed image data through the hardware encoder, and encoding the original image data in the processed image data to obtain desktop encoded data includes:

Read the processed image data according to the alignment step by the hardware encoder

The original image data is encoded by the hardware encoder to obtain the desktop encoded data.

An image processing device for a virtual desktop, the device comprising:

The processing unit is configured to determine the current display size of the virtual desktop after receiving an operation instruction representing a change in the display size of the virtual desktop on the client;

The processing unit is further configured to expand the original image data of the virtual desktop based on the alignment step of the hardware encoder of the server and the current display size, to obtain processed image data; The alignment step represents the number of bytes of data read by the hardware encoder each time;

The processing unit is further configured to read the processed image data through the hardware encoder, and encode the original image data in the processed image data to obtain desktop coded data;

A sending unit, configured to send the desktop encoded data to the client.

A server, the server includes: a processor, a memory and a communication bus;

The communication bus is used to realize the communication connection between the processor and the memory;

The processor is configured to execute the image processing program of the virtual desktop in the memory, so as to realize the steps of the above-mentioned method for processing the image of the virtual desktop.

A storage medium, the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors, so as to realize the above-mentioned steps of the image processing method for a virtual desktop.

The image processing method, device, server, and storage medium of the virtual desktop provided in the embodiments of the present application can determine the current display size of the virtual desktop after receiving an operation instruction representing a change in the display size of the virtual desktop on the client, Then, based on the alignment step size of the hardware encoder on the server side and the current display size, the original image data of the virtual desktop is expanded to obtain the processed image data, and then the processed image data is read by the hardware encoder, and processed The original image data in the processed image data is encoded to obtain the desktop encoded data, and the desktop encoded data is sent to the client; thus, since the processed image data is based on the alignment step of the hardware encoder, the original image data of the virtual desktop is It is obtained after expansion processing, so the data size of the processed image data matches the alignment step of the hardware encoder, so the processed image data can be read according to the alignment step through the hardware encoder, and there is no need to read it separately Bytes, the rate of reading data is high, and only the original image data in the processed image data is encoded during encoding, which will not increase redundant operations, thus reducing the time consumed by processing and improving data encoding. efficiency, thereby improving the overall performance of the virtual desktop, and solving the problem in related technologies that after the size of the cloud desktop changes, the encoder cannot read the information in the screen according to the fixed number of bytes, and can only read word by word, resulting in encoding The efficiency of reading information by the reader is reduced and the overall time-consuming problem of encoding is increased.

Description of drawings

FIG. 1 is a schematic flowchart of an image processing method for a virtual desktop provided in an embodiment of the present application;

FIG. 2(a) and FIG. 2(b) are schematic diagrams showing changes in the display size of the virtual desktop in an image processing method for a virtual desktop provided by an embodiment of the present application;

FIG. 3 is a schematic flowchart of another image processing method for a virtual desktop provided by an embodiment of the present application;

FIG. 4 is a schematic flowchart of another image processing method for a virtual desktop provided by an embodiment of the present application;

FIG. 5 is a schematic flowchart of a method for processing an image of a virtual desktop provided in another embodiment of the present application;

FIG. 6 is a schematic structural diagram of an image processing device for a virtual desktop provided by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a server provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

It should be understood that references to "an embodiment of the present application" or "the foregoing embodiment" throughout the specification mean that a specific feature, structure or characteristic related to the embodiment is included in at least one embodiment of the present application. Therefore, appearances of "in the embodiment of the present application" or "in the foregoing embodiment" throughout the specification do not necessarily refer to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the order of execution, and the execution order of each process should be determined by its functions and internal logic, rather than the implementation process of the embodiments of the present application. constitute any limitation. The serial numbers of the above embodiments of the present application are for description only, and do not represent the advantages and disadvantages of the embodiments.

Unless otherwise specified, the electronic device executes any step in the embodiments of the present application, and may be a processor of the electronic device executes the step. It should also be noted that the embodiment of the present application does not limit the order in which the electronic device executes the following steps. In addition, the methods for processing data in different embodiments may be the same method or different methods. It should also be noted that any step in the embodiments of the present application can be executed independently by the electronic device, that is, when the electronic device executes any step in the following embodiments, it may not depend on the execution of other steps.

It should be understood that the specific embodiments described here are only used to explain the present application, not to limit the present application.

The embodiment of the present application provides an image processing method of a virtual desktop, which can be applied to a server, as shown in FIG. 1 , the method includes the following steps:

Step 101: After receiving an operation instruction representing a change in the display size of the virtual desktop on the client, determine the current display size of the virtual desktop.

Wherein, the virtual desktop is a desktop that is virtualized on the server and can be accessed through the client.

In the embodiment of the present application, the virtual desktop may be a desktop that is virtualized on the server and can be accessed through the client, and the virtual desktop is also called a cloud desktop. The operation instruction is used to indicate that the display size of the virtual desktop on the client changes. The current display size of the virtual desktop may be the changed display size of the virtual desktop. In a feasible implementation, when the virtual desktop changes from a full-screen state to a window state, determine the display of the virtual desktop on the client If the size changes, the current display size of the virtual desktop is the display size in the window state, where the window state means that the virtual desktop is in a non-full-screen state; when the virtual desktop changes from the first window state to the second window state, In this case, it is determined that the display size of the virtual desktop on the client has changed. At this time, the current display size of the virtual desktop is the display size in the second window state, wherein the display size in the first window state is the same as that in the second window state. different display sizes, that is, the display size in the first window state may be larger or smaller than the display size in the second window state; when the virtual desktop is switched from the first display screen to the In the case of the second display screen, it is determined that the display size of the virtual desktop changes on the client. At this time, the current display size of the virtual desktop is the display size switched to the second display screen, wherein the screen size of the first display screen is the same as that of the second display screen. Displays vary in screen size. Exemplarily, as shown in Figure 2(a) and Figure 2(b), the virtual desktop is in the first window state in Figure 2(a), and the virtual desktop is in the second window state in Figure 2(b), obviously, the The display size in the first window state shown in Figure 2(a) is different from the display size in the second window state shown in Figure 2(b), if the virtual desktop starts from the first window state shown in Figure 2(a) Change to the second window state shown in FIG. 2( b ), then it is determined that the display size of the virtual desktop on the client changes.

In an achievable manner, an operation command representing a change in the display size of the virtual desktop can be sent to the server through external devices such as the mouse and keyboard of the client where the virtual desktop is located, and then the server receives the operation command sent by the client. After the instruction, the current display size of the virtual desktop is determined based on the operation instruction.

Step 102 , based on the alignment step of the hardware encoder on the server side and the current display size, expand the original image data of the virtual desktop to obtain processed image data.

Among them, the alignment step represents the number of bytes of data read by the hardware encoder each time.

In the embodiment of the present application, the original image data can be the image data displayed on the client after the changed virtual desktop, for example, the original image data can be the window when the virtual desktop is in the second window state in Figure 2(b) display content in . The processed image data is the image data after processing the original image data based on the alignment step of the hardware encoder and the current display size. At this time, the processed image data is the image data that matches the alignment step of the hardware encoder , that is, the hardware encoder can read the processed image data at one time; since the hardware encoder reads the image data according to the alignment step, in order to prevent the hardware encoder from reading the original image data at one time, it is necessary Based on the alignment step of the hardware encoder and the current display size, the original image data is expanded, so that the processed image data can be read by the hardware encoder at one time, so that there is no need to read the data word by word. The hardware encoder is used to process the original image data. Further, the hardware encoder is used to read the image data displayed on the client by the virtual desktop, and compress the image data and transmit it to the client. By compressing the image data , reduce the communication traffic of the cloud desktop system, and increase the data transmission rate; in a feasible implementation, the hardware encoder can be Advanced Video Coding (Advanced Video Coding, H. (High Efficiency Video Coding, H.265), of course, the hardware encoder may also be other encoders, which is not limited in this embodiment of the present application. The alignment step size (Alignment) of the hardware encoder represents the number of bytes of data read in sequence. The alignment step size can be set in advance, generally can be set to a multiple of 2, and can be set to 32 or other values according to actual business needs. This application The embodiment does not limit this; usually, after the alignment step of the hardware encoder is set, it will not change arbitrarily, that is, it is fixed.

In this embodiment of the application, the original image data is expanded based on the alignment step of the hardware encoder and the current display size to obtain the processed image data whose data size matches the alignment step, so that the display size of the virtual desktop In the case of a change, the changed image data (that is, the original image data) can also be read according to the alignment step of the hardware encoder, and there will be no bytes that need to be read separately, so as to improve the data reading rate , and then increase the encoding rate, thereby improving the overall performance of the cloud desktop system.

In a feasible implementation, the alignment step size of the hardware encoder can be determined, and then the original image data is expanded according to the alignment step size and the current display size, so as to obtain a processed image whose data size matches the alignment step size. image data.

Step 103, read the processed image data through the hardware encoder, and encode the original image data in the processed image data to obtain desktop coded data.

In the embodiment of the present application, the desktop encoded data is the data after encoding the original image data in the processed image data, that is, when encoding with a hardware encoder, the processed image data is not encoded, but It is to encode only the original image data in the processed image data. Read the processed image data through the hardware encoder, and encode the original image data in the processed image data; in fact, although the processed image data is read, only the original image data is encoded Encoding, so that since the data size of the processed image data matches the alignment step of the hardware encoder, the processed image data can be read according to the alignment step through the hardware encoder, and there will be no need to read it separately Bytes, the rate of reading data is high; during encoding, the original image data is normally encoded, and the rest of the data except the original image data is not encoded, so the operation of encoding the original image data is guaranteed, and the obtained The desktop encoding data is only the information after encoding the original image data, and no redundant operations will affect the desktop cloud system, thereby improving the data encoding efficiency and improving the overall performance of the cloud desktop system.

In a practicable manner, the processed image data may be read by a hardware encoder according to the alignment step, and the original image data in the processed image data may be encoded to obtain desktop encoded data.

In other embodiments of the present application, an encoding acceleration card can be set in the server, and the encoding acceleration card can be a chip specially used for encoding data, so that the encoding acceleration card is executed on the encoding acceleration card through the hardware encoder to read The step of processing the processed image data and encoding the original image data in the processed image data to obtain desktop coded data can further improve the coding efficiency and reduce the resource occupancy rate of the cloud desktop system.

Step 104, sending the desktop encoded data to the client.

In this embodiment of the application, after the desktop encoding data is sent to the client, the client can generate original image data on the target desktop based on the desktop encoding data; After receiving the data, the coded desktop data is decoded to obtain the original image data, and then the original image data is displayed on the virtual desktop of the client.

In the embodiment of this application, when the display size of the virtual desktop changes in the related art, the display content corresponding to the changed virtual desktop does not match the step size of the target encoder, and there is a need to read it separately bytes, the rate of reading information is low; and the image processing method of the virtual desktop in the embodiment of the present application reads the processed image data through the hardware encoder is the data that matches the alignment step of the hardware encoder, Therefore, the processed image data can be read through the alignment step, and there are no bytes that need to be read separately. The rate of reading data is higher than the rate of reading data in the related art, and only the processed image is processed. The original image data in the data is processed, so redundant operations will not affect the desktop cloud system, reduce the time consumed in processing, improve the information coding efficiency, and then improve the overall performance of the cloud desktop system.

In the image processing method of the virtual desktop provided by the embodiment of the present application, since the processed image data is obtained by expanding the original image data of the virtual desktop based on the alignment step of the hardware encoder, the data of the processed image data The size matches the alignment step of the hardware encoder, so the processed image data can be read by the hardware encoder according to the alignment step, there are no bytes that need to be read separately, and the rate of reading data is high. And when encoding, only the original image data in the processed image data is encoded, and no redundant operations are added, so the time consumed for processing is reduced, the data encoding efficiency is improved, and the overall performance of the virtual desktop is improved. In the related technology, after the size of the cloud desktop changes, the encoder cannot read the information in the screen according to the fixed number of bytes, but can only read word by word, which reduces the efficiency of the encoder to read information and increases the overall coding efficiency. time consuming problem.

Based on the foregoing embodiments, an embodiment of the present application provides an image processing method for a virtual desktop, as shown in FIG. 3 , the method includes the following steps:

Step 201, after receiving the operation command representing the change of the display size of the virtual desktop on the client, the server sends the coordinates of the virtual desktop on the display screen of the client and the display size of the virtual desktop before the change, Determines the current display size.

Wherein, the display size changes, including: the virtual desktop changes from the full screen state to the window state; or, the virtual desktop changes from the first window state to the second window state; or, the virtual desktop switches from the first display screen to the second display screen .

In the embodiment of the present application, the operation command may carry the coordinates displayed on the display screen of the virtual desktop on the client, so that the coordinates displayed on the client can be determined, and then according to the display coordinates and the virtual desktop before the change Display size, to determine the current display size. The current display size is the display size after the virtual desktop changes. For example, the display size before no change can be the display size of the virtual desktop shown in Figure 2(a) in the first window state, and the display coordinates are the virtual desktop shown in Figure 2(b) in the second window state The coordinates below, the current display size is the display size of the virtual desktop shown in Figure 2(b) in the second window state.

In an achievable manner, the display size of the virtual desktop before the change can be obtained from the server's own storage, and then the change of the virtual desktop can be determined according to the display coordinates carried in the operation instruction and the display size before the change situation, and then determine the display size of the virtual desktop after the change (ie, the current display size).

Step 202, the server obtains the virtual width and virtual height of the processed image data based on the current display size and alignment step.

In the embodiment of the present application, the virtual width and virtual height of the processed image data are obtained after processing the current display size based on the alignment step. In a practicable manner, the current display size may be expanded based on the alignment step to obtain the virtual width and virtual height of the processed image data.

Wherein, step 202 can be realized through the following steps:

Step 202a, the server determines the alignment step of the hardware encoder.

Among them, the alignment step represents the number of bytes of data read by the hardware encoder each time.

In the embodiment of this application, the step size of the hardware encoder can be set in advance, so that the server can perform subsequent processing based on the alignment step size of the hardware encoder.

Step 202b, the server processes the actual width and height of the virtual desktop based on the alignment step to obtain the virtual width and virtual height.

Wherein, the current display size includes actual width and actual height.

In the embodiment of this application, since the hardware encoder reads data according to the alignment step, it is necessary to adjust the display size of the original image data to the display size that matches the alignment step, so as to facilitate subsequent image reading data. The actual width and actual height in the current display size can be the width and height of the virtual desktop shown in Figure 2(b) in the second window state, and the virtual width and virtual height are obtained after processing the actual width and actual height Width and height to match the alignment step.

In a practicable manner, the actual width and the actual height may be respectively mapped based on the alignment step to obtain virtual width and virtual height matching the alignment step.

In a feasible way, if the alignment step is 32, the actual width is 63, and the actual height is 31, then the virtual height obtained by mapping the actual height based on the alignment step is 32, and the actual width is mapped based on the alignment step The virtual width obtained by mapping is 64; that is, the adjusted virtual width and virtual height based on the alignment step must be divisible by 32, and at this time, both the virtual height and virtual height match the alignment step.

Step 203, the server performs expansion processing on the original image data according to the virtual width and virtual height, and obtains the processed image data.

In the embodiment of this application, the purpose of expanding the original image data according to the virtual width and virtual height is to match the processed image data with the alignment step, so that when the data is read by the hardware encoder, it can be read at one time. , there will be no bytes that need to be read separately to improve the efficiency of data reading.

In this embodiment of the application, the display size corresponding to the actual width and height (that is, the current display size) can be called the first display size, and the virtual width and virtual height can be called the second display size, because the second The display size is the display size after expanding the first display size based on the alignment step, so the second display size is larger than the first display size. At this time, the original image data whose display size is the first display size needs to be expanded to display The processed image data whose size is the second display size; since the second display size matches the alignment step, the information size of the obtained processed image data matches the alignment step.

In a practicable manner, content in the original image data may be filled according to the virtual width and virtual height to obtain processed image data.

Step 204, the server reads the processed image data through the hardware encoder according to the alignment step.

In the embodiment of the present application, since the processed image data matches the alignment step of the hardware encoder, the processed image data can be read through the hardware encoder according to the alignment step, and there is no need to read it separately. bytes, increasing the rate at which data is read.

Step 205, the server encodes the original image data through the hardware encoder to obtain desktop encoded data.

In the embodiment of this application, only the original image data in the processed image data can be encoded by the hardware encoder to realize the operation of encoding the original image data, without adding the operation of encoding the filling content, so there will be no The generation of redundant operations will affect the desktop cloud system.

Step 206, the server sends the desktop coded data to the client.

In this embodiment of the application, the first display size is adjusted based on the alignment step of the hardware encoder to obtain a second display size that matches the step, and then the first display information is expanded based on the second display size to obtain the data size The processed image data that matches the alignment step. In this way, when encoding, the processed image data is read by the hardware encoder according to the alignment step, which can improve the data reading rate. Only the processed image data is processed by the hardware encoder. The original image data in the image data is encoded, and the operation of encoding the supplementary content is not increased, so redundant operations will not affect the desktop cloud system, thereby improving the data encoding efficiency and improving the overall performance of the cloud desktop system.

It should be noted that, for descriptions of the same steps and content in this embodiment as in other embodiments, reference may be made to the descriptions in other embodiments, and details are not repeated here.

In the image processing method of the virtual desktop provided by the embodiment of the present application, since the processed image data is obtained by expanding the original image data of the virtual desktop based on the alignment step of the hardware encoder, the data of the processed image data The size matches the alignment step of the hardware encoder, so the processed image data can be read by the hardware encoder according to the alignment step, there are no bytes that need to be read separately, and the rate of reading data is high. And when encoding, only the original image data in the processed image data is encoded, and no redundant operations are added, so the time consumed for processing is reduced, the data encoding efficiency is improved, and the overall performance of the virtual desktop is improved. In the related technology, after the size of the cloud desktop changes, the encoder cannot read the information in the screen according to the fixed number of bytes, but can only read word by word, which reduces the efficiency of the encoder to read information and increases the overall coding efficiency. time consuming problem.

Based on the foregoing embodiments, an embodiment of the present application provides an image processing method for a virtual desktop, as shown in FIG. 4 , the method includes the following steps:

Step 301. After receiving the operation command representing the change of the display size of the virtual desktop on the client, the server sends the virtual desktop according to the coordinates of the virtual desktop displayed on the client's display screen and the display size of the virtual desktop before the change. Determines the current display size.

Wherein, the display size changes, including: the virtual desktop changes from the full screen state to the window state; or, the virtual desktop changes from the first window state to the second window state; or, the virtual desktop switches from the first display screen to the second display screen .

In the embodiment of this application, the server and the client can communicate through the target transmission protocol, the target transmission protocol includes but not limited to SPICE protocol, Remote Desktop Protocol (Remote Desktop Protocol, RDP), personal computer network interconnection protocol ( Personal Computer over Internet Protocol, PCoIP), Independent Computing Architecture (Independent Computing Architecture, ICA) protocol, High Definition Experience (High Definition Experience, HDX) protocol, Remote Frame Buffer (Remote Frame Buffer, RFB) protocol.

In a practicable manner, the display coordinates carried in the operation instruction can be obtained through the object transfer protocol (SPICE protocol), and then the current display size can be obtained based on the display coordinates and the display size before the change.

Step 302, the server determines the alignment step of the hardware encoder.

Among them, the step size represents the number of bytes of data read by the target encoder each time.

Step 303, the server determines the difference between the alignment step and the actual width and height.

Wherein, the current display size includes actual width and actual height.

In the embodiment of the present application, the second display size (that is, virtual width and virtual height) matches the alignment step, that is, the second display size can be divisible by the alignment step, that is, the second display size and the alignment step The relationship between is a multiple relationship, so the difference between the alignment step and the first display size (that is, the actual width and actual height) can be determined based on the multiple relationship; in a feasible way, if the alignment step is 32 , the actual width of the virtual desktop is 63, and the actual height in the virtual current display ruler is 31, then the difference between the alignment step and the actual width and height is 1.

Step 304, the server expands the actual width and height according to the difference to obtain the virtual width and height.

In this embodiment of the application, the actual width of the first display size and the corresponding difference can be calculated, and the actual height of the first display size and the corresponding difference can be calculated to obtain the virtual height at the second display size and imaginary width; in an achievable manner, the actual width and the corresponding difference can be added, and the actual height and the corresponding difference can be added to obtain the imaginary width and the imaginary height (that is, the second display size).

Step 305, the server expands the original image data according to the virtual width and virtual height, and obtains the expanded image data.

In the embodiment of the present application, the original image data is expanded based on the virtual width and virtual height, that is, the display size of the original image data is expanded to a second display size (ie virtual width and virtual height).

In step 306, the server fills the expanded image data based on the target character to obtain processed image data.

In this embodiment of the present application, the target character may be a character that does not affect the display content of the first display information. In this way, content filled in the expanded first display information based on the target character will not affect subsequent encoding. In a feasible manner, the target character may be 0. In this case, the expanded part in the expanded image data is all filled with 0 to obtain the processed image data.

In the embodiment of this application, since the processed image data is stored in the memory of the server, and the data is stored in multiples of the alignment step, the processed image data is also stored in memory alignment , there will be no free memory.

Step 307, the server reads the processed image data through the hardware encoder according to the alignment step.

In the embodiment of this application, the hardware encoder can be called through the target transmission protocol to read the processed image data according to the alignment step, that is, the H265 encoder can be called through the SPICE protocol to read the processed image data according to the alignment step .

Step 308, the server encodes the original image data through the hardware encoder to obtain desktop encoded data.

In the embodiment of the present application, the hardware encoder is invoked through the target transmission protocol to read the processed image data to the encoding accelerator card according to the alignment step, so that the original image data in the processed image data can be executed on the encoding accelerator card. Perform coding operations to obtain desktop coded data.

Step 309, the server sends the desktop coded data to the client.

In the embodiment of the present application, the processed image data is obtained by padding the expanded image data based on the target characters. The filled content will not affect the original image data, so that only the processed image data can be processed by the hardware encoder. The original image data in the image is encoded, and the operation of filling content encoding is not added, so redundant operations will not affect the desktop cloud system, thereby improving the data encoding efficiency and improving the overall performance of the cloud desktop system.

As shown in Figure 5, the user can arbitrarily scale the desktop size of the cloud desktop through the client. At this time, the client will represent the operation command of the change in the size of the cloud desktop to the server; after receiving the operation command, the server can Obtain the actual width and height of the cloud desktop through the SPICE protocol, and then adjust the actual width and height of the cloud desktop according to the alignment step of the H265 encoder to obtain the aligned virtual height and virtual height, and then based on the virtual height and virtual height Expand the original image data to obtain the processed image data, read the processed image data according to the alignment step through the H265 encoder, and encode the original image data in the processed image data to obtain the desktop coded data, send The desktop coded data is sent to the client, so that the client displays the content corresponding to the original image data on the cloud desktop based on the desktop coded data.

It should be noted that, for descriptions of the same steps and content in this embodiment as in other embodiments, reference may be made to the descriptions in other embodiments, and details are not repeated here.

In the image processing method of the virtual desktop provided by the embodiment of the present application, since the processed image data is obtained by expanding the original image data of the virtual desktop based on the alignment step of the hardware encoder, the data of the processed image data The size matches the alignment step of the hardware encoder, so the processed image data can be read by the hardware encoder according to the alignment step, there are no bytes that need to be read separately, and the rate of reading data is high. And when encoding, only the original image data in the processed image data is encoded, and no redundant operations are added, so the time consumed for processing is reduced, the data encoding efficiency is improved, and the overall performance of the virtual desktop is improved. In the related technology, after the size of the cloud desktop changes, the encoder cannot read the information in the screen according to the fixed number of bytes, but can only read word by word, which reduces the efficiency of the encoder to read information and increases the overall coding efficiency. time consuming problem.

Based on the aforementioned embodiments, this embodiment of the present application provides an image processing device for a virtual desktop. The image processing device for a virtual desktop can be applied to the image processing method for a virtual desktop provided in the embodiments corresponding to FIGS. 1 and 3-4. Refer to As shown in Figure 6, the image processing device 4 of the virtual desktop can include:

The processing unit 41 is configured to determine the current display size of the virtual desktop after receiving an operation instruction representing a change in the display size of the virtual desktop on the client;

The processing unit 41 is also used to expand the original image data of the virtual desktop based on the alignment step of the hardware encoder on the server side and the current display size to obtain processed image data; the alignment step represents the hardware encoder each time The number of bytes of read data;

The processing unit 41 is further configured to read the processed image data through a hardware encoder, and encode the original image data in the processed image data to obtain desktop coded data;

The sending unit 42 is configured to send the desktop encoded data to the client.

In other embodiments of the application, the display size changes, including:

The virtual desktop changes from a full-screen state to a windowed state; or,

The virtual desktop changes from the first window state to the second window state; or,

The virtual desktop switches from the first display to the second display.

In other embodiments of the present application, the processing unit 41 is also configured to implement the following steps:

Determine the current display size according to the coordinates displayed on the client's display screen of the virtual desktop sent by the client and the display size of the virtual desktop before it changes;

Correspondingly, the processing unit 41 is also configured to implement the following steps:

Based on the current display size and alignment step, obtain the virtual width and virtual height of the processed image data;

The original image data is expanded according to the virtual width and virtual height to obtain the processed image data.

In other embodiments of the present application, the processing unit 41 is also configured to implement the following steps:

Determine the alignment step size of the hardware encoder;

Process the actual width and height of the virtual desktop based on the alignment step to obtain the virtual height and height; the current display size includes the actual width and height.

In other embodiments of the present application, the processing unit 41 is also configured to implement the following steps:

Determine the difference between the alignment step and the actual width and height;

The actual width and height are expanded according to the difference to obtain the virtual width and virtual height; the virtual width and virtual height are integer multiples of the alignment step.

In other embodiments of the present application, the processing unit 41 is also configured to implement the following steps:

Expand the original image data according to the virtual height and virtual width to obtain the expanded image data;

The expanded image data is filled according to the target character to obtain the processed image data.

In other embodiments of the present application, the processing unit 41 is also configured to implement the following steps:

Read the processed image data through the hardware encoder according to the alignment step;

The original image data is encoded by a hardware encoder to obtain desktop encoded data.

It should be noted that, for a specific description of the steps performed by each unit, reference may be made to the implementation process in the image processing method for a virtual desktop provided in the embodiments corresponding to FIGS. 1 and 3-4 , which will not be repeated here.

In the image processing device for the virtual desktop provided by the embodiment of the present application, since the processed image data is obtained by expanding the original image data of the virtual desktop based on the alignment step of the hardware encoder, the data of the processed image data The size matches the alignment step of the hardware encoder, so the processed image data can be read by the hardware encoder according to the alignment step, there are no bytes that need to be read separately, and the data is read at a high rate. And when encoding, only the original image data in the processed image data is encoded, and no redundant operations are added, so the time consumed for processing is reduced, the data encoding efficiency is improved, and the overall performance of the virtual desktop is improved. In the related technology, after the size of the cloud desktop changes, the encoder cannot read the information in the screen according to the fixed number of bytes, but can only read word by word, which reduces the efficiency of the encoder to read information and increases the overall coding efficiency. time consuming problem.

Based on the foregoing embodiments, the embodiments of the present application provide a server, which can be applied to the image processing method of the virtual desktop provided in the embodiments corresponding to Figures 1 and 3-4, as shown in Figure 7, the service Terminal 5 may include: processor 51, memory 52 and communication bus 53, wherein:

The communication bus 53 is used to realize the communication connection between the processor 51 and the memory 52;

The processor 51 is used to execute the image processing program of the virtual desktop in the memory 52, so as to realize the following steps:

After receiving an operation instruction representing a change in the display size of the virtual desktop on the client, determine the current display size of the virtual desktop;

Based on the alignment step of the server-side hardware encoder and the current display size, the original image data of the virtual desktop is expanded to obtain the processed image data; the alignment step represents the number of bytes of data read by the hardware encoder each time ;

The processed image data is read by a hardware encoder, and the original image data in the processed image data is encoded to obtain desktop encoded data;

Send desktop encoded data to client.

In other embodiments of the application, the display size changes, including:

The virtual desktop changes from a full-screen state to a windowed state; or,

The virtual desktop changes from the first window state to the second window state; or,

The virtual desktop switches from the first display to the second display.

In other embodiments of the present application, the processor 51 is used to execute the image processing program of the virtual desktop in the memory 52 to determine the current display size of the virtual desktop, so as to implement the following steps:

Determine the current display size according to the coordinates displayed on the client's display screen of the virtual desktop sent by the client and the display size of the virtual desktop before it changes;

Correspondingly, the processor 51 is used to execute the image processing program of the virtual desktop in the memory 52 based on the alignment step of the hardware encoder on the server side and the current display size, and expand the original image data of the virtual desktop to obtain the processed image data to achieve the following steps:

Based on the current display size and alignment step, obtain the virtual width and virtual height of the processed image data;

The original image data is expanded according to the virtual width and virtual height to obtain the processed image data.

In other embodiments of the present application, the processor 51 is used to execute the image processing program of the virtual desktop in the memory 52 based on the current display size and the alignment step to obtain the virtual width and virtual height of the processed image data, to achieve the following steps:

Determine the alignment step size of the hardware encoder;

Process the actual width and height of the virtual desktop based on the alignment step to obtain the virtual height and height; the current display size includes the actual width and height.

In other embodiments of the present application, the processor 51 is used to execute the image processing program of the virtual desktop in the memory 52 to process the actual width and actual height of the virtual desktop based on the alignment step to obtain the virtual height and virtual height, so as to Implement the following steps:

Determine the difference between the alignment step and the actual width and height;

The actual width and height are expanded according to the difference to obtain the virtual width and virtual height; the virtual width and virtual height are integer multiples of the alignment step.

In other embodiments of the present application, the processor 51 is used to execute the image processing program of the virtual desktop in the memory 52 to perform expansion processing on the original image data according to the virtual height and the virtual height to obtain the processed image data, so as to realize the following step:

Expand the original image data according to the virtual height and virtual width to obtain the expanded image data;

The expanded image data is filled according to the target character to obtain the processed image data.

In other embodiments of the present application, the processor 51 is used to execute the image processing program of the virtual desktop in the memory 52, read the processed image data through the hardware encoder, and process the original image data in the processed image data Perform encoding to obtain desktop encoding data to achieve the following steps:

Read the processed image data through the hardware encoder according to the alignment step;

The original image data is encoded by a hardware encoder to obtain desktop encoded data.

It should be noted that, for a specific description of the steps executed by the processor, reference may be made to the implementation process in the image processing method for a virtual desktop provided in the embodiments corresponding to FIGS. 1 and 3 to 4 , which will not be repeated here.

In the server provided by the embodiment of this application, since the processed image data is obtained by expanding the original image data of the virtual desktop based on the alignment step of the hardware encoder, the data size of the processed image data is the same as that of the hardware encoding The alignment step of the encoder is matched, so the processed image data can be read according to the alignment step by the hardware encoder, there is no byte that needs to be read separately, the rate of reading data is high, and only Coding the original image data in the processed image data will not increase redundant operations, thus reducing the time consumed by processing, improving the efficiency of data encoding, thereby improving the overall performance of the virtual desktop, and solving the problems in related technologies. After the size of the cloud desktop changes, the encoder cannot read the information in the screen according to the fixed number of bytes, but can only read word by word, which reduces the efficiency of the encoder to read information and increases the overall time-consuming problem of encoding .

Based on the foregoing embodiments, the embodiments of the present application provide a storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors, so as to realize the The steps of the image processing method of the virtual desktop provided in the embodiment of 4.

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowcharts and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

The above descriptions are only preferred embodiments of the present application, and are not intended to limit the protection scope of the present application.

Claims (10)

1. An image processing method of a virtual desktop, characterized in that, the method is applied to a server, including: After receiving an operation instruction representing a change in the display size of the virtual desktop on the client, determine the current display size of the virtual desktop; Based on the alignment step of the hardware encoder of the server and the current display size, the original image data of the virtual desktop is expanded to obtain processed image data; the alignment step represents the hardware encoding The number of bytes of data read by the device each time; Reading the processed image data through the hardware encoder, and encoding the original image data in the processed image data to obtain desktop encoded data; Send the desktop encoded data to the client. 2. The method according to claim 1, wherein the change in the display size comprises: The virtual desktop changes from a full-screen state to a windowed state; or, The virtual desktop changes from the first window state to the second window state; or, The virtual desktop switches from the first display to the second display. 3. The method according to claim 1, wherein the determining the current display size of the virtual desktop comprises: determining the current display size according to the coordinates of the virtual desktop sent by the client and displayed on the display screen of the client and the display size of the virtual desktop before any change; Correspondingly, based on the alignment step of the hardware encoder of the server and the current display size, the original image data of the virtual desktop is expanded to obtain processed image data, including: Obtaining a virtual width and a virtual height of the processed image data based on the current display size and the alignment step; The original image data is expanded according to the virtual width and the virtual height to obtain the processed image data. 4. The method according to claim 3, wherein the obtaining the virtual width and virtual height of the processed image data based on the current display size and the alignment step size comprises: determining an alignment step size of the hardware encoder; The actual width and the actual height of the virtual desktop are processed based on the alignment step to obtain the virtual height and the virtual height; the current display size includes the actual width and the actual height. 5. The method according to claim 4, wherein said processing the actual width and actual height of the virtual desktop based on the alignment step to obtain the virtual height and the virtual height comprises: determining a difference between the alignment step and the actual width and the actual height; Expanding the actual width and the actual height according to the difference to obtain the imaginary width and the imaginary height; the imaginary width and the imaginary height are integer multiples of the alignment step. 6. The method according to claim 3, characterized in that said expanding the original image data according to said virtual height and said virtual height to obtain processed image data comprises: expanding the original image data according to the virtual height and the virtual width to obtain expanded image data; The expanded image data is filled according to the target character to obtain the processed image data. 7. The method according to any one of claims 1-6, wherein the processed image data is read by the hardware encoder, and the original image data in the processed image data is Image data is encoded to obtain desktop encoded data, including: Reading the processed image data through the hardware encoder according to the alignment step; The original image data is encoded by the hardware encoder to obtain the desktop encoded data. 8. An image processing device for a virtual desktop, characterized in that the device comprises: The processing unit is configured to determine the current display size of the virtual desktop after receiving an operation instruction representing a change in the display size of the virtual desktop on the client; The processing unit is further configured to expand the original image data of the virtual desktop based on the alignment step of the hardware encoder of the server and the current display size, to obtain processed image data; The alignment step represents the number of bytes of data read by the hardware encoder each time; The processing unit is further configured to read the processed image data through the hardware encoder, and encode the original image data in the processed image data to obtain desktop coded data; A sending unit, configured to send the desktop encoded data to the client. 9. A server, characterized in that, the server includes: a processor, a memory, and a communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute the image processing program of the virtual desktop in the memory, so as to realize the steps of the image processing method of the virtual desktop according to any one of claims 1-7. 10. A storage medium, characterized in that one or more programs are stored in the storage medium, and the one or more programs can be executed by one or more processors, so as to implement any one of claims 1-7. A step of the image processing method of a virtual desktop.