CN121879653A - A control method and system - Google Patents

Abstract

This application discloses a control method and system, relating to the field of terminal technology. The display interface of the control device independently presents the projection windows corresponding to multiple projection devices. The control device obtains the user's drag operation on the first display object in the first projection window of the first projection device. Based on the drag operation and a global view layer independent of each projection window, the control device displays a global drag animation corresponding to the drag operation. The global drag animation is configured to follow the user's drag operation trajectory and move across each projection window to drag the first display object to the second projection window of the destination second projection device.

Description

Control method and system

Technical Field

The present application relates to the field of terminal technologies, and in particular, to a control method and system.

Background

With the popularization of wireless screen-throwing technology, a plurality of terminal devices (such as screen-throwing devices of notebook computers, smart phones or tablet computers) can project screen contents of the terminal devices onto a display device (such as a large intelligent screen, a conference tablet and the like, and the terminal devices can also be called as control devices) and respectively present the screen contents in an independent screen-throwing window mode, so that the requirement of cross-device cooperative operation is induced, and the object dragging operation of the cross-screen-throwing window becomes a natural interaction mode.

However, when a user tries to drag a display object in one screen-throwing window to another screen-throwing window, a serious drag effect cracking problem is caused due to the fact that the drag effect rendering tasks are dispersed in each screen-throwing device, and the smoothness of user experience is reduced.

Disclosure of Invention

In view of the above problems, the present application provides the following solutions:

The first aspect of the present application provides a control method, applied to a control device, where a display interface of the control device independently presents screen projection windows corresponding to a plurality of screen projection devices, and the method includes:

acquiring a drag operation of a user on a first display object in a first screen-throwing window of first screen-throwing equipment;

And displaying global drag effects corresponding to the drag operations based on the drag operations and global view layers independent of the screen throwing windows, wherein the global drag effects are configured to move across the screen throwing windows along the drag operation tracks of the users so as to drag the first display object to a second screen throwing window of a target second screen throwing device.

In one possible implementation, the method further comprises:

and responding to the drag operation, and sending a first drag event to the first screen throwing device so that the screen throwing window data of the first screen throwing device does not comprise the original drag effect of the first display object.

In one possible implementation, the method further comprises:

Receiving a first transmission message sent by the first screen throwing device based on the first drag event, wherein the first transmission message comprises drag effect visual data of the first display object;

And performing display rendering based on the drag effect visual data in the first transmission message, and displaying global drag effect on the global view layer.

In one possible implementation, the first transmission message further includes data storage information for locating and accessing file data corresponding to the first display object, where the data storage information includes a storage path of the file data corresponding to the first display object in a network file system shared directory, and the method further includes:

And responding to the drag release operation of the global drag effect, triggering a second screen throwing device to directly access the storage path shared by the first screen throwing device according to the network file system protocol through a direct communication channel between the second screen throwing device and the first screen throwing device, and acquiring file data corresponding to the first display object, wherein the second screen throwing device is a screen throwing device corresponding to a second screen throwing window where a drag release position is located.

In one possible implementation, the triggering implementation manner for triggering the second screen-throwing device to acquire the file data corresponding to the first display object includes any one of the following:

Determining a second screen throwing device corresponding to a second screen throwing window where a drag release position is located, and sending a data acquisition message to the second screen throwing device, wherein the data acquisition message comprises the data storage information so that the second screen throwing device responds to the data acquisition message to acquire file data corresponding to the first display object;

and sending a drag release event to each screen throwing device, so that each screen throwing device responds to the cached data storage information after determining that the screen throwing device belongs to a second screen throwing device based on the drag release event, and acquiring file data corresponding to the first display object.

In one possible implementation, the control device and the plurality of screen throwing devices access a message bus in a publish-subscribe mode, and subscribe to a preset screen throwing theme in the message bus, so that the first screen throwing device responds to the first dragging event and publishes the first transmission message to the screen throwing theme through the message bus;

The receiving a first transmission message sent by the first screen-throwing device based on the first drag event includes:

And receiving the first transmission message under the screen throwing theme through the message bus.

In one possible implementation, the data acquisition message is transmitted through a screen-on communication channel established based on a screen-on protocol between the control device and the second screen-on device;

The drag release event is sent to each of the screen throwing devices through a message bus of a publish-subscribe mode.

The second aspect of the present application provides a control method applied to a first screen-throwing device in a plurality of screen-throwing devices, where screen-throwing windows corresponding to the plurality of screen-throwing devices are independently presented on a display interface of the control device, and the method includes:

The method comprises the steps of receiving a first drag event sent by the control equipment, wherein the first drag event is generated by the control equipment in response to a drag operation of a user on a first display object in a first screen-throwing window of the first screen-throwing equipment;

intercepting a native drag effect triggered by the first drag event on the first screen throwing device, so that the screen throwing window data of the first screen throwing device does not comprise the native drag effect of the first display object;

And displaying global drag effect corresponding to the drag operation on a display interface of the control device, wherein the global drag effect is configured to move across each screen throwing window along a drag operation track of the user so as to drag the first display object to a second screen throwing window of a target second screen throwing device.

In one possible implementation, the intercepting the native drag effect triggered by the first drag event on the first screen-drop device includes any one of the following implementation manners:

responding to a dynamic effect interception instruction corresponding to the first dragging event, and controlling a dragging dynamic effect rendering module in the first screen throwing equipment to be in a closed state so as to prevent the generation of a primary dragging dynamic effect;

And prohibiting the first screen throwing device from sending screen throwing data of the original drag effect to the control device so that the original drag effect is not displayed in the first screen throwing window, wherein the screen throwing data is generated by a drag effect rendering module in the first screen throwing device in response to the first drag event.

A third aspect of the present application provides a control system comprising a control device, and a plurality of screen-projecting devices communicatively connected to the control device, wherein:

the display interface of the control device independently presents the screen projection windows corresponding to the screen projection devices,

The control apparatus includes:

The user interface is used for acquiring the drag operation of a user on a first display object in a first screen projection window of the first screen projection device;

And the display screen is used for displaying the global drag effect corresponding to the drag operation based on the drag operation and the global view layers independent of the screen throwing windows, and the global drag effect is configured to move across the screen throwing windows along the drag operation track of the user so as to drag the first display object to the second screen throwing window of the target second screen throwing device.

Drawings

The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.

FIG. 1 is a schematic diagram of a control system according to an embodiment of the present application;

fig. 2 is a schematic diagram of a scenario of a control system according to a second embodiment of the present application;

fig. 3 is a schematic diagram of a control system according to a third embodiment of the present application;

FIG. 4 is a flow chart of a control method according to an embodiment of the present application;

FIG. 5 is a schematic view of an alternative scenario in which a display object is dragged across a projection window in a control method according to an embodiment of the present application;

fig. 6 is a flow chart of a control method according to a second embodiment of the present application;

Fig. 7 is a schematic diagram of a scenario based on MQTT message synchronization in a control method according to an embodiment of the present application;

Fig. 8 is a flow chart of a control method according to a third embodiment of the present application;

fig. 9 is a schematic structural diagram of a control device according to a first embodiment of the present application;

fig. 10 is a schematic structural diagram of a control device according to a second embodiment of the present application.

Detailed Description

Embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. The terminology used in the description of the embodiments of the application herein is for the purpose of describing particular embodiments of the application only and is not intended to be limiting of the application. As one of ordinary skill in the art can know, with the development of technology and the appearance of new scenes, the technical scheme provided by the embodiment of the application is also applicable to similar technical problems.

The terms "first," "second," and the like in the description and in the above drawings, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and are merely illustrative of the manner in which embodiments of the application have been described in connection with the description of the objects having the same attributes. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.

The problem described in the background art part of the present application is that when a display object in one screen throwing window (virtual window) presented by a control device is dragged into the other screen throwing window, the drag effect on the display object is only displayed in the two screen throwing windows, once the drag position leaves the boundaries of the two screen throwing windows, the drag effect disappears, and even different drag effects can be displayed in the two different screen throwing windows, so that the user perceives a continuous and seamless drag process in vision, but the operation of jumping, which is interrupted by the boundaries of the screen throwing windows, greatly damages the substantivity of interaction and the fluency of user experience. In order to solve the problems, the embodiment of the application provides a control method which can realize the visual seamless display of the dragging effect in the process of dragging an object crossing a screen throwing window. The following describes a control method and an application scenario thereof in detail with reference to the accompanying drawings.

Referring to fig. 1, an architecture diagram of a control system according to an embodiment of the present application may include a control device 110, and a plurality of (at least two) screen-throwing devices 120 communicatively connected to the control device 110, where a display interface of the control device 110 is configured to independently present screen-throwing windows corresponding to the plurality of screen-throwing devices 120, so that a user can complete a cross-device data transmission scheme in a multi-device screen-throwing environment directly by dragging a display object between the plurality of screen-throwing windows displayed by the control device.

In practical application, the control device 110 and each screen device 120 may be connected in the same local area network (such as an intranet, a home WIFI network, etc.), or a wired screen connection or a wireless screen connection may be established through a wide area network, so as to implement data transmission between the control device 110 and the corresponding screen device 120. It should be understood that after each screen throwing device 120 establishes a communication connection with the control device 110 through a screen throwing technology, the screen throwing device 120 may send screen throwing data to the control device 110, such as a real-time video stream, to be synchronously displayed in a corresponding screen throwing window (virtual screen throwing window) displayed on a display interface of the control device 110, and then, may directly operate a display object in the screen throwing window, without directly operating the corresponding screen throwing device 120, and the present application will not be described in detail about a process of acquiring and transmitting the screen throwing data.

According to the above analysis, referring to the schematic view of the display interface of the control device 110 shown in fig. 2 for the scene of each of the plurality of screen capturing devices, since each of the screen capturing windows of different screen capturing devices 120 is simultaneously presented on the same display interface, in the case that data of one screen capturing device needs to be transmitted to another screen capturing device, as shown in fig. 2, a user may directly drag a display object (an object, such as an icon, a file name, or other identifier, presented by the data to be transmitted in the interactive interface) in one screen capturing window in the control device 110 into the corresponding other screen capturing window, without going to the location of the screen capturing device from the data source to operate the same. In the process of dragging the display object across the screen throwing window, the control device 110 and the screen throwing device 120 corresponding to the screen throwing window can execute the control method provided by the application, and the continuous and seamless visual dragging effect is displayed in the control device 110, so that the user cannot perceive the switching of the screen throwing window, and the dragging process is finished at one time.

In some embodiments, under the control system architecture, for each of the screen-projection windows corresponding to the plurality of screen-projection devices 120, as shown in fig. 2, the screen-projection windows may be simultaneously displayed in the display interface of the control device 110, so that a user can conveniently and intuitively see at least one display object displayed in each screen-projection window, and select a display object (denoted as a first display object) corresponding to the required transmission data from the display object, so as to complete the drag operation across the screen-projection window. Optionally, especially in the case of a large number of screen projection devices 120, the user may select that a portion of the screen projection windows is minimized or otherwise enter a hidden state (e.g., a portion of the screen projection windows overlap), and the like, and another portion of the screen projection windows is presented in the display interface of the control device 110, so that the user can clearly read the content displayed in this portion of the screen projection windows. In this way, the application can simultaneously present the screen throwing windows corresponding to the source equipment and the destination equipment of the data to be transmitted on the display interface, and then select the display object to carry out the drag operation. The application does not limit the independent display mode of the screen throwing window corresponding to each of the plurality of screen throwing devices.

According to the above analysis, the control device 110 according to the present application may include a user interface 111 and a display screen 112, where the user interface 111 may be used to obtain a drag operation of a user on a first display object in a first screen window of a first screen device. The first screen-throwing window may be any one of a plurality of screen-throwing windows currently presented by the display interface of the control device 110, and the first screen-throwing device is a screen-throwing device corresponding to the first screen-throwing window, that is, is a source device of data to be transmitted currently, and may dynamically determine which screen-throwing device is the first screen-throwing device according to the change of the actual dragging requirement, and record the screen-throwing window corresponding to the first screen-throwing device as the first screen-throwing window.

The user interface 111 may be a hardware interface of an input component such as a keyboard, a mouse, a touch screen, a touch pen, or an audio collector, and supports a user to select the first display object to execute a drag operation by using the input component, and it should be understood that implementation manners of the drag operation on the first display object may be different for different types of input components, which is not limited in the present application. In addition, during the process that the user finishes the operation of dragging the first display object through a voice command, the audio collector can collect the input audio of the user, so that the intelligent program (such as an intelligent agent or an intelligent assistant) of the control device can recognize that the input audio is a task of dragging the first display object across the screen throwing window, and the task is automatically executed, namely the first display object is dragged out of the first screen throwing window to the target screen throwing window (which can be recorded as a second screen throwing window).

The display screen 112 in the control device 110 may display a global drag effect corresponding to a drag operation on the first display object based on the drag operation and a global view layer independent of each of the drop windows. The global drag effect is configured to move across each screen throwing window along a drag operation track of a user so as to drag a first display object to a second screen throwing window of a target second screen throwing device.

In order to solve the problem that in the conventional object drag operation, drag effect is only displayed in a corresponding screen throwing window, so as to cause drag effect splitting, the control device 10 may create a global view layer independent of each screen throwing window through a system-level graphical interface of the control device 110, where a display layer level of the global view layer is higher than a display layer level of each screen throwing window, so that the global drag effect is not blocked by any screen throwing window, so that the global view layer is a new top-level image with the highest display priority. In this way, based on the global drag effect of the drag operation of the global view layer display on the first display object, the global drag effect is ensured to be always located on all the screen throwing windows physically, the moving track of the global drag effect is a continuous curve on the display interface and is not influenced by the boundary of the screen throwing window below, and a user intuitively sees that one display object flies smoothly from one place (the original display position in the first screen throwing window) to the other place (the target position in the second screen throwing window) of the display screen 112, so that real visual seamless is realized.

In one possible implementation, at the same time, for one drag operation, only one global drag effect is presented, so that multiple effect conflicts are avoided, i.e. the uniformity of the global drag effect is realized. In the one-time drag operation, the display position (moving track) of the global drag effect on the display screen can be determined and dynamically updated based on the operation position of the drag operation of the user on the display interface, so that visual continuity when crossing different screen-throwing window boundaries is realized. Optionally, the global drag effect may include at least one of a first display object, a file data type identifier corresponding to the first display object, a drag state prompt, and the like.

Regarding the implementation of the respective contents described above by the user interface 111 and the display screen 112 in the control device 110, reference may be made to the description of the respective parts of the method embodiments described below, and the embodiments of the present application will not be described in detail herein. Further, the control device 110 may further include at least one first memory for storing a plurality of first computer instructions for implementing the control method, and a message or data from each of the screen projection devices 120, and at least one first processor for executing the first computer instructions to implement the control method on the control device 110 side.

In some embodiments, for the above-mentioned screen device 120 may include at least one second memory and at least one second processor, where the second memory may be used to store file data that can be shared with other screen devices, messages from other screen devices or control devices, and second computer instructions for implementing a control method executed by the screen device side, where the second processor executes the second computer instructions to implement the steps of the control method executed by the screen device side, and the implementation process may refer to the description of the method embodiments below, where this embodiment is not described in detail herein.

In order to achieve message synchronization between the control device 110 and the multiple screen-throwing devices 120 connected by communication, in some embodiments, as shown in fig. 3, a message server and a message client matched with the message server may be deployed in the control device 110, and a message client matched with the message server and capable of communicating is deployed in each screen-throwing device 120, so that message synchronization between the control device 110 and each screen-throwing device 120 is achieved through a message bus in a publish-subscribe mode.

Optionally, the control device 110 and the multiple screen-throwing devices 120 connected by communication can implement real-time and reliable message synchronization through an MQTT (Message Queuing Telemetry Transport, message queue telemetry transmission, a lightweight message transmission protocol based on a publish/subscribe mode) message bus, so as to meet the communication requirement of the terminal device with limited resources. The message server may be an MQTT server and the message client may be an MQTT client, so that the complexity of point-to-point communication is avoided based on a one-to-many message distribution mechanism. In the process of message synchronization, a hierarchical theme structure may be adopted to ensure accuracy and expansibility of message routing, and in this embodiment, a screen-throwing theme may be created in advance in the message bus (corresponding to a session, so as to configure a one-to-one theme/session identifier), so that a plurality of screen-throwing devices 120 communicatively connected to the control device 110 may subscribe to a preset screen-throwing theme, where the control device 110 may record respective device identifiers (such as a device ID) of the screen-throwing devices 120 subscribing to the screen-throwing theme, for example, store the theme identifier in association with each device identifier, and implement status management of each device accordingly, and the implementation process of the present application will not be described in detail.

In this way, after any device subscribed to a preset screen theme publishes a message, for example, the message is sent to the screen theme on the message bus, and other devices subscribed to the screen theme can receive the message under the screen theme through the message bus so as to analyze the content of the message and complete the task to be executed. The present application does not describe the processing logic of the control device 110 after receiving the message according to, but not limited to, this method, and supports fault tolerance and recovery mechanisms for the control device 110 and each screen-throwing device 120, so as to recover in time when communication is abnormal, thereby meeting the control requirements of the corresponding devices. In addition, in the face of the expansion requirement of the system, for example, after the screen projection equipment is newly added, the screen projection equipment is controlled to subscribe to the screen projection theme, so that the published message under the screen projection theme can be obtained, and the implementation process is not described in detail. It should be noted that, the method for implementing message synchronization between the control device 110 and each screen-throwing device 120 includes, but is not limited to, MQTT communication method.

Further, in order to support point-to-point transmission between different screen throwing devices 120, a direct communication channel may be established between different screen throwing devices 120, so that in the process of drag operation and drag release operation of a first display object crossing a screen throwing window, by means of the above-described message synchronization manner, it is determined that a second screen throwing device needs to acquire file data corresponding to the first display object shared by the first screen throwing device, the file data is acquired through the corresponding direct communication channel. Alternatively, each of the screen-throwing devices 120 may be mounted on a Network file system (NFS, network FILE SYSTEM) to share a directory, so that the second screen-throwing device directly accesses a storage path of the file data shared by the first screen-throwing device through a direct communication channel with the first screen-throwing device, and reads or copies the file data. The application does not detail the detailed process of how to realize data sharing between different screen throwing devices in the NFS mode. Of course, data transmission between different screen throwing devices in a multi-device screen throwing environment can be realized by adopting other network sharing modes, and the application is not limited to the above.

It should be understood that the control system and the structure of the control device included in the control system shown in fig. 1-3 do not limit the control system and the device in the embodiment of the present application, and in practical applications, the control device and the screen-throwing device may further include more or fewer components than those shown in the drawings, or may combine some components. Moreover, the control device 110 in the control system provided by the present application may be any one of a mobile phone, a tablet computer, a wearable device with a display function and a wireless communication function, a vehicle-mounted terminal device, a Virtual Reality (VR) device, an augmented reality (augmented reality, AR) device, a personal computer (personal computer, PC), a notebook computer, a netbook, a smart home device (such as a smart television, etc.), etc., the screen device 120 may be a terminal device with data processing, storage and communication capabilities, and multiple screen devices may be used as independent data sending ends or data receiving ends, and the same screen device may switch roles according to a scene. The screen-throwing device can also comprise, but is not limited to, terminal devices in the form of products, and can be determined according to actual screen-throwing scene requirements. In addition, the control device 110 may further include components such as various sensors, a power management module, etc., and the present application does not describe the composition of the control system and the composition of each device included in the control system in a one-to-one way manner.

In connection with the control system provided by the above-described embodiment of the present application, a control method suitable for the control system will be described below.

Referring to fig. 4, a flow chart of a control method according to an embodiment of the present application is provided, where the method according to the embodiment of the present application may be applied to the control device 110, and in combination with the above description of the control system, in a multi-device screen-projection environment, the control device 110 is communicatively connected to the plurality of screen-projection devices 120, and screen-projection windows corresponding to the plurality of screen-projection devices 120 are independently presented on a display interface of the control device 110, based on this, as shown in fig. 4, the method according to the embodiment may include:

Step S41, obtaining a drag operation of a user on a first display object in a first screen projection window of first screen projection equipment;

And step S42, displaying the global drag effect corresponding to the drag operation based on the drag operation and the global view layers independent of the screen throwing windows, wherein the global drag effect is configured to move across the screen throwing windows along the drag operation track of the user so as to drag the first display object to the second screen throwing window of the target second screen throwing device.

In the embodiment of the application, in combination with the above analysis, a user hopes to seamlessly move the first display object from the screen throwing window of one screen throwing device to the screen throwing window of the other screen throwing device through one continuous drag operation on the control device, so that the user can visually perceive one continuous and seamless drag effect, the jump feeling is not generated due to the interruption of the boundaries of the screen throwing windows, namely, the drag effect visual seamless display is realized on the display screen of the control device, and the smoothness of user experience is improved.

The first display object may be any visual element that may be used by a user to perform a drag operation, and has interactivity, visibility, and state variability, and may be associated with data in a screen device corresponding to the screen window, that is, have data association. The method can include, but is not limited to, application interface elements such as application icons, window thumbnails or interface controls, file system objects such as folder identifications/file identifications (icons/shortcuts), media contents such as video preview frames, audio waveform diagrams, image thumbnails or other image identifications, and any one of review objects such as search result sets, multi-selection object groups or intelligent recommendation items.

In one possible implementation, a user selects a first display object presented in a first screen window through an input component such as a mouse button, a touch pen and the like on a display interface of the control device, and starts a drag operation by this, for example, clicking and holding the first display object with a mouse to start drag movement, an input event manager of the control device may continuously capture a drag operation position of the input component on the display screen through an interface of an operating system bottom layer, for example, a position of a cursor in an absolute coordinate system of the display screen, while a window manager of the control device may calculate, based on the position, which screen window the first display object belongs to, and may further determine, through a screen session mapping table, that the screen window corresponds to the screen window of the first screen device, which is denoted as the first screen window, so that it may be identified that the user is performing drag operation on the first display object in the first screen window of the first screen device, but is not limited to such a drag operation capturing and identifying method.

After the control device identifies the drag operation, a new transparent top view layer with the highest display priority can be created or called by the view manager, and is marked as a global view layer, so that the view manager is ensured to be displayed on all the screen throwing windows, namely the global view layer is independent of each screen throwing window, so that a dynamically controllable global drag effect is created and displayed in the global view layer through a dynamic effect engine, the initial position of the dynamic controllable global drag effect is accurately aligned with the position when the drag operation starts, and then the display position of the global drag effect is synchronously updated along with the position change of the captured drag operation, so that the global drag effect can smoothly track the movement track of the drag operation of a user and directly move across each screen throwing window, namely the global drag effect is ensured to move to the screen throwing window boundary in real time and not disappear, the global drag effect continuously moves in the global view layer in the whole drag operation process, and the virtual drag effect cannot pass through each screen throwing window, so that the different screen throwing window boundaries can not pass through in a sense.

In one possible implementation, for the drag effect visual data for implementing the global drag effect, the drag effect visual data may be rendering data of a specified dynamic effect/default dynamic effect preconfigured by the control device, or may be rendering data from the first screen throwing device, so as to implement consistency between the global drag effect of the control device and the native drag effect of the screen throwing device, which is not limited in the present application. Optionally, for different types of display objects, the type of the global drag effect displayed by the control device may be the same or different, and may be determined according to actual requirements.

Optionally, during the whole drag operation process of the first display object, the control device captures that the first display object is dragged into a certain screen throwing window according to, but not limited to, the method described above, and can adjust the visual style (such as highlighting, etc.) of the screen throwing window or output the corresponding screen throwing device identifier in a voice/popup text mode, so as to remind the user of the screen throwing window of which screen throwing device is currently dragged, so that the user can clearly operate the screen throwing window, avoid mistakenly dragging the target screen throwing window, and the like, and realize the certainty of drag operation. Of course, when the first display object is captured and dragged to the invalid area, a visual cue (such as color change or dithering) can be provided through the global dragging effect, so that misoperation can be prevented. The method for realizing the visual state change in the process of dragging the first display object across the screen-throwing window is not limited, and can be flexibly configured or adjusted according to actual requirements.

According to the analysis, through one continuous drag operation, the first display object is dragged into the second screen throwing window, namely, the screen throwing window corresponding to the target screen throwing device (marked as the second screen throwing device), the mouse/touch can be directly released, the drag release operation is captured by the input event manager, and the screen throwing window where the drag release position is located can still be determined to be the second screen throwing window of the second screen throwing device according to the position calculation method of the drag operation.

Therefore, in the multi-device screen-throwing environment provided by the embodiment of the application, in combination with the schematic view of the display object dragging scene across the screen-throwing window shown in fig. 5, the user directly drags the first display object (such as the file icon of the document a.docx shown in fig. 5) in the first screen-throwing window (office equipment window) corresponding to the first screen-throwing device (such as the screen-throwing device a shown in fig. 5) in the display interface of the control device, and the user perceives the same object as the object dragged on the single device without adapting to the particularity of the multi-window environment, and the intuitive operation improves the interaction straightness. In addition, in one drag process, the control device continuously displays the global drag effect along the drag operation moving track of the user based on the drag operation and the global view layer independent of each screen throwing window, can continuously and smoothly cross the boundaries of each screen throwing window, cannot break or disappear, determines the visual continuity of the drag effect, and improves the fluency of user experience.

Wherein, the global drag effect of the application is intensively rendered by the control equipment and displayed on the global view layer to solve the problems of asynchronous and discontinuous drag effect caused by independent rendering of each screen throwing equipment, the application solves the error caused by coordinate conversion of a plurality of window coordinate systems by unifying the display screen coordinate systems, thereby ensuring that the overall dragging effect display position is aligned with the dragging operation position and realizing seamless visual experience of dragging the cross-window display object. In addition, according to the display frequency of the display screen of the control device, the display fluency of the overall drag effect is ensured, and the rendering state of the drag effect is coordinated relative to a plurality of devices.

Referring to fig. 6, a flow chart of a control method according to a second embodiment of the present application is still applicable to the control apparatus, and as shown in fig. 6, the method according to the present embodiment may include:

step S61, obtaining a drag operation of a user on a first display object in a first screen projection window of first screen projection equipment;

Step S62, responding to the drag operation, and sending a first drag event to the first screen throwing device so that the screen throwing window data of the first screen throwing device does not comprise the original drag effect of the first display object;

In the process of executing the drag operation on the first display object in the first screen throwing window, in general, after the first screen throwing device obtains a corresponding first drag event, the first drag event is responded, the original drag effect of the first display object is displayed on the local display screen of the first screen throwing device, which is equivalent to the drag effect generated by dragging the first display object on the display screen of the first screen throwing device, so that the screen throwing data of the original drag effect is synchronously displayed to the first screen throwing window in the screen throwing mode, the original drag effect is displayed on the view layer where the first screen throwing window is located, the global drag effect rendered by the control device and displayed on the global view layer overlaps with the control device, vision is disordered, and resource waste is caused.

In this way, the control device can include relevant task information for intercepting and suppressing the original drag effect of the first screen throwing device in the process of sending the first drag event to the first screen throwing device, so that the first screen throwing device analyzes the first drag event, accurately intercepts and suppresses the original drag effect, the screen throwing window data of the first screen throwing device does not include the original drag effect of the first display object, at the moment, the screen throwing data transmitted to the control device in real time by the first screen throwing device can not include the data for directly displaying the original drag effect on the display view layer of the first screen throwing window, and can be the drag effect data for representing the original drag effect itself.

In a possible implementation method, the first drag event does not contain the task information described in the upper section either, the first screen throwing device determines that the obtained first screen throwing event is generated in response to the drag operation of the first display object in the first screen throwing window displayed by the control device, but not in response to the drag operation of the first display object in the self interface of the first screen throwing device, and then the original response logic of the first drag event is intercepted to prevent the generation of the original drag effect, or the mode that the data (dynamic effect data) for directly displaying the original drag effect on the display view layer of the first screen throwing window, which is generated by rendering, is fed back to the control device is prevented, so that the screen throwing window data of the first screen throwing device does not comprise the original drag effect of the first display object, and the problem of dynamic effect conflict in the multi-device screen throwing environment is solved. It should be noted that, the method for implementing the interception of the original drag effect by the first screen throwing device is not limited, and the method can be implemented in combination with an interception technology supported by a system platform of the first screen throwing device, for example, interception of system messages through a Hook mechanism and window subclassing, interception of system events through an event monitor and CGEVENTTAP, and the like.

In some embodiments, the control device may send the first drag event directly to the first screen device through a screen-throwing communication channel created based on a screen-throwing protocol between the control device and the first screen-throwing device, for example, through a SmartConnect (a proprietary screen-throwing protocol may provide an efficient bidirectional communication channel, implement low-delay transmission of events) screen-throwing reverse channel to perform transmission of the first drag event, and a mechanism of precisely intercepting the native drag effect of the first screen-throwing event, but is not limited thereto. Optionally, the first drag event may also be sent to each of the screen capturing devices through a message bus in the publish-subscribe mode, where the first screen capturing device may include a start position of a drag operation (an initial display position of the first display object), so that each of the screen capturing devices calculates whether the start position is located in its corresponding screen capturing window according to the start position, and if so, the first screen capturing device performs the above-described interception operation on the native drag effect as the first screen capturing device.

Wherein, because the control device and the screen throwing device can have the difference of display parameters such as resolution, scaling and the like, accurate coordinate conversion can be carried out, so that the screen throwing device determines whether the received initial position is positioned in the corresponding screen throwing window, the control equipment can realize the alignment of the display content in each screen throwing window and the screen display content/to-be-displayed content of the corresponding screen throwing equipment, so that the screen throwing effect is ensured, and the application does not detail the coordinate conversion process.

Step S63, a first transmission message sent by the first screen throwing device based on a first drag event is received, wherein the first transmission message comprises drag effect visual data of a first display object;

step S64, display rendering is carried out based on the drag effect visual data in the first transmission message, and the global drag effect is presented on the global view layer;

In one possible implementation, the control device is configured to render and display the drag effect visual data of the global drag effect, which may be from a storage device of the control device itself, such as drag effect visual data determined after the drag effect is preconfigured or defaulted, so as to determine the consistency of the global drag effect in the whole drag operation process, without focusing on differences between platforms of the screen throwing devices.

Preferably, in order to realize personalized display of drag effects of display objects of different screen throwing devices, the drag effect visual data may originate from a source device of a first display object, that is, a first screen throwing device, in this case, in a process of the first screen throwing device responding to a first drag event, the drag effect visual data (that is, native drag effect visual data) of the first display object may also be determined, for example, a first display object screenshot or a preconfigured drag effect icon, a thumbnail or a drag effect lightweight code, etc., and then a first transmission message including the drag effect visual data, that is, the first transmission message may also include drag effect visual data, so that the control device parses the first transmission message, renders a unified drag effect (global drag effect) on a global view layer based on the drag effect visual data, so that the global drag effect is highly similar or completely consistent to a native drag effect displayed by a user in a local drag operation of the first screen throwing device, thereby reducing a cognitive load of the user, and implementing seamless rendering of the application.

In the process of drag operation of the first display object by the user, as shown in fig. 5, the control device may detect the position of the drag operation in real time, that is, the position where the first display object is dragged to, may perform display rendering based on the position and drag effect visual data, and display the global drag effect on the global view layer at the position, or dynamically adjust the global drag effect displayed in the global view layer based on the position, without repeating rendering to cause resource waste, thereby implementing that the global drag effect moves across the screen throwing window along the drag operation movement track of the user, eliminating the visual splitting problem, ensuring natural and smooth operation, and facilitating intuitive operation of the user.

In one possible implementation, the global drag effect may be obtained by a rendering engine displaying and rendering drag effect visual data. The global View layer (such as the highest display level of the Z-axis display layer) independent of each screen throwing window can also be created on the top display layer, the drag effect visual data is used as texture and drawn on the global View layer, so that a visual entity of the global drag effect is instantiated, for example, a method startDragAndDrop (which is a View class method provided by an Android system and is used for starting drag and drop operation, allowing a user to drag View elements through touch gestures and drop the View elements to target positions) starts system-level drag, and a received drag effect screenshot (a drag effect visual data) is drawn into a coherent drag effect (global drag effect) following the movement track of the drag operation through DragShadowBuilder (an abstract class and is used for creating and updating drag shadows during the drag operation, and the shadows identify the dragged display objects) so as to ensure that the global drag effect spans different screen throwing windows without cutting and delay.

In some embodiments, the first transmission message may be transmitted through a message bus in a publish-subscribe mode, for example, through an MQTT communication bus, and the first transmission message is sent to a preset screen-throwing topic (topic/dragFile drag file) in the message bus, so that each device subscribed to the screen-throwing topic can acquire a message published to the screen-throwing topic, for example, the first transmission message. It should be noted that, in a multi-device screen-projection environment, other broadcasting methods may be adopted to realize message synchronization between multiple devices, and the application is only illustrated by taking an MQTT-based message synchronization method as an example.

For example, as shown in fig. 7, the control device and the plurality of screen throwing devices access a message bus in a publish-subscribe mode (in fig. 7, an MQTT message bus is taken as an example for illustration), subscribe to a preset screen throwing topic (such as topic/dragFile, etc.) in the message bus, and respond to a first drag event by using the first screen throwing device to publish a first transmission message to the screen throwing topic through the message bus, that is, the first screen throwing device issues a first transmission message to a screen throwing topic in a message server (such as an MQTT server shown in fig. 7) through a self-deployed message client (such as an MQTT client shown in fig. 7), and the control device receives the first transmission message under the screen throwing topic through the message bus, that is, where the MQTT client deployed by the control device acquires the first transmission message from the subscribed screen throwing topic, and meanwhile, other screen throwing devices subscribed to the screen throwing topic (such as a second screen throwing device shown in fig. 7, and other screen throwing devices may also include other screen throwing devices) may also acquire the subscribed to the screen throwing topic from the MQTT server through the self-deployed by using the self-MQT client (such as a first drag client) and wait for releasing the message in a subsequent transmission topic (such as a buffer queue or other forms).

Step S65, in response to the drag release operation of the global drag effect, triggering the second screen throwing device to acquire file data corresponding to the first display object through a direct communication channel between the second screen throwing device and the first screen throwing device based on data storage information contained in the first transmission message.

Continuing the analysis, as shown in the scenario of fig. 5, the user drags the first display object from the first screen-throwing window (such as the office window shown in fig. 5) to the destination screen-throwing window (such as the conference device window shown in fig. 5, which may be any other screen-throwing window different from the first screen-throwing window and denoted as the second screen-throwing window), and can release the drag operation on the first display object, such as lifting the mouse button/leaving the touch screen with the touch pen, etc., the control device obtains the drag release operation on the first display object, and ends the global drag effect in response to the drag release operation, such as removing the global view layer by calling the graphics interface, so as to clear the global drag effect from the display screen, so that the complete life cycle of one global drag effect is ended.

In the embodiment of the application, the drag operation is performed on the first display object, which is expected to directly transmit the file data (such as one or more of image data, audio data, text data or video data) corresponding to the first display object in the first screen throwing device to the second screen throwing device, and when the drag release operation on the global drag effect is performed by the user, the screen throwing device corresponding to the screen throwing window where the position is located can be determined based on the position (i.e. the drag release position) of the drag release operation, so that the screen throwing device is informed of directly acquiring the file data corresponding to the first display object from the first screen throwing device, and the control device does not need to forward the file data.

Therefore, the control device can process light-weight position mapping and graphic rendering, and forwards the first transmission message among a plurality of screen throwing devices, and the first transmission message is not involved in the transmission of the file data corresponding to the first display object, but is directly completed by different screen throwing devices through the direct communication channel, so that the separation of the message control flow and the data flow is realized.

The data storage information in the first transmission message is used for positioning and accessing file data corresponding to the first display object, and may include a storage path of the file data in the NFS shared directory, so, as shown in fig. 7, in a drag release operation response process, the control device may trigger the second screen throwing device (i.e., the screen throwing device corresponding to the second screen throwing window where the drag release position is located) to directly access, through a direct communication channel between the second screen throwing device and the first screen throwing device, the storage path shared by the first screen throwing device (such as an absolute path of the file data in the NFS shared directory) according to the NFS protocol, so as to obtain the file data corresponding to the first display object, that is, download the file data from the NFS shared directory to the local area of the second screen throwing device, thereby quickly implementing point-to-point quick transmission of the file data point without transferring to the control device.

The implementation process of mounting the NFS shared catalogs by the plurality of screen throwing devices comprises two links of direct communication channel establishment and NFS connection establishment, after connection establishment, the second screen throwing device executes NFS mounting operation, and after the NFS shared catalogs of the first screen throwing device are mapped into a local virtual catalogue, after a storage path shared by the first screen throwing device is required to be acquired, the storage path of the virtual catalogue can be directly accessed, and file data corresponding to a first display object in the first screen throwing device can be directly read, but the implementation method is not limited to the data transmission implementation method.

In some embodiments, in the implementation process of triggering the second screen throwing device by the control device to acquire the file data, the control device may directly determine the second screen throwing device corresponding to the second screen throwing window where the drag release position is located, and directly send a data acquisition message to the second screen throwing device, where the data acquisition message includes the data storage information, so as to directionally notify the second screen throwing device of acquiring the file data under the data storage information. For example, the global view layer of the control device triggers a drag release event, identifies a second screen-throwing window of the second screen-throwing device by the drag release position, immediately terminates the global drag effect of the system level, and simultaneously transmits a data acquisition message containing data storage information to the second screen-throwing device through a screen-throwing communication channel established between the control device and the second screen-throwing device based on a screen-throwing protocol, such as transmitting the data acquisition message through a SmartConnect screen-throwing reverse channel, so that the second screen-throwing device can download file data corresponding to the first display object to the local area based on the data storage information through a direct communication channel between the control device and the first screen-throwing device.

In one possible implementation, the data acquisition message may further include a second device identifier of the second screen device, and in the distributed message broadcast, each screen device judges that the each screen device belongs to the second screen device according to the second device identifier, so that file data corresponding to the first display object can be acquired according to the method described above, and data shared by the first screen device to other screen devices (third screen device) is avoided from being acquired by mistake. The method can further comprise a first device identifier (such as a device ID, etc. which can be from a first transmission message sent by the first screen device) of the first screen device, so that the second screen device can verify whether the second screen device has access rights for accessing file data corresponding to a first display object from the first screen device according to the first device identifier, if the second screen device does not have the access rights, the step of acquiring the file data is forbidden, and if the second screen device does not have the access rights, the step of acquiring the file data is executed, thereby improving the data access security.

Optionally, in another implementation process that the control device triggers the second screen throwing device to acquire the file data, the control device determines a drag release event including a drag release position according to, but not limited to, the method described above, and sends the drag release event to each screen throwing device, so that after each screen throwing device determines that the screen throwing device belongs to the second screen throwing device based on the drag release event, the corresponding file data of the first display object is acquired in response to the cached data storage information, for example, an NFS protocol is called, and the file data corresponding to the first display object is downloaded to the local area from the NFS sharing target according to the previously stored NFS path of the file data from the first screen throwing device. It should be noted that, the triggering implementation manner of triggering the second screen-throwing device to acquire the file data corresponding to the first display object by the control device includes, but is not limited to, the two manners described above.

In some embodiments, the first transmission message sent by the first screen device may further include a drag start timestamp of a drag operation, so that each screen device and the control device do not detect a drag release operation/drag release event corresponding to the global drag effect/the first display object within a preset buffer duration, clear a message stored for the first display object within the preset buffer duration, for example, the first transmission message, release the occupied memory resource, especially, the first transmission message contains a larger data amount, for example, contains the drag effect visual data, and clear invalid message data in time under the condition that more memory resources are occupied, so as to avoid the processing of other messages from being affected due to the shortage of memory resources, but the application is not limited to the memory resource release trigger mechanism described in the application, and clear temporary stored message data when the memory resource occupancy reaches a threshold value.

Referring to fig. 8, a flow chart of a control method according to a third embodiment of the present application is provided, where the method according to the present embodiment may be applied to a first screen device of a plurality of screen devices, that is, a screen device that is a data sending end, where the plurality of screen devices are connected to the same control device in a communication manner, and respective corresponding screen windows independently present a display interface of the control device. In such a multi-device screen-projection environment, as shown in fig. 8, the control method executed by the first screen-projection device may include:

Step S81, a first drag event sent by the control device is received, wherein the first drag event is generated by the control device in response to a drag operation of a user on a first display object in a first screen window of the first screen device;

The manner of generating the first drag event may refer to the description of the corresponding part of the control method embodiment described from the control device side, which may be generated by the control device itself in response to the drag operation and then sent to the first screen capturing device, may be generated by the first screen capturing device itself after detecting the drag operation, and so on, and the implementation process of this embodiment will not be described in detail herein.

Step S82, intercepting the original drag effect triggered by the first drag event on the first screen throwing device, so that the screen throwing window data of the first screen throwing device does not comprise the original drag effect of the first display object;

and step 83, displaying global drag effect corresponding to drag operation on a display interface of the device in cooperation with the control device, wherein the global drag effect is configured to move across each screen throwing window along a drag operation track of a user so as to drag the first display object to a second screen throwing window of a target second screen throwing device.

In the embodiment of the application, in order to solve the problems of conflict of dynamic effects and poor operation consistency in dragging across screen-throwing windows, the method and the device provided by the application are used for intercepting the native dragging effect of the first screen-throwing device in time after receiving a first dragging event, so that the native dragging effect is prevented from being displayed on a display screen of the first screen-throwing device, and is overlapped with the global dragging effect which is displayed on a global view layer of the control device and independent of each screen-throwing window when the native dragging effect is synchronously displayed on the display view where the first screen-throwing window is located. Therefore, the interception step executed by the first screen throwing device is matched with the control device to only display the global dragging effect on the display interface of the first screen throwing device, so that the role of the first screen throwing device in the whole set of cross-device dragging scheme is defined, namely the data sending end rather than the dragging effect rendering end, the dragging effect conflict is avoided, and the cross-window display object dragging interactive experience is improved.

It should be noted that the method for implementing how the first screen projection device intercepts the original drag effect is not limited. In one possible implementation, the first screen throwing device may respond to a dynamic effect interception instruction corresponding to the first drag event, for example, a dynamic effect interception instruction generated and sent by the control device or generated by a rule stored in advance by the first screen throwing device, so as to control the drag dynamic effect rendering module in the first screen throwing device to be in a closed state (for example, switch from an open state to a closed state, or keep the closed state, and prohibit the state from switching), so as to prevent the generation of the native drag dynamic effect of the first screen throwing device. Optionally, the first screen throwing device may also generate, by the self drag effect rendering module, screen throwing data of the original drag effect, such as original drag effect visual data, in response to the first drag event, but prohibit the first screen throwing device from sending the screen throwing data to the control device, so that the original drag effect is not displayed in the first screen throwing window, phenomena such as flicker or jump of the original drag effect are eliminated from the root, overlapping and collision problems of simultaneous display of multiple sets of drag effects are avoided, and the platform-crossing experience consistency is ensured, and interactive experience and performance are improved.

Having described a control method provided by an embodiment of the present application, an apparatus for executing the control method will be described below.

Referring to fig. 9, a schematic structural diagram of a control device provided by an embodiment of the present application, where the control device may be applied to a control device, as shown in fig. 9, the control device provided by this embodiment may include:

The screen display module 91 is configured to independently present screen display windows corresponding to each of the plurality of screen display devices on a display interface of the control device;

a drag operation obtaining module 92, configured to obtain a drag operation of a user on a first display object in a first screen window of a first screen device;

The global drag effect display module 93 is configured to display a global drag effect corresponding to the drag operation based on the drag operation and a global view layer independent of each screen-throwing window, where the global drag effect is configured to be capable of moving across each screen-throwing window along a drag operation track of the user, so as to drag the first display object to a second screen-throwing window of a destination second screen-throwing device.

In some embodiments, the control device may further include:

The first drag event sending module is used for responding to the drag operation and sending a first drag event to the first screen throwing device so that the screen throwing window data of the first screen throwing device does not comprise the original drag effect of the first display object.

In some embodiments, the control device may further include:

The first transmission message receiving module is used for receiving a first transmission message sent by the first screen throwing device based on the first drag event, wherein the first transmission message comprises drag effect visual data of the first display object;

and the display rendering module is used for performing display rendering based on the drag effect visual data in the first transmission message, and displaying the global drag effect on the global view layer.

In some embodiments, the first transmission message may further include data storage information for locating and accessing file data corresponding to the first display object, where the data storage information may include a storage path of the file data corresponding to the first display object in a network file system shared directory. Based on this, the control device may further include:

The drag release triggering module is used for responding to drag release operation of the global drag effect, triggering a second screen throwing device to directly access the storage path shared by the first screen throwing device according to the network file system protocol through a direct communication channel between the second screen throwing device and the first screen throwing device, and acquiring file data corresponding to the first display object, wherein the second screen throwing device is corresponding to a second screen throwing window where a drag release position is located.

In one possible implementation, the drag release triggering module may include:

The first determining unit is used for determining second screen throwing equipment corresponding to a second screen throwing window where the drag release position is located;

The data acquisition message comprises the data storage information, so that the second screen projection device responds to the data acquisition message to acquire file data corresponding to the first display object.

Alternatively, the data acquisition message may be transmitted through a screen-projection communication channel established between the control device and the second screen-projection device based on a screen-projection protocol.

In one possible implementation, the drag release triggering module may also include:

And the drag release event sending unit is used for sending drag release events to each screen throwing device so that each screen throwing device can acquire file data corresponding to the first display object in response to the cached data storage information after determining that the screen throwing device belongs to the second screen throwing device based on the drag release event.

Alternatively, the drag release event may be sent to each of the screen-casting devices through a message bus of a publish-and-subscribe mode, but is not limited thereto.

In some embodiments, the control device and the plurality of screen-throwing devices access a message bus in a publish-subscribe mode, and subscribe to a screen-throwing theme preset in the message bus, so that the first screen-throwing device responds to the first drag event and publishes the first transmission message to the screen-throwing theme through the message bus. Based on this, the first transmission message receiving module may include:

and the first receiving unit is used for receiving the first transmission message under the screen-throwing theme through the message bus.

Referring to fig. 10, a schematic structural diagram of a control device provided by a second embodiment of the present application may be applied to a first screen-throwing device of a plurality of screen-throwing devices, where respective screen-throwing windows corresponding to the plurality of screen-throwing devices are independently presented on a display interface of the control device, as shown in fig. 10, the control device provided by this embodiment may include:

the first drag event receiving module 101 is configured to receive a first drag event sent by the control device, where the first drag event is generated by the control device in response to a drag operation of a user on a first display object in a first screen window of the first screen device;

The interception module 102 is configured to intercept a native drag effect triggered by the first drag event on the first screen capturing device, so that the screen capturing window data of the first screen capturing device does not include the native drag effect of the first display object;

the configuration processing module 103 is configured to cooperate with the control device to display a global drag effect corresponding to the drag operation on a display interface of the control device, where the global drag effect is configured to move across each screen projection window along a drag operation track of the user, so as to drag the first display object to a second screen projection window of a target second screen projection device.

Alternatively, the interception module 102 may include any one of the following control units:

the first control unit is used for responding to the dynamic effect interception instruction corresponding to the first dragging event and controlling a dragging dynamic effect rendering module in the first screen throwing device to be in a closed state so as to prevent the generation of the original dragging dynamic effect;

The first screen throwing device is used for throwing screen throwing data of the original drag effect to the first screen throwing device, the second control unit is used for prohibiting the first screen throwing device from sending the screen throwing data of the original drag effect to the control device, so that the original drag effect is not displayed in the first screen throwing window, and the screen throwing data are generated by a drag effect rendering module in the first screen throwing device in response to the first drag event.

The embodiment of the application also provides a computer program product, which comprises computer readable instructions, wherein the computer readable instructions, when running on a computer device (a control device or a screen throwing device), cause the computer device to realize any control method executed by the corresponding device side provided by the embodiment of the application. The computer program product may be stored on a readable storage medium, such as a floppy disk, a usb disk, a removable hard disk, a ROM (Read-Only Memory), a RAM (Random Access Memory ), a magnetic disk, or an optical disk of a computer, where the instructions include instructions for causing a computer device (which may be a personal computer, a training device, or a network device) to perform the task processing method described in the embodiments of the present application.

The embodiment of the application also provides a computer readable storage medium, which carries one or more computer programs, and when the one or more computer programs are executed by a computer device (a control device or a screen throwing device), the computer device can realize any control method executed by the corresponding device side provided by the embodiment of the application. The present application is not limited to the product form of the computer-readable storage medium.

Finally, it should be noted that the above-described embodiments of the apparatus are merely illustrative, wherein the elements described as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the drawings of the embodiment of the device provided by the application, the connection relation between the modules represents that the modules have communication connection, and can be specifically implemented as one or more communication buses or signal lines.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. From the above description of the embodiments, it will be apparent to those skilled in the art that the present application may be implemented by means of software plus necessary general purpose hardware, or of course by means of special purpose hardware including application specific integrated circuits, special purpose CPUs, special purpose memories, special purpose components, etc. When implemented in software, may be implemented in whole or in part in the form of a computer program product. In this specification, each embodiment is described in a progressive or parallel manner, and each embodiment is mainly described by a difference from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. The apparatus, system and device disclosed in the embodiments are relatively simple to describe, and the relevant parts refer to the description of the method section because they correspond to the methods disclosed in the embodiments.

Claims (10)

1. A control method applied to a control device, wherein a display interface of the control device independently presents screen projection windows corresponding to a plurality of screen projection devices, and the method comprises the following steps:

acquiring a drag operation of a user on a first display object in a first screen-throwing window of first screen-throwing equipment;

And displaying global drag effects corresponding to the drag operations based on the drag operations and global view layers independent of the screen throwing windows, wherein the global drag effects are configured to move across the screen throwing windows along the drag operation tracks of the users so as to drag the first display object to a second screen throwing window of a target second screen throwing device.

2. The method of claim 1, the method further comprising:

and responding to the drag operation, and sending a first drag event to the first screen throwing device so that the screen throwing window data of the first screen throwing device does not comprise the original drag effect of the first display object.

3. The method of claim 2, the method further comprising:

Receiving a first transmission message sent by the first screen throwing device based on the first drag event, wherein the first transmission message comprises drag effect visual data of the first display object;

And performing display rendering based on the drag effect visual data in the first transmission message, and displaying global drag effect on the global view layer.

4. The method of claim 3, wherein the first transfer message further includes data storage information for locating and accessing file data corresponding to the first display object, the data storage information including a storage path of the file data corresponding to the first display object in a network file system shared directory, the method further comprising:

And responding to the drag release operation of the global drag effect, triggering a second screen throwing device to directly access the storage path shared by the first screen throwing device according to the network file system protocol through a direct communication channel between the second screen throwing device and the first screen throwing device, and acquiring file data corresponding to the first display object, wherein the second screen throwing device is a screen throwing device corresponding to a second screen throwing window where a drag release position is located.

5. The method of claim 4, wherein triggering the second screen-casting device to obtain the file data corresponding to the first display object includes any one of the following:

Determining a second screen throwing device corresponding to a second screen throwing window where a drag release position is located, and sending a data acquisition message to the second screen throwing device, wherein the data acquisition message comprises the data storage information so that the second screen throwing device responds to the data acquisition message to acquire file data corresponding to the first display object;

and sending a drag release event to each screen throwing device, so that each screen throwing device responds to the cached data storage information after determining that the screen throwing device belongs to a second screen throwing device based on the drag release event, and acquiring file data corresponding to the first display object.

6. The method according to any one of claims 3-5, wherein the control device and the plurality of screen throwing devices each access a message bus in a publish-subscribe mode and subscribe to a preset screen throwing topic in the message bus, so that the first screen throwing device issues the first transmission message to the screen throwing topic through the message bus in response to the first drag event;

The receiving a first transmission message sent by the first screen-throwing device based on the first drag event includes:

And receiving the first transmission message under the screen throwing theme through the message bus.

7. The method according to claim 5, wherein:

the data acquisition message is transmitted through a screen-throwing communication channel established between the control equipment and the second screen-throwing equipment based on a screen-throwing protocol;

The drag release event is sent to each of the screen throwing devices through a message bus of a publish-subscribe mode.

8. A control method applied to a first screen-throwing device of a plurality of screen-throwing devices, wherein screen-throwing windows corresponding to the screen-throwing devices are independently presented on a display interface of the control device, and the method comprises:

The method comprises the steps of receiving a first drag event sent by the control equipment, wherein the first drag event is generated by the control equipment in response to a drag operation of a user on a first display object in a first screen-throwing window of the first screen-throwing equipment;

intercepting a native drag effect triggered by the first drag event on the first screen throwing device, so that the screen throwing window data of the first screen throwing device does not comprise the native drag effect of the first display object;

And displaying global drag effect corresponding to the drag operation on a display interface of the control device, wherein the global drag effect is configured to move across each screen throwing window along a drag operation track of the user so as to drag the first display object to a second screen throwing window of a target second screen throwing device.

9. The method of claim 8, the intercepting a native drag effect triggered by the first drag event on the first screen-drop device, comprising any one of the following implementations:

responding to a dynamic effect interception instruction corresponding to the first dragging event, and controlling a dragging dynamic effect rendering module in the first screen throwing equipment to be in a closed state so as to prevent the generation of a primary dragging dynamic effect;

And prohibiting the first screen throwing device from sending screen throwing data of the original drag effect to the control device so that the original drag effect is not displayed in the first screen throwing window, wherein the screen throwing data is generated by a drag effect rendering module in the first screen throwing device in response to the first drag event.

10. A control system comprising a control device, and a plurality of screen-casting devices communicatively connected to the control device, wherein:

the display interface of the control device independently presents the screen projection windows corresponding to the screen projection devices,

The control apparatus includes:

The user interface is used for acquiring the drag operation of a user on a first display object in a first screen projection window of the first screen projection device;

And the display screen is used for displaying the global drag effect corresponding to the drag operation based on the drag operation and the global view layers independent of the screen throwing windows, and the global drag effect is configured to move across the screen throwing windows along the drag operation track of the user so as to drag the first display object to the second screen throwing window of the target second screen throwing device.