CN115914210A - File transmission method, device and system based on virtual network - Google Patents

Abstract

The invention discloses a virtual network-based file transmission method, device and system, and relates to the technical field of data transmission. In the virtual network-based file transfer method, the data forwarding device receives data in a specific message format sent from the client, the data forwarding device forwards the data in the specific message format to the cloud computer, and the data forwarding device sends the data in the specific message format to the server The listening port, the data forwarding device receives data in a specific message format and writes it into a designated file. This virtual network-based file transfer method transmits data to the conversion device in a cross-network application scenario. Compared with the traditional method, the bandwidth loss is lower and the transmission speed is higher, and the existing network environment will not be modified. Intrusive, good compatibility, based on application layer message encapsulation and analysis, widely applicable to various operating systems, no need to switch windows back and forth between local and cloud computers, and supports batch file transmission.

Description

A virtual network-based file transfer method, device and system

technical field

The invention relates to the technical field of data transmission, in particular to a virtual network-based file transmission method, device and system.

Background technique

In the actual use of cloud computers, there is often a need for data transmission, but the network in the cloud computer is a private network virtualized by openstack, which can only access the Internet through the cloud computer, and cannot transmit to the cloud computer through the Internet. data, so we cannot directly access this cloud computer through the IP address virtualized by the cloud computer, let alone upload files to the cloud computer through ordinary file transfer tools (such as SFTP, SCP, SSH, etc.) to transfer files to the cloud The existing computer solution is to forward data through the interaction between the cloud computer client and the underlying communication service port (virtualization process port) in the cloud computer image, which can realize the local computer sending data to the cloud computer.

However, this transmission scheme has the following problems. First, the virtualization interaction of the cloud computer client is a full-screen interaction. To transfer files, you need to switch back and forth between the local and virtualization windows, which is cumbersome. The virtualization channel will be occupied when transferring data, resulting in virtualization The display operation is stuck, the more data, the longer it takes, and the experience effect is not good. During the data transmission, the cloud computer is operated at the same time or the network is unstable. The data transmission may be interrupted. After the interruption, manual intervention is required to perform retransmission, which will affect the use. experience.

Contents of the invention

Aiming at the deficiencies of the prior art, the present invention provides a virtual network-based file transfer method, device and system, which solves the problem of high bandwidth loss and poor compatibility caused by traditional cloud computer file transfer, which cannot be widely used and various operations system, and at the same time realize the problem that the file transfer operation is simple and the file transfer can be performed in batches.

In order to achieve the above object, the present invention is achieved through the following technical solutions: a virtual network-based file transfer method, characterized in that it includes:

S1: Receive data in a specific message format sent from the client;

S2: forward the data of the specific message format to the cloud computer;

S3: Send data in a specific message format to the server listening port;

S4: Receive data in a specific message format and write it into a specified file.

Further, the data of the specific message format consists of a 4-digit long message identification header, a 2-digit long message type, a 4-digit long message length, a 4-digit long message sequence number and message content.

Further, in the step S1, data in a specific message format sent from the client is received, wherein the specific reception is as follows:

The client creates a message according to the message body format encapsulation channel and sends it to the forwarding device. The forwarding device parses the message body according to the message type, extracts the session ID in the message body, establishes a Socket persistent connection, and stores the current persistent connection channel ID in the memory;

The client encapsulates the real data to be sent into a data forwarding message and sends it to the forwarding device, and the forwarding device parses the message type as a forwarding message.

Further, after the client establishes a connection with the forwarding device, the forwarding device combines the channel ID of the client with the user identifier, the user ID and the cloud computer ID into a KV key-value pair and saves it in the client memory object Map.

Further, the client encapsulates the channel creation message according to the specific message format message structure, sets the message type to 2, and the message content is the JSON serialized string of the user channel connection authentication message object. After the assembly is completed, the message is sent to the forwarding device for forwarding. After the device finishes processing the message, it returns a response message CLOSED or ESTABLISHED.

Further, the server encapsulates the channel creation message according to the specific message format message structure, sets the message type to 1, and the message content is the JSON serialized string of the user channel connection authentication message object. After the assembly is completed, the message is sent to the forwarding device for forwarding After the device finishes processing the message, it returns a response message CLOSED or ESTABLISHED.

Further, in the step S3, the data in a specific message format is sent to the server, wherein the specific sending is as follows:

The server creates a message by encapsulating the channel according to the message body format and sends it to the forwarding device. The forwarding device parses the message body according to the message type, extracts the session ID in the message body, establishes a Socket persistent connection, and stores the current persistent connection channel ID in the memory.

Further, the data in the specific message format received in the step S4 is written into a specified file, wherein the specific writing is as follows:

The forwarding device receives the message and parses it: according to the specific message format message structure, after the forwarding device receives the message, the message decoder reads the first 4 digits of the message as the source of the message, and only trusted sources process the message;

The forwarding device processes the authentication message type: if the message type is 1 or 2, deserialize the parsed message content into a user channel connection authentication message object through JSON;

The forwarding device processes the forwarded message type: if the message type is 3, the content of the message is no longer deserialized, and the destination channel is intelligently matched directly according to the sending channel ID;

Destination end receives data and writes it to disk: the destination end receives the message sent by the forwarding device and writes it directly to the disk. After the message is processed, it actively disconnects to complete the entire data file transmission.

A virtual network-based file transfer system, including:

A client module, a data forwarding module, a server, a client identification module, a target identification module, a message data forwarding module, a long connection session maintenance module and a flow control module;

The flow control module performs flow control on the high water level and low water level in the data receiving thread, and after the high water level of the data to be sent in the buffer, suspends the reading thread work until the data in the buffer is lower than the low water level and resumes the receiving thread work, Continue to receive messages.

A file transfer device based on a virtual network, applied to a client included in a file transfer system, the file transfer system also includes a data forwarding device and a cloud computer, the cloud computer includes a server; including:

Sending module: used to receive data in a specific message format sent from the client;

Forwarding module: used to forward data in a specific message format to the cloud computer;

The second sending module: used to send data in a specific message format to the listening port of the server;

Receiving module: used to receive data in a specific message format and write it into a specified file.

The present invention has the following beneficial effects:

The virtual network-based file transfer method transmits data to the conversion device in a cross-network application scenario. Compared with the traditional method, the bandwidth loss is lower and the transmission speed is higher, and the existing network environment will not be modified. Intrusive, good compatibility, based on application layer packet encapsulation and analysis, widely applicable to various operating systems, no need to switch windows back and forth between local and cloud computers, supports batch file transfer, and supports file transfer retry after network disconnection , the optimization is more prominent in the weak network environment.

Of course, any product implementing the present invention does not necessarily need to achieve all the above-mentioned advantages at the same time.

Description of drawings

Fig. 1 is a flow chart of the file transmission method of the present invention;

Fig. 2 is a schematic diagram of the message format of the present invention;

Fig. 3 is a schematic diagram of message interaction flow in the present invention;

Fig. 4 is a schematic diagram of the intelligent matching process at the server end of the present invention;

Fig. 5 is a schematic diagram of the status of the message type parsing process of the present invention;

FIG. 6 is a schematic diagram of a data forwarding process in the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

In describing the present invention, it is to be understood that the terms "opening", "upper", "lower", "thickness", "top", "middle", "length", "inner", "surrounding" etc. Indicating orientation or positional relationship is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the components or elements referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as limiting the present invention .

Please refer to Fig. 1, the embodiment of the present invention provides a kind of technical scheme: a kind of file transmission method based on virtual network, comprising:

S1: Receive data in a specific message format sent from the client;

S2: forward the data of the specific message format to the cloud computer;

S3: Send data in a specific message format to the server listening port;

S4: Receive data in a specific message format and write it into a specified file.

Specifically, as shown in FIG. 2 , the data of a specific message format consists of a 4-digit long message identification header, a 2-digit long message type, a 4-digit long message length, a 4-digit long message sequence number and message content.

In this embodiment, the message content is the real data message to be forwarded, the message content is parsed according to the message type, the connection session ID of the client and the target end is identified, and the client and the target end are matched through the same session ID to perform Data forwarding; the content of the message whose type is data forwarding is forwarded directly without parsing.

Specifically, as shown in Figure 3, in step S1, data in a specific message format sent from the client is received, wherein the specific reception is as follows:

The client creates a message according to the message body format encapsulation channel and sends it to the forwarding device. The forwarding device parses the message body according to the message type, extracts the session ID in the message body, establishes a Socket persistent connection, and stores the current persistent connection channel ID in the memory;

The client encapsulates the real data to be sent into a data forwarding message and sends it to the forwarding device.

In this embodiment, when the forwarding device parses the message type as a forwarded message, it no longer parses the message body, obtains the session ID corresponding to the channel through the currently connected channel ID, addresses the channel with the same session ID at the target end according to the session ID, and addresses After connecting to the target channel, the current message is directly forwarded to the target channel to realize message forwarding across the network.

Specifically, in step S3, send data in a specific message format to the server, wherein the specific sending is as follows:

As shown in Figure 3, the server creates a message based on the message body format encapsulation channel and sends it to the forwarding device. The forwarding device parses the message body according to the message type, extracts the session ID in the message body, establishes a Socket persistent connection, and stores the current persistent connection channel ID in the in memory.

Specifically, as shown in Figure 4, after the client establishes a connection with the forwarding device, the forwarding device combines the channel ID of the client with the user ID, user ID and cloud computer ID into a KV key-value pair and stores it in the client memory object Map.

In this embodiment, as shown in Figure 4, when the client requests to forward data, the forwarding device obtains the connection channel ID through the connection of the client, searches for the channel ID corresponding to the client memory object Map through the channel ID, and if found, passes the corresponding user The ID (user ID + cloud computer ID) searches for the corresponding user ID in the memory object Map of the server. If it is not found, it will prompt that the server does not exist. Otherwise, the channel ID of the server can be obtained, and then data can be sent to the channel of the server. To achieve the purpose of automatically addressing the server channel and forwarding data.

Specifically, as shown in Figure 5, the server encapsulates the channel creation message according to the specific message format message structure, sets the message type to 1, and the message content is the JSON serialized string of the user channel connection authentication message object. After the assembly is completed, the message is sent to The forwarding device (SYN-SENT), returns a response message (CLOSED or ESTABLISHED) after the forwarding device processes the message (5.4, 5.5).

Specifically, as shown in Figure 5, the client encapsulates the channel creation message according to the specific message format message structure, sets the message type to 2, and the message content is the JSON serialized string of the user channel connection authentication message object. After the assembly is completed, the message is sent to The forwarding device (SYN-SENT), returns a response message (CLOSED or ESTABLISHED) after the forwarding device processes the message (5.4, 5.5).

In this implementation, as shown in Figure 5, the client data forwarding message follows the specific message format message structure, sets the message type to 3, and the message content is the real data of the file transmitted by the user. The format is assembled into a message and sent to the forwarding device (SYN-SENT). The forwarding device returns a response message (CLOSED or ESTABLISHED) after asynchronously processing the message (5.4, 5.6), and the client actively disconnects after sending the message (FIN_WAIT1).

A virtual network-based file transfer system, including:

A client module, a data forwarding module, a server, a client identification module, a target identification module, a message data forwarding module, a long connection session maintenance module and a flow control module;

The client module is mainly used to log in to the synchronization tool through the Internet to send requests to the cloud computer;

The data forwarding module is mainly used to identify, create, convert and analyze the data of the client and server;

The flow control module performs flow control on the high water level and low water level in the data receiving thread, and after the high water level of the data to be sent in the buffer, suspends the work of the reading thread until the data in the buffer is lower than the low water level, resumes the work of the receiving thread, and continues to receive information;

The target identification module is mainly used to identify the forwarded message type;

The server mainly creates a message according to the message format encapsulation channel and sends it to the data forwarding module;

The message data forwarding module is mainly used to forward the content of different types of message identification.

Specifically, in the step S4, the data of the specific message format received is written into the specified file, wherein the specific writing is as follows:

As shown in Figure 5, the forwarding device receives the message and parses it: according to the specific message format message structure, after the forwarding device receives the message (SYN_RCVD), the message decoder reads the first 4 digits of the message as the source of the message, and only the credible The source only processes the message;

The forwarding device processes the authentication message type: if the message type is 1 or 2, deserialize the parsed message content into a user channel connection authentication message object through JSON;

The forwarding device processes the forwarded message type: if the message type is 3, the content of the message is no longer deserialized, and the destination channel is intelligently matched directly according to the sending channel ID;

Destination end receives data and writes it to disk: the destination end receives the message sent by the forwarding device and writes it directly to the disk. After the message is processed, it actively disconnects to complete the entire data file transfer (FIN_WAIT1).

In this embodiment, the first 4 digits of the message are read by the message decoder as the source of the message, and only credible sources process the message, and continue to read the 2 digits as the message type. Currently, 3 types are used (1-destination end Connection request, 2-client connection request, 3-client forwarding request), continue to read the message length and message sequence number according to the message type, and read the real message according to the message length;

Realize legal authentication of the user channel by processing the message object. The authenticated connection is allowed to establish a channel (ESTABLISHED), and the long-term connection of the channel is saved in the memory. The connection that fails the authentication rejects the connection and disconnects the current connection (CLOSED);

If the message type is 3, the content of the message will no longer be deserialized, and the destination channel will be intelligently matched directly according to the sending channel ID; if the destination is not addressed, the current connection will be disconnected (CLOSED), and the destination will be parsed directly The outgoing message content is forwarded to the destination channel (SYN-SENT).

A virtual network-based file transfer device, comprising:

Sending module: used to receive data in a specific message format sent from the client;

The client creates a message according to the message body format encapsulation channel and sends it to the forwarding device. The forwarding device parses the message body according to the message type, extracts the session ID in the message body, establishes a Socket persistent connection, and stores the current persistent connection channel ID in the memory;

The client encapsulates the real data to be sent into a data forwarding message and sends it to the forwarding device, and the forwarding device parses the message type as a forwarding message.

Forwarding module: used to forward data in a specific message format to the cloud computer;

After the client establishes a connection with the forwarding device, the forwarding device combines the client's channel ID and user ID (user ID+cloud computer ID) into a KV key-value pair and stores it in the client memory object Map.

The second sending module: used to send data in a specific message format to the listening port of the server;

The server creates a message by encapsulating the channel according to the message body format and sends it to the forwarding device. The forwarding device parses the message body according to the message type, extracts the session ID in the message body, establishes a Socket persistent connection, and stores the current persistent connection channel ID in the memory.

Receiving module: used to receive data in a specific message format and write it into a specified file.

After receiving the message sent by the forwarding device, the destination end directly writes it to the disk, and actively disconnects after the message is processed to complete the entire data file transfer (FIN_WAIT1).

When in use, as shown in Figure 6, first, the client logs in to the synchronization tool through the Internet, and the synchronization tool sends a request to the cloud computer business management platform to verify user information and obtain a list of cloud computer information. After successful login, register the synchronization service to the management node. Obtain historical synchronization tasks, tasks that have not been completed and are in the state of synchronization are automatically started, continue to resume the breakpoint, call the industry management interface service to send signaling to the corresponding cloud computer, select the file to be transferred to the designated Taiyun computer, and then pass the cloud computer The mechanism notifies the cloud computer to start the transmission server. If the cloud computer where the server is located has the ability to access the external network, it will directly register with the management node through the external network. If it does not have the external network capability, it will forward the request to the management node through the 100.64.3.254 proxy node , register and establish a long-term connection, the client refreshes the online status of the cloud computer, obtains the directory structure, and starts the server-side transmission service when transferring files. If the cloud computer has external network capabilities, it directly connects to the synchronization node through the external network. If If there is no external network capability, the proxy node forwards the request to the synchronization node through 100.64.3.254, and establishes a long connection. When the server transmission service starts, the client establishes a long connection with the synchronization node through the public network, and sends the transmission data to the synchronization node. The synchronization node automatically matches the server channel to forward the client data to the server to complete data synchronization across network segments. At this time, after the server transmission service receives the data from the synchronization node, it writes the data into a local file with the same path. Complete the entire transfer process of the data file.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is a relationship between these entities or operations. There is no such actual relationship or order between them. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or device.

The preferred embodiments of the invention disclosed above are only to help illustrate the invention. The preferred embodiments are not exhaustive in all detail, nor are the inventions limited to specific embodiments described. Obviously, many modifications and variations can be made based on the contents of this specification. This description selects and specifically describes these embodiments in order to better explain the principle and practical application of the present invention, so that those skilled in the art can well understand and utilize the present invention. The invention is to be limited only by the claims, along with their full scope and equivalents.

Claims (10)

1. A file transmission method based on a virtual network is characterized by comprising the following steps:

s1, receiving data in a specific message format sent by a client;

s2, forwarding the data in the specific message format to a cloud computer;

s3, sending data in a specific message format to a server monitoring port;

and S4, receiving data in a specific message format and writing the data into a specified file.

2. The file transmission method based on the virtual network according to claim 1, wherein: the data of the specific message format consists of a 4-bit length message identification head, a 2-bit length message type, a 4-bit length message length, a 4-bit length message sequence number and message contents.

3. The file transmission method based on the virtual network as claimed in claim 1, wherein: in step S1, data in a specific message format sent from the client is received, where the specific receiving is as follows:

the client packages the channel establishment message according to the message body format and sends the message to the forwarding device, the forwarding device analyzes the message body according to the message type, extracts the session ID in the message body, establishes Socket long connection and stores the current long connection channel ID into the memory;

the client encapsulates real data to be sent into a data forwarding message and sends the data forwarding message to the forwarding device, and the forwarding device analyzes the message type into the forwarding message.

4. The file transmission method based on the virtual network according to claim 3, wherein: after the client is connected with the forwarding device, the forwarding device stores the channel ID and the user identification of the client, and the KV key value pair synthesized by the user ID and the cloud computer ID in the memory object Map of the client.

5. The file transmission method based on the virtual network as claimed in claim 3, wherein: the client side packaging channel creating message sets the message type to be 2 according to a specific message format message structure, the message content is a JSON serialized character string of a user channel connection authentication message object, the message is sent to a forwarding device after the assembly is completed, and the forwarding device returns a response message CLOSED or ESTABISHED after the message is processed.

6. The file transmission method based on the virtual network as claimed in claim 4, wherein: the server side packaging channel creation message sets the message type to be 1 according to a specific message format message structure, the message content is a JSON serialized character string of a user channel connection authentication message object, the message is sent to a forwarding device after the assembly is completed, and the forwarding device returns a response message CLOSED or ESTABLECHED after the message is processed.

7. The file transmission method based on the virtual network according to claim 1, wherein: in step S3, data in a specific message format is sent to the server, where the specific sending is as follows:

the server side creates a message according to the message format encapsulation channel and sends the message to the forwarding device, the forwarding device analyzes the message body according to the message type, the session ID in the message body is extracted, the Socket long connection is established, and the current long connection channel ID is stored in the memory.

8. The file transmission method based on the virtual network as claimed in claim 1, wherein: in step S4, the data write-in specified file in the specific message format is received, where the specific write-in is as follows:

the forwarding device receives and analyzes the message: according to a message structure with a specific message format, after a forwarding device receives a message, a message decoder reads the front 4 bits of the message as a message source, and only a credible source processes the message;

the forwarding device processes the authentication message type: if the message type is 1 or 2, deserializing the analyzed message content into a user channel connection authentication message object through JSON;

the forwarding device processes the forwarding message type: if the message type is 3, the message content is not deserialized, and the destination end channel is intelligently matched directly according to the ID of the sending channel;

the destination receives data and writes into the disk: the target end receives the message sent by the forwarding device and then directly writes the message into the disk, and the connection is actively disconnected after the message is processed, so that the whole data file transmission can be completed.

9. A file transfer system based on a virtual network, comprising:

the system comprises a client module, a data forwarding module, a server, a client identification module, a target identification module, a message data forwarding module, a long connection session holding module and a flow control module;

the flow control module controls the high water level and the low water level in the data receiving thread, suspends the work of the reading thread after the high water level of the data to be sent in the buffer area, resumes the work of the receiving thread until the data in the buffer area is lower than the low water level, and continues to receive the message.

10. A file transfer apparatus based on a virtual network, comprising:

a sending module: the system comprises a client, a server and a server, wherein the client is used for receiving data in a specific message format sent by the client;

a forwarding module: the data forwarding device is used for forwarding data in a specific message format to the cloud computer;

a second sending module: the monitoring port is used for sending data in a specific message format to the server side;

a receiving module: for receiving data written in a specific message format.

Images