CN121387633B

CN121387633B - Data synchronous backup method and system for distributed mobile storage cluster

Info

Publication number: CN121387633B
Application number: CN202511970483.2A
Authority: CN
Inventors: 周正贤; 郭益铭
Original assignee: Shenzhen Haojie Innovation Electronics Co ltd
Current assignee: Shenzhen Haojie Innovation Electronics Co ltd
Priority date: 2025-12-25
Filing date: 2025-12-25
Publication date: 2026-03-06
Anticipated expiration: 2045-12-25
Also published as: CN121387633A

Abstract

The invention discloses a data synchronous backup method and system of a distributed mobile storage cluster, which relate to the technical field of mobile distributed storage and comprise the steps of cooperatively calculating clock offset and drift of each node based on ultra-wideband two-way ranging results of mobile nodes in the distributed mobile storage cluster, forming a distributed relative clock under the conditions of no external time source and fixed master control nodes, calling the distributed relative clock to obtain a logic time stamp when any mobile node generates data change, generating differential information through a local conflict-free replication data engine, binding the logic time stamp with the differential information to form an increment synchronous item and storing the increment synchronous item into a to-be-synchronized list, comparing the maximum logic time stamp in the to-be-synchronized list of the two mobile nodes when the two mobile nodes enter a communication range, and initiating a synchronous request by a backward direction leading party.

Description

Data synchronous backup method and system for distributed mobile storage cluster

Technical Field

The invention relates to the technical field of mobile distributed storage, in particular to a data synchronous backup method and system of a distributed mobile storage cluster.

Background

In the prior art, final consistency models based on version vectors or logic clocks have been widely used, such as Ceph, dynamo, etc. distributed storage systems. Such schemes typically rely on a coordinated global time source (e.g., NTP, PTP) or a centralized coordination node to enable orderly update and conflict merge between copies by logical time stamps or version numbers. In a fixed network environment, the mechanism can effectively ensure the final consistency of data.

However, when the final consistency model is migrated into a mobile ad hoc network consisting of vehicles, drones, portable nodes, etc., its underlying assumptions are no longer true. The method has the advantages that connection among the mobile cluster nodes is intermittent and cannot depend on a unified external time source or a stable master control node, non-negligible offset and drift exist in local clocks of all the nodes, so that the actual sequence relation of events is difficult to be reflected correctly after network splitting and recombination of traditional logic clocks or version vectors, disordered or conflict can occur in data update sequences among the nodes under the condition that no external time source exists, consistency needs to be recovered through additional full comparison or version reordering, transmission overhead and conflict resolution complexity are increased, precious short-time communication opportunities are wasted, and convergence speed and reliability of final consistency are affected, and therefore a method for realizing consistent backup of a mobile distributed storage cluster through a relative clock cooperation and difference data synchronization mechanism among the nodes under the condition that no external time source exists is needed.

Disclosure of Invention

The present invention has been made in view of the above-described problems occurring in the prior art.

Therefore, the invention provides a data synchronous backup method of a distributed mobile storage cluster, which solves the problems of synchronous conflict and low efficiency caused by clock misalignment in a mobile environment.

In order to solve the technical problems, the invention provides the following technical scheme:

In a first aspect, the present invention provides a method for synchronously backing up data in a distributed mobile storage cluster, including,

Based on ultra-wideband two-way ranging results of mobile nodes in the distributed mobile storage cluster, calculating clock offset and drift of each node cooperatively to form a distributed relative clock under the conditions of no external time source and fixed master control node;

When any mobile node changes data, the distributed relative clock is called to acquire a logic time stamp, a local conflict-free copy data engine is used for generating difference information, the logic time stamp is bound with the difference information, and an increment synchronization item is formed and stored in a to-be-synchronized list;

When two mobile nodes enter a communication range, comparing the maximum logic time stamp in the lists to be synchronized of the two mobile nodes, and initiating a synchronous request by a leading party in a backward direction;

And receiving and analyzing the increment synchronization item behind the step, and completing the synchronous backup by applying the difference information through the local conflict-free copy data engine.

As an optimal scheme of the data synchronous backup method of the distributed mobile storage cluster, the method for acquiring the ultra-wideband two-way ranging result comprises the following steps of,

Respectively starting the sending and receiving functions of the ultra-wideband signals for each mobile node in the distributed mobile storage cluster, defining the broadcasting period time, and broadcasting the ultra-wideband signals to other mobile nodes in the detection range in each period;

when each mobile node receives ultra-wideband signals of other mobile nodes, recording a receiving time stamp of the ultra-wideband signals, and correspondingly matching the receiving time stamp with a transmitting time stamp;

and calculating the bidirectional distance based on the round trip propagation delay and the signal propagation rate of the ultra-wideband signals of each mobile node to obtain a bidirectional distance measurement result between the mobile nodes.

As an optimal scheme of the data synchronous backup method of the distributed mobile storage cluster, the method for forming the distributed relative clock comprises the following steps of,

Based on the two-way distance measurement results among the mobile nodes, selecting a plurality of mobile nodes with the smallest distance in the two-way distance measurement results as adjacent mobile nodes, and constructing a distance topological relation by taking the adjacent mobile nodes as vertexes and the distance measurement distance as a weighted edge;

Selecting adjacent mobile nodes, of which the relative change rate of the two-way ranging result variance is lower than the average change level in a plurality of continuous ranging periods and the time change rate variance of the two-way ranging result tends to converge in a sliding time window, as reference nodes by utilizing a distance topological relation;

Calculating initial offset and drift rate of the local clock and the reference node clock based on the two-way ranging time sequence difference between the reference node and the mobile node, and performing joint fitting on the initial offset and drift rate between the local clock and the reference node clock to obtain global clock calibration parameters;

And correcting the difference value of the local clock time based on the global clock calibration parameter, and synchronously adjusting the counting rate of the local clock until the deviation between the local clock and the reference node clock converges to the minimum variation range to form a distributed relative clock without an external time source and under the condition of fixing a main control node.

As a preferable scheme of the data synchronous backup method of the distributed mobile storage cluster, the method for acquiring the logic time stamp comprises the following steps of,

When the data of a certain mobile node is changed, the distributed relative clock acquires the local clock time of the current moment of the mobile node;

and correcting the difference value of the local clock time according to the global clock calibration parameters recorded in the distributed relative clock to obtain a globally calibrated logic time value, and taking the logic time value as a logic time stamp.

As an optimal scheme of the data synchronous backup method of the distributed mobile storage cluster, the method for forming the incremental synchronous entries into the to-be-synchronized list comprises the following steps of,

Executing state snapshot on the current local storage data of the mobile node with the changed data to obtain a static data mirror image, taking the static data mirror image as data before the change, comparing the static data mirror image with the changed data content, and outputting the data granularity difference;

When the difference information is extracted, recording a logic time stamp when the data change occurs, and taking the logic time stamp as a unique time sequence identifier of the difference information;

binding the logic time stamp with the corresponding difference information to form an increment synchronization item capable of uniquely identifying the data change sequence and the content, and writing the increment synchronization item into a list to be synchronized according to the logic time stamp.

As a preferable scheme of the data synchronous backup method of the distributed mobile storage cluster, the method for comparing the maximum logic time stamp in the list to be synchronized of the two parties comprises the following steps of,

After two mobile nodes enter a communication range, respectively reading all increment synchronization items in a list to be synchronized of the two mobile nodes, and extracting a logic time stamp carried by each increment synchronization item;

based on the extracted logical time stamps, respectively obtaining the maximum logical time stamps in the lists to be synchronized of the two mobile nodes, and using the maximum logical time stamps as time sequence identifiers for the latest data change of the two mobile nodes;

comparing the maximum logic time stamp of one mobile node with the maximum logic time stamp of the other mobile node, wherein the mobile node with the large logic time stamp is the leading party, and the mobile node with the large logic time stamp is the lagging party.

As an optimized scheme of the data synchronous backup method of the distributed mobile storage cluster, the method for searching incremental synchronous items with logic time stamps larger than the largest logic time stamp behind in the list to be synchronized by the leading party and sending the incremental synchronous items to the behind according to the logic time stamp sequence comprises the following steps,

The backward part initiates a synchronous request to the leading part by taking the maximum logic time stamp as a reference, and the request content comprises a local maximum logic time stamp and a list to be synchronized;

After receiving the synchronization request, the leading party searches all increment synchronization items with the logic time stamp larger than the maximum logic time stamp behind from the self list to be synchronized based on the maximum logic time stamp behind carried in the synchronization request;

and the leading party sorts the retrieved increment synchronous entries according to the sequence of the logic time stamps, and sends the increment synchronous entries to the trailing party one by one.

As a preferable scheme of the data synchronous backup method of the distributed mobile storage cluster, the method for receiving and analyzing the increment synchronous entries behind the method comprises the following steps of,

Based on the read logical time stamp, caching each increment synchronization item according to the time sequence of the logical time stamp to form a queue to be analyzed;

And analyzing increment synchronization items one by one according to the sequence of the logic time stamps from the queue to be analyzed, and extracting the difference information in the increment synchronization items.

As a preferable scheme of the data synchronous backup method of the distributed mobile storage cluster, the invention comprises the following steps:

the method of applying delta information by a local collision-free replication data engine includes,

Determining a corresponding data change sequence according to a logic time stamp in the difference information by a local conflict-free copy data engine, and sequentially executing difference update on local storage data of the backward mobile node according to the data change sequence;

When detecting that the data related to the difference information has a modification record locally, the local conflict-free copy data engine compares the logic time stamps of the change records of the two parties and performs conflict-free combination according to the sequence of the logic time stamps;

The local conflict-free copy data engine updates a state snapshot of local storage data of the backward mobile node, synchronously writes the latest logic time stamp into a list to be synchronized, and generates a data consistency view of the current mobile node based on the updated state snapshot.

In a second aspect, the present invention provides a data synchronous backup system for a distributed mobile storage cluster, comprising,

The clock synchronization module is used for cooperatively calculating clock offset and drift of each node based on ultra-wideband two-way ranging results of mobile nodes in the distributed mobile storage cluster to form a distributed relative clock under the condition of no external time source and fixed master control node;

The system comprises a data generation module, a local conflict-free copying data engine, a data generation module, a data synchronization module and a data synchronization module, wherein the data generation module is used for calling a distributed relative clock to acquire a logic time stamp when any mobile node generates data change;

The synchronous coordination module is used for comparing the maximum logic time stamp in the lists to be synchronized of the two mobile nodes when the two mobile nodes enter the communication range, and initiating a synchronous request by a leading party in the backward direction;

And the synchronization module is used for receiving and analyzing the increment synchronization items at the backward position, and completing the synchronization backup by applying the difference information through the local conflict-free copy data engine.

The invention has the beneficial effects that by introducing a distributed relative clock formed based on a two-way ranging result into a mobile cluster, self-organizing time coordination under the conditions of no external time source and fixed master control nodes is realized, each mobile node can self-calibrate local clock offset and drift according to ranging topology and time sequence difference, so that uniform logic time sequence reference is maintained in a dynamic network, and meanwhile, granularity difference before and after data change is bound with logic time stamp in a delta information form to generate an increment synchronization item through a conflict-free copy and delta synchronization mechanism driven by logic time stamp, so that synchronous backup can be completed only by transmitting necessary change contents between nodes. The method and the device are combined, so that the method and the device can ensure the global consistency of event time sequences in a mobile environment with intermittent communication and frequent topology change, reduce redundancy and conflict of data synchronization, realize consistent backup of data with low cost and high robustness, and remarkably improve the autonomous synchronization capability and consistency maintenance efficiency of the mobile distributed storage cluster.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a method for synchronously backing up data of a distributed mobile storage cluster according to the present invention;

FIG. 2 is a schematic diagram of a data synchronous backup system of a distributed mobile storage cluster according to the present invention;

FIG. 3 is a flow chart of generating a distributed relative clock in the present invention;

FIG. 4 is a flow chart of generating incremental sync entries in the present invention.

Detailed Description

In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways other than those described herein, and persons skilled in the art will readily appreciate that the present invention is not limited to the specific embodiments disclosed below.

Further, reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic can be included in at least one implementation of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.

Referring to fig. 1,2, 3 and 4, an embodiment of the present invention provides a data synchronization backup method of a distributed mobile storage cluster, including the following steps:

the method for acquiring the ultra-wideband two-way ranging result comprises,

And defining broadcasting period time, and broadcasting the ultra-wideband signals to other mobile nodes in the detection range in each period.

Specifically, each mobile node enters a standby synchronous state after being started, detects the occupation condition of surrounding channels, automatically selects a frequency band (3.1-4.8 GHz) with lower interference as a working channel, and then verifies the stability and the receiving sensitivity of a communication link by sending a test pulse packet. After confirming that the link establishment is stable, the mobile node loads a unique identifier and a local initial timestamp into an ultra-wideband signal load;

Furthermore, the broadcasting period time can be dynamically configured according to the factors such as the node moving speed, the ranging accuracy requirement, the channel occupancy rate and the like. For example, when the mobile node is in a low-speed or stationary state, the broadcasting period time can be set to 500 ms-1 s to reduce the communication overhead, while in a scenario that the mobile node moves at a high speed or the topology changes rapidly, the broadcasting period time can be shortened to 100 ms-200 ms, in each broadcasting period, the mobile node actively broadcasts a UWB signal packet carrying self-identification and timestamp information to other mobile nodes within the detection range, and the other mobile nodes return response packets immediately after receiving the broadcast signal.

When each mobile node receives ultra wideband signals of other mobile nodes, the receiving time stamp of the ultra wideband signals is recorded, the receiving time stamp is correspondingly matched with the transmitting time stamp, and the round trip propagation delay of the ultra wideband signals is calculated based on the matched transmitting time stamp and receiving time stamp.

It should be noted that, when the mobile node receives the ultra wideband signals sent by other mobile nodes, it will read the sending time stamp carried in the signals and record the local receiving time stamp at the same time, and then match one-to-one according to the unique identifier (including the sending node ID and the serial number) in the signal packet. The receiving mobile node searches a sending record consistent with the serial number in a local record table, and pairs the receiving time stamp with the corresponding sending time stamp. Based on the time stamp obtained by matching, the round trip propagation delay of the ultra-wideband signal is calculated by combining the response time difference during the round trip of the signal, and the calculation formula is as follows:

;

wherein, the Is the round trip propagation delay of the ultra wideband signal,Is the timestamp of the response signal received by the original transmitting mobile node,Is the time stamp of the sending mobile node sending the ultra wideband signal,Is the timestamp of the response signal sent by the receiving mobile node,Is the time stamp of the reception of the signal by the receiving mobile node.

Calculating a bidirectional distance based on the round trip propagation delay and the signal propagation rate of the ultra-wideband signals of each mobile node to obtain a bidirectional distance measurement result between the mobile nodes;

It should be noted that the expression for calculating the bidirectional distance is:

;

wherein, the Is a mobile nodeWith mobile nodesThe distance between the two-way lines,Is the transmission rate of ultra-wideband signals in the air)。

The method of forming the distributed relative clock includes,

The clock offset and drift among the nodes are influenced by the motion state and channel fluctuation, so that the logic time is inconsistent, the time drift accumulation causes the event ordering error and the synchronization delay to be increased, and further the distributed data consistency and the reliability of the cooperative operation are influenced.

And constructing a distance topological relation by taking the adjacent mobile nodes as vertexes and the distance measurement distance as a weighted edge.

It should be noted that, by indexing the two-way ranging results obtained by calculation between each pair of mobile nodes according to the number of the mobile nodes and sequentially filling the two-way ranging results into the corresponding matrix row and column positions to construct a complete inter-mobile node distance matrix;

Further, all adjacent mobile nodes are taken as vertexes, a mobile node pair with a ranging error lower than that of the exemplary 10 cm is selected according to the distance matrix between the adjacent mobile nodes, and a weighted undirected graph is constructed by taking the ranging result as a weighted edge weight, so that a distance topological relation reflecting the spatial distribution of the nodes is formed, wherein the smaller the edge weight is, the closer the physical distance between the nodes is.

And selecting adjacent mobile nodes, of which the relative change rate of the two-way ranging result variance is lower than the average change level in a plurality of continuous ranging periods and the two-way ranging result time change rate variance tends to converge in a sliding time window, as reference nodes by utilizing the distance topological relation.

It should be noted that, the calculation formula of the two-way ranging result variance in a plurality of continuous ranging periods is:

;

wherein, the Is a mobile nodeWith mobile nodesBetween and are continuousThe two-way ranging result variance over the ranging period,Is the calculated total number of consecutive ranging periods,Is the firstMobile node in secondary ranging periodWith mobile nodesIs used for the two-way ranging result,Is a mobile nodeWith mobile nodesAt the average value of the continuous ranging results,AndIs any two different mobile nodes and,Is the sequence number of the current ranging period;

the decision formula for the two-way ranging result variance with the relative change rate lower than the average change level in a plurality of continuous ranging periods is as follows:

;

wherein, the Is indicative of a mobile node pairIs a function of the relative rate of change of the ranging variance,Is the firstTwo-way ranging result variances within a sliding time window,Is the firstTwo-way ranging result variances within a sliding time window,Is the average of the two-way ranging result variances relative to the rate of change for all mobile nodes,Is the sequence number of the sliding time window;

the time change rate variance calculation formula of the two-way ranging result is as follows:

;

wherein, the Is a mobile node pairIs the variance of the time rate of change of the two-way ranging result,Is the firstSecondary and firstThe rate of change between the secondary two-way ranging results,Is a mobile node pairThe average rate of change of the two-way ranging results over a continuous plurality of ranging periods,AndIs a mobile node pair with adjacent two ranging periodsIs used for the two-way ranging result,Is the time interval between two adjacent ranging periods;

the time change rate variance tends to converge in the sliding time window;

;

wherein, the Is the current sliding time window (the firstIndividual) the variance of the time rate of change of the two-way ranging result,Is the previous sliding time window (the firstIndividual) the variance of the time rate of change of the two-way ranging result,Is a time rate of change variance convergence threshold (exemplary 0.05);

When the number of windows continuously meeting the time change rate variance convergence threshold exceeds 80% of the example, determining that the variance of the two-way ranging result time change rate of the mobile node pair tends to converge within the sliding time window;

Through the process, the adjacent mobile nodes with high ranging stability, small variance fluctuation and convergence trend of the time change rate variance in the continuous ranging period can be screened out, and the adjacent mobile nodes are determined as reference nodes.

Calculating initial offset and drift rate of the local clock and the reference node clock based on the two-way ranging time sequence difference between the reference node and the mobile node; and performing joint fitting on the initial offset and the drift rate between the local clock and the reference node clock to obtain global clock calibration parameters.

It should be noted that the expression for calculating the initial offset of the local clock and the reference node clock is:

;

wherein, the Is the firstMobile node in secondary ranging periodLocal clock and reference nodeThe initial offset between the clocks is set to be equal,Is the firstSecondary ranging mobile nodeThe time stamp recorded under the local clock,Is the firstSecondary ranging mobile nodeA timestamp recorded under the local clock;

The drift rate calculation formula between the local clock and the reference node clock is as follows:

;

wherein, the Is a mobile nodeLocal clock and reference nodeThe rate of the clock drift is such that,The initial offsets of clocks in two adjacent ranging periods,Is the time interval of the reference node during two adjacent ranging weeks;

Further, the waste water is collected in a continuous state Mobile node during a ranging periodWith each reference nodeInitial clock offset sequence therebetweenAnd clock drift rate sequence;

For each reference nodeAssigning a weight valueThe weight is determined based on the ranging stability of the reference node in the sliding time window, wherein the higher the ranging stability is, the larger the weight value is allocated;

Based on Performing weighted joint fitting on initial offsets and drift rates from all reference nodes, in particular mobile nodesGlobal clock drift correction factor of (a)And global clock offset correction valueRespectively calculating the average value of the sequences corresponding to each reference nodeAnd (3) with) Obtained by a weighted average of (c), the calculation formula is:

;

Wherein the summation traverses all reference nodes Thereby, it is possible to obtain a mobile nodeGlobal clock calibration parameters for clock calibration。

The mobile node uses the global clock offset correction value to compensate the current reading of the local clock in real time, adjusts the crystal oscillator driving frequency of the local clock or the increment step length of the software timer according to the global clock offset correction factor in proportion, repeatedly executes the calculation process of the clock offset and the drift rate in each subsequent ranging period, and compares the new calculation result with the currently used global clock calibration parameter;

When the calculated absolute value of the offset between the local clock and the reference node clock is maintained below a preset offset threshold value, for example, the absolute value of the offset is continuously lower than 100 microseconds, and the variation trend of the drift rate tends to be stable, for example, the absolute value of the drift rate is smaller than 0.1ppm and the variation of the drift rate of adjacent periods is smaller than 0.01ppm, the local clock of the mobile node is judged to be calibrated. The local clock of the mobile node becomes a part of the distributed relative clock, and all the mobile nodes finish the local clock calibration according to the same flow to jointly form the distributed relative clock without external time source and under the condition of fixed master control node.

The distance topological relation is constructed based on two-way ranging, stable reference nodes are selected, offset and drift joint fitting is carried out, a self-calibrated distributed relative clock is formed, and dynamic alignment of time references among the nodes is achieved; the mechanism can maintain clock consistency without an external time source, and improves the data synchronization precision and the cooperative stability of the mobile cluster under the dynamic topology.

The method of obtaining a logical timestamp includes,

When the data of a certain mobile node is changed, the distributed relative clock acquires the local clock time of the current moment of the mobile node.

When the distributed relative clock obtains the local clock time of the mobile node at the current time, the mobile node directly reads the current count value of the internal timer, and takes the count value as the local clock time of the current time.

It should be noted that, the distributed relative clock performs a difference correction operation on the local clock time according to the recorded global clock calibration parameter, and obtains a globally calibrated logic time value by multiplying the local clock time by a drift rate compensation factor and adding an offset correction value, where the drift rate compensation is used to correct the timing rate error, and the offset correction is used to eliminate the initial time deviation, so that the logic time can accurately reflect the globally unified time sequence.

The method for forming the increment synchronization entry into the to-be-synchronized list includes,

The existing distributed data synchronization method is dependent on full data comparison or centralized time stamping, cannot accurately capture data difference under the environments of frequent change of mobile nodes and unstable network, lacks of fine recognition of granularity levels before and after data change, and causes high synchronous data redundancy and bandwidth occupation, and meanwhile, time stamps are attached to local time of nodes, so that time sequence disorder and conflict update risks exist, and data consistency and synchronization efficiency are affected.

And executing a state snapshot on the current local storage data of the mobile node with the changed data to obtain a static data mirror image, taking the static data mirror image as the data before the change, comparing the static data mirror image with the changed data content, and outputting the data granularity difference.

After the change data carrying the logic timestamp is input into a local conflict-free copy data engine, the conflict-free copy data engine immediately invokes a state snapshot mechanism when detecting a write-in or update operation trigger signal, and executes complete copy on a data object currently stored locally by a mobile node;

Further, when the static data mirror image is used as data before modification and is compared with the content of the data after modification, the old value recorded in the static data mirror image is compared with the new value stored in the storage after modification based on the mapping relation between the field level of the data object and the storage key value, the content difference is detected according to the data granularity (field, record or node level) in the comparison process, and the field identification, the change type and the change value which are changed are extracted to form the data granularity difference.

And when the difference information is extracted, recording a logic time stamp when the data change occurs, and taking the logic time stamp as a unique time sequence identifier of the difference information.

It should be noted that, by traversing corresponding fields or records of static data mirror images and changed data objects, comparing old values and new values of each field or node, identifying specific change content including field identification, change type (new addition, deletion and modification) and change value, then encoding the identified change content according to its position and hierarchy in the data structure, generating differential information, forming unique corresponding data differential entries based on the differential information, recording logical time stamps corresponding to the occurrence of data changes while generating differential entries, binding the logical time stamps with the differential entries, and making each differential information have unique time sequence identification.

When the logic time stamp and the corresponding differential information are bound, the logic time stamp and the differential information are stored in the structure by adopting the key value, the logic time stamp is used as a key, the differential information is used as a value, after the binding is completed, the formed increment synchronous entries are sequentially written into the tail position of the list to be synchronized according to the ascending order of the logic time stamp, and the list to be synchronized is maintained by adopting a first-in first-out queue structure, so that the increment synchronous entries are ensured to be stored according to the sequence of the logic time stamp.

The method realizes the granularity level identification of data change and accurate time sequence binding by a differential extraction mechanism based on a logic time stamp, so that each change has a unique logic sequence, and by combining a local conflict-free copy data engine, incremental synchronous entries can be quickly generated under a centerless coordination condition, thereby remarkably reducing redundant transmission quantity and synchronous delay and improving the controllability and reliability of distributed data consistency and a synchronous process.

The method for comparing the maximum logical time stamps in the list to be synchronized of both parties includes,

And after the two mobile nodes enter the communication range, respectively reading all the increment synchronization items in the lists to be synchronized of the two mobile nodes, and extracting the logic time stamp carried by each increment synchronization item.

It should be noted that, the mobile node first accesses the local database or the memory cache area storing the increment synchronization entries, and traverses all the increment synchronization entries in the list to be synchronized. For each incremental synchronization entry in the list to be synchronized, the mobile node parses its data structure format, which typically contains a header field and payload data. In the parsing process, the mobile node locates to a specific field position storing the logic timestamp in the data structure, extracts the timestamp value in binary or numerical form of the specific field, and obtains the logic timestamp carried by each increment synchronization item.

Based on the extracted logical time stamps, the maximum logical time stamps in the lists to be synchronized of the two mobile nodes are respectively obtained and used as time sequence identifiers of the latest data changes of the two mobile nodes.

It should be noted that, all the logical time stamps extracted from the list to be synchronized are compared, and the logical time stamp with the largest value is selected as the largest logical time stamp of the mobile node, where the largest logical time stamp represents the logical time when the last data change of the mobile node occurs.

It should be noted that, two mobile nodes exchange their maximum logical timestamp values through communication, and compare the values of the two maximum logical timestamps, and mark the mobile node with the larger logical timestamp value as the leading party and the mobile node with the smaller logical timestamp value as the trailing party.

The leading party retrieves incremental synchronization entries in the to-be-synchronized list having a logical timestamp greater than a maximum logical timestamp of the trailing party, and sends the incremental synchronization entries to the trailing party in logical timestamp order, including,

And the backward part initiates a synchronous request to the leading part by taking the maximum logic time stamp as a reference, wherein the request content comprises the local maximum logic time stamp and a list to be synchronized.

It should be noted that, when the backward mobile node encapsulates the synchronization request message, the maximum logic timestamp value recorded in the local to-be-synchronized list is used as a core parameter, and meanwhile, the summary information of the to-be-synchronized list is packaged together. The summary information includes the total number of incremental sync entries currently stored in the list to be synchronized, the percentage of list storage capacity occupied, and the logical timestamp of the earliest one of the incremental sync entries. The synchronization request message is organized in a particular data frame format and sent to the lead mobile node over an established communication link, such as an ultra wideband wireless channel, between the mobile nodes. The data frame format comprises three parts of a frame head, load data and a frame tail check code, wherein the frame head defines the type of a message as a synchronous request, the load data field sequentially stores parameters such as a maximum logic time stamp, a total number of list entries, a list capacity percentage, an earliest logic time stamp and the like, and the frame tail adopts a cyclic redundancy check code to ensure the integrity of data transmission.

After receiving the synchronization request, the leading party retrieves all increment synchronization items with the logic time stamp larger than the lag maximum logic time stamp from the self list to be synchronized based on the lag maximum logic time stamp carried in the synchronization request.

Specifically, the leading mobile node checks the received data stream, and first checks whether the data stream length reaches the expected length defined by the frame header. Then, the leading mobile node calculates the cyclic redundancy check code of the load data in the data stream, and compares the calculated result with the check code carried by the frame tail of the data stream, if the calculated result is consistent with the check code carried by the frame tail of the data stream, the synchronous request message format is judged to be complete and the transmission is correct;

Further, when the leading mobile node parses the synchronization request message body, the leading mobile node locates the field offset address storing the maximum logical timestamp behind according to the frame structure of the synchronization request message. The leading mobile node reads the byte data with fixed length from the address, decodes the byte data into integer values according to a predefined coding rule (such as big endian or little endian byte order), wherein the integer values are specific values of the trailing maximum logic time stamp;

furthermore, the leading mobile node uses the specific value of the maximum logical timestamp behind as a filtering condition, performs full-quantity scanning operation on the local to-be-synchronized list, compares the logical timestamp bound by each increment synchronization item in the to-be-synchronized list item by item, and screens out all items meeting the condition that the logical timestamp is greater than the maximum logical timestamp behind to form a to-be-synchronized item set.

Specifically, the leading mobile node converts the ordered incremental synchronization entries one by one into a format suitable for network transmission. Each incremental sync entry is encapsulated as an independent data packet, the data packet structure containing a header, a payload, and a trailer. The packet header field contains a destination mobile node identifier, a source mobile node identifier, a data packet sequence number, a total packet number, and a current packet sequence number. The payload field stores the complete contents of the delta-sync entry, including the logical timestamp and corresponding delta information. The tail field contains a cyclic redundancy check code for error detection;

The leading mobile node sequentially sends the serialized data packets to the trailing mobile node through the established ultra-wideband communication link. The transmission process adopts a transmission protocol with an acknowledgement mechanism, for example, after a single data packet is transmitted, the leading mobile node waits for the trailing mobile node to return an acknowledgement receiving signal, and if the acknowledgement signal is not received within a set time limit, the leading mobile node retransmits the data packet until the transmission is successful or the maximum retry number is reached. All data packets are transmitted strictly following the ordered sequence, ensuring that the following mobile node can receive and process incremental sync entries in the logical timestamp sequence.

The method of receiving and parsing delta sync entries behind includes,

And caching the increment synchronization items according to the time sequence of the logic time stamp based on the read logic time stamp to form a queue to be analyzed.

It should be noted that, during the receiving process, the backward mobile node immediately analyzes the packet header of each successfully received and checked data packet to obtain the sequence number of the data packet, and re-obtains the complete incremental synchronization entry. The laggard mobile node then parses the header structure of the delta-sync entry and reads the logical timestamp value encoded therein. When newly arrived increment synchronous items are inserted, the correct insertion position of the items in the queue is determined by comparing the logic timestamp values, thereby ensuring that the items in the queue to be analyzed always follow the global logic time sequence.

It should be noted that, the processing thread of the backward mobile node continuously checks the queue to be resolved. When the queue to be parsed is not empty, the processing thread fetches one increment synchronization entry from the head of the queue (i.e., the position where the logical timestamp value is the smallest). The backward mobile node performs deep parsing of the entry, including verifying the entry integrity, decoding the compressed format (if any) of the delta information, and restoring the data change operations (e.g., field updates, record inserts or deletes) contained in the delta information to an explicit set of operation instructions. The resolved differential information operation instruction set and the corresponding logic time stamp are temporarily stored in the memory to wait for being applied to the local storage.

Methods of applying delta information by a local collision-free replication data engine include,

Determining a corresponding data change sequence according to the logic time stamp in the difference information by a local conflict-free copy data engine; and sequentially performing delta updates on the locally stored data of the backward mobile node according to the data change order.

It should be noted that, the local conflict-free replication data engine maintains an operation log, and each delta information to be applied is assigned a globally unique serial number according to its logical timestamp. The local conflict-free replication data engine creates an ordered execution plan according to the strict ordering of the serial numbers. In the application process, the local conflict-free copy data engine decodes the difference information into specific database operation instructions, such as PUT and DELETE operations aiming at key value storage or field level UPDATE operations aiming at a document database, and sequentially executes the database operation instructions in a local storage transaction to ensure that each change is completed in an atomic operation, thereby accurately reproducing the data change of a leading party into a local storage behind the leading party.

When detecting that the data related to the difference information has a modification record locally, the local conflict-free copy data engine compares the logic time stamps of the change records of the two parties and performs conflict-free combination according to the sequence of the logic time stamps.

It should be noted that before applying the change, the conflict-free replication data engine will query the metadata of the target data item to check whether there is a modification record locally generated by the lag behind that the logical timestamp is later than the last synchronization reference but earlier than the entry currently to be applied, and when such a conflict is detected, the conflict-free replication data engine starts the merge process, i.e. first unconditionally applies the change with the logical timestamp larger because it represents a more recent event, and for conflicts with the same logical timestamp (very low probability event), a deterministic algorithm based on the change source node identifier (such as prioritized by the node identifier dictionary) is adopted to decide to generate the merged result. The merged data item will be marked as resolved conflict state and the latest logical timestamp corresponding to the merged data item will be recorded.

It should be noted that, locking the local storage data of the backward mobile node to ensure the data stationarity, then traversing all the data items in the backward mobile node, recording the complete content, the metadata and the corresponding latest logic time stamp of each data item, and serializing the complete content, the metadata and the corresponding latest logic time stamp to generate a new state snapshot file with a version identifier;

Furthermore, after the state snapshot is generated, the local conflict-free copy data engine loads the latest snapshot file, analyzes and index-reconstructs all data items and metadata thereof contained in the latest snapshot file, in the analysis process, the conflict-free copy data engine carries out global sorting on the data items according to the logic time stamps to ensure consistency of the data items in time dimension, only the latest valid version is reserved for the data items with multiple versions or repeated records according to the new and old relations of the logic time stamps and conflict resolution strategies, the latest valid version is marked as a current visible state, a key-to-version mapping table is established for key-value type data, the dependency relation and parent-child reference relation among reconstructed fields are guaranteed for document type or hierarchical type data, integrity and reference correctness of snapshot internal structures are guaranteed, and all the valid data items, the latest metadata thereof and the corresponding logic time stamps are uniformly packaged into a consistent mapping structure to form a data consistency view of the current mobile node.

The embodiment also provides a data synchronous backup system of the distributed mobile storage cluster, which comprises:

The embodiment also provides computer equipment, which is suitable for the situation of the data synchronous backup method of the distributed mobile storage cluster, and comprises a memory and a processor, wherein the memory is used for storing computer executable instructions, and the processor is used for executing the computer executable instructions to realize the data synchronous backup method of the distributed mobile storage cluster.

The computer device may be a terminal comprising a processor, a memory, a communication interface, a display screen and input means connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

The present embodiment also provides a storage medium having a computer program stored thereon, which when executed by a processor implements a method for implementing data synchronous backup of a distributed mobile storage cluster as proposed in the above embodiment, where the storage medium may be implemented by any type of volatile or non-volatile storage device or combination thereof, such as a static random access Memory (Static Random Access Memory, SRAM for short), an electrically erasable Programmable Read-Only Memory (ELECTRICALLY ERASABLE PROGRAMMABLE READ-Only Memory, EEPROM for short), an erasable Programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM for short), a Programmable Read-Only Memory (PROM for short), a Read-Only Memory (ROM for short), a magnetic Memory, a flash Memory, a magnetic disk, or an optical disk.

In summary, the invention realizes self-organizing time coordination under the conditions of no external time source and fixed master control nodes by introducing a distributed relative clock formed based on a two-way ranging result into a mobile cluster, each mobile node can self-calibrate local clock offset and drift according to ranging topology and time sequence difference so as to maintain uniform logic time sequence reference in a dynamic network, and meanwhile, granularity difference before and after data change is bound with logic time stamp in a delta information form to generate increment synchronization items through a conflict-free copy and delta synchronization mechanism driven by logic time stamp, so that synchronous backup can be completed only by transmitting necessary change contents between nodes. The method and the device are combined, so that the method and the device can ensure the global consistency of event time sequences in a mobile environment with intermittent communication and frequent topology change, reduce redundancy and conflict of data synchronization, realize consistent backup of data with low cost and high robustness, and remarkably improve the autonomous synchronization capability and consistency maintenance efficiency of the mobile distributed storage cluster.

It should be noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention may be modified or substituted without departing from the spirit and scope of the technical solution of the present invention, which is intended to be covered in the scope of the claims of the present invention.

Claims

1. A data synchronization and backup method for a distributed mobile storage cluster, characterized in that it includes:

Based on the ultrawideband bidirectional ranging results of mobile nodes in the distributed mobile storage cluster, the clock offset and drift of each node are calculated collaboratively to form a distributed relative clock under the condition of no external time source and fixed master control node. The specific steps are as follows.

For each mobile node in the distributed mobile storage cluster, the transmission and reception functions of ultra-wideband signals are activated; and a broadcast period is defined to broadcast ultra-wideband signals to other mobile nodes within the detection range in each period.

When each mobile node receives an ultra-wideband signal from another mobile node, it records the reception timestamp of the ultra-wideband signal and matches the reception timestamp with the transmission timestamp; based on the matched transmission timestamp and reception timestamp, it calculates the round-trip propagation delay of the ultra-wideband signal.

Based on the round-trip propagation delay and signal propagation rate of the ultra-wideband signal of each mobile node, the two-way distance is calculated to obtain the two-way ranging results between the mobile nodes.

Based on the bidirectional ranging results between each mobile node, several mobile nodes with the smallest distance in the bidirectional ranging results are selected as adjacent mobile nodes; and a distance topology relationship is constructed with adjacent mobile nodes as vertices and ranging distances as weighted edges.

Using distance topology, adjacent moving nodes that have a relative rate of change of variance of bidirectional ranging results that is lower than the average rate of change in multiple consecutive ranging cycles, and whose time rate of change variance of bidirectional ranging results tends to converge within the sliding time window, are selected as reference nodes.

Based on the difference in bidirectional ranging time series between the reference node and the mobile node, the initial offset and drift rate between the local clock and the reference node clock are calculated; and the initial offset and drift rate between the local clock and the reference node clock are jointly fitted to obtain the global clock calibration parameters.

Based on the global clock calibration parameters, the local clock time is corrected for the difference, and the counting rate of the local clock is adjusted synchronously until the deviation between the local clock and the reference node clock converges to the minimum range, forming a distributed relative clock under the condition of no external time source and fixed master node.

When any mobile node experiences a data change, it calls the distributed relative clock to obtain a logical timestamp; it generates differential information through the local conflict-free data replication engine; and it binds the logical timestamp with the differential information to form an incremental synchronization entry and stores it in the synchronization list.

When two mobile nodes enter the communication range, they compare the maximum logical timestamp in their respective synchronization lists. The lagging node initiates a synchronization request to the leading node. The leading node retrieves incremental synchronization entries in the synchronization list whose logical timestamps are greater than the lagging node's maximum logical timestamp and sends them to the lagging node in logical timestamp order.

The lagging party receives and parses the incremental synchronization entries, applies the differential information through the local conflict-free replication data engine, and completes the synchronization backup.

2. The data synchronization and backup method for a distributed mobile storage cluster as described in claim 1, characterized in that: the method for obtaining the logical timestamp includes,

When the data of a mobile node changes, the distributed relative clock obtains the local clock time of that mobile node at the current moment;

Based on the global clock calibration parameters recorded in the distributed relative clock, the local clock time is corrected for the difference to obtain the globally calibrated logical time value; and the logical time value is used as the logical timestamp.

3. The data synchronization and backup method for a distributed mobile storage cluster as described in claim 2, characterized in that: the method of forming incremental synchronization entries and storing them in the list to be synchronized includes,

Input the changed data with logical timestamps into the local conflict-free replication data engine; perform a state snapshot on the current local storage data of the data-changed moving node to obtain a static data image; use the static data image as the data before the change and compare it with the data content after the change to output the data granularity difference.

Based on the differences in data granularity, the content identifiers before and after the current mobile node change are compared to extract the difference information. When extracting the difference information, the logical timestamp when the data change occurs is recorded, and this logical timestamp is used as the unique time sequence identifier of the difference information.

Bind the logical timestamp with the corresponding differential information to form an incremental synchronization entry that can uniquely identify the order and content of data changes; write the incremental synchronization entry into the list to be synchronized according to the logical timestamp.

4. The data synchronization and backup method for a distributed mobile storage cluster as described in claim 3, characterized in that: the method for comparing the largest logical timestamp in the lists to be synchronized by both parties includes,

Once two mobile nodes enter the communication range, read all incremental synchronization entries in the synchronization list of the two mobile nodes respectively, and extract the logical timestamp carried by each incremental synchronization entry.

Based on the extracted logical timestamps, the maximum logical timestamps in the synchronization lists of the two mobile nodes are obtained respectively, and used as the time sequence identifiers of the latest data changes of the two mobile nodes.

Compare the maximum logical timestamp of one moving node with the maximum logical timestamp of the other moving node; the moving node with the larger logical timestamp is the leading party, and the other moving node is the lagging party.

5. The data synchronization and backup method for a distributed mobile storage cluster as described in claim 4, characterized in that: the method by which the leading party retrieves incremental synchronization entries in the synchronization list whose logical timestamps are greater than the maximum logical timestamp of the lagging party, and sends them to the lagging party in logical timestamp order, includes,

The lagging party initiates a synchronization request to the leading party based on its own maximum logical timestamp. The request includes the local maximum logical timestamp and the list to be synchronized.

After receiving a synchronization request, the leading party retrieves all incremental synchronization entries whose logical timestamps are greater than the maximum logical timestamp of the lagging party from its own list of pending synchronization, based on the maximum logical timestamp of the lagging party carried in the synchronization request.

The leading party sorts the retrieved incremental synchronization entries according to their logical timestamps and sends them to the lagging party one by one.

6. The data synchronization and backup method for a distributed mobile storage cluster as described in claim 5, characterized in that: the method for the lagging party to receive and parse incremental synchronization entries includes,

After receiving an incremental synchronization entry, the lagging mobile node reads the logical timestamp of each incremental synchronization entry in sequence; based on the read logical timestamps, it caches each incremental synchronization entry in the order of the logical timestamps to form a queue to be parsed.

The incremental synchronization entries are parsed one by one from the queue to be parsed in order of logical timestamp, and the differential information is extracted.

7. The data synchronization and backup method for a distributed mobile storage cluster as described in claim 1, characterized in that: the method of applying differential information through a local conflict-free replication data engine includes,

The local conflict-free data replication engine determines the corresponding data change order based on the logical timestamp in the differential information; and performs differential updates on the local storage data of the lagging moving node in sequence according to the data change order.

When it is detected that the data involved in the difference information has already been modified locally, the local conflict-free replication data engine compares the logical timestamps of the change records of both parties and performs a conflict-free merge based on the order of the logical timestamps.

The local conflict-free replication data engine updates the state snapshot of the local storage data of the lagging mobile node and synchronously writes the latest logical timestamp into the list to be synchronized; based on the updated state snapshot, a data consistency view of the current mobile node is generated.

8. A data synchronization and backup system for a distributed mobile storage cluster, based on the data synchronization and backup method for a distributed mobile storage cluster according to any one of claims 1 to 7, characterized in that it includes:

The clock synchronization module is used to collaboratively calculate the clock offset and drift of each node based on the ultra-wideband bidirectional ranging results of mobile nodes in the distributed mobile storage cluster, so as to form a distributed relative clock under the condition of no external time source and fixed master control node.

The differential generation module is used to obtain a logical timestamp by calling a distributed relative clock when any mobile node undergoes a data change; generate differential information through a local conflict-free data replication engine; and bind the logical timestamp with the differential information to form an incremental synchronization entry and store it in the list to be synchronized.

The synchronization coordination module is used to compare the maximum logical timestamps in the synchronization lists of the two mobile nodes when they enter the communication range. The lagging node initiates a synchronization request to the leading node. The leading node retrieves incremental synchronization entries in the synchronization list whose logical timestamps are greater than the maximum logical timestamps of the lagging node and sends them to the lagging node in logical timestamp order.

The synchronization module is used by the lagging party to receive and parse incremental synchronization entries, and apply the differential information through the local conflict-free replication data engine to complete the synchronization backup.