CN115878269B

CN115878269B - Cluster migration methods, related devices and storage media

Info

Publication number: CN115878269B
Application number: CN202211739454.1A
Authority: CN
Inventors: 请求不公布姓名
Original assignee: Beijing Real AI Technology Co Ltd
Current assignee: Beijing Real AI Technology Co Ltd
Priority date: 2022-12-31
Filing date: 2022-12-31
Publication date: 2025-12-12
Anticipated expiration: 2042-12-31
Also published as: CN115878269A

Abstract

The embodiment of the application relates to the field of cloud service, and provides a cluster migration method, a related device and a storage medium, wherein the cluster migration method is applied to migration control nodes in a cluster migration system, the cluster migration system further comprises a source cluster and a target container environment which are separately deployed, the target container environment comprises a plurality of first nodes, the source cluster comprises a plurality of second nodes, the number of the first nodes is the same as that of the second nodes, each first node is added into the source cluster to obtain a transition cluster, the transition cluster comprises all the first nodes and all the second nodes, all partition data in each second node are migrated to each first node in the transition cluster, the first nodes correspond to the second nodes one by one, each second node stores a plurality of partition data, configuration information of all the nodes in the transition cluster is updated to obtain a target cluster, and the target cluster only comprises the first nodes.

Description

Cluster migration method, related device and storage medium

Technical Field

The embodiment of the application relates to the technical field of cloud service, in particular to a cluster migration method, a related device and a storage medium.

Background

The advent of container technology has enabled the delivery and deployment of applications and middleware to be more convenient and faster than traditional virtual machine or physical machine environment-based delivery and deployment, and to be more easily integrated into the development and operational integration (Development and Operations, devOps) ecology of the container environment-based.

Kafka is a widely used high-throughput, high-reliability message middleware, and nowadays with the popularization of cloud-based concepts, more and more users deploy Kafka in a container environment. However, for the original business system in the enterprise, how to migrate the Kafka cluster from the virtual machine or physical machine environment to the container environment on the premise that the business is basically not interrupted is not provided with a proper scheme.

Disclosure of Invention

The embodiment of the application provides a cluster migration method, a related device and a storage medium, which can migrate a cluster service program from a virtual machine or physical machine environment to a container environment on the premise of not interrupting service.

In a first aspect, an embodiment of the present application provides a cluster migration method, applied to a migration control node in a cluster migration system, where the cluster migration system further includes a source cluster and a target container environment that are separately deployed, where the target container environment includes a plurality of first nodes, the source cluster includes a plurality of second nodes, and the number of the first nodes is the same as the number of the second nodes, where the method includes:

Adding each first node into the source cluster to obtain a transition cluster, wherein the transition cluster comprises all first nodes and all second nodes;

In the transition cluster, migrating all partition data in each second node to each first node, wherein the first nodes are in one-to-one correspondence with the second nodes, and each second node stores a plurality of partition data;

And updating configuration information of all nodes in the transition cluster to obtain a target cluster, wherein the target cluster only comprises the first node.

In a second aspect, an embodiment of the present application provides a control node, which is applied to a cluster migration system, and has a function of implementing a cluster migration method corresponding to the first aspect. The functions may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above, which may be software and/or hardware.

In one embodiment, the control node is applied to a cluster migration system, the cluster migration system further comprises a source cluster and a target container environment which are separately deployed, the target container environment comprises a plurality of first nodes, the source cluster comprises a plurality of second nodes, and the number of the first nodes is the same as the number of the second nodes;

the receiving and transmitting module is configured to send an instruction for joining the source cluster to each first node to obtain a transition cluster, wherein the transition cluster comprises all first nodes and all second nodes;

The processing module is configured to migrate the data of all the partitions in each second node to each first node in the transition cluster, wherein the first nodes are in one-to-one correspondence with the second nodes, and each second node stores a plurality of partition data;

The processing module is further configured to update configuration information of all nodes in the transition cluster to obtain a target cluster, wherein the target cluster only comprises the first node.

In a third aspect, embodiments of the present application provide a computer-readable storage medium comprising instructions which, when run on a computer, cause the computer to perform the cluster migration method according to the first aspect.

In a fourth aspect, an embodiment of the present application provides a computing device, including a memory, a processor, and a computer program stored on the memory and capable of running on the processor, where the processor implements the cluster migration method according to the first aspect when executing the computer program.

Compared with the prior art, in the embodiment of the application, the migration control node obtains a complete transition cluster by adding the newly-built first node in the target container environment into the source cluster in the virtual machine or physical machine environment, then completes migration of partition data in all second nodes of the source cluster to the first node in the transition cluster, and finally shuts down the source cluster. Because the data migration operation in the embodiment of the application is completed in the same cluster, not among different clusters in the prior art, the cross-domain transmission of data is avoided, the transmission efficiency of the data is improved, and the network resources and time are saved. In addition, in the embodiment of the application, the network configuration information of the new and old cluster nodes is updated through the migration control node, so that the conversion of the data transmission destination address after the data migration is realized, which is equivalent to the data transmission between the service party (consumer and producer) and the cluster through a fixed transfer station instead of the direct connection mode of the service party and the cluster in the prior art, thereby enabling the service party to be respectively connected with the old node or the new node through the transfer station before the cluster migration starts to after the cluster migration starts, avoiding the possible jitter in the cluster migration process, realizing the hot migration of the cluster and avoiding the service interruption.

Drawings

The objects, features and advantages of embodiments of the present application will become readily apparent from the detailed description of the embodiments of the present application read with reference to the accompanying drawings. Wherein:

FIG. 1 is a schematic diagram of a cluster migration system according to an embodiment of the present application;

FIG. 2 is a schematic flow chart of a cluster migration method according to an embodiment of the present application;

FIG. 3 is a schematic diagram illustrating a node change process of cluster migration according to the cluster migration method in an embodiment of the present application;

FIG. 4 is a flowchart of a cluster migration method for migrating data of a second node to a first node according to an embodiment of the present application;

FIG. 5 is a flowchart illustrating a method for migrating data of a second partition to a first node according to an embodiment of the present application;

FIG. 6 is a schematic diagram illustrating a partition change process of data migration in a cluster migration method according to an embodiment of the present application;

FIG. 7 is a signaling interaction diagram of a cluster migration method according to an embodiment of the present application;

FIG. 8 is a schematic diagram of a control node according to an embodiment of the present application;

FIG. 9 is a schematic diagram of a computing device in accordance with an embodiment of the present application;

fig. 10 is a schematic structural diagram of a server according to an embodiment of the present application.

In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

Detailed Description

The terms first, second and the like in the description and in the claims of embodiments of the application and in the above-described figures are used for distinguishing between similar objects (e.g., a first node and a second node each represented by a different node, and vice versa) and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or modules is not necessarily limited to those listed or explicitly listed or inherent to such process, method, article, or apparatus, but may include other steps or modules that may not be listed or inherent to such process, method, article, or apparatus, and the partitioning of such modules by embodiments of the application may include only one logical partitioning, and may be implemented in additional partitions, such as a plurality of modules may be combined or integrated into another system, or some features may be omitted or not implemented. In addition, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling between modules via interfaces, and the communication connection may be in electrical or other similar forms, which are not limited in this embodiment. The modules or sub-modules described as separate components may or may not be physically separate, may or may not be physical modules, or may be distributed in a plurality of circuit modules, and some or all of the modules may be selected according to actual needs to achieve the purposes of the embodiment of the present application.

The embodiment of the application provides a cluster migration method, a related device and a storage medium, which can be applied to a cluster migration system. The migration control node is at least used for adding each first node into the source cluster to obtain a transition cluster, and migrating all partition data in each second node to each first node in the transition cluster. The source cluster is deployed in a virtual machine or physical machine environment and comprises first nodes, wherein cluster service programs and data are deployed by the first nodes. The migration control node may be an application program that migrates all partition data in each second node to each first node, or a server that installs an application program that migrates all partition data in each second node to each first node, respectively.

The scheme of the embodiment of the application can be realized based on cloud technology, and particularly relates to the technical fields of cloud computing, cloud storage, databases and the like in the cloud technology, and the technical fields of cloud computing, cloud storage, databases and the like are respectively described as follows:

Cloud technology (Cloud technology) refers to a hosting technology for integrating hardware, software, network and other series resources in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The cloud technology (Cloudtechnology) can form a resource pool based on the general terms of network technology, information technology, integration technology, management platform technology, application technology and the like applied by the cloud computing business model, and is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing. According to the embodiment of the application, the target migration data can be stored through the cloud technology.

Cloud storage (cloud storage) is a new concept that extends and develops in the concept of cloud computing, and a distributed cloud storage system (hereinafter referred to as a storage system for short) refers to a storage system that integrates a large number of storage devices (storage devices are also referred to as storage nodes) of various types in a network to work cooperatively through application software or application interfaces through functions such as cluster application, grid technology, and a distributed storage file system, so as to provide data storage and service access functions for the outside. In the embodiment of the application, the information such as network configuration and the like can be stored in the storage system, so that the server can conveniently call the information.

At present, a storage method of a storage system is that logical volumes are created, and when the logical volumes are created, a physical storage space is allocated to each logical volume, where the physical storage space may be a disk of a certain storage device or a plurality of storage devices. The client stores data on a certain logical volume, that is, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as a data Identification (ID) and the like, the file system writes each object into a physical storage space of the logical volume, and the file system records storage position information of each object, so that when the client requests to access the data, the file system can enable the client to access the data according to the storage position information of each object.

The storage system allocates physical storage space for a logical volume, specifically, the physical storage space is divided into stripes in advance according to the set of capacity estimation of objects stored in the logical volume (the estimation often has a large margin with respect to the capacity of the objects actually to be stored) and redundant array of independent disks (RAID, redundant Array of INDEPENDENT DISK), and one logical volume can be understood as one stripe, so that the physical storage space is allocated for the logical volume.

The Database (Database), which can be considered as an electronic filing cabinet, is a place for storing electronic files, and users can perform operations such as adding, inquiring, updating, deleting and the like on the data in the files. A "database" is a collection of data stored together in a manner that can be shared with multiple users, with as little redundancy as possible, independent of the application.

The Database management system (DBMS for short, english: database MANAGEMENT SYSTEM) is a computer software system designed for managing databases, and generally has basic functions of storage, interception, security assurance, backup and the like. The database management system may be categorized by the database model it supports, such as relational, XML (Extensible MarkupLanguage ), or by the type of computer supported, such as server clusters, mobile phones, or by the query language used, such as SQL (structured query language (Structured QueryLanguage), XQuery), or by the energy impact focus, such as maximum scale, maximum speed, or other classification means.

In the prior art, a Kafka cluster node is often bound with a service party (a producer node or a consumer node) through a real network address, when the Kafka cluster needs to be migrated from an original deployment environment to a new deployment environment (for example, from a physical machine environment to a container environment), the service of the service party is often interrupted in the migration process due to the fact that the real network address of the cluster node is not changeable, so that the system of the service party is stopped, great inconvenience is brought to a user, and moreover, in the prior art, when the cluster is migrated, the data of an old cluster is often migrated to the new cluster in a cross-domain manner, the data transmission flow is long, and more transmission resources are needed to be consumed.

Compared with the prior art, the method and the device can add the newly-built first node in the target container environment into the source cluster to obtain a transition cluster, then complete migration of partition data in all second nodes of the source cluster to the first node in the transition cluster, and finally shut down the source cluster. Because the data migration operation in the embodiment of the application is completed in the same cluster, not among different clusters in the prior art, the cross-domain transmission of data is avoided, the transmission efficiency of the data is improved, and the network resources and time are saved. In addition, in the embodiment of the application, the network configuration information of the new and old cluster nodes is updated through the migration control node, so that the conversion of the data transmission destination address after the data migration is realized, which is equivalent to the data transmission between the service party (consumer and producer) and the cluster through a fixed transfer station instead of the direct connection mode of the service party and the cluster in the prior art, thereby enabling the service party to be respectively connected with the old node or the new node through the transfer station before the cluster migration starts to after the cluster migration starts, avoiding the possible jitter in the cluster migration process, realizing the hot migration of the cluster and avoiding the service interruption.

In some embodiments, the cluster migration method provided by the embodiment of the present application may be implemented based on a cluster migration system shown in fig. 1. The cluster migration system may include a migration control node 01, a first node 02, and a second node 03.

The migration control node 01 is configured to combine the first node 02 and the second node 03 to obtain a transition cluster, migrate partition data in the second node 03 to the first node 02 in the transition cluster, update configuration information of the first node 02 and the second node 03 in the transition cluster after the partition data is migrated, and shut down the second node 03 to obtain a target cluster only including the first node 02.

It should be noted that, the server according to the embodiment of the present application may be an independent physical server, or may be a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and an artificial intelligence platform.

Referring to fig. 2, fig. 2 is a flow chart of a cluster migration method according to an embodiment of the application. The method can be applied to a cluster migration system, and is executed by a migration control node, and a cluster service program is migrated from a source cluster (such as a virtual machine or a physical machine environment) to a target cluster (such as a container environment) on the premise of not interrupting service. The cluster migration system further comprises a source cluster and a target container environment which are separately deployed, wherein the target container environment comprises a plurality of first nodes, the source cluster comprises a plurality of second nodes, the number of the first nodes is the same as that of the second nodes, and the cluster migration method comprises the following steps of 101-103:

and step 101, adding each first node into the source cluster to obtain a transition cluster.

Wherein the transition cluster comprises all first nodes and all second nodes. In an embodiment of the present application, the source cluster may be a physical server cluster or a virtual server cluster, where a cluster service program may be deployed, and the cluster service program may provide service support for operation of a service system, and may be, for example, a messaging middleware such as ActiveMQ, rabbitMQ, rocketMQ and Kafka, and the like, configured to coordinate data transmission operations between a data producer (e.g., a server) and a data consumer (e.g., a client). In some application scenarios, it may be desirable to migrate a cluster service in a source cluster to a new cluster for reasons such as improving performance of the service deployment environment, so that the cluster service is deployed in a machine environment with higher performance.

In performing cluster migration, it is necessary to explicitly migrate the destination of the work (e.g., new nodes relative to the original nodes in the source cluster), i.e., where the cluster servers and data are to be migrated. Thus, in the embodiment of the application, each first node (i.e., the node in the target cluster) created in the target container environment can be acquired, and the first node can be used for receiving part of data of the cluster service program in the source cluster, for example, one first node can receive all data of one second node (i.e., the brooker in the Kafka) in the source (Kafka) cluster. It will be appreciated that in the embodiment of the present application, all data of each second node in the source cluster may need to be migrated to a new node, so that a plurality of first nodes equal in number to the second nodes may be created in the target container environment so as to be able to accept all data of the source cluster.

It should be noted that, in the embodiment of the present application, the target container environment may be established in advance by a third party (or a cluster maintainer) before the cluster migration, or may be established by a migration control node, which is not limited in the embodiment of the present application. The target container environment may be a unit constructed using Docker technology for providing runtime environment support for program services. In some alternative embodiments, the migration control node may also create a target container environment, and create a preset number of first nodes in the target container environment.

After defining the source cluster and the destination (i.e., each first node) involved in the cluster migration, each first node and each second node may be placed into a cluster to obtain a transition cluster, so as to complete the data migration work in the transition cluster. Because the data migration operation in the embodiment of the application is completed in the same cluster, not among different clusters in the prior art, the cross-domain transmission of data is avoided, the transmission efficiency of the data is improved, and the network resources and time are saved.

In the embodiment of the present application, for example, referring to fig. 3, a first node may be added to a source cluster to obtain a transition cluster, or each second node may be separated from the source cluster and form a transition cluster different from the source cluster with each first node, which may be set by a person skilled in the art according to actual needs, and this is not limited herein.

In some alternative embodiments, the specific manner of adding each first node to the source cluster to obtain the transition cluster may be that each first node is registered in a control node of the source cluster, so that each first node may be controlled by the control node in the source cluster, and thus the transition cluster is obtained.

In consideration of the fact that the control node in the source cluster is essentially selected from the second nodes (i.e. is pre-registered by a second node), that is, the control node is also a certain node (for example, may be a physical server or a virtual server) in the source cluster, and belongs to an object that needs to be shut down after the cluster migration work is completed. Therefore, when the cluster migration work proceeds to a certain progress, the control node needs to be replaced in order to maintain the normal operation of the new cluster (i.e., the target cluster) after the source cluster is shut down. Thus, in the embodiment of the present application, one candidate control node may be obtained from each first node, so that the candidate control node takes over the original control node in the source cluster at a preset time, that is, the candidate cluster includes one main control node (that is, the original control node in the source cluster) and one candidate control node (obtained based on one first node).

In the prior art, each Broker node in the Kafka cluster will register together at the Zookeeper node, and a temporary node is generated from each Broker node, that is, only one Broker node will register successfully (become the master control node of the source cluster), and the registration actions of other nodes will fail. Thus, the temporary node that successfully registers with the Zookeeper node becomes the control node (Controller) of the Kafka cluster, which can register with the Zookeeper node to monitor the Watch service, i.e. the Controller can monitor all information of other Broker nodes. If the corresponding Broker node of the Controller is down, the temporary nodes registered on the Zookeeper node disappear, at this time, all the Broker nodes are registered on the Zookeeper node together, and because only one Broker node is successfully registered and the other Broker nodes are failed, the Broker successfully registered on the Zookeeper node as the temporary node becomes a new Controller of the Kafka cluster.

Specifically, once the Broker node corresponding to the original Controller is down, the Broker node corresponding to the new Controller reads the states of all Partition parts on the down Broker node, and selects one copy Replica in the Isr list as the primary Partition part Leader (if all Replica in the Isr list are not available, one Replica available outside the Isr list is selected as the Partition Leader, and if all Replica of the Partition is down, the new Partition Leader is set to-1, and any Replica in the Isr is waited for to recover and is selected as the Partition Leader, or the Replica recovered first is selected as the Partition Leader). In addition, the original Controller can inform the Zookeeper node of the message of downtime of the corresponding Broker node, and the Zookeeper node can inform other Broker nodes so as to register and select a new Controller.

It can be seen that in the prior art, the source cluster only selects a new master control node from the second nodes when the master control node fails. In the embodiment of the application, the cluster migration work is required to be carried out on the premise of ensuring that the service is not interrupted, and each second node of the source cluster is actively shut down when the cluster migration work is completed, so that the problem of frequent replacement of the control nodes can be possibly generated by adopting a control node handover mode in the prior art, for example, after the historical main control node of the source cluster is shut down, a new main control node is still generated in the second node, so that the new control node still needs to be actively shut down, and a main control node selection step of a further round is further carried out.

In order to realize the orderly handover of the main control node in the cluster migration work, the cluster migration is ensured not to influence the normal operation of the service system. In the embodiment of the application, after each first node joins the source cluster to obtain the transition cluster, one candidate control node (i.e. obtained by registering one first node) is selected from each first node of the transition cluster, and the candidate control node can synchronize configuration information in the main control node and take over the main control node when a preset instruction is received. It will be appreciated that the candidate control nodes may be generated in a similar manner to the prior art, that is, may be registered by each first node on the Zookeeper node, or may be randomly selected from each first node by the migration control node, which is not limited herein.

It can be understood that after the candidate control node in the embodiment of the present application is generated, control information of the master control node, for example, operation information of each node in the transition cluster, may be obtained in real time, so as to take over the master control node at any time. In some alternative embodiments, if the master control node fails, the candidate control node may immediately take over the master control node to become a new master control node, and simultaneously generate new candidate control nodes (from the first nodes) so as to ensure that the control nodes are generated in the first nodes until the second nodes are turned off, and no additional workload is brought about due to the second nodes of the source cluster.

In consideration, during the handover process of the new and old control nodes, some control configuration information may not be completely migrated due to different sources of the new control node (i.e., the candidate control node of the transition cluster) and the old control node (i.e., the master control node of the transition cluster), and errors may occur, so that the new control node may not implement control correctly. In order to discover the migration error in advance, repeated operations caused by continuous error work after the migration error are avoided as much as possible, in some alternative embodiments, the migration progress of the cluster can be monitored, and when the migration progress is performed, a preset instruction is sent to a candidate control node by the migration control node, so that the candidate control node can actively take over the main control node to control each node in the transition cluster. Therefore, whether the new control node has a problem or not can be found in advance, and the service can not be provided as the old control node, so that the problem can be traced back as early as possible when the problem occurs, and the availability of the migration system is improved.

Having described one possible way to replace a control node in a cluster migration job, the following continues to describe how data in a second node is migrated to a first node.

And 102, in the transition cluster, migrating all partition data in each second node to each first node respectively.

The first nodes are in one-to-one correspondence with the second nodes, and each second node comprises a plurality of partition data.

In the embodiment of the present application, the source cluster may be a Kafka cluster, and each second node may be a Broker node, so each second node may include multiple topics Topic, and each Topic may include multiple Partition. It is contemplated that in order to migrate the data of each second node entirely into the new node, a preset number of first nodes are created in the target container environment in step 101. Therefore, the first nodes can be in one-to-one correspondence with the second nodes, so that each second node has a corresponding data migration destination node (namely the first node), and the orderly performance of data migration work is ensured.

When data in a second node needs to be migrated to a corresponding first node, in order to ensure that the operation of a service system is not interrupted, namely the second node can continuously provide data support service, different data migration strategies can be adopted according to the data activity state in the second node, for example, copies of the active data can be migrated, and the situation that the normal service operation is influenced by data occupation is avoided. Referring to fig. 4, migrating all data in a second node to a first node corresponding to the second node may include steps 201-202:

Step 201, according to the historical consumption record of each consumer node, acquiring a first partition and a second partition from all partitions of the second node.

The historical consumption times of the first partition are not larger than a preset value, and the historical consumption times of the second partition are larger than the preset value.

In the embodiment of the application, in order to determine the active state of the data to be migrated, the characteristic that the data of each partition can only be consumed by one consumer node in the Kafka cluster can be utilized, and the active data partition and the inactive data partition are obtained by counting the historical consumption records of the consumer node.

Specifically, when data migration to a second node is needed, historical consumption records of all consumer nodes can be obtained, recorded data of the second node is analyzed, the consumed times of each partition in the second node are determined, if the consumed times of one partition are not more than the preset value (for example, 10), the data of the partition can be considered to be accessed infrequently, namely, the partition can be determined to be a first partition (inactive partition), and similarly, if the consumed times of one partition are more than the preset value, the data of the partition can be considered to be accessed frequently, namely, the partition can be determined to be a second partition (active partition).

It can be appreciated that in order to ensure that the data active state determined according to the historical consumption record is attached to the current situation (i.e. the situation during data migration), active data and inactive data during data migration can be accurately distinguished. In some alternative embodiments, the consumption record for a period of history closest to the current time may be obtained as a history consumption record, e.g., the consumption record for a consumer node within 7 days prior to the data migration may be obtained as a history consumption record.

And 202, migrating the data of each first partition to the first node according to a first migration strategy, and migrating the data of each second partition to the first node according to a second migration strategy.

In the embodiment of the application, after the active state of each partition in the second node to be migrated is determined, the data in each partition can be migrated to the first node corresponding to the second node according to the preset migration strategy. Specifically, for an inactive first partition, a first migration policy may be adopted to migrate data therein to a corresponding first node indifferently, and for an active second partition, a second migration policy may be adopted to migrate data therein to a corresponding first node indifferently.

It should be noted that, even in the inactive first partition, the state in which the data is consumed is not completely consistent. Specifically, in the Kafka cluster, each Broker node includes a plurality of partitions, which include not only one data but a data sequence of the same subject, and the sequential data generated by the producer node may be additionally stored in the form of a queue after the previous data to form the data sequence.

For example, in a Kafka cluster, each partition may exist in the form of a folder, and multiple groups of segment files may be included in each partition folder, each group segement of files containing an index, log, and timeindex files, where log files are locations where messages are actually stored, and index and timeindex files are index files for retrieving messages. After receiving the message generated by the producer node, the message is written sequentially to segement, and only the last segement can perform the write operation.

It can be seen that a partition includes multiple data (i.e., segment files), and the number of times that multiple data in the partition are consumed by consumers is not exactly the same, i.e., each data is also differentiated by liveness. However, although the liveness of the respective data in the first partition is different, the data is not consumed frequently, and can be attributed to the inactive data, so that the data of the first partition can be migrated into the first partition completely without distinction by adopting a unified first migration strategy.

In step 101 it is mentioned that in the cluster migration, not only the data stored on each second node in the source cluster needs to be migrated, but also the control nodes in the source cluster need to be changed, i.e. the candidate control node obtained in the first node takes over the master control node obtained based on the second node.

In consideration of that each first partition is an inactive partition, the first partition can be directly and integrally migrated to a corresponding first node, instead of the migration operation of a second partition, which is to distinguish different data and process the data in different manners, namely, the data migration speed of the first partition may be faster and can be completed earlier, and if the first partition in each second node completes migration, the candidate control node takes over the main control node, so that possible problems in cluster migration may be found earlier and backtracking may be performed in time.

Thus, in some optional embodiments, the migration progress of all the first partitions of each second node may be obtained (in real time or at a fixed time), and if the migration progress of all the first partitions meets a preset condition (for example, the migration progress indicates that the data of each first partition has been migrated into the first node), the preset instruction is sent to the candidate control node, so that the candidate control node may take over the main control node, so as to test whether the candidate control node can correctly control each node in the transition cluster, and normally complete the data support service of the whole cluster.

Similarly to the case of the first partition, the second partition also includes a plurality of data, and the liveness of these data is not exactly the same. For example, in some second partitions there may be active data and inactive data, the active data may be consumed frequently by consumer nodes, e.g. some data may be accessed by consumer nodes more than the preset value during a history period, while inactive data may be consumed rarely by consumers, e.g. the number of times it is consumed by consumer nodes does not exceed the preset value during a history period. Therefore, the second migration strategy can be adopted for the active data and the inactive data in the second partition to carry out differential migration, the access of the active data by the consumer node during data migration is not affected, and the migration of the inactive data can be completed faster.

Referring to fig. 5, in some alternative embodiments, when the data of a second partition is differentially migrated to the first node according to the second migration policy, active data in the second partition may be differentially processed from inactive data, specifically including steps 2021-2024:

step 2021, acquiring active data in the second partition.

In the embodiment of the present application, the active data in the second partition may also be determined according to the historical consumption record, that is, according to the number of times each data in the second partition is consumed by the consumer node in the historical period. If the number of times a certain data is consumed exceeds the preset value, it may be determined as active data.

And step 2022, storing the copy of the active data and the first incremental data in a buffer.

Wherein the buffer is for providing data services to producer and consumer nodes, the first incremental data being generated by the producer node during a first period of time.

In embodiments of the present application, it is contemplated that active data is consumed frequently by consumers, i.e., it may still need to be consumed by consumers at the time of data migration. In order to not influence the active data to be migrated, the data service is continuously provided for the consumer node, the copy of the active data can be obtained and then stored into the buffer zone, the data service is provided for the consumer through the buffer zone, the operation of the business system is not interrupted, and each data in the second partition can be completely migrated to the first node.

It will be appreciated that in a Kafka cluster, each partition may include multiple copies so that in the event of a failure of a certain (primary) partition, the upper level becomes the new (primary) partition. Thus, in the embodiment of the present application, when active data in a second partition in a second node is migrated, a copy of the second partition may be acquired, and based on the copy of the second partition, a copy of the active data may be acquired, and stored in a cache region, so as to provide data services during data migration.

In the Kafka cluster, when a certain node fails, each partition in the node still holds a full data backup, and can replace the failed (main) partition to continue to bear data service, wherein one partition and its copy are stored on different nodes. For example, if the Kafka cluster includes 3 nodes, partition A1 in node a may include 2 copies, which may be stored in node B and node C, respectively. Therefore, in some alternative embodiments, the active data in the second partition to be migrated can be directly copied to obtain a copy thereof and stored in the cache region, so that data processing can be realized more quickly and efficiently.

In order to ensure that data service support is continuously provided for the business system in the cluster migration process, the message production requirement of the producer node needs to be considered in the embodiment of the application. In the embodiment of the present application, after a second node starts to perform data migration (for example, starts to migrate to a period between obtaining a target partition corresponding to the second node, that is, a first period), a message newly generated by a producer node that needs to be stored in the second node may be referred to as first incremental data, where the first incremental data may also be stored in a buffer area, so as to continue to provide data services for the producer node and the consumer node, and not affect normal data migration.

In the embodiment of the application, the data migration is performed in a partition unit. Therefore, in order to ensure that each partition can provide data service in time when data migration is performed, in some alternative embodiments, a buffer area may be set corresponding to each partition. That is, the second partitions in which each second node can be arranged have the same number of cache areas, so as to provide data cache services for each second partition one-to-one, so that each second partition can continuously support the data services of the consumer node and the producer node during data migration without any obstacle.

And step 2023, migrating all data in the second partition to the first node to obtain a target partition.

Wherein the storage order of all the data in the second partition is the same as that in the target partition.

Due to the arrangement of the buffer area, the data of the second partition can be integrally migrated when the data of the second partition is migrated, the active data does not need to be considered how to provide data consumption support for the consumer node, and new messages generated by the producer node can be continuously accepted.

Referring to fig. 6, in the embodiment of the present application, when migrating data in a second partition, data migration may be performed according to storage locations of respective data in the partition, so as to ensure that after the data in the second partition is migrated to a first node, a target partition completely consistent with the second partition may be obtained, that is, data in the target partition, a storage location and a storage order of the data are the same as those of the corresponding second partition.

Step 2024, migrating the first incremental data in the buffer area to the target partition.

In the embodiment of the present application, after all data in a second partition to be migrated is migrated to a first node, a target partition may be obtained, however, the data in the target partition is not complete at this time, and a message (i.e. the first increment number in the buffer area) generated by the producer node after the data migration operation is started needs to be stored in the target partition. Specifically, since the data of each next partition of one brooker node in the Kafka cluster is additionally stored according to the time sequence, after all the data in one second partition is migrated to the first node to obtain the corresponding target partition, the first incremental data in the buffer area corresponding to the second partition can be directly additionally stored after the last data in the target partition.

It is contemplated that the action of the producer node producing the message may be ongoing, i.e., while storing the old delta data (i.e., the first delta data) in the cache region into its corresponding target partition, new delta data is also being produced, i.e., the delta data in the cache region is increasing. In order to avoid data collision in the buffer, in some alternative embodiments, the first incremental data in the buffer may be stored in the target partition, and the second incremental data generated by the producer node in this period (i.e., the second period may be connected to the first period in time sequence so as to keep the data migration operation going on) may be stored in the buffer, and the second incremental data may be stored in a preset position of the target partition, where the preset position is determined based on the first incremental data in the buffer.

For example, a second partition comprises 3 data segments, a1, a2 and a3, at time t1, 3 data segments in the second partition are started to be migrated to corresponding first nodes, at time t2, the producer node generates 2 new messages, a4 and a5, thereby storing first incremental data, a4 and a5, in a buffer zone of the second partition, at time t3 (i.e. the first time period comprises time t 1-time t 3), a1, a2 and a3 are completely migrated successfully, a target partition of the second partition is obtained, a1, a2 and a3 are stored in the target partition, at time t4 (i.e. the second time period), first incremental data, a4 and a5 in the buffer zone are started to be migrated to the target partition, and a message, a6 (i.e. the second incremental data) is newly generated by the producer node, thereby determining a6 th data position, where a6 should be stored in the target partition, according to the two first incremental data included in the buffer zone, and starting to store a6 to the corresponding position in the target partition. That is, in the embodiment of the present application, after the target partition is obtained, the new message generated by the producer node is stored in the corresponding target partition, rather than in the buffer area.

And step 103, updating configuration information of all nodes in the transition cluster to obtain a target cluster.

Wherein the target cluster only comprises the first node.

In the embodiment of the application, after the data of all the partitions in each second node are migrated to the corresponding first node, the second node can stop each second node without continuing to provide data service. For example, the cluster node addresses corresponding to the producer node and the consumer node may be changed from the address of the second node to the address of the first node.

It is considered that in some scenarios, the real network address (e.g., the real IP address) of each node in the cluster may not be changed, and if the real network address is directly docked with the producer node or the consumer node, the cluster migration cannot be implemented without interrupting the service. Thus, in an embodiment of the present application, an intermediate address may be mapped to the real network address of the cluster node by means of address mapping, and then a connection is established with the producer node or consumer node using the intermediate address.

Specifically, in the embodiment of the application, each second node establishes communication connection with the consumer node and the producer node through a preset network address mapping, wherein the preset network address mapping comprises a mapping of a real network address of each second node and a preset network address, and the preset network address comprises one of a domain name, a virtual network address or a forwarding node address. Based on the configuration information, the update of the configuration information of each node in the transition cluster can be realized by updating the real network address of each second node in the preset network address mapping to the real network address of each first node, so that each second node in the transition cluster is shut down, and the target cluster is obtained.

For example, if the service party (consumer node and producer node) accesses the source cluster through the domain name, the real network address of the first node in the target container environment is added to the domain name map, if the service party accesses the source cluster through the virtual network address, the back end host address of the virtual network address proxy needs to be adjusted (that is, the real network address of each second node in the source cluster is adjusted to the real network address of each first node), and if the service party uses traffic forwarding, the IP Tables rules need to be adjusted, which will not be described in detail.

In this way, in the final stage of cluster migration, the real network address of the old cluster node is unbinding with the intermediate address, and then the old cluster node is bound with the real network address of the new cluster node, so that the switching of the cluster nodes corresponding to the consumer node and the producer node can be realized. In the embodiment of the application, because the consumer node and the producer node are in communication connection with the intermediate address, even if the cluster node bound by the intermediate address is changed, the consumer node and the producer node are noninductive and cannot cause interruption of service.

Although the embodiment of fig. 2 of the present application describes a cluster migration method that can be implemented without interrupting the service using the migration control node as an execution body, the present application is not limited thereto. In some alternative embodiments, the migration control node may also send instructions to each second node in the source cluster and each first node newly built in the target container environment, and the first node, the second node and the migration control node interact with each other to complete migration of the entire cluster service program. Referring to fig. 7, in some alternative embodiments, data in a second node may be migrated to a first node by the following steps 301-305:

in step 301, the migration control node sends a first instruction to the first node.

Wherein the first instruction may be to instruct the first node to join the source cluster.

In step 302, the first node receives the first instruction and joins the source cluster based on the first instruction.

Step 303, the migration control node sends a second instruction to the second node.

The second instruction may be configured to instruct the second node to begin data migration.

Step 304, the second node receives the second instruction, and migrates the data to the first node based on the second instruction.

Step 305, the second node sends a message to the migration control node.

Wherein the message may be used to indicate that the data in the second node has been entirely migrated to the first node.

And 306, the migration control node receives the message and updates configuration information of all nodes in the transition cluster based on the message to obtain the target cluster.

In the embodiment of the application, a complete transition cluster is obtained by adding a newly built first node in a target container environment into a source cluster in a virtual machine or physical machine environment, then migration of partition data in all second nodes of the source cluster to the first node is completed in the transition cluster, and finally the source cluster is shut down. Because the data migration operation in the embodiment of the application is completed in the same cluster, not among different clusters in the prior art, the cross-domain transmission of data is avoided, the transmission efficiency of the data is improved, and the network resources and time are saved. In addition, in the embodiment of the application, the network configuration information of the new and old cluster nodes is updated through the migration control node, so that the conversion of the data transmission destination address after the data migration is realized, which is equivalent to the data transmission between the service party (consumer and producer) and the cluster through a fixed transfer station instead of the direct connection mode of the service party and the cluster in the prior art, thereby enabling the service party to be respectively connected with the old node or the new node through the transfer station before the cluster migration starts to after the cluster migration starts, avoiding the possible jitter in the cluster migration process, realizing the hot migration of the cluster and avoiding the service interruption.

The above describes a cluster migration method in the embodiments of the present application, and a cluster migration system (e.g., a server cluster) that executes the cluster migration method is described below.

Referring to fig. 8, a schematic diagram of a control node shown in fig. 8 may be applied to a server cluster, and configured to migrate a cluster service program from a source cluster (e.g., a virtual machine or a physical machine environment) to a target cluster (e.g., a container environment) without interrupting a service. The control node in the embodiment of the present application can implement the steps corresponding to the cluster migration method performed in the embodiment corresponding to fig. 2 described above. The functions realized by the cluster migration system can be realized by hardware, and can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the functions described above, which may be software and/or hardware. The control node may include a transceiver module 601 and a processing module 602, and the node may further include a display module (not shown in fig. 5), and the functional implementation of the processing module 602 and the transceiver module 601 may refer to the operations performed in the embodiment corresponding to fig. 2, which are not described herein. For example, the processing module 602 may be configured to control operations of transceiving, acquiring, etc. of the transceiving module 601.

In some embodiments, the control node 60 is applied to a cluster migration system, and the cluster migration system further includes a source cluster and a target container environment which are separately deployed, wherein the target container environment includes a plurality of first nodes, the source cluster includes a plurality of second nodes, and the number of the first nodes is the same as the number of the second nodes;

The transceiver module 601 is configured to send an instruction for joining the source cluster to each first node to obtain a transition cluster, wherein the transition cluster comprises all first nodes and all second nodes;

The processing module 602 is further configured to migrate, in the transition cluster, data of all partitions in each second node to each first node, where the first nodes are in one-to-one correspondence with the second nodes, and each second node stores multiple partition data;

The processing module 602 is further configured to update configuration information of all nodes in the transition cluster to obtain a target cluster, where the target cluster includes only the first node.

In some embodiments, the transition cluster includes a master control node and a candidate control node, the master control node is registered in advance by a second node, and the candidate control node is registered by a first node;

And the candidate control nodes are used for synchronizing configuration information in the main control node and taking over the main control node when a preset instruction is received.

In some embodiments, each second node establishes a communication connection with the consumer node and the producer node through a preset network address map;

the preset network address mapping comprises mapping of a real network address and a preset network address of each second node, wherein the preset network address comprises one of a domain name, a virtual network address and a forwarding node address;

the processing module 602 is further configured to update the real network address of each second node in the preset network address map to the real network address of each first node.

In some embodiments, the processing module 602 is further configured to obtain, from all partitions of the second node, a first partition and a second partition according to a historical consumption record of each consumer node, where a historical consumption number of the first partition is not greater than a preset value, and a historical consumption number of the second partition is greater than the preset value, migrate data of each first partition to the first node according to a first migration policy, and migrate data of each second partition to the first node according to a second migration policy.

In some embodiments, the processing module 602 is further configured to obtain all active data in the second partition, each of the active data being consumed by the consumer node more than the preset value for a historical period of time, and store a copy of each of the active data in a cache with first incremental data, wherein the cache is configured to provide data services to the producer node and the consumer node, the first incremental data being generated by the producer node for a first period of time;

The processing module 602 is further configured to migrate all data in the second partition to the first node to obtain a target partition, where a storage order of all data in the second partition is the same as that in the target partition, and migrate the first incremental data in the buffer area to the target partition.

In some embodiments, the processing module 602 is further configured to store the second incremental data in a preset location of the target partition after the migration of all data in the second partition to the first node to obtain the target partition;

The second incremental data is generated by the producer node in a second time period, the second time period is adjacent to the first time period in time sequence and is not earlier than the first time period, and the preset position is determined based on the first incremental data in the buffer zone.

In some embodiments, the processing module 602 is further configured to obtain migration progress of all first partitions of each second node before the updating the configuration information of all nodes in the transition cluster to obtain the target cluster, and send the preset instruction to the candidate control node if the migration progress of all first partitions meets a preset condition.

In the embodiment of the application, under the control of a processing module, a control node obtains a complete transition cluster by adding a newly-built first node in a container environment into a source cluster in a virtual machine or physical machine environment, then migration of partition data in all second nodes of the source cluster to the first node is completed in the transition cluster, and finally the source cluster is shut down. Because the data migration operation in the embodiment of the application is completed in the same cluster, not among different clusters in the prior art, the cross-domain transmission of data is avoided, the transmission efficiency of the data is improved, and the network resources and time are saved. In addition, in the embodiment of the application, the network configuration information of the new and old cluster nodes is updated through the control node, so that the conversion of the data transmission destination address after the data migration is realized, which is equivalent to the data transmission between the service party (consumer and producer) and the cluster through a fixed transfer station, rather than the direct connection mode of the service party and the cluster in the prior art, so that the service party can be respectively connected with the old node or the new node through the transfer station before the cluster migration starts to after the cluster migration starts, thereby avoiding the possible jitter in the cluster migration process, realizing the hot migration of the cluster and avoiding the service interruption.

The control node 60 in the embodiment of the present application is described above in terms of a modularized functional entity, and the servers performing the cluster migration method in the embodiment of the present application are described below in terms of hardware processing, respectively.

It should be noted that, in the embodiment of the control node of the present application, the entity device corresponding to the transceiver module 601 shown in fig. 8 may be an input/output unit, a transceiver, a radio frequency circuit, a communication module, an input/output (I/O) interface, etc., and the entity device corresponding to the processing module 602 may be a processor. The control node 60 shown in fig. 8 may have a structure as shown in fig. 9, and when the control node 60 shown in fig. 8 has a structure as shown in fig. 9, the processor and the transceiver in fig. 9 can implement the same or similar functions as the processing module 602 and the transceiver module 601 provided in the foregoing node embodiment corresponding to the control node, and the memory in fig. 9 stores a computer program that needs to be invoked when the processor executes the foregoing cluster migration method.

Referring to fig. 10, fig. 10 is a schematic diagram of a server structure according to an embodiment of the present application, where the server 1100 may have a relatively large difference due to different configurations or performances, and may include one or more central processing units (english: central processing units, english: CPU) 1122 (e.g., one or more processors) and a memory 1132, and one or more storage mediums 1130 (e.g., one or more mass storage devices) storing application programs 1142 or data 1144. Wherein the memory 1132 and the storage medium 1130 may be transitory or persistent. The program stored on the storage medium 1130 may include one or more modules (not shown), each of which may include a series of instruction operations on a server. Still further, the central processor 1122 may be provided in communication with a storage medium 1130, executing a series of instruction operations in the storage medium 1130 on the server 1100.

The Server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input-output interfaces 1158, and/or one or more operating systems 1141, such as Windows Server, mac OS X, unix, linux, freeBSD, and the like.

The steps performed by the server in the above embodiments may be based on the structure of the server 1100 shown in fig. 10. For example, the steps performed by the control node 60 shown in fig. 10 in the above-described embodiments may be based on the server structure shown in fig. 10. For example, the CPU 1122 may perform the following operations by calling instructions in the memory 1132:

Sending an instruction to each first node through the input/output interface 1158 to enable each first node to join the source cluster to obtain a transition cluster, wherein each first node is deployed in the target container environment, each second node is deployed in the source cluster, and the number of the first nodes is the same as the number of the second nodes;

Migrating all partitioned data in each second node to each first node, wherein the first nodes and the second nodes are in one-to-one correspondence, and each second node comprises a plurality of partitions;

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.

It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, apparatuses and modules described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein.

In the embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or modules, which may be in electrical, mechanical, or other forms.

The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, the flow or functions according to the embodiments of the present application are fully or partially produced. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk Solid STATE DISK (SSD)), etc.

The foregoing describes the technical solution provided by the embodiments of the present application in detail, and specific examples are applied to the embodiments of the present application to explain the principles and implementation modes of the embodiments of the present application, and the description of the above embodiments is only for helping to understand the methods and core ideas of the embodiments of the present application, and meanwhile, for those skilled in the art, according to the ideas of the embodiments of the present application, there are various changes in the specific implementation manners and application ranges, so the disclosure should not be interpreted as limiting the embodiments of the present application.

Claims

1. A cluster migration method is applied to a migration control node in a cluster migration system, the cluster migration system further comprises a source cluster and a target container environment which are separately deployed, the target container environment comprises a plurality of first nodes, the source cluster comprises a plurality of second nodes, and the number of the first nodes is the same as that of the second nodes, and the method comprises the following steps:

2. The method of claim 1, wherein the transition cluster comprises a master control node and candidate control nodes, the master control node being pre-registered by a second node, the candidate control nodes being registered by a first node;

3. The method of claim 1, wherein each second node establishes a communication connection with the consumer node and the producer node through a preset network address map;

the updating the configuration information of all nodes in the transition cluster comprises the following steps:

and updating the real network address of each second node in the preset network address mapping to the real network address of each first node.

4. A method as claimed in any one of claims 1 to 3, wherein migrating all partition data in a second node to a first node comprises:

According to the historical consumption records of all consumer nodes, acquiring a first partition and a second partition from all partitions of the second node, wherein the historical consumption times of the first partition are not more than a preset value, and the historical consumption times of the second partition are more than the preset value;

migrating the data of each first partition to the first node according to a first migration strategy; and migrating the data of each second partition to the first node according to a second migration strategy.

5. The method of claim 4, wherein migrating data of a second partition to the first node according to the second migration policy comprises:

acquiring all active data in the second partition, wherein the number of times that each active data is consumed by a consumer node in a historical period exceeds the preset value;

storing copies of each active data with first incremental data, wherein the cache is configured to provide data services to producer and consumer nodes, the first incremental data being generated by the producer node during a first period of time;

migrating all data in the second partition to the first node to obtain a target partition, wherein the storage sequence of all data in the second partition is the same as that in the target partition;

and migrating the first increment data in the cache region to the target partition.

6. The method of claim 5, wherein after said migrating all data in said second partition to said first node to obtain a target partition, the method further comprises:

storing the second incremental data into a preset position of the target partition;

the second incremental data is generated by the producer node in a second time period, the second time period is adjacent to the first time period in time sequence and is later than the first time period, and the preset position is determined based on the first incremental data in the buffer zone.

7. The method of claim 4, wherein the updating the configuration information of all nodes in the transition cluster, before obtaining the target cluster, further comprises:

acquiring migration progress of all first partitions of each second node;

And if the migration progress of all the first partitions meets the preset conditions, sending a preset instruction to the candidate control nodes.

8. A control node for a cluster migration system, the cluster migration system further comprising a source cluster and a target container environment which are separately deployed, wherein the target container environment comprises a plurality of first nodes, the source cluster comprises a plurality of second nodes, the number of the first nodes is the same as the number of the second nodes, and the control node comprises:

9. The control node of claim 8, wherein the transition cluster comprises a master control node and candidate control nodes, the master control node being pre-registered by a second node, the candidate control nodes being registered by a first node;

10. The control node of claim 8, wherein each second node establishes a communication connection with the consumer node and the producer node through a preset network address map;

the processing module is further configured to update the real network address of each second node in the preset network address map to the real network address of each first node.

11. The control node according to any of claims 8-10, wherein the processing module is further configured to obtain, from all partitions of the second node, a first partition and a second partition according to a historical consumption record of each consumer node, a historical consumption number of the first partition being not greater than a preset value, a historical consumption number of the second partition being greater than the preset value, and migrate data of each first partition to the first node according to a first migration policy, and migrate data of each second partition to the first node according to a second migration policy.

12. The control node of claim 11, wherein the processing module is further configured to obtain all active data in the second partition, each active data being consumed by the consumer node more than the preset value for a historical period of time, and store a copy of each active data with first incremental data, wherein the cache is configured to provide data services to the producer node and the consumer node, the first incremental data being generated by the producer node for a first period of time;

The processing module is further configured to migrate all data in the second partition to the first node to obtain a target partition, wherein the storage sequence of all data in the second partition is the same as that in the target partition, and migrate the first incremental data in the cache region to the target partition.

13. The control node of claim 12, wherein the processing module is further configured to store second incremental data in a preset location of a target partition after the migration of all data in the second partition to the first node to obtain the target partition;

14. The control node of claim 11, wherein the processing module is further configured to obtain migration progress of all first partitions of each second node before the updating the configuration information of all nodes in the transition cluster to obtain the target cluster, and send a preset instruction to the candidate control node if the migration progress of all first partitions meets a preset condition.

15. A computing device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of claims 1-7 when the computer program is executed.

16. A computer readable storage medium comprising instructions which, when run on a computer, cause the computer to perform the method of any of claims 1-7.

17. A computer program product comprising instructions, the computer program product comprising program instructions which, when run on a computer or a processor, cause the computer or the processor to perform the method of any of claims 1-7.

18. A chip system, comprising:

A communication interface for inputting and/or outputting information;

A processor for executing a computer executable program to cause a device on which the chip system is installed to perform the method of any one of claims 1-7.