CN115292051B - Hot migration method, device and application of GPU (graphics processing Unit) resource POD (POD) - Google Patents

Abstract

This application proposes a method, device and application for hot migration of a GPU resource POD, including the following steps: the GPU client sends a migration request; the migration component in the migration POD caches the metadata after receiving the migration request The index is sent to the migration receiving component that receives the POD; after receiving the migration request, the migration component in the migration POD transmits the high-frequency memory block and the low-frequency memory block to the migration receiving component; after the GPU server receives the migration request, Adjust the priority of the task; the receiving POD performs a hash check on the received file, and confirms the stability of the GPU with the GPU server. This solution solves the problem of GPU container image transmission, achieves fast copying of CPU status, and maintains GPU task scheduling, so as to achieve the purpose of fast hot migration of GPU resource POD in the algorithm container cloud.

Description

A thermal migration method, device and application of a GPU resource POD

technical field

The present application relates to the technical field of algorithm container cloud platform, in particular to a method, device and application for hot migration of GPU resource containers.

Background technique

With the development of cloud computing technology, with the advantages of standardized operating environment, rapid deployment and operation and maintenance, and flexible allocation on demand, Docker and Kubernetes technologies have become the standards for many enterprise application delivery. At the same time, with the generalization of GPUs, the use of GPUs in the field of deep learning is increasing. The algorithm container cloud that combines the two emerges as the times require. It can quickly provide GPU resources, achieve the delivery of standardized algorithm applications, and perform in graphics and image rendering and parallelism. Both computing and artificial intelligence have played an important role.

Algorithmic operations in the field of deep learning often have the characteristics of complex tasks and long time-consuming. When PODs (container groups) need to be scheduled and adjusted due to resource adjustments, work priority adjustments, or failures, the reset of the running status will lead to The interruption of the algorithm operation leads to the failure of the calculation process of the algorithm. At the same time, because the state of the container cannot be maintained, the flexible scheduling and collocation of the algorithm container cannot be achieved, which reduces the utilization rate of the container cloud platform.

In the algorithm container cloud cluster, Pod is the foundation of all business types. It is a combination of one or more containers. For specific applications, Pod is their logical host. Pod contains multiple algorithm containers related to business. Unlike cloud hosts, containers are process-based and cannot be separated and hot-migrated as easily as kvm cloud hosts. In the native Kubernetes container cloud platform, new pods are often started and old pods are destroyed through yaml configuration files The form of migration is completed, and for the newly generated Pod, it often does not have the original running state.

In some open source projects, there is a way to live migrate the CPU through CRIU (Checkpoint/Restore In Userspace): checkpoint the old container, stop the container, transfer the image file to the destination host, and restore the container state. Technology-related research focuses on how to maintain and restore the state of the memory CPU. There are various CPU optimization methods that compress memory pages to optimize network transmission and reduce duplicate copies, but these solutions are only applicable to CPU resource containers.

In the CPU container migration method, there is also an image optimization scheme. One is preloading, which preloads the image on each server in the cluster, thereby reducing the time for image transmission. The other is based on the overlay layering of the container image. The principle is to judge the difference between the mirror warehouse and the local by layering, and reduce the transmission of repeated overlays.

The CPU is the central processing unit, which is the computing and control core of the computer system. There are 25% of the ALU (computing unit), 25% of the Control (control unit), and 50% of the Cache (caching unit). The GPU is the graphics processing unit. 90% of the ALU (computing unit), 5% of the Control (control unit), 5% of the Cache (cache unit). Due to the great difference between the CPU resource container and the GPU resource container, the hot migration method applicable to the CPU resource container currently on the market cannot be directly applied to the GPU resource container. In other words, the two CPU solutions mentioned above None of them are applicable to GPU containers, because of the large size of GPU container images, preloading is a great waste of resources, and some container configuration options are set to imagePullPolicy: Always (that is, the image is still pulled), preloading does not work, Moreover, the synchronization of UpperDir (the uppermost layer) cannot be solved, and the layering of the second overlay, in the case of a large base, is not particularly meaningful to reduce individual repeated layers, and there is also a synchronization problem of UpperDir.

Specifically, the image size of a GPU container is much larger than that of a general image. Taking the tensorflow image in deep learning as an example, its official image size on dockerhub is 3.2G, compared with the CPU common image Centos is only 234M, the former is the latter 14 times that of a container, and this is only a native image of a container in the POD. Considering the operations of CPU and GPU resource calls and status records of the algorithm container group, its transmission capacity is far greater than that of pure CPU containers. If the general image transmission scheme is used, The delay interruption of the container group will be at the level of minutes or even hours, which is unacceptable in a production environment.

At the same time, the GPU container needs to consider not only the CPU state, but also the GPU state. In the algorithm cloud platform, a Pod often contains multiple algorithms, which are deployed in their respective containers to form Pods of an algorithm cluster. At the same time, because the GPU Due to the high cost of resources, a GPU is often shared by different algorithm groups, and its complexity prevents the current POD hot migration solution for CPU resources from being translated to GPU resource PODs.

To sum up, the current CPU hot migration solution is not suitable for GPU resource containers. The current CPU hot migration solution used in GPU resource containers has problems with the transfer of GPU container image files, fast copy of CPU status, and maintenance of GPU task scheduling. these three issues.

Contents of the invention

The embodiment of the present application provides a method, device and application for thermal migration of GPU resource POD, which solves the problem of GPU container image transmission, achieves fast copying of CPU status, maintains GPU task scheduling, and realizes rapid thermal migration of algorithm container cloud GPU resource POD. migrate.

In the first aspect, an embodiment of the present application provides a method for hot migration and release of a GPU resource POD. The method includes:

The GPU client sends a migration request to the migrating POD and the GPU server that need to be hot-migrated, and the control server of the algorithm container cloud platform confirms receiving the POD through the migration request;

After receiving the migration request, the migration component in the migration POD checks the metadata tree index of the image file in the migration POD to obtain a metadata index cache, and receives the metadata cache index from the POD Migration receiving component, the migration receiving component receiving the POD establishes a file system in the receiving POD according to the metadata cache index, and puts the data corresponding to the metadata cache index into the file system;

Collect the process tree information in the migration POD, and add addressing code in each process of the process tree information, the addressing code is copied to the address space of the corresponding memory when the corresponding process calls the memory and Record the change frequency of each memory in real time, divide the memory into a high-frequency memory block and a low-frequency memory block according to the change frequency, and transmit the high-frequency memory block and the low-frequency memory block as a memory state file to the migration receiver in the component.

In the second aspect, the embodiment of the present application provides a GPU resource POD hot migration device, including:

Request module: the GPU client sends a migration request to the migrating POD and the GPU server that need to be hot-migrated, and the control server of the algorithm container cloud platform confirms receiving the POD through the migration request;

File migration module: after receiving the migration request, the migration component in the migration POD checks the metadata tree index of the mirror file in the migration POD to obtain a metadata index cache, and caches the metadata The migration receiving component of the index receiving POD, the migration receiving component of receiving the POD establishes a file system in the receiving POD according to the metadata cache index, and puts the data corresponding to the metadata cache index into the file in the system;

Memory migration module: collect the process tree information in the migration POD, and add addressing code in each process of the process tree information, and the addressing code is copied to the corresponding memory when the corresponding process calls the memory address space and record the change frequency of each memory in real time, divide the memory into high-frequency memory block and low-frequency memory block according to the change frequency, and transfer the high-frequency memory block and the low-frequency memory block as memory state files to In the migration receiving component.

In a third aspect, the embodiment of the present application provides an electronic device, including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute a GPU resource POD. Thermal migration method.

In a fourth aspect, an embodiment of the present application provides a readable storage medium, where a computer program is stored in the readable storage medium, and the computer program includes a program code for controlling a process to execute the process, and the process includes a A hot migration method for GPU resource PODs.

Main contribution and innovation of the present invention are as follows:

The embodiment of this application introduces the concepts of metadata and data, and cuts the storage layer into fixed-size blocks according to the metadata to generate a flattened metadata tree index. The receiving container does not need to be downloaded in all image files like traditional containers. After it can be started again, it can migrate the mirror file system of the POD through the network call according to the metadata tree index, and cache it after the call. Download the complete image file of the migrated POD in the background. When the complete image file is downloaded, the metadata tree index is replaced, which saves the time for network transmission of large-volume images, thereby solving the problem of GPU container image transmission.

In the embodiment of the present application, the hash value is confirmed for each memory block by subdividing the memory blocks, and the memory is divided into high-frequency memory blocks and low-frequency memory blocks at a certain frequency. The low-frequency memory block is transmitted first, and then the high-frequency memory block is transmitted. The memory blocks are merged and compressed. When the frequency of the high-frequency memory block is reduced, it is transmitted in the form of a low-frequency memory block, or after a time step, so as to reduce the transmission overhead and enhance the stability of the memory. The effect of fast copy of GPU state;

In the embodiment of this application, the migration request of the migration POD for the GPU resource Pod is sent to the GPU server through the GPU client, and the GPU server returns the migration structure, and the GPU state of the load is changed to only need to maintain the GPU state on the GPU server. And re-link to the receiving POD, reducing the coupling between CPU and GPU, so as to realize the state maintenance of GPU.

The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below, so as to make other features, objects, and advantages of the application more comprehensible.

Description of drawings

The drawings described here are used to provide a further understanding of the application and constitute a part of the application. The schematic embodiments and descriptions of the application are used to explain the application and do not constitute an improper limitation to the application. In the attached picture:

FIG. 1 is a flow chart of a method for hot migration of a GPU resource POD according to an embodiment of the present application;

FIG. 2 is a structural block diagram of a thermal migration device for a GPU resource POD according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application.

detailed description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. Implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with aspects of one or more embodiments of the present specification as recited in the appended claims.

It should be noted that in other embodiments, the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or less steps than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step in other embodiments describe.

In order to better understand the technical points of this solution, here is an explanation of the nouns that appear in this solution:

Sidecar container (sidecar container): A sidecar container is a container that should run alongside the main container in a pod, this sidecar container extends and enhances the functionality of the current container without changing it. Today, we know that we use container technology to wrap all dependencies so that applications can run anywhere. Said sidecar container can add some functionality to the current container without touching or changing the main container.

Image file: The main resource content in the POD is stored in the image file of the POD, and the image file needs to be run through the POD.

Time step: Time step refers to the difference between two time points before and after. In this solution, it is a set time difference. When the memory frequency of the high-frequency memory block does not decrease to a certain extent within the set time difference, the High-frequency memory blocks are transferred directly.

Embodiment one

The embodiment of this application provides a method for hot migration of GPU resource POD, which solves the problem of GPU container image transmission, achieves fast copying of CPU status, maintains GPU task scheduling, and realizes fast hot migration of algorithm container cloud GPU resource POD, specifically , with reference to Figure 1, the method includes:

The GPU client sends a migration request to the migrating POD and the GPU server that need to be hot-migrated, and the control server of the algorithm container cloud platform confirms receiving the POD through the migration request;

After receiving the migration request, the migration component in the migration POD checks the metadata tree index of the image file in the migration POD to obtain a metadata index cache, and receives the metadata cache index from the POD Migration receiving component, the migration receiving component receiving the POD establishes a file system in the receiving POD according to the metadata cache index, and puts the data corresponding to the metadata cache index into the file system;

Collect the process tree information in the migration POD, and add addressing code in each process of the process tree information, the addressing code is copied to the address space of the corresponding memory when the corresponding process calls the memory and Record the change frequency of each memory in real time, divide the memory into a high-frequency memory block and a low-frequency memory block according to the change frequency, and transmit the high-frequency memory block and the low-frequency memory block as a memory state file to the migration receiver in the component.

In some embodiments, in the step of "the control server of the algorithm container cloud platform confirms receiving the POD through the migration request", the control server confirms receiving the POD according to the pointing parameter of the migration request, if the migration request If the pointing parameter is empty, the most suitable server node will be automatically allocated as the receiving POD according to the resources and network topology.

Specifically, the control server analyzes the parameters in the migration request to check whether it contains the pointing parameters of the receiving POD. The platform confirms the resource balance of the receiving POD. If the resource balance satisfies the migration condition, the receiving POD is determined. If the resource balance does not meet the migration condition, the migration request fails and a failure result is returned.

Specifically, when the pointing parameter of the migration request is empty, it is considered that the user needs to migrate the migration POD, but does not specify to receive the POD. The amount of resources and network topology to find the most suitable receiving POD.

Further, the resource amount and resource balance of the POD in the container cloud platform can be confirmed through etcd (distributed registration service center).

In some embodiments, since the resource amount of the receiving POD is not synchronized with the resource balance of the receiving POD recorded in the container cloud platform in real time, after confirming the receiving of the POD, the kubelet (control component) of the receiving POD Send a confirmation request to confirm again whether the resource balance in the received POD meets the migration condition.

That is to say, after the step of "the control server of the algorithm container cloud platform confirms receiving the POD through the migration request", a step is included: sending a confirmation request to the control component of the receiving pod, the content of the confirmation request is: confirmation Whether the resource margin of the receiving pod satisfies the migration condition.

In some embodiments, in the step of "the migration component in the migration POD receives the migration request", the migration component of the migration POD joins in the algorithm container cloud platform to obtain the migration request.

In some embodiments, in the step of "checking the metadata tree index of the image file in the migration POD to obtain the metadata cache index", the sidecar container in the migration POD updates the image file in the migration POD The metadata tree index of the file is retrieved to obtain the metadata index cache.

Specifically, the sidecar container retrieves the used data blocks in the metadata tree, and generates the metadata index based on the database where the data blocks are located.

The metadata dendrogram index is a dendrogram index generated according to the data in the image file, and the metadata index cache is the tree corresponding to the used data slice in the metadata dendrogram index Chart index.

Specifically, the data in the mirror file in the migration POD is divided into fixed-size slices, and these data slices are stored in the data layer, and the corresponding metadata dendrogram index is generated according to the data slices in the data layer, Retrieving the metadata dendrogram index, retrieving data slices used in the metadata dendrogram index, and generating a metadata cache index based on these data slices, and storing the metadata cache index corresponding to The data slices are sent to the file system of the receiving POD.

Specifically, the metadata dendrogram index is a self-verifying hash tree, and using the self-verifying hash tree as the metadata dendrogram index can avoid , the content of the file system is inconsistent with the content of the image file; in addition, the transmission is carried out by indexing, so that the rapid transmission of the image file is achieved, and there is no need to wait for the minute-level or even hour-level overall transmission of the large-volume image, and the index The existence of the network also avoids the situation that the image file is inconsistent with the content in the file system due to the impact of network interruption or interruption, thereby completing the rapid transmission of the GPU image file.

In some embodiments, in "the migration receiving component of the receiving POD creates a file system in the receiving POD according to the metadata tree index and the metadata cache index, and stores the metadata cache index Put the corresponding metadata into the file system" step, when the receiving POD uses the file system, it first uses the corresponding data slice through the metadata cache index, when it needs to use the For data slices other than the data slices corresponding to the metadata cache index, the data slices in the image file of the migrated POD are invoked for use through the metadata dendrogram index in the form of network calls.

In some embodiments, in the step of "transmitting the high-frequency memory block and the low-frequency memory block to the migration receiving component as a memory state file", the low-frequency memory block is compressed and combined as a memory The state file is transmitted to the migration receiving component, and when the change frequency of the high-frequency memory block is reduced to the change frequency of the low-frequency memory block, it is regarded as a low-frequency memory block, compressed, merged and transmitted as a memory state file to the migration receiving component, or directly transfer the high-frequency memory block to the migration receiving component after a set time step.

Specifically, in order to display the change frequency of the high-frequency memory block and the low-frequency memory block higher, the frequency change of the high-frequency memory block and the low-frequency memory block is represented by a hash value.

Directly transmitting the high-frequency memory block to the migration receiving component after the set time step means: if the change frequency of the high-frequency memory block is still not changed to a low-frequency memory block after a set time period, in order To ensure the integrity of memory information transmission, it is still necessary to directly transmit high-frequency memory blocks to the migration receiving component.

Further, the migration receiving component merges the high-frequency memory block and the low-frequency memory block, analyzes the resources in the high-frequency memory block and the low-frequency memory block, and The resources of the low-frequency memory block recursively generate a process tree in the receiving POD, prepare a namespace, fill the resources of the high-frequency memory block and the low-frequency memory block in the namespace, and create a socket .

Specifically, the socket is a network programming interface. In this solution, the receiving POD is connected to the network through the socket, and the network and the receiving POD interact through the socket.

Further, after the resources are filled in the received POD, the memory mapping in the migration stage is canceled, and the addressing codes in the high-frequency memory block and the low-frequency memory block are cleared.

Specifically, by judging the change frequency of the memory block, the memory block is divided into high-frequency and low-frequency, and the low-frequency memory block is compressed, combined and transmitted, and the high-frequency memory block is sorted, and the high-frequency memory block is waited until the high-frequency The frequency of change of the sub-memory block is reduced, or the high-frequency memory block is transmitted after the time step, which reasonably reduces the transmission overhead, enhances the stability of memory transmission, and achieves fast copying of the memory state.

In some embodiments, after the GPU server receives the migration request, it traverses the task queue of the migrated POD, uses the number of iterations as the minimum scheduling unit, and performs input and output analysis on the teams in the task queue And the context switch, and make a forward call to the GPU kernel, and then calculate the update dimension of the task gradient backward, and confirm the priority of the task in the task queue.

Specifically, using the GPU server to receive the migration request is to separate the CPU from the GPU, reducing the coupling between the two, forwarding the migration request of the POD migration to the server through the client, and the server returns the calculation result of the GPU. Change the complex GPU state transfer to only need to maintain the GPU state on the server side, and relink the migration POD.

Further, tasks with many input and output operations, frequent context switching, and time-consuming tasks are adjusted to the front of the queue as low-priority tasks for priority execution, so that blocking occurs during the migration process, and high-priority tasks are given priority when the migration is completed and the task is restored. Implemented to improve GPU utilization during normal operation.

Specifically, in order to keep the running of the GPU in a relatively stable state and to facilitate docking with new container nodes, tasks can be artificially created, and the artificially created tasks can be inserted into a queue to freeze GPU process scheduling.

In some embodiments, after the migration of the data in the file system and the memory state file is completed, a hash check is performed on it, and when the hash check is passed, a pass request is sent to the algorithm container cloud platform The control server, the control server connects with the GPU server through the API and combines the network proxy component and DNS service of the algorithm cloud platform to perform network status replacement, and notifies the algorithm container cloud platform to destroy the migration POD.

Further, before the hash verification is performed, the data in the file system should be the same as the complete image file of the migrated POD.

As mentioned above, compared with the existing technology, a GPU resource POD hot migration method provided by this solution can achieve:

1. Solve the problem of GPU container image transmission:

Introduce the concepts of metadata and data. Metadata is the description of data. Specifically, the data in the metadata layer is a self-verifying hash tree, and the data layer is divided into fixed-size slices. All data is divided into data slices and stored in the data layer, and corresponding metadata dendrogram indexes are generated at the same time. When the GPU container group starts live migration, the metadata tree index of the image file is retrieved, the used data slices are retrieved, and a metadata index cache is generated based on these used data slices.

Send the metadata index cache to the receiving POD, and the receiving POD creates a file system on the receiving POD according to the flattened metadata tree index and the metadata index cache, and stores the metadata index The data corresponding to the cache is synchronized to the file system, and the receiving POD first uses the corresponding data through the metadata index cache to use the data in the file system. If necessary, use data other than the data corresponding to the metadata index cache , the image file in the migration POD is called in the form of a network call through the metadata tree index, and the existence of the metadata index cache avoids the problem of network interruption and the need for retransmission. On the other hand , the verification function of the hash also avoids the problem that the data in the migrating POD image file is inconsistent with the data in the receiving POD file system. The minute-level or even hour-level overall transmission of the volume image, and the existence of the metadata tree index also avoid the problem of inconsistency between the image file and the data in the file system caused by the impact of network interruption or interruption.

2. Fast copy of CPU state:

Memory state copy: According to the recursive collection of the process tree information formed by the container process and its sub-processes of the migration POD process, inject addressing code into the process of the migration POD, when the container process calls memory mapping, the addressing code The address code will be copied to the corresponding memory address space, record the frequency of subsequent memory changes, distinguish high-frequency memory blocks and low-frequency memory blocks according to the frequency of memory changes, and make a hash representation, compress and merge the low-frequency memory blocks, Transfer to the migration receiving component of the receiving POD. For the high-frequency memory block, after its calling frequency decreases, it will be transferred to a low-frequency memory block for transmission, or it will be transferred to the migration receiving component of the receiving POD after a time step. in the component.

Restoration of memory state: Merge transmitted high-frequency memory blocks and low-frequency memory blocks, analyze resources in the process, recursively generate process trees that need to be restored, prepare namespaces, fill memory data, create sockets, and cancel memory in the copy stage Mapping and clearing the injected addressing code, combining the high-frequency memory block with the low-frequency memory block reduces transmission overhead and enhances the stability of memory transmission.

3. GPU state maintenance

By using the client to send the migration request and the server to receive the migration request, the CPU and GPU are separated, the coupling between the two is reduced, and the POD and GPU are separated. Specifically, the migration request of the migrated POD to the GPU is forwarded to the server through the client, and the server returns a migration result calculated by the GPU. This method changes the complex GPU state transfer to only need to maintain the GPU state on the server and relink the receiving POD.

In particular, the following optimizations have been made to the running state of the GPU on the server to make it more suitable for live migration. The number of iterations is used as the smallest unit of scheduling to distinguish GPU high-priority tasks from low-priority tasks. Low-priority tasks often have more input and output and context switching. During the migration process, by adjusting the task priority, the low-priority tasks Move to the front of the queue. At the same time, optional, artificially created tasks are inserted into the queue to freeze the GPU process scheduling. The priority execution of low-priority tasks keeps the GPU running in a relatively stable state, which is convenient for docking with the receiving POD. When the migration is completed and the task is restored, the high-priority tasks are prioritized to improve the utilization of the GPU during normal operation.

By solving the problem of GPU container image transmission, achieving fast copy of CPU state, and maintaining the state of GPU, the purpose of fast hot migration of GPU resource POD is achieved. Second-level hot migration is basically imperceptible to users, and the maintenance of running status is of great significance for algorithms with long task chains and long computing times, which enables the algorithm container to achieve flexible scheduling and collocation, and enhances the robustness of the algorithm container platform and resource utilization efficiency.

Embodiment two

Based on the same idea, referring to Figure 2, this application also proposes a thermal migration device for GPU resource POD, including:

Request module: the GPU client sends a migration request to the migrating POD and the GPU server that need to be hot-migrated, and the control server of the algorithm container cloud platform confirms receiving the POD through the migration request;

File migration module: after receiving the migration request, the migration component in the migration POD checks the metadata tree index of the mirror file in the migration POD to obtain a metadata index cache, and caches the metadata The migration receiving component of the index receiving POD, the migration receiving component of receiving the POD establishes a file system in the receiving POD according to the metadata cache index, and puts the data corresponding to the metadata cache index into the file in the system;

Memory migration module: collect the process tree information in the migration POD, and add addressing code in each process of the process tree information, and the addressing code is copied to the corresponding memory when the corresponding process calls the memory address space and record the change frequency of each memory in real time, divide the memory into high-frequency memory block and low-frequency memory block according to the change frequency, and transfer the high-frequency memory block and the low-frequency memory block as memory state files to In the migration receiving component.

Embodiment Three

This embodiment also provides an electronic device, referring to FIG. 3 , including a memory 404 and a processor 402, the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to perform any one of the methods described above. steps in the example.

Specifically, the processor 402 may include a central processing unit (CPU), or an Application Specific Integrated Circuit (ASIC for short), or may be configured to implement one or more integrated circuits in the embodiments of the present application.

Wherein, the memory 404 may include a mass memory 404 for data or instructions. For example without limitation, the memory 404 may include a hard disk drive (HardDiskDrive, referred to as HDD), a floppy disk drive, a solid state drive (SolidStateDrive, referred to as SSD), flash memory, optical disk, magneto-optical disk, tape or Universal Serial Bus (UniversalSerialBus, abbreviated as USB) drive or a combination of two or more of the above. Storage 404 may include removable or non-removable (or fixed) media, where appropriate. Memory 404 may be internal or external to the data processing arrangement, where appropriate. In a particular embodiment, memory 404 is a non-volatile (Non-Volatile) memory. In a specific embodiment, the memory 404 includes a read-only memory (Read-Only Memory, ROM for short) and a random access memory (Random Access Memory, RAM for short). In appropriate cases, the ROM can be mask programmed ROM, programmable ROM (Programmable Read-Only Memory, referred to as PROM), erasable PROM (Erasable Programmable Read-Only Memory, referred to as EPROM), electrically erasable PROM (Electrically Erasable Programmable Read -OnlyMemory, referred to as EEPROM), electrically rewritable ROM (Electrically Alterable Read-OnlyMemory, referred to as EAROM) or flash memory (FLASH) or a combination of two or more of these. In appropriate cases, the RAM can be a static random access memory (Static Random-Access Memory, referred to as SRAM) or a dynamic random access memory (Dynamic Random Access Memory, referred to as DRAM), where the DRAM can be a fast page mode dynamic random access Memory 404 (FastPageModeDynamicRandomAccessMemory, referred to as FPMDRAM), extended data output dynamic random access memory (ExtendedDateOutDynamicRandomAccessMemory, referred to as EDODRAM), synchronous dynamic random access memory (SynchronousDynamicRandom-AccessMemory, referred to as SDRAM), etc.

The memory 404 may be used to store or cache various data files required for processing and/or communication, and possibly computer program instructions executed by the processor 402 .

The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the GPU resource POD hot migration methods in the foregoing embodiments.

Optionally, the above-mentioned electronic device may further include a transmission device 406 and an input-output device 408 , wherein the transmission device 406 is connected to the above-mentioned processor 402 , and the above-mentioned input-output device 408 is connected to the above-mentioned processor 402 .

Transmission device 406 may be used to receive or transmit data via a network. The specific example of the above network may include a wired or wireless network provided by the communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, NIC for short), which can be connected to other network devices through a base station so as to communicate with the Internet. In an example, the transmission device 406 may be a radio frequency (Radio Frequency, RF for short) module, which is used to communicate with the Internet in a wireless manner.

Input and output devices 408 are used to input or output information. In this embodiment, the input information may be a migration request for POD migration, high-frequency memory blocks and low-frequency memory blocks, metadata tree index, etc., and the output information may be migration results, hash verification results, etc.

Optionally, in this embodiment, the processor 402 may be configured to execute the following steps through a computer program:

S101. The GPU client sends a migration request to the migrating POD and the GPU server that need hot migration, and the control server of the algorithm container cloud platform confirms receiving the POD through the migration request;

S102. After receiving the migration request, the migration component in the migration POD checks the metadata tree index of the mirror file in the migration POD to obtain a metadata index cache, and receives the metadata cache index The migration receiving component of the POD, the migration receiving component of the receiving POD establishes a file system in the receiving POD according to the metadata cache index, and puts the data corresponding to the metadata cache index into the file system ;

S103. Collect process tree information in the migrated POD, and add an addressing code in each process of the process tree information, and the addressing code is copied to the address of the corresponding memory when the corresponding process calls the memory Space and record the change frequency of each memory in real time, divide the memory into high-frequency memory block and low-frequency memory block according to the change frequency, and transmit the high-frequency memory block and the low-frequency memory block as memory state files to the Migrate in the receiving component.

It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the foregoing embodiments and optional implementation manners, and details will not be repeated in this embodiment.

In general, the various embodiments may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto. Although various aspects of the present invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, as non-limiting examples, these blocks, devices, systems, techniques or methods described herein may be presented in hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof.

Embodiments of the invention may be implemented by computer software executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and/or macros can be stored on any device-readable data storage medium and they include program instructions for performing certain tasks. A computer program product may include one or more computer-executable components configured to perform an embodiment when the program is run. One or more computer-executable components may be at least one software code or a portion thereof. Also in this regard, it should be noted that any blocks of the logic flow as in FIG. 3 may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. Software may be stored on physical media such as memory chips or memory blocks implemented within the processor, magnetic media such as hard or floppy disks, and optical media such as eg DVD and its data variants, CD. Physical media are non-transient media.

Those skilled in the art should understand that the various technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, all possible combinations of the various technical features in the above embodiments are not described. There is no contradiction in the combination, and all should be considered as within the scope of the description.

The above examples only express several implementation modes of the present application, and the description thereof is relatively specific and detailed, but should not be construed as limiting the scope of the present application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be determined by the appended claims.

Claims (9)

1. A hot migration method of GPU resource POD comprises the following steps:

the method comprises the steps that a GPU client sends a migration request to a migration POD and a GPU server side which need to be subjected to thermal migration, and a control server of an algorithm container cloud platform confirms to receive the POD through the migration request;

after receiving the migration request, a migration component in the migration POD checks a metadata tree graph index of an image file in the migration POD to obtain a metadata cache index, a sidecar container in the migration POD retrieves the metadata tree graph index of the image file in the migration POD to obtain the metadata cache index, the metadata tree graph index is generated according to a data slice in the image file, the metadata cache index is a tree graph index corresponding to a used data slice in the metadata tree graph index, the metadata cache index is sent to a migration receiving component of the receiving POD, the migration receiving component of the receiving POD establishes a file system in the receiving POD according to the metadata cache index, and data corresponding to the metadata cache index is placed in the file system;

the method comprises the steps of collecting process tree information in a migration POD, adding an addressing code in each process of the process tree information, copying the addressing code to an address space of a corresponding memory when the corresponding process calls the memory, recording the change frequency of each memory in real time, dividing the memory into a high-frequency memory block and a low-frequency memory block according to the change frequency, and transmitting the high-frequency memory block and the low-frequency memory block to a migration receiving assembly as memory state files.

2. A method according to claim 1, wherein in the step of "control server of algorithm container cloud platform confirms receiving POD by said migration request", said control server confirms receiving POD according to pointing parameter of said migration request, if pointing parameter of said migration request is empty, then automatically allocating optimum server node as receiving POD according to resource and network topology.

3. The method according to claim 1, wherein in the step of "after the migration request is received by the migration component in the migration POD", the migration component of the migration POD is added to the algorithm container cloud platform to obtain the migration request.

4. The method according to claim 1, wherein in the step of creating, by the migration receiving component of the receiving POD, a file system in the receiving POD according to the metadata tree index and the metadata cache index, and placing metadata corresponding to the metadata cache index into the file system, the receiving POD uses its corresponding data slice through the metadata cache index first when using the file system, and calls, in a network call manner, a data slice in the image file of the migration POD for use through the metadata tree map index when needing to use a data slice other than the data slice corresponding to the metadata cache index.

5. The method according to claim 1, wherein in the step of "transmitting the high-frequency memory block and the low-frequency memory block as the memory state files to the migration receiving component", the low-frequency memory block is compressed and merged and then transmitted as the memory state files to the migration receiving component, and when a change frequency of the high-frequency memory block decreases to the change frequency of the low-frequency memory block, the high-frequency memory block is regarded as the low-frequency memory block, compressed and merged and then transmitted as the memory state files to the migration receiving component, or the high-frequency memory block is directly transmitted to the migration receiving component after a time step.

6. The method according to claim 1, wherein after the GPU server receives the migration request, traversing a task queue of the migration POD, performing input/output analysis and context switching on queues in the task queue with an iteration number as a minimum scheduling unit, and calling a GPU kernel forward, and then determining priorities of tasks in the task queue by calculating update dimensions of task gradients backward.

7. A device for hot migration of GPU resource PODs, comprising:

a request module: the method comprises the steps that a GPU client sends a migration request to a migration POD and a GPU server side which need to be subjected to thermal migration, and a control server of an algorithm container cloud platform confirms to receive the POD through the migration request;

a file migration module: after receiving the migration request, a migration component in the migration POD checks a metadata tree map index of an image file in the migration POD to obtain a metadata cache index, a sidecar container in the migration POD retrieves the metadata tree map index of the image file in the migration POD to obtain the metadata cache index, the metadata tree map index is generated according to data slices in the image file, the metadata cache index is a tree map index corresponding to a used data slice in the metadata tree map index, the metadata cache index is sent to a migration receiving component of the receiving POD, and the migration receiving component of the receiving POD establishes a file system in the receiving POD according to the metadata cache index and places data corresponding to the metadata cache index into the file system;

a memory migration module: the method comprises the steps of collecting process tree information in a migration POD, adding an addressing code in each process of the process tree information, copying the addressing code to an address space of a corresponding memory when the corresponding process calls the memory, recording the change frequency of each memory in real time, dividing the memory into a high-frequency memory block and a low-frequency memory block according to the change frequency, and transmitting the high-frequency memory block and the low-frequency memory block serving as memory state files to a migration receiving assembly.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to perform a method for hot migration of GPU resources POD according to any of claims 1-6.

9. A readable storage medium having stored therein a computer program comprising program code for controlling a process to execute a process, the process comprising a method for hot-migration of a GPU resource POD according to any of claims 1-6.

Images