CN119718216B

CN119718216B - Method and device for storing metadata on disk

Info

Publication number: CN119718216B
Application number: CN202510224462.XA
Authority: CN
Inventors: 李超; 于建彬; 周春法
Original assignee: Suzhou Metabrain Intelligent Technology Co Ltd
Current assignee: Suzhou Metabrain Intelligent Technology Co Ltd
Priority date: 2025-02-27
Filing date: 2025-02-27
Publication date: 2025-05-06
Anticipated expiration: 2045-02-27
Also published as: CN119718216A

Abstract

The application discloses a metadata disc-dropping method and device, which relate to the technical field of storage software, because a virtual address set and an address set identifier with a corresponding relation are configured, and the virtual address and the physical address with the corresponding relation are stored in the address set identifier, after the virtual addresses of a plurality of target metadata are acquired, the corresponding address set identifiers are matched for the virtual address set where each target virtual address is located in the virtual address set and the address set identifier with the corresponding relation, then the physical addresses corresponding to each target virtual address are searched in the target address set indicated by each target set identifier in parallel, and the physical addresses are searched in a large range including a plurality of small ranges in parallel instead of the physical addresses in parallel, so that the addressing speed of the metadata is accelerated, the technical problem of lower disc-dropping efficiency of the metadata is solved, and the technical effect of improving the disc-dropping efficiency of the metadata is achieved.

Description

Metadata disc-dropping method and device

Technical Field

The application relates to the technical field of storage software, in particular to a method and a device for dropping metadata.

Background

Metadata is used as data of data, and describes information such as characteristics, structures, positions, rights and the like of the data, so that the metadata can help us to better organize, search and understand the data. Metadata disk drop management refers to the fact that metadata is persisted from a memory to external physical storage such as a disk, which is important for ensuring the integrity and persistence of data, and particularly in the field of data storage and management, metadata of the data can be ensured not to be lost even under the condition of system failure, so that the recoverability and consistency of the data are ensured. The current metadata disc-dropping method is dependent on serial searching of a plurality of metadata external storage space addresses, and in the case of a large amount of metadata to be disc-dropped, the disc-dropping method consumes a large amount of time resources and operation resources, and the disc-dropping efficiency of the metadata is not high.

Disclosure of Invention

The application provides a method and a device for dropping metadata, which at least solve the problem of lower efficiency of dropping metadata in the related technology.

The application provides a metadata disc-dropping method which comprises the steps of obtaining virtual addresses of a plurality of target metadata to be dropped in a memory to obtain a plurality of target virtual addresses, matching corresponding address set identifiers for virtual address sets where all target virtual addresses are located in a virtual address set and address set identifiers with corresponding relations to obtain a plurality of target set identifiers, wherein physical address sets indicated by the address set identifiers store physical addresses corresponding to all virtual addresses in a target external storage space in the corresponding virtual address sets, searching physical addresses corresponding to all target virtual addresses in the target address sets indicated by all target set identifiers in parallel to obtain a plurality of target physical addresses, and dropping the plurality of target metadata onto the target physical addresses corresponding to the target external storage space.

The application further provides a metadata disc-dropping device, which comprises an acquisition module, a matching module and a disc-dropping module, wherein the acquisition module is used for acquiring virtual addresses of a plurality of target metadata to be dropped in a memory to obtain a plurality of target virtual addresses, the matching module is used for matching corresponding address set identifiers for the virtual address sets where all target virtual addresses are located in a virtual address set and an address set identifier with corresponding relations to obtain a plurality of target set identifiers, the physical address set indicated by the address set identifier stores physical addresses corresponding to all virtual addresses in a target external storage space in the corresponding virtual address set, the search module is used for searching physical addresses corresponding to all target virtual addresses in the target address set indicated by all target set identifiers in parallel to obtain a plurality of target physical addresses, and the disc-dropping module is used for dropping the plurality of target metadata onto the corresponding target physical addresses in the target external storage space.

The application also provides electronic equipment, which comprises a memory and a processor, wherein the memory is used for storing a computer program, and the processor is used for realizing the step of the disc-dropping method of any metadata when executing the computer program.

The application also provides a computer readable storage medium in which a computer program is stored, wherein the computer program when executed by a processor implements the steps of the method for dropping metadata of any of the above.

The application also provides a computer program product comprising a computer program which when executed by a processor implements the steps of the landing method for any of the above metadata.

According to the application, as the virtual address set and the address set identifier with the corresponding relation are configured, and the virtual address and the physical address with the corresponding relation are stored in the address set identifier, after the virtual addresses of a plurality of target metadata to be dropped in the memory are obtained, the corresponding address set identifier is matched with the virtual address set in which each target virtual address is positioned in the virtual address set and the address set identifier with the corresponding relation, then the physical address corresponding to each target virtual address is searched in the target address set indicated by each target set identifier in parallel, and the mode of searching the physical address corresponding to the virtual address in each small range in parallel is used for replacing the mode of searching the physical address in a large range comprising a plurality of small ranges in series, so that the addressing speed of the metadata is accelerated, the technical problem that the dropping efficiency of the metadata is lower can be solved, and the technical effect of improving the dropping efficiency of the metadata can be achieved.

Drawings

For a clearer description of embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described, it being apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to the drawings without inventive effort for those skilled in the art.

FIG. 1 is a hardware block diagram of a server device of a method for dropping metadata according to an embodiment of the present application;

FIG. 2 is a flow chart of a method of dropping metadata in accordance with an embodiment of the present application;

FIG. 3 is a schematic diagram of a method for dropping metadata onto a physical disk according to an embodiment of the present application;

FIG. 4 is a mapping relationship between a physical address set and a memory and a physical disk according to an embodiment of the present application;

FIG. 5 is a diagram illustrating a first physical address set capacity adjustment according to an embodiment of the present application;

FIG. 6 is a diagram illustrating a second physical address set capacity adjustment according to an embodiment of the present application;

fig. 7 is a block diagram of a structure of a disc-drop apparatus of metadata according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without making any inventive effort are within the scope of the present application.

It should be noted that in the description of the present application, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. The terms "first," "second," and the like in this specification are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order.

The present application will be further described in detail below with reference to the drawings and detailed description for the purpose of enabling those skilled in the art to better understand the aspects of the present application.

The specific application environment architecture or specific hardware architecture upon which the execution of the drop method of metadata depends is described herein.

The method embodiments provided in the embodiments of the present application may be executed in a server apparatus or similar computing device. Taking the example of running on a server device, fig. 1 is a hardware block diagram of a server device of a method for dropping metadata according to an embodiment of the present application. As shown in fig. 1, the server device may include one or more (only one is shown in fig. 1) processors 102 (the processor 102 may include, but is not limited to, a microprocessor MCU, a programmable logic device FPGA, or the like processing means) and a memory 104 for storing data, wherein the server device may further include a transmission device 106 for communication functions and an input-output device 108. It will be appreciated by those of ordinary skill in the art that the architecture shown in fig. 1 is merely illustrative and is not intended to limit the architecture of the server apparatus described above. For example, the server device may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.

The memory 104 may be used to store a computer program, for example, a software program of application software and a module, such as a computer program corresponding to a starting method of an operating system in an embodiment of the present application, and the processor 102 executes the computer program stored in the memory 104 to perform various functional applications and data processing, that is, implement the above-mentioned method. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located with respect to the processor 102, which may be connected to the server device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 106 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of a server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, simply referred to as a NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is configured to communicate with the internet wirelessly.

The embodiment of the application provides a method for dropping metadata, which is described in detail by combining the execution flow of the method for dropping metadata.

The following explains the terms of art appearing in the present application:

Metadata defines that Metadata (Metadata) is "data of data" that describes information of characteristics, structure, content, context, quality, format, storage location, owner, access rights, and the like of data. Metadata is useful to aid in the management and understanding of data, and can be considered as ancillary information to support the retrieval, access, control, integration, and maintenance of data. Metadata is a very critical component in data management and information systems that can improve discoverability, accessibility, understandability, and manageability of data, thereby ensuring efficient utilization of data and implementation of data governance.

Drop disc definition-drop disc refers to the process of persisting data stored in a volatile storage (e.g., memory) onto a non-volatile storage medium (e.g., hard disk, SSD, etc.). In computer systems, memory is volatile, i.e., the data it stores is lost after the system is powered down or restarted, while external storage devices such as hard disks are persistent storage locations for the data. Thus, landing a disk is a critical step in ensuring data security and persistence, and it writes data from memory to disk without losing data even if the system experiences a failure.

Memory is also known as main memory or RAM (random access memory) and is the volatile memory used in computers to store running programs and data. The memory is directly communicated with the CPU, and the read-write speed of the memory is far faster than that of external storage devices such as a hard disk and the like, so that the memory is a quick temporary storage space when a computer processes data. The access speed of the memory is high, but the data can disappear after the power is off, so that the important data is required to be persisted by matching with a disk-dropping mechanism.

External storage space defines the storage space provided by storage devices in a computer system other than memory, such as hard disk drives, solid State Drives (SSDs), USB flash drives, network storage, and the like. These storage devices typically provide a large amount of persistent storage, with data remaining after power failure. Because of their slower speed, they are mainly used to store large amounts of data that are not accessed frequently or that do not require frequent modification.

In this embodiment, a method for playing metadata is provided, and fig. 2 is a flowchart of a method for playing metadata according to an embodiment of the present application, as shown in fig. 2, where the method includes the following steps:

Step S202, obtaining virtual addresses of a plurality of target metadata of a disk to be dropped in a memory to obtain a plurality of target virtual addresses;

Step S204, matching corresponding address set identifiers for virtual address sets where all target virtual addresses are located from virtual address sets and address set identifiers with corresponding relations to obtain a plurality of target set identifiers, wherein physical address sets indicated by the address set identifiers store physical addresses corresponding to all virtual addresses in the corresponding virtual address sets in a target external storage space;

step S206, searching the physical addresses corresponding to the target virtual addresses in the target address sets indicated by the target set identifiers in parallel to obtain a plurality of target physical addresses;

And step S208, landing the plurality of target metadata on the corresponding target physical addresses in the target external storage space.

Through the steps, as the virtual address set and the address set identifier with the corresponding relation are configured, and the virtual address and the physical address with the corresponding relation are stored in the address set identifier, after the virtual addresses of a plurality of target metadata to be dropped in the memory are obtained, the corresponding address set identifier is matched with the virtual address set in which each target virtual address is positioned in the virtual address set and the address set identifier with the corresponding relation, then the physical address corresponding to each target virtual address is searched in the target address set indicated by each target set identifier in parallel, and the parallel mode of searching the physical address corresponding to the virtual address in each small range is used for replacing the serial mode of searching the physical address in a large range comprising a plurality of small ranges, so that the addressing speed of the metadata is accelerated, the technical problem that the dropping efficiency of the metadata is lower can be solved, and the technical effect of improving the dropping efficiency of the metadata is achieved.

Optionally, in this embodiment, the method of dropping metadata includes, but is not limited to, being cooperatively performed by one or more components of a Central Processing Unit (CPU), a memory management unit (Memory Management Unit, MMU), a direct memory access Controller (DMA Controller), an Operating System (OS), a file System (FILE SYSTEM), a dedicated memory Controller, a multi-core processor, or a parallel processing unit.

Optionally, in this embodiment, the method includes, but is not limited to, designing an intelligent metadata management module (SMART METADATA MANAGEMENT, SMM) that mainly applies a programmable logic device such as an ARM on a board card, and executing the method for dropping metadata in the present application together with the detection module, the front-end module, the main control module, the back-end module, and the disk array module. The SMM module may dynamically manage the detection module (for detecting CPU utilization, memory utilization, CPU model, memory model, etc.), the front-end module (for IO queue management), the master-end module (for metadata management), the back-end module (for metadata disk-drop management), and the disk array module (for hard disk management). The front end module processes IO read-write request in real time, the detection module obtains data of related detection objects, the data are transmitted to the main control module, the main control module processes the current environment configuration data, a corresponding metadata processing method is selected, and the back end module executes the metadata processing method of the main control module to implement metadata landing writing operation. In addition, the SMM can synchronize the current intelligent metadata management strategy in real time through the detection module, the front end module, the master control module, the back end module and the disk array module.

In the embodiment provided in step S202, the metadata is persisted from the memory to the target external storage space, and the metadata is not limited to the metadata being first dropped to the target external storage space or the metadata being changed and then modified to be covered by the metadata being dropped to the target external storage space, which is not particularly limited in the present application.

Optionally, in this embodiment, the virtual address is used to indicate a storage location of the target metadata in the memory.

Optionally, in this embodiment, obtaining virtual addresses of a plurality of target metadata of the disc to be landed in the memory includes, but is not limited to, implementing metadata management at a software level. Specifically, in the memory, the mapping relationship of the metadata is maintained through a data structure (such as a hash table, a tree, a linked list, etc.). When metadata needs to be dropped, software (such as an operating system, a file system or an application program) obtains a virtual address of the metadata in the memory by querying the data structures. For example, a metadata index table may be maintained that contains an identifier for each piece of metadata and its corresponding virtual address of memory. The address acquisition mechanism may be triggered automatically when the software performs a metadata write operation. For example, when system calls or API functions are called to write metadata, the functions internally perform metadata lookup and virtual address retrieval. In particular, this may involve traversing the data structure of metadata management until a record of the target metadata is found and the virtual address is read therefrom. The specific steps of the software layer implementation include maintaining a hash table, taking the ID of metadata as a key and taking the virtual address as a value, inquiring the hash table by the metadata ID to obtain the virtual address when a metadata disc-dropping request exists, reading the metadata from the memory by using the obtained virtual address, and then writing the metadata into the target external storage space by the software.

Optionally, in this embodiment, obtaining virtual addresses of multiple target metadata of the to-be-dropped disc in the memory further includes, but is not limited to, acceleration and optimization at a hardware level. Specifically, at the hardware level, the address acquisition process may be optimized by a dedicated hardware acceleration unit. For example, the foregoing intelligent metadata management module (SMM) may be integrated with a hardware accelerator for quickly finding and retrieving virtual addresses of metadata. When the system detects a metadata write operation, the hardware accelerator can retrieve the virtual address of the target metadata directly from the internal cache or private storage without requiring complex software processing by the CPU. The specific steps of the hardware level implementation include preloading metadata management structures inside the SMM module for fast lookup of virtual addresses of metadata. When a metadata-drop signal is received, the SMM module looks up the virtual address of the target metadata directly from the preloaded data structure through internal hardware logic.

In the embodiment provided in step S204, the set of virtual addresses, i.e. the set of virtual addresses in the memory, may include, but is not limited to, one or more virtual address segments, or a plurality of virtual addresses that are not consecutive, or a combination of the two, which is not specifically limited in the present application.

Alternatively, in this embodiment, the address set identifier is used to uniquely identify (or uniquely indicate) a set of physical addresses, one set of physical addresses having its address set identifier uniquely determined. For example, the address set of physical address set 1 is identified as Page1, and the address set of physical address set 2 is identified as Page2.

Optionally, in this embodiment, each physical address is used to indicate each storage location in the target external storage space.

Optionally, in this embodiment, the physical address sets store virtual addresses and physical addresses having a corresponding relationship, and the virtual addresses and physical addresses having a corresponding relationship stored in each physical address set are different from each other.

Alternatively, in this embodiment, the capacities of the virtual address and the physical address that can be stored in each physical address set and have a corresponding relationship may be the same or different. Including but not limited to setting the same capacity for each set of physical addresses in the case where there is little gap in access requirements for each metadata, and setting different capacities for each set of physical addresses in the case where there is a large gap in access requirements for each metadata.

Optionally, in this embodiment, matching the corresponding address set identifier for the virtual address set where each target virtual address is located from the virtual address set and the address set identifier having a corresponding relationship includes, but is not limited to, implementation in any one of the following three manners:

the first implementation mode is that the hash table is utilized for fast matching:

When the system is initialized, a hash table is built for each virtual address set, the key of the hash table is the identifier of the virtual address set, and the value is the corresponding physical address set identifier. When the target virtual address is obtained, the virtual address set identifier to which the address belongs is firstly determined, and then the corresponding physical address set identifier is quickly matched through the inquiry of a hash table.

The second implementation mode is that the multi-level index based on the tree structure:

An index is constructed using a tree structure (e.g., a B-tree, B + tree, or Trie tree), wherein nodes of the index store mappings of virtual address set identifications and corresponding physical address set identifications, and leaf nodes store specific physical address information. For each target virtual address to be dropped, recursively traversing the tree structure from the root node according to the virtual address set identifier to which the target virtual address belongs until the corresponding physical address set identifier is found.

Implementation three, using special hardware accelerator:

A dedicated hardware accelerator is designed to accelerate the addressing and landing process of metadata. In the initialization stage, the mapping relation between the virtual address set identifier and the physical address set identifier is loaded into a cache of the hardware accelerator. When the target virtual address is received, the hardware accelerator directly performs matching and searching operations without CPU intervention of a software layer. The accelerator quickly finds out the corresponding address set identifier through internal comparison and logic operation, and directly accesses the hardware area storing the physical address set.

Each of the above implementations has its specific application scenarios and advantages. The hash table implementation provides fast matching when data is randomly distributed, the tree structure implementation is suitable for scenes where data is orderly distributed and hierarchical recursive lookup is required, and the special hardware accelerator is excellent in high-performance and high-concurrency applications. In practical application, the most suitable implementation mode can be selected according to the performance requirement, the data distribution characteristic and the hardware resource of the system.

In the embodiment provided in step S206, the physical addresses corresponding to the target virtual addresses are searched in parallel from the target address sets indicated by the target set identifiers, including, but not limited to, directly recording a plurality of virtual addresses and physical addresses with corresponding relations in the target address sets, and then, directly searching the physical addresses corresponding to the target virtual addresses in the target address sets indicated by the target set identifiers in parallel, or further recording a multi-layer mapping structure in the target address sets, and then, searching the physical addresses corresponding to the target virtual addresses in the small sets, wherein the address small sets directly recording the virtual addresses and the physical addresses with corresponding relations are found in the lowest layer by layer.

Alternatively, in this embodiment, the step S206 may be performed only if the address set identifier corresponding to the virtual address set where each target virtual address is located is found to be a plurality of different address set identifiers, or the address set identifier corresponding to each target virtual address may be found in the target time period, and the step S206 may be performed after the target time period is completed.

Alternatively, in the present embodiment, the physical address may be stored in a storage space other than the memory, for example, may be stored in the target external storage space, or may be stored in another external storage space. The virtual address set and the address set identifier with the corresponding relation can be stored in the memory, and the double-layer mapping of the addresses can be realized by matching with the physical address set indicated by the address set identifier recorded outside the memory, so that the manageable hard disk space (namely the target external storage space) of the storage system can be greatly expanded. When the hard disk (namely, the target external storage space) to be managed is enlarged, the occupied space of the hard disk management data (namely, the virtual address set and the address set identification with the corresponding relation) on the memory is not required to be enlarged, and the memory space of the physical address set indicated by the address set identification is only required to be expanded outside the memory, and then the content of the mapping relation in the memory is correspondingly modified, so that the management of the hard disk space (namely, the target external storage space) in a larger range can be realized.

Optionally, in this embodiment, searching, in parallel, the physical address corresponding to each target virtual address from the target address set indicated by each target set identifier may be implemented, but is not limited to, by the following three ways:

the implementation mode one is that based on parallel searching of the multi-core processor:

With multi-core processors in modern computer systems, a separate lookup thread is assigned to each target set identification. Each thread is responsible for looking up a mapping of virtual addresses to physical addresses from a corresponding set of target addresses. After all threads complete searching, the collected multiple target physical addresses are summarized, and the next metadata disc-falling operation is prepared.

The second implementation mode is to use a special hardware accelerator to search in parallel:

A dedicated hardware accelerator, such as FPGA (Field-Programmable gate array) or ASIC (Application-SPECIFIC INTEGRATED Circuit), was developed to accelerate the parallel lookup process of virtual addresses to physical addresses. The hardware accelerator comprises a plurality of searching units, and each searching unit can independently perform searching operation. Each searching unit reads data in parallel from the address set indicated by the target set identifier, and the matched physical address is quickly found through built-in high-speed comparison logic. Furthermore, the search result can be directly processed by the hardware accelerator without returning to the CPU for further processing, and the accelerator directly writes the metadata into the appointed physical address, thereby reducing the burden of the CPU and the data transmission delay.

Implementation mode III, parallel searching based on heterogeneous computing platform:

The search tasks are assigned to a plurality of stream processors (STREAMING MULTIPROCESSORS) of the GPU using the high-parallel computing power of the GPU. The GPU can process the searching of a large number of target virtual addresses at the same time, and the searching efficiency is remarkably improved. Mapping the target virtual address set and the corresponding physical address set into the GPU memory, slicing the data, and enabling each stream processor to be responsible for searching a part of data. Once the search is completed, the results are transferred directly from the GPU memory to the main memory by DMA (direct memory access) or other techniques, and then uniformly dropped to the target external memory space. The method can fully utilize the parallel computing advantage of the GPU and accelerate data processing.

By the method, the physical address searching efficiency in the metadata disc-falling process can be remarkably improved, and further the overall storage performance and the system response speed are improved. Each implementation has its unique advantages, and the specific choice of which should be determined by the hardware configuration, software environment, and specific performance requirements of the metadata-based landing system.

In the embodiment provided in step S208, the plurality of target metadata are landed onto the corresponding target physical addresses in the target external storage space, including but not limited to, according to the division of the physical address sets, when the metadata stored in the virtual addresses corresponding to all the physical addresses in one physical address set need to be landed, all the metadata corresponding to one physical address set are landed together, or when the plurality of target physical addresses are searched in parallel from the target address set indicated by each target set identifier, the metadata corresponding to the plurality of target physical addresses are landed together into the target external storage space in parallel.

Optionally, in this embodiment, the landing of the plurality of target metadata onto the corresponding target physical address in the target external storage space is achieved by, but not limited to, the following three ways:

implementation one, using DMA (Direct Memory Access ) technology:

The DMA controller is configured to recognize the target physical address and communicate directly with the external memory space without CPU intervention. Once the target physical address is obtained, the DMA controller simultaneously starts a plurality of write channels to write a plurality of metadata in parallel to the corresponding physical address of the external storage. The method avoids the bottleneck of the CPU in the data transmission process and improves the writing speed. After the writing is completed, the DMA controller sends an interrupt or completion signal to the CPU, and the CPU processes the signals according to the requirement, but the influence of the data transmission process on the CPU is minimal.

The implementation mode II is based on parallel writing of multithreading/multiprocessing:

At the software level, one or more write threads or processes are created for each target physical address, utilizing multi-threading or concurrent programming techniques supported by modern operating systems. Each thread or process independently writes metadata to a specified physical address in external storage space. The system may utilize the parallel processing capabilities of the multi-core processor to perform multiple write tasks simultaneously. The operating system is responsible for managing synchronization between threads or processes, avoiding data competition and conflict, and ensuring atomicity and consistency of write operations.

Implementation three, write acceleration using hardware accelerator (e.g., FPGA or ASIC):

Hardware accelerator configuration the mapping information of the target physical address and metadata is loaded into the accelerator, either custom or with existing hardware accelerators, such as FPGAs or ASICs. The hardware accelerator directly writes the metadata into the appointed physical address of the external storage space according to the configuration information, and a CPU at a software level is not required to participate in the data transmission process, so that the writing speed is greatly improved, and the delay is reduced. After the writing is completed, the hardware accelerator feeds back writing states to the CPU through a specific interface, and the CPU performs subsequent processing according to the states, such as updating metadata states or releasing memory resources.

Each implementation mode has specific advantages and application scenes that the DMA technology is suitable for scenes with large and continuous data transmission quantity, the burden of a CPU can be effectively reduced, the multithreading/multiprocessing technology is more suitable for tasks of distributed or concurrent processing, the parallel capability of a multi-core processor can be fully utilized, and a hardware accelerator (such as an FPGA or an ASIC) has extremely high performance and efficiency of data processing, but has higher design and implementation cost. In practical application, the most suitable implementation mode should be selected according to the specific requirements and resource conditions of the system.

FIG. 3 is a schematic diagram of a method for dropping metadata onto a physical disk according to an embodiment of the present application. As shown in fig. 3, metadata in the memory needs to be dropped into the physical disk, where the physical disk is the aforementioned target external storage space, and the physical address set may be stored in an intermediate management layer except the memory and the physical disk. When the metadata in the memory needs to be stored in the physical disk, the physical address set recorded in the middle management layer can be utilized to find the position of the physical address set of the physical address corresponding to the virtual address from the virtual address in the memory, then the physical address corresponding to the virtual address is found at the corresponding position of the middle management layer, and after the accurate physical address is determined, the metadata in the memory can be flushed down to the physical disk, so that the lasting storage is realized.

As an optional implementation manner, matching corresponding address set identifiers for virtual address sets where all target virtual addresses are located in a virtual address set and address set identifiers with corresponding relations to obtain a plurality of target set identifiers, wherein the method comprises the steps of searching a plurality of landing address sets where all target virtual addresses fall in the virtual address set, searching target set identifiers corresponding to the landing address sets respectively in the virtual address set and the address set identifiers with corresponding relations to obtain a plurality of target set identifiers.

Optionally, in this embodiment, after a virtual address of metadata in a memory is obtained, a set of landing addresses where the virtual address falls may be first searched from multiple virtual address sets, and after the dropped virtual address set is determined, a corresponding target set identifier is searched according to a one-to-one correspondence between each virtual address set and each address set identifier.

Through the steps, a plurality of landing address sets in which each target virtual address falls are searched in the virtual address sets, and address set identifiers corresponding to the landing address sets are searched from the virtual address sets and the address set identifiers with corresponding relations, so that each target virtual address can be ensured to be accurately mapped to the correct physical address of the target virtual address in the external storage space. This exact matching mechanism reduces the likelihood of data write errors and improves the reliability of data management.

As an optional implementation manner, searching physical addresses corresponding to the target virtual addresses in the target address sets indicated by the target set identifiers in parallel to obtain a plurality of target physical addresses comprises obtaining corresponding target address sets from a plurality of target intermediate addresses in an additional external storage space, wherein the corresponding physical address sets are stored in the intermediate addresses in the additional external storage space, the address set identifiers comprise intermediate addresses, the target set identifiers comprise target intermediate addresses, and searching the target physical addresses in the target address sets in parallel.

Alternatively, in the present embodiment, each set of physical addresses may be, but is not limited to being, stored on other storage locations, and the address set identification may be, but is not limited to, an intermediate address that is used to indicate a specific storage location of the set of physical addresses.

As an optional implementation manner, searching the physical addresses corresponding to the target virtual addresses in the target address sets indicated by the target set identifiers in parallel comprises repeatedly executing the following steps with each target address set as a candidate address set until the target virtual addresses corresponding to the candidate address set are all found to the corresponding target physical addresses, traversing the target virtual addresses corresponding to the candidate address set as reference virtual addresses, and matching the corresponding physical addresses for the reference virtual addresses from the virtual addresses and the physical addresses with the corresponding relation.

Optionally, in this embodiment, including but not limited to creating a corresponding thread for each target address set, repeating the steps in the thread until the target virtual addresses corresponding to the corresponding target address sets find the corresponding target physical addresses, traversing the target virtual addresses corresponding to the corresponding target physical address sets as reference virtual addresses, and matching the corresponding physical addresses for the reference virtual addresses from the virtual addresses and the physical addresses having the corresponding relationship. The virtual addresses and the physical addresses with the corresponding relation output by each thread are stored in the same storage area, and the method can trigger the disk dropping operation of the metadata corresponding to all the physical addresses in the storage area under the condition that the storage area is filled.

As an optional implementation manner, the step of landing the plurality of target metadata on the target physical addresses corresponding to the target external storage space comprises the step of landing the candidate metadata corresponding to the plurality of candidate virtual addresses on the target physical addresses corresponding to the target external storage space in parallel under the condition that the physical addresses corresponding to the plurality of candidate virtual addresses are searched from all target address sets in parallel, wherein the plurality of target virtual addresses comprise the plurality of candidate virtual addresses, the plurality of target metadata comprise the plurality of candidate metadata, and target address sets corresponding to the candidate virtual addresses are different.

Through the steps, after a plurality of physical addresses are searched from each target address set in parallel, metadata corresponding to the physical addresses can be landed in the target external storage space in parallel according to the physical addresses obtained simultaneously, and the parallel writing of the metadata can be realized on the basis of parallel addressing, so that the landing efficiency of the metadata is further improved.

The method comprises the steps of obtaining a plurality of target subset identifications by matching corresponding physical address subset identifications for virtual address subsets where target virtual addresses are located in the virtual address subsets with corresponding relations in the target address subsets indicated by the target subset identifications, obtaining a plurality of target physical addresses by searching physical addresses corresponding to the target virtual addresses in parallel from target address subsets indicated by the target set identifications, wherein the virtual address sets are divided into a plurality of virtual address subsets, the physical address sets comprise a plurality of physical address subsets, and obtaining the target physical addresses by matching the physical addresses for the target virtual addresses in the target address subsets indicated by the target subset identifications from the virtual addresses and the physical addresses with corresponding relations.

Alternatively, in this embodiment, but not limited to, a plurality of virtual address subsets and physical address subset identifiers having a corresponding relationship may be stored in the physical address set, and then virtual addresses and physical addresses having a corresponding relationship may be stored in the target address subsets indicated by the respective physical address subset identifiers.

By the steps, the two-layer mapping is expanded into the three-layer mapping, and the management range of the target external storage space is further expanded under the condition that the storage position of the management data of the target external storage space is recorded in the memory and is unchanged.

FIG. 4 is a mapping relationship between a physical address set and a memory and a physical disk according to an embodiment of the present application. As shown in FIG. 4, the mapping ratio of the memory to the intermediate management layer may be set to 1:16, that is, one virtual address set and address set identifier with a corresponding relationship in the memory correspond to 16 physical address sets in the intermediate management layer, and the mapping ratio of the intermediate management layer to the physical disk may also be set to 1:16, that is, one virtual address subset and physical address set identifier with a corresponding relationship in each physical address set in the intermediate management layer correspond to 16 physical addresses in the physical disk. By means of the arrangement, on the one hand, on the basis of searching in each target address set in parallel, searching actions of physical addresses can be executed in each target address subset in parallel, on the other hand, occupation of memory space and data writing to a memory can be reduced to a greater extent, and according to the proportion, writing to a physical disk for 256 times only involves writing management data to a middle management layer for 16 times and writing metadata disc-falling management data to a memory for 1 time, wherein the metadata disc-falling management data is the virtual address set and address set identification with the corresponding relation.

As an optional implementation manner, before matching the corresponding address set identifier for the virtual address set where each target virtual address is located from the virtual address set and the address set identifier with the corresponding relationship, the method further comprises the steps of establishing a one-to-one correspondence relationship between each virtual address in the memory and each physical address in the target external storage space, storing the virtual addresses and the physical addresses with the corresponding relationship into a plurality of physical address sets, wherein the physical address sets are located in the additional external storage space, establishing a one-to-one correspondence relationship between each address set identifier and each virtual address set, and storing the virtual address sets and the address set identifiers with the corresponding relationship into the memory.

Optionally, in this embodiment, the additional external storage space includes, but is not limited to, a storage space that is divided from the target external storage space, and may also be an intermediate management layer storage space that is newly added in the memory and the target external storage space (i.e., does not belong to the memory or the target external storage space).

Optionally, in this embodiment, one or more intermediate key management layers (hereinafter, simply referred to as intermediate management layers) are added between the memory and the target external storage space. When the intermediate management layer is a plurality of layers, the mapping relationship is also converted into a plurality of layers, for example, when the intermediate management layer is a 2-layer (including the intermediate management layer 1 and the intermediate management layer 2), the mapping relationship between the virtual address and the physical address is converted from one-layer mapping relationship between the memory and the target external storage space into a three-layer mapping relationship between the memory and the intermediate management layer 1-the intermediate management layer 2-the target external storage space, and the management range of the target external storage space can be further expanded by increasing the layer number of the intermediate management layer. The optimal middle management layer number M may be calculated by the SMM management module by monitoring the memory resources of the system and the target external storage space resources when, but not limited to, starting up the thin pool volume. The aforementioned intermediate management layer may be divided from the target external storage space (typically, SSD disk), and there are multiple pages (i.e., a set of physical addresses), where the number of entries stored in each page is not the same, for example, 6 gears may be set, 1 in one page has 1k entries, 2 in one page has 2k entries, 3 in one page has 4k entries, 4 in one page has 8k entries, 5 in one page has 16k entries, and 6 in one page has 32k entries. The lower the gear is, the fewer the number of entries in one page is, and the addressing efficiency and the parallel writing efficiency are higher when the metadata is written. Here, one entry may correspond to one virtual address and physical address having a correspondence relationship, or may correspond to a plurality of virtual addresses and physical addresses having a correspondence relationship, and in the case where one entry corresponds to a plurality of virtual addresses and physical addresses having a correspondence relationship, the number of virtual addresses and physical addresses having a correspondence relationship corresponding to the respective entries may be set to be the same, or may be different. The number of the pages in each gear can be flexibly set according to the requirement, for example, each of the gears 1-6 can be set to comprise 100 pages, or the gears 1-3 can be set to comprise 50 pages, and the gears 4-6 can be set to comprise 100 pages.

Alternatively, in the present embodiment, including but not limited to, storing in memory the following information, virtual address 1-1000_page1_middle management layer address 1, virtual address 1001-2000_page2_middle management layer address 2, virtual address 2001-3000_page3_middle management layer address 3, virtual address XX-xx_ pageX _middle management layer address X, storing in one page (i.e., a set of physical addresses) of the middle management layer (i.e., the additional external storage space), the following information, namely, the ent1_physical address 1, the ent2_physical address 2, the ent3_physical address 3, the number entryX _physical address X, where one entry corresponds to one virtual address.

Through the steps, a plurality of virtual addresses and physical addresses with corresponding relations are stored in the additional external storage space, only a plurality of virtual address sets and address set identifiers with corresponding relations are stored in the memory, and under the condition that the memory space is unchanged, the management range of the target external storage space is enlarged through two-layer mapping.

The method comprises the steps of storing a plurality of virtual addresses and physical addresses with corresponding relations into a plurality of physical address sets, determining the number of landing gear according to a landing capacity parameter, wherein the landing capacity parameter is used for indicating the landing capacity of metadata, the number of the landing gear is the number of types of the physical address sets divided according to the size of the capacity, the capacity of the physical address sets is used for indicating the number of physical addresses and virtual addresses with corresponding relations allowed to be stored in the corresponding physical address sets, dividing the physical addresses in a target external storage space into a plurality of physical address sets matched with the capacity of the physical address sets according to the sequence from low to high, the capacity of the physical address sets is matched with the number of the landing gear, and the physical addresses in the physical address sets with larger capacity of the physical address sets in the physical address sets are all higher than the physical addresses in the physical address sets with smaller capacity.

Optionally, in this embodiment, including but not limited to, after monitoring the memory resources and the target external storage space resources of the system, the SMM management module calculates the optimal middle management layer number M, and then obtains the number of shift positions K (i.e. the number of dropped shift positions) of the theoretical middle management layer according to the hardware configuration of the CPU, the memory, the hard disk and the middle management layer number (because the number of pages owned by each shift position is determined, the more the number of shift positions, the more objects the system needs to process in parallel, the higher the requirement on the processing performance of the system hardware, and the higher the writing speed of metadata.

Alternatively, in the present embodiment, including but not limited to, after determining the number of drop gear steps, a lower physical address is assigned to a lower gear (i.e., smaller capacity) set of physical addresses, and a higher physical address is assigned to a higher gear set of physical addresses.

Optionally, in this embodiment, the capacities of the plurality of physical address sets are matched with the number of landing gear positions, that is, the number of kinds of capacities of the plurality of physical address sets is the number of landing gear positions, for example, the capacities of the plurality of physical address sets are respectively 1k, 2k, 3k, and the number of landing gear positions matched with the plurality of physical address sets is 3.

Optionally, in this embodiment, including but not limited to, the default gear corresponding capacity and the number of physical address sets owned by each gear have been configured in the disc-drop system of metadata, and the default setting of the capacity of the physical address sets is to be set from the lowest gear and can only be set continuously, so that after the number of disc-drop gears is determined, the capacity configuration schemes of the plurality of physical address sets can be determined.

Through the steps, the physical address sets with different capacities are set, so that the addressing efficiency in each physical address set is different, the disc-dropping process of metadata with different read-write frequency requirements can be dealt with, and the disc-dropping of the metadata can be more flexible.

As an optional implementation manner, establishing the one-to-one correspondence between each address set identifier and each virtual address set comprises recording virtual addresses stored in each physical address set to obtain a plurality of physical address sets and virtual address sets with corresponding relations, and establishing the one-to-one correspondence between each address set identifier and each virtual address set according to the corresponding relations between each physical address set and each virtual address set.

As an optional implementation manner, after the plurality of target metadata are dropped onto the corresponding target physical addresses in the target external storage space, the method further comprises detecting access information of each metadata stored in the target external storage space, wherein the access information is used for indicating the condition that the corresponding metadata is accessed, adjusting the physical addresses and the virtual addresses which are stored in each physical address set and have the corresponding relation to be matched with the access information to obtain a reorganization adjustment result, wherein the reorganization adjustment result is used for indicating the condition of the virtual addresses stored in each physical address set after the reorganization adjustment, and modifying the virtual address set corresponding to the address set identification according to the reorganization adjustment result.

Alternatively, in the present embodiment, the access information of the metadata may be, but not limited to, used to indicate how frequently the corresponding metadata is accessed.

Optionally, in this embodiment, the physical addresses and virtual addresses stored in each physical address set and having a corresponding relationship are adjusted to match with the access information, that is, metadata, indicated by the access information, of which the accessed frequency is greater than that indicated by the access information is stored in a physical address set with more convenient access, and metadata, indicated by the access information, of which the accessed frequency is greater than that indicated by the access information is stored in a physical address set with relatively more time-consuming access. Because the small-capacity physical address set traverses a small range when addressing, and the large-capacity physical address set traverses a large range when addressing, metadata in the small-capacity physical address set can be more convenient to access than metadata in the large-capacity physical address set. Therefore, the physical addresses and virtual addresses stored in the physical address sets and having the corresponding relation are adjusted to be matched with the access information, namely, the metadata which is indicated by the access information and is accessed frequently is stored in the physical address set with small capacity, and the metadata which is indicated by the access information and is accessed relatively idle is stored in the physical address set with large capacity.

Alternatively, in this embodiment, after the access information of each metadata stored in the target external storage space is detected, but not limited to, the storage position of the metadata in the target external storage space may be selected to be adjusted without adjusting the content recorded in the physical address set, that is, metadata that is more frequently accessed may be adjusted to a physical address corresponding to a physical address space with a small capacity, and the virtual address of the metadata in the memory may be correspondingly modified.

Through the steps, the storage scheme of the address information of the metadata is adjusted according to the condition that the metadata is accessed, so that the storage of the address information of the metadata is more matched with the access condition of the metadata, the metadata with frequent access can be accessed more quickly, and the access efficiency of the metadata is improved.

As an optional implementation manner, the physical addresses and virtual addresses stored in the physical address sets and having corresponding relations are adjusted to be matched with access information, the method comprises the steps of sorting the metadata stored in a target external storage space according to access parameters of metadata included in the access information to obtain a metadata sequence, wherein the access parameters are used for indicating the frequency of access of the metadata corresponding to the access parameters in a current time period, establishing a reorganization corresponding relation between the metadata stored in the target external storage space and the physical address sets according to the positions of the metadata stored in the target external storage space in the metadata sequence and the capacities of the physical address sets, wherein the capacities of the physical address sets are used for indicating the number of the physical addresses and the virtual addresses stored in the corresponding physical address sets and the capacity of the physical address sets corresponding to the metadata with the more frequent access indicated by the access parameters in the reorganization corresponding relation, and writing the physical addresses and the virtual addresses corresponding to the physical address sets corresponding to the reorganization corresponding relation.

Alternatively, in the present embodiment, the ordering of metadata is an ordering that does not deduplicate.

Alternatively, in the present embodiment, including but not limited to, at initialization, an intermediate management layer from low gear to high gear is created according to the order of physical addresses from low to high. When metadata is written, the metadata reading and writing times are recorded, when the metadata is written for the first time, the middle management layer is written gradually from low gear to high gear, and the target external storage space is written sequentially from a low physical address to a high physical address. After a certain period of time, the intermediate management layer is reorganized according to the read-write times of the metadata, namely, intermediate management data (namely, virtual address and physical address with corresponding relation) of metadata with high read-write times are written into the low-gear page, and intermediate management data of metadata with low read-write times are written into the high-gear page. After recombination, intermediate management data of high-frequency read-write metadata are stored in a low-gear page, and the quantity of information stored in the low-gear page is small, so that the concurrent access quantity can be increased and the address addressing efficiency in a single page can be improved by the adjustment.

As an optional implementation manner, rewriting the physical address and the virtual address corresponding to each metadata stored in the target external storage space into the physical address set corresponding to the reorganization corresponding relation instruction comprises backing up the physical address and the virtual address with the corresponding relation stored in the reference address set into the backup set, wherein the reference address set is the physical address set to be rewritten, and rewriting the physical address and the virtual address with the corresponding relation corresponding to each metadata corresponding to the reference address set indicated by the reorganization corresponding relation from the backup set into the reference address set.

Through the steps, the physical address set to be rewritten is stored in the backup set, and then rewritten into the reference address set according to the reorganization corresponding relation from the backup set, so that the loss of data caused in the rewriting process is avoided.

As an optional implementation manner, modifying the virtual address set corresponding to the address set identifier according to the reorganization adjustment result comprises determining virtual addresses stored in each address set according to the reorganization adjustment result to obtain a plurality of modified address sets corresponding to each physical address set, and modifying the virtual address set corresponding to the address set identifier indicating the physical address set into the corresponding modified address set.

Through the steps, after the virtual addresses and the physical addresses which are stored in each physical address set and have the corresponding relation are rewritten, the virtual address set corresponding to the physical address set is correspondingly modified, so that the correctness of the corresponding relation between the addresses is ensured, and metadata cannot be normally stored in the target external storage space due to the fact that the wrong mapping relation is used.

The method further comprises the steps of comparing a size relation between the disc-dropping parameter and a disc-dropping parameter threshold, wherein the disc-dropping parameter is used for indicating the disc-dropping load condition of metadata of a current disc to be dropped, and adjusting the capacity of each physical address set in a mode of adjusting the current disc-dropping parameter to the disc-dropping parameter threshold when the number of times that the disc-dropping parameter is continuously detected to be larger than the disc-dropping parameter threshold or the number of times that the disc-dropping parameter is continuously detected to be smaller than the disc-dropping parameter threshold is larger than or equal to the number of times threshold, wherein the capacity of the physical address set is used for indicating the number of physical addresses and virtual addresses which are allowed to be stored in the corresponding physical address set and have the corresponding relation.

Alternatively, in this embodiment, the landing parameter may include, but is not limited to, one or more of a CPU resource real-time usage, a memory resource real-time usage, and a metadata writing speed.

Alternatively, in this embodiment, the landing parameter threshold may include, but is not limited to, one or more of a CPU resource theoretical utilization, a memory resource theoretical utilization, and a metadata theoretical writing speed.

Alternatively, in this embodiment, the drop parameter threshold may be set manually, or may be determined according to different target external storage space types (for example, different hard disk types), target external storage space performance (for example, hard disk performance), CPU performance, memory capacity and memory rate, and intermediate management layer number, including, but not limited to, recording the drop parameter threshold into a theoretical matrix table after determining the above drop parameter threshold, and storing the matrix table in an SMM register corresponding to the SMM management module.

Through the steps, the capacity of each physical address set is adjusted according to the landing load condition of the metadata to be landed at present, so that the possibility of system breakdown is reduced on the premise of pursuing the landing efficiency of the metadata under the condition that the continuous operation of a landing system of the metadata is large in concurrency but cannot cause system breakdown, and the landing efficiency of the metadata is improved to a certain extent.

As an optional implementation mode, the capacity of each physical address set is adjusted by adjusting the current disc-drop parameter to the disc-drop parameter threshold, and the method comprises the steps of searching physical address sets with capacity smaller than that of other address sets in a plurality of physical address sets to obtain a plurality of merging address sets under the condition that the number of times that the disc-drop parameter is continuously detected to be larger than or equal to the disc-drop parameter threshold is larger than or equal to the disc-drop parameter threshold, wherein the capacity is used for indicating the number of physical addresses and virtual addresses which are stored in the corresponding physical address sets and have corresponding relations, the other address sets are physical address sets except the merging address sets in the plurality of physical address sets, merging the plurality of merging address sets to obtain an integrating address set, establishing the corresponding relation between the virtual address set corresponding to the merging address set identifier and the integrating address set identifier, wherein the merging address set identifier is the address set identifier indicating the integrating address set, and releasing the corresponding relation between the merging address set identifier and the corresponding virtual address set.

Optionally, in this embodiment, the number of times that the tray parameter is continuously detected to be greater than or equal to the tray parameter threshold is greater than or equal to the number threshold, that is, the tray system is considered to be currently operated in a state that the concurrency pressure is too high, and the concurrency pressure needs to be reduced, so that the concurrency processing pressure of the system is reduced by adopting a mode of merging address sets.

Optionally, in this embodiment, merging the multiple merged address sets includes, but is not limited to, merging the multiple merged address sets into one merged address set, or merging the multiple merged address sets into a smaller number of multiple merged address sets.

Fig. 5 is a diagram illustrating a physical address set capacity adjustment according to an embodiment of the present application. As shown in fig. 5, page n and Page n+1 are combined to obtain the integrated address set Page n ¹.

The method comprises the steps of searching a physical address set with the largest capacity in a plurality of physical address sets to obtain a split address set under the condition that the number of times that a current disc-drop parameter is smaller than a disc-drop parameter threshold is larger than or equal to the number of times threshold is continuously detected, wherein the capacity is used for indicating the number of physical addresses and virtual addresses which are stored in the corresponding physical address set and have a corresponding relation, splitting the split address set into a plurality of physical address sets to obtain a plurality of divided address sets, establishing the corresponding relation between the virtual address set corresponding to the split address set identifier and the plurality of divided set identifiers, wherein the split address set identifier is an address set identifier for indicating the split address set, the divided set identifier is an address set identifier for indicating the divided address set, and releasing the corresponding relation between the split address set identifier and the corresponding virtual address set.

Optionally, in this embodiment, the number of times that the disc-falling parameter is continuously detected to be smaller than the disc-falling parameter threshold is greater than or equal to the number threshold, that is, the disc-falling system is considered to be currently operated in a state that the concurrency pressure is too small, and the concurrency amount needs to be improved, so that the concurrency processing capability of the system is improved by adopting a mode of splitting the address set.

Optionally, in this embodiment, splitting the split address set into the plurality of physical address sets includes, but is not limited to, equally dividing the split address set into the plurality of physical address sets.

FIG. 6 is a diagram illustrating a second physical address set capacity adjustment according to an embodiment of the present application. As shown in fig. 6, page t is the split address set, and divided address sets Page t ¹ and Page t ¹ +1 can be obtained by equally dividing Page t.

As an alternative implementation mode, comparing the size relation between the landing parameters and the landing parameter threshold comprises detecting a first parameter and a second parameter, and obtaining the first parameter threshold and the second parameter threshold, wherein the first parameter is used for indicating the utilization condition of the operation resources by the current metadata landing, the second parameter is used for indicating the utilization condition of the memory resources by the current metadata landing, comparing the size relation between the first parameter and the first parameter threshold, and comparing the size relation between the second parameter and the second parameter threshold, determining that the detected landing parameter is larger than the landing parameter threshold when the first parameter is larger than the first parameter threshold and the second parameter is larger than the second parameter threshold, and determining that the detected landing parameter is smaller than the landing parameter threshold when the first parameter is smaller than the first parameter threshold and the second parameter is smaller than the second parameter threshold.

Optionally, in this embodiment, the setting monitoring module obtains the real-time writing speed of metadata, the real-time utilization rate of CPU resources and the real-time utilization rate of memory resources, and obtains the corresponding theoretical writing processing speed of metadata, the theoretical utilization rate of CPU resources and the theoretical utilization rate of memory resources (preset values established according to historical data) from the SMM register according to the hardware configuration of the corresponding CPU, memory and hard disk and the intermediate management layer number. If the real-time utilization rate of CPU resources and the real-time utilization rate of memory resources are smaller than the theoretical utilization rate of CPU resources and the theoretical utilization rate of memory resources for 3 times (3 check periods), one page of the gear 6 (the highest gear) is equally divided into two pages, and the two pages are combined into the gear 5 for management. Therefore, the number of low-gear pages is increased, and the concurrent processing efficiency of the system is improved. After the page of the gear 6 is split, the page of the gear 5 is split again, and the steps are circulated in sequence. If the real-time utilization rate of CPU resources and the real-time utilization rate of memory resources are continuously 3 times (3 check periods) larger than the theoretical utilization rate of CPU resources and the theoretical utilization rate of memory resources, 2 pages of the gear 1 are combined into one page for management, namely, the two pages of the gear 1 are combined into 1 page of the gear 2, so that the management object of the pages is promoted, and the concurrent processing pressure of the system is reduced.

As an alternative implementation mode, acquiring the first parameter threshold value and the second parameter threshold value comprises detecting whether a user threshold value set is detected, wherein the user threshold value set is a parameter threshold value set by a user, extracting the first parameter threshold value and the second parameter threshold value from the user threshold value set when the user threshold value set is detected, and determining the first default threshold value as the first parameter threshold value and the second default threshold value as the second parameter threshold value when the user threshold value set is detected to be absent.

Optionally, in this embodiment, the user threshold set is a parameter threshold set by a user, including, but not limited to, a metadata theoretical writing speed set by a user, a CPU resource theoretical usage rate, a memory resource theoretical usage rate, and the like.

Optionally, in this embodiment, the corresponding relationship between the threshold gear and the parameter threshold is set, and the lower the threshold gear, the lower the corresponding metadata theoretical writing speed, CPU resource theoretical utilization rate and memory resource theoretical utilization rate, and the higher the threshold gear, the higher the corresponding metadata theoretical writing speed, CPU resource theoretical utilization rate and memory resource theoretical utilization rate. The user can select different threshold gear according to the demand, when selecting high threshold gear, namely representing that the user hopes to acquire high metadata writing speed, namely the metadata writing theoretical speed, the CPU resource theoretical utilization rate and the memory resource theoretical utilization rate are higher. When the low threshold gear is selected, the user hopes to ensure the safety and stability of the whole system preferentially, the metadata writing speed priority is not high, namely the metadata writing theoretical speed, the CPU resource theoretical utilization rate and the memory resource theoretical utilization rate are lower.

Through the steps, a more flexible triggering scheme of capacity adjustment is set, and a user can set triggering time of capacity adjustment according to requirements instead of the operation condition of the metadata disc-landing system, so that flexibility of metadata disc-landing is improved.

Through the above, the mode of directly adding the intermediate management layer between the memory and the external storage space realizes multi-level mapping management, expands the management range, supports the page mapping management of a plurality of gears, realizes the parallel writing of metadata, and can further improve the addressing and parallel writing speed according to service cold and hot data. The method has the advantages that the space where the metadata writing speed of the current system can be finely adjusted is detected by comparing the hardware theoretical utilization rate and the metadata theoretical writing speed, the shifting of the pages is realized by combining and disassembling the multi-level pages of the middle management layer, the intelligent metadata management is realized, and the basic management capability is provided for improving the system service writing speed.

From the description of the above embodiments, it will be clear to a person skilled in the art that the method according to the above embodiments may be implemented by means of software plus the necessary general hardware platform, but of course also by means of hardware, but in many cases the former is a preferred embodiment.

The embodiment of the application also provides a device for dropping metadata, fig. 7 is a block diagram of the structure of the device for dropping metadata according to the embodiment of the application, as shown in fig. 7, and the device comprises:

The obtaining module 702 is configured to obtain virtual addresses of a plurality of target metadata of a disc to be dropped in a memory, so as to obtain a plurality of target virtual addresses;

a matching module 704, configured to match corresponding address set identifiers for virtual address sets where each target virtual address is located from the virtual address sets and the address set identifiers that have a corresponding relationship, to obtain a plurality of target set identifiers, where a physical address set indicated by the address set identifier stores physical addresses corresponding to each virtual address in the corresponding virtual address set in a target external storage space;

A searching module 706, configured to search, in parallel, for physical addresses corresponding to each target virtual address from the target address set indicated by each target set identifier, to obtain multiple target physical addresses;

and the landing module 708 is configured to landing the plurality of target metadata onto the corresponding target physical addresses in the target external storage space.

By the device, as the virtual address set and the address set identifier with the corresponding relation are configured, and the virtual address and the physical address with the corresponding relation are stored in the address set identifier, after the virtual addresses of a plurality of target metadata to be dropped in the memory are obtained, the corresponding address set identifier is matched with the virtual address set in which each target virtual address is positioned in the virtual address set and the address set identifier with the corresponding relation, then the physical address corresponding to each target virtual address is searched in the target address set indicated by each target set identifier in parallel, and the parallel mode of searching the physical address corresponding to the virtual address in each small range is used for replacing the serial mode of searching the physical address in a large range comprising a plurality of small ranges, so that the addressing speed of the metadata is accelerated, the technical problem that the dropping efficiency of the metadata is lower can be solved, and the technical effect of improving the dropping efficiency of the metadata is achieved.

As an optional implementation mode, the matching module comprises a first searching unit and a second searching unit, wherein the first searching unit is used for searching a plurality of landing address sets in which each target virtual address falls from the virtual address sets, and the second searching unit is used for searching target set identifiers corresponding to the plurality of landing address sets respectively from the virtual address sets and the address set identifiers with corresponding relations to obtain a plurality of target set identifiers.

As an optional implementation manner, the searching module comprises an acquiring unit and a third searching unit, wherein the acquiring unit is used for acquiring corresponding target address sets from a plurality of target intermediate addresses in an additional external storage space, the corresponding physical address sets are stored in the intermediate addresses in the additional external storage space, the address set identifier comprises the intermediate addresses, the target set identifier comprises the target intermediate addresses, and the third searching unit is used for searching the target physical addresses from the target address sets in parallel.

The searching module further comprises an executing unit, wherein the executing unit is used for repeatedly executing the following steps with each target address set as a candidate address set until the corresponding target physical address is searched for each target virtual address corresponding to the candidate address set, wherein the target virtual address corresponding to the candidate address set is traversed to serve as a reference virtual address, and the corresponding physical address is matched with the reference virtual address from the virtual address and the physical address which have the corresponding relation.

As an optional implementation manner, the landing module comprises a landing unit, a landing unit and a landing unit, wherein the landing unit is used for landing candidate metadata corresponding to a plurality of candidate virtual addresses onto target physical addresses corresponding to a target external storage space in parallel under the condition that physical addresses corresponding to the plurality of candidate virtual addresses are searched out from each target address set in parallel, the plurality of target virtual addresses comprise the plurality of candidate virtual addresses, the plurality of target metadata comprise the plurality of candidate metadata, and target address sets corresponding to the candidate virtual addresses are different.

The searching module further comprises a first matching unit, a second matching unit and a target physical address matching unit, wherein the first matching unit is used for matching corresponding physical address subset identifications for the virtual address subset where each target virtual address is located in each target address subset identification in parallel in each target address set, the target address subset identifications are obtained, the virtual address set is divided into a plurality of virtual address subsets, the physical address set comprises a plurality of physical address subsets, and the second matching unit is used for matching physical addresses for the target virtual addresses in the target address subsets indicated by each target subset identification in parallel from the virtual addresses and the physical addresses with the corresponding relations, and the target physical addresses are obtained.

As an optional implementation mode, the device further comprises a first building module, a first storage module and a second building module, wherein the first building module is used for building one-to-one correspondence between each virtual address in the memory and each physical address in the target external storage space, the first storage module is used for storing a plurality of virtual addresses and physical addresses with correspondence into a plurality of physical address sets, the physical address sets are located in the additional external storage space, the second building module is used for building one-to-one correspondence between each address set identifier and each virtual address set, and the second storage module is used for storing a plurality of virtual address sets and address set identifiers with correspondence into the memory.

The first storage module comprises a first determining unit and a dividing unit, wherein the first determining unit is used for determining the number of landing gears according to a landing capacity parameter, the landing capacity parameter is used for indicating the landing capacity of metadata, the number of the landing gears is the number of types of physical address sets divided according to the size of capacity, the capacity of the physical address sets is used for indicating the number of physical addresses and virtual addresses which are allowed to be stored in the corresponding physical address sets and have a corresponding relation, and the dividing unit is used for dividing the physical addresses in the target external storage space into a plurality of physical address sets matched with the capacity of the physical address sets according to the sequence from low to high, wherein the capacity of the physical address sets is matched with the number of the landing gears, and the physical addresses in the physical address sets with larger capacity of the physical address sets in the physical address sets are all higher than the physical addresses in the physical address sets with smaller capacity of the physical address sets.

As an optional implementation manner, the second building module comprises a recording unit and a first building unit, wherein the recording unit is used for recording virtual addresses stored in each physical address set to obtain a plurality of physical address sets and virtual address sets with corresponding relations, and the first building unit is used for building one-to-one corresponding relations between each address set identifier and each virtual address set according to the corresponding relations between each physical address set and each virtual address set.

The device further comprises a detection module for detecting access information of each metadata stored in the target external storage space, wherein the access information is used for indicating the condition that the corresponding metadata is accessed, a first adjustment module for adjusting the physical addresses and virtual addresses with corresponding relations stored in each physical address set to be matched with the access information so as to obtain a reorganization adjustment result, wherein the reorganization adjustment result is used for indicating the condition of the virtual addresses stored in each physical address set after adjustment, and a modification module for modifying the virtual address set corresponding to the address set identification according to the reorganization adjustment result.

As an optional implementation manner, the first adjustment module comprises a sorting unit, a second establishing unit and a rewriting unit, wherein the sorting unit is used for sorting all metadata stored in the target external storage space according to access parameters of metadata included in access information to obtain a metadata sequence, the access parameters are used for indicating the frequency of access of metadata corresponding to the access parameters in a current time period, the second establishing unit is used for establishing a reorganization corresponding relation between all metadata stored in the target external storage space and all physical address sets according to the positions of all metadata stored in the metadata sequence and the capacities of all physical address sets in the target external storage space, the capacities of the physical address sets are used for indicating the number of physical addresses and virtual addresses with corresponding relation stored in the corresponding physical address sets, and the rewriting unit is used for rewriting the physical addresses and the virtual addresses corresponding to all metadata stored in the target external storage space into the physical address sets indicated by the reorganization corresponding relation as the capacities of the metadata corresponding frequent metadata indicated by the access parameters in the reorganization corresponding relation are smaller.

The rewrite unit is further used for backing up the physical address and the virtual address which are stored in the reference address set and have the corresponding relation into the backup set, wherein the reference address set is the physical address set to be rewritten, and rewriting the physical address and the virtual address which have the corresponding relation and correspond to each metadata corresponding to the reference address set indicated by the reorganization corresponding relation into the reference address set from the backup set.

As an optional implementation manner, the modification module comprises a second determination unit and a modification unit, wherein the second determination unit is used for determining virtual addresses stored in each address set according to the reorganization adjustment result to obtain a plurality of modified address sets corresponding to each physical address set, and the modification unit is used for modifying the virtual address set corresponding to the address set identifier indicating the physical address set into the corresponding modified address set.

The device further comprises a comparison module for comparing the size relation between the disc-dropping parameter and the disc-dropping parameter threshold, wherein the disc-dropping parameter is used for indicating the disc-dropping load condition of the metadata of the current disc to be dropped, and a second adjustment module for adjusting the capacity of each physical address set in a mode of adjusting the current disc-dropping parameter to the disc-dropping parameter threshold when the number of times that the disc-dropping parameter is continuously detected to be larger than the disc-dropping parameter threshold or the number of times that the disc-dropping parameter is continuously detected to be smaller than the disc-dropping parameter threshold is larger than or equal to the number of times threshold, wherein the capacity of the physical address set is used for indicating the number of physical addresses and virtual addresses which are allowed to be stored in the corresponding physical address set and have the corresponding relation.

The second adjusting module comprises a fourth searching unit, a merging unit and a third establishing unit, wherein the fourth searching unit is used for searching a physical address set with capacity smaller than that of other address sets in a plurality of physical address sets to obtain a plurality of merging address sets when the number of times that the disc-falling parameter is larger than the disc-falling parameter threshold is continuously detected to be larger than or equal to the number of times threshold, the capacity is used for indicating the quantity of physical addresses and virtual addresses which are stored in the corresponding physical address sets and have corresponding relations, the other address sets are physical address sets except the merging address sets in the plurality of physical address sets, the merging unit is used for merging the plurality of merging address sets to obtain a merging address set, the third establishing unit is used for establishing the corresponding relation between the virtual address set corresponding to the merging address set identifier and the merging address set identifier, the merging address set identifier is an address set identifier indicating the merging address set, and the merging set identifier is used for indicating the address set of the merging address set, and the first relieving unit is used for relieving the corresponding relation between the merging address identifier and the corresponding virtual address set.

The second adjusting module further comprises a fifth searching unit, a splitting unit and a fourth establishing unit, wherein the fifth searching unit is used for searching a physical address set with the largest capacity in the physical address sets to obtain a splitting address set under the condition that the number of times that the disc-drop parameter is smaller than the disc-drop parameter threshold is larger than or equal to the number of times threshold continuously detected, the capacity is used for indicating the number of physical addresses and virtual addresses which are stored in the corresponding physical address set and have corresponding relations, the splitting unit is used for splitting the splitting address set into the physical address sets to obtain the dividing address sets, the fourth establishing unit is used for establishing the corresponding relations between the virtual address sets corresponding to the splitting address set identifiers and the dividing address set identifiers, the dividing set identifiers are address set identifiers for indicating the dividing address sets, and the second canceling unit is used for canceling the corresponding relations between the splitting address set identifiers and the corresponding virtual address sets.

As an optional implementation mode, the comparison module comprises a detection unit, a comparison unit and a third determination unit, wherein the detection unit is used for detecting a first parameter and a second parameter and obtaining a first parameter threshold and a second parameter threshold, the first parameter is used for indicating the utilization condition of operation resources by the current metadata disc, the second parameter is used for indicating the utilization condition of memory resources by the current metadata disc, the comparison unit is used for comparing the size relation between the first parameter and the first parameter threshold and comparing the size relation between the second parameter and the second parameter threshold, the third determination unit is used for determining that the disc-falling parameter is larger than the disc-falling parameter threshold when the first parameter is larger than the first parameter threshold and the second parameter is larger than the second parameter threshold, and the fourth determination unit is used for determining that the disc-falling parameter is smaller than the disc-falling parameter threshold when the first parameter is smaller than the first parameter threshold and the second parameter is smaller than the second parameter threshold.

As an alternative implementation mode, the detection unit is further used for detecting whether a user threshold set exists, wherein the user threshold set is a parameter threshold set by a user, extracting a first parameter threshold and a second parameter threshold from the user threshold set when the user threshold set exists is detected, and determining the first default threshold as the first parameter threshold and the second default threshold as the second parameter threshold when the user threshold set does not exist.

The description of the features in the embodiment corresponding to the disc-dropping device of the metadata can be referred to the related description of the embodiment corresponding to the disc-dropping method of the metadata, which is not described herein in detail.

An embodiment of the application also provides an electronic device comprising a memory in which a computer program is stored and a processor arranged to run the computer program to perform the steps of the drop method embodiment of any of the metadata described above.

Embodiments of the present application also provide a computer readable storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of the drop method embodiment of any of the metadata described above when run.

In an exemplary embodiment, the computer readable storage medium may include, but is not limited to, a U disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a removable hard disk, a magnetic disk, or an optical disk, etc. various media in which a computer program may be stored.

Embodiments of the present application also provide a computer program product comprising a computer program which, when executed by a processor, implements the steps of a drop method embodiment of any of the metadata described above.

Embodiments of the present application also provide another computer program product comprising a non-volatile computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of a drop method embodiment of any of the metadata described above.

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The above describes in detail a metadata landing disc provided by the present application. The principles and embodiments of the present application have been described herein with reference to specific examples, the description of which is intended only to facilitate an understanding of the method of the present application and its core ideas. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the application can be made without departing from the principles of the application and these modifications and adaptations are intended to be within the scope of the application as defined in the following claims.

Claims

1. A method for storing metadata on a disk, characterized in that:

Acquire virtual addresses of multiple target metadata to be written to disk in the memory to obtain multiple target virtual addresses; match the corresponding address set identifier for the virtual address set where each of the target virtual addresses is located from the virtual address set and the address set identifier having a corresponding relationship to obtain multiple target set identifiers, wherein the physical address set indicated by the address set identifier stores the physical addresses corresponding to each virtual address in the corresponding virtual address set in the target external storage space; search for the physical address corresponding to each of the target virtual addresses from the target address sets indicated by each of the target set identifiers in parallel to obtain multiple target physical addresses; write the multiple target metadata to the corresponding target physical addresses in the target external storage space;

Among them, the matching of the corresponding address set identifier for the virtual address set where each of the target virtual addresses is located from the virtual address set and the address set identifier having a corresponding relationship to obtain multiple target set identifiers includes: searching the virtual address set for the multiple disk address sets into which each of the target virtual addresses falls; searching the target set identifiers corresponding to the multiple disk address sets respectively from the virtual address set and the address set identifier having a corresponding relationship to obtain multiple target set identifiers.

2. The method according to claim 1, characterized in that the step of searching for the physical address corresponding to each of the target virtual addresses from the target address sets indicated by each of the target set identifiers in parallel to obtain a plurality of target physical addresses comprises:

Acquire the corresponding target address set from a plurality of target intermediate addresses in the additional external storage space, wherein the corresponding physical address set is stored in the intermediate address in the additional external storage space, the address set identifier includes the intermediate address, and the target set identifier includes the target intermediate address;

The target physical address is searched from each of the target address sets in parallel.

3. The method according to claim 1, characterized in that the step of searching for the physical address corresponding to each of the target virtual addresses from the target address sets indicated by each of the target set identifiers in parallel comprises:

The following steps are repeatedly performed using each target address set as a candidate address set until the corresponding target physical address is found for each target virtual address corresponding to the candidate address set:

Traversing the target virtual addresses corresponding to the candidate address set as reference virtual addresses;

A corresponding physical address is matched for the reference virtual address from virtual addresses and physical addresses having a corresponding relationship.

4. The method according to claim 3, wherein the step of writing the plurality of target metadata to the target physical addresses corresponding to the target external storage space comprises:

When the physical addresses corresponding to multiple candidate virtual addresses are found in parallel from each of the target address sets, the candidate metadata corresponding to the multiple candidate virtual addresses are written to the disk in parallel to the corresponding target physical addresses in the target external storage space, wherein the multiple target virtual addresses include multiple candidate virtual addresses, the multiple target metadata include multiple candidate metadata, and the target address sets corresponding to each of the candidate virtual addresses are different.

5. The method according to claim 1, characterized in that the step of searching for the physical address corresponding to each of the target virtual addresses from the target address sets indicated by each of the target set identifiers in parallel to obtain a plurality of target physical addresses comprises:

In parallel, in each of the target address sets, a corresponding physical address subset identifier is matched for the virtual address subset where each of the target virtual addresses is located from the virtual address subset identifiers and the physical address subset identifiers having a corresponding relationship, to obtain a plurality of target subset identifiers, wherein the virtual address set is divided into a plurality of virtual address subsets, and the physical address set includes a plurality of physical address subsets;

In parallel, a physical address is matched for the target virtual address from virtual addresses and physical addresses having a corresponding relationship in the target address subsets indicated by the target subset identifiers to obtain a plurality of target physical addresses.

6. The method according to claim 1, characterized in that before matching the corresponding address set identifier for the virtual address set where each of the target virtual addresses is located from the virtual address sets and address set identifiers having a corresponding relationship, the method further comprises:

Establishing a one-to-one correspondence between each virtual address in the memory and each physical address in the target external storage space;

Storing a plurality of virtual addresses and physical addresses having corresponding relationships into a plurality of physical address sets, wherein the physical address sets are located in an additional external storage space;

Establishing a one-to-one correspondence between each of the address set identifiers and each of the virtual address sets;

A plurality of corresponding virtual address sets and address set identifiers are stored in the memory.

7. The method according to claim 6, wherein storing a plurality of virtual addresses and physical addresses having a corresponding relationship into a plurality of physical address sets comprises:

Determine the number of disk placement positions according to a disk placement capacity parameter, wherein the disk placement capacity parameter is used to indicate the disk placement capacity of metadata, the number of disk placement positions is the number of types of the physical address set divided according to capacity, and the capacity of the physical address set is used to indicate the number of physical addresses and virtual addresses with corresponding relationships that are allowed to be stored in the corresponding physical address set;

The physical addresses in the target external storage space are divided into a plurality of physical address sets matching the capacity of the physical address sets in order from low to high, wherein the capacity of the plurality of physical address sets matches the number of disk positions, and the physical addresses in a physical address set with a larger capacity in each of the physical address sets are higher than the physical addresses in a physical address set with a smaller capacity.

8. The method according to claim 6, wherein the step of establishing a one-to-one correspondence between each of the address set identifiers and each of the virtual address sets comprises:

Recording the virtual addresses stored in each of the physical address sets to obtain a plurality of the physical address sets and the virtual address sets having a corresponding relationship;

A one-to-one correspondence between each of the address set identifiers and each of the virtual address sets is established according to the correspondence between each of the physical address sets and each of the virtual address sets.

9. The method according to claim 1, characterized in that after writing the plurality of target metadata to the target physical address corresponding to the target external storage space, the method further comprises:

Detecting access information of each metadata stored in the target external storage space, wherein the access information is used to indicate a situation in which the corresponding metadata is accessed;

Adjusting the corresponding physical addresses and virtual addresses stored in each of the physical address sets to match the access information, and obtaining a reorganization adjustment result, wherein the reorganization adjustment result is used to indicate the virtual addresses stored in each of the physical address sets after adjustment;

The virtual address set corresponding to the address set identifier is modified according to the reorganization adjustment result.

10. The method according to claim 9, wherein the step of adjusting the corresponding physical addresses and virtual addresses stored in each of the physical address sets to match the access information comprises:

sorting the metadata stored in the target external storage space according to the access parameters of the metadata included in the access information to obtain a metadata sequence, wherein the access parameters are used to indicate the frequency of access to the metadata corresponding to the access parameters in a current time period;

Establishing a reorganized correspondence between each metadata stored in the target external storage space and each physical address set according to the position of each metadata stored in the target external storage space in the metadata sequence and the capacity of each physical address set, wherein the capacity of the physical address set is used to indicate the number of physical addresses and virtual addresses having a corresponding relationship stored in the corresponding physical address set, and in the reorganized correspondence, the more frequently accessed metadata indicated by the access parameter is, the smaller the capacity of the physical address set corresponding to the metadata;

The physical addresses and virtual addresses corresponding to each metadata stored in the target external storage space are rewritten into the corresponding physical address set indicated by the reorganized correspondence relationship.

11. The method according to claim 10, characterized in that rewriting the physical addresses and virtual addresses corresponding to each metadata stored in the target external storage space to the corresponding physical address set indicated by the reorganized correspondence relationship comprises:

Backing up the corresponding physical addresses and virtual addresses stored in the reference address set to the backup set, wherein the reference address set is the physical address set to be rewritten;

The physical addresses and virtual addresses with corresponding relationships corresponding to the metadata corresponding to the reference address set indicated by the reorganized corresponding relationship are rewritten from the backup set to the reference address set.

12. The method according to claim 9, wherein modifying the virtual address set corresponding to the address set identifier according to the reorganization adjustment result comprises:

Determine the virtual addresses stored in each of the address sets according to the reorganization and adjustment results, and obtain a plurality of changed address sets corresponding to each of the physical address sets;

The virtual address set corresponding to the address set identifier indicating the physical address set is modified to the corresponding changed address set.

13. The method according to claim 1, characterized in that the method further comprises:

Comparing the relationship between the disk placement parameter and the disk placement parameter threshold, wherein the disk placement parameter is used to indicate the disk placement load of the metadata to be currently placed on disk;

When the number of consecutive detections that the disk landing parameter is greater than the disk landing parameter threshold, or the number of consecutive detections that the disk landing parameter is less than the disk landing parameter threshold, is greater than or equal to the number threshold, the capacity of each physical address set is adjusted by adjusting the current disk landing parameter toward the disk landing parameter threshold, wherein the capacity of the physical address set is used to indicate the number of corresponding physical addresses and virtual addresses allowed to be stored in the corresponding physical address set.

14. The method according to claim 13, characterized in that the step of adjusting the capacity of each of the physical address sets by adjusting the current disk placement parameter toward the disk placement parameter threshold comprises:

In the case where the number of times that the disk placement parameter is continuously detected to be greater than the disk placement parameter threshold is greater than or equal to the number threshold, searching for the physical address set whose capacity is smaller than the capacity of other address sets in the multiple physical address sets to obtain multiple merged address sets, wherein the capacity is used to indicate the number of physical addresses and virtual addresses having a corresponding relationship stored in the corresponding physical address set, and the other address set is a physical address set in the multiple physical address sets except the merged address set;

Merging a plurality of the merged address sets to obtain an integrated address set;

Establishing a correspondence between a virtual address set corresponding to a merged set identifier and an integrated set identifier, wherein the merged set identifier is an address set identifier indicating the merged address set, and the integrated set identifier is an address set identifier indicating the integrated address set;

The correspondence between the merged set identifier and the corresponding virtual address set is cancelled.

15. The method according to claim 13, characterized in that the step of adjusting the capacity of each of the physical address sets by adjusting the current disk placement parameter toward the disk placement parameter threshold comprises:

In the case where the number of times that the disk placement parameter is continuously detected to be less than the disk placement parameter threshold is greater than or equal to the number threshold, searching for the physical address set with the largest capacity among the multiple physical address sets to obtain a split address set, wherein the capacity is used to indicate the number of physical addresses and virtual addresses having a corresponding relationship stored in the corresponding physical address set;

Splitting the split address set into multiple physical address sets to obtain multiple partition address sets;

Establishing a correspondence between a virtual address set corresponding to a split set identifier and a plurality of partition set identifiers, wherein the split set identifier is the address set identifier indicating the split address set, and the partition set identifier is the address set identifier indicating the partition address set;

The correspondence between the split set identifier and the corresponding virtual address set is cancelled.

16. The method according to claim 13, characterized in that the step of comparing the relationship between the plate placement parameter and the plate placement parameter threshold comprises:

Detecting a first parameter and a second parameter and obtaining a first parameter threshold and a second parameter threshold, wherein the first parameter is used to indicate the utilization of operation resources by the current metadata storage, and the second parameter is used to indicate the utilization of memory resources by the current metadata storage;

Comparing the magnitude relationship between the first parameter and the first parameter threshold, and comparing the magnitude relationship between the second parameter and the second parameter threshold;

When the first parameter is greater than the first parameter threshold and the second parameter is greater than the second parameter threshold, it is determined that the plate placement parameter is greater than the plate placement parameter threshold;

When the first parameter is less than the first parameter threshold and the second parameter is less than the second parameter threshold, it is determined that the plate landing parameter is detected to be less than the plate landing parameter threshold.

17. The method according to claim 16, characterized in that the obtaining of the first parameter threshold and the second parameter threshold comprises:

Detecting whether there is a user threshold set, wherein the user threshold set is a parameter threshold set by a user;

In a case where the user threshold set is detected to exist, extracting the first parameter threshold and the second parameter threshold from the user threshold set;

In case that the user threshold set is detected to be absent, a first default threshold is determined as the first parameter threshold and a second default threshold is determined as the second parameter threshold.

18. A metadata storage device, comprising:

An acquisition module is used to acquire virtual addresses of multiple target metadata to be written to the disk in the memory, and obtain multiple target virtual addresses; a matching module is used to match the corresponding address set identifier for the virtual address set where each of the target virtual addresses is located from the virtual address set and the address set identifier having a corresponding relationship, and obtain multiple target set identifiers, wherein the physical address set indicated by the address set identifier stores the physical addresses corresponding to each virtual address in the corresponding virtual address set in the target external storage space; a search module is used to search in parallel for the physical addresses corresponding to each of the target virtual addresses from the target address sets indicated by each of the target set identifiers, and obtain multiple target physical addresses; a disk writing module is used to write the multiple target metadata to the corresponding target physical addresses in the target external storage space;

Among them, the matching module includes: a first search unit, used to search for multiple disk address sets into which each target virtual address falls from the virtual address set; a second search unit, used to search for the target set identifiers corresponding to the multiple disk address sets respectively from the virtual address sets and address set identifiers with corresponding relationships, and obtain multiple target set identifiers.

19. An electronic device, comprising:

Memory for storing computer programs;

A processor, configured to implement the steps of the method for writing metadata to a disk as claimed in any one of claims 1 to 17 when executing the computer program.

20. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, wherein when the computer program is executed by a processor, the steps of the method for storing metadata on a disk as described in any one of claims 1 to 17 are implemented.

21. A computer program product, comprising a computer program, characterized in that when the computer program is executed by a processor, the steps of the method for storing metadata in a disk as claimed in any one of claims 1 to 17 are implemented.