CN112000487B - Scrub pressure adjusting method, device and medium - Google Patents

Scrub pressure adjusting method, device and medium Download PDF

Info

Publication number
CN112000487B
CN112000487B CN202010821283.1A CN202010821283A CN112000487B CN 112000487 B CN112000487 B CN 112000487B CN 202010821283 A CN202010821283 A CN 202010821283A CN 112000487 B CN112000487 B CN 112000487B
Authority
CN
China
Prior art keywords
scrub
task
cluster
process pressure
objects
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010821283.1A
Other languages
Chinese (zh)
Other versions
CN112000487A (en
Inventor
李彦博
孟祥瑞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
IEIT Systems Co Ltd
Original Assignee
Inspur Electronic Information Industry Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inspur Electronic Information Industry Co Ltd filed Critical Inspur Electronic Information Industry Co Ltd
Priority to CN202010821283.1A priority Critical patent/CN112000487B/en
Publication of CN112000487A publication Critical patent/CN112000487A/en
Application granted granted Critical
Publication of CN112000487B publication Critical patent/CN112000487B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/52Program synchronisation; Mutual exclusion, e.g. by means of semaphores
    • G06F9/524Deadlock detection or avoidance
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083Techniques for rebalancing the load in a distributed system

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Measuring Fluid Pressure (AREA)
  • Feedback Control In General (AREA)

Abstract

本发明实施例公开了一种scrub压力调整方法、装置和介质,获取集群的进程压力;依据集群的进程压力以及scrub任务的执行条件,触发scrub状态机,此时系统会自动对scrub任务进行加锁操作,为了降低本次加锁操作占用的时长,可以根据集群的进程压力,调整scrub任务中执行本次扫描任务的对象个数;依据对象个数执行本次扫描任务。每完成一次扫描任务则根据集群的进程压力,调整scrub任务中执行本次扫描任务的对象个数,直至scrub任务包含的所有扫描任务全部完成则结束操作。根据进程压力自动调整当前所需扫描的对象个数,有效的降低了scrub任务对前端业务影响。

Figure 202010821283

The embodiment of the present invention discloses a method, device and medium for adjusting the scrub pressure, which acquires the process pressure of the cluster; triggers the scrub state machine according to the process pressure of the cluster and the execution conditions of the scrub task, and the system automatically adds the scrub task at this time. Lock operation, in order to reduce the time taken by this lock operation, you can adjust the number of objects in the scrub task to perform this scan task according to the process pressure of the cluster; perform this scan task according to the number of objects. Each time a scan task is completed, the number of objects in the scrub task to perform the scan task is adjusted according to the process pressure of the cluster, and the operation ends when all the scan tasks included in the scrub task are completed. According to the process pressure, the number of objects to be scanned is automatically adjusted, which effectively reduces the impact of the scrub task on the front-end business.

Figure 202010821283

Description

Scrub pressure adjusting method, device and medium
Technical Field
The present invention relates to the field of distributed storage technologies, and in particular, to a method and an apparatus for regulating scrub pressure, and a computer-readable storage medium.
Background
For distributed storage clusters, data consistency between copies needs to be guaranteed. In the cluster operation process, data inconsistency between copies can be caused due to various reasons, a data consistency check (scrub) provides a mechanism for sensing the data inconsistency between the copies, and the scrub compares the scanned data between the copies and shows the data inconsistency when the data inconsistency is found.
The scrub operation needs to be processed by adding the pg lock when being executed, the front-end service of the cluster also needs to be executed by adding the pg lock, the scrub operation and the front-end service belong to mutual exclusion operation due to the relation of the pg lock, and when a large number of scrubs trigger, the front-end service of the cluster can be seriously influenced.
It can be seen that how to reduce the impact of the scrub task on the front-end service is a problem to be solved by those skilled in the art.
Disclosure of Invention
An object of the embodiments of the present invention is to provide a method and an apparatus for regulating scrub pressure, and a computer-readable storage medium, which can reduce the influence of a scrub task on a front-end service.
To solve the above technical problem, an embodiment of the present invention provides a scrub pressure adjustment method, including:
acquiring the process pressure of a cluster;
triggering a scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task;
according to the process pressure of the cluster, adjusting the number of objects for executing the scanning task in the scrub task;
and executing the scanning task according to the number of the objects, and adjusting the number of the objects executing the scanning task in the scrub task according to the process pressure of the cluster every time the scanning task is completed until all the scanning tasks contained in the scrub task are completed, and ending the operation.
Optionally, the triggering the scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task includes:
when the PG is in a task execution state and the current time meets the execution time of a scrub task, judging whether the process pressure of the cluster belongs to a preset high pressure range;
when the process pressure of the cluster does not belong to a preset high pressure range, triggering a scrub state machine;
and when the process pressure of the cluster belongs to a preset high pressure range, triggering a scrub state machine when the scrub task reaches the cut-off time.
Optionally, after the scanning task is executed according to the number of the objects, the method further includes:
judging whether the process pressure of the cluster belongs to a preset high pressure range or not;
and when the process pressure of the cluster belongs to a preset high pressure range, waiting for a preset time and then executing the step of adjusting the number of objects for executing the scanning task in the scrub task according to the process pressure of the cluster.
Optionally, the adjusting, according to the process pressure of the cluster, the number of objects in the scrub task for executing the scanning task includes:
searching the number of objects matched with the process pressure of the cluster from a pre-established corresponding relation list, and taking the number of the objects as the number of the objects for executing the scanning task in the scrub task; wherein, the corresponding relation list records the corresponding relation between different process pressure ranges and the number of objects.
Optionally, after the scanning task is executed according to the number of the objects, the method further includes:
judging whether the object data acquired by executing the scanning task is consistent with the object data fed back by the standby equipment or not;
and when the object data acquired by executing the scanning task is inconsistent with the object data fed back by the standby equipment, marking the scrub task in an error state.
The embodiment of the invention also provides a scrub pressure adjusting device which comprises an acquisition unit, a trigger unit, an adjusting unit and an execution unit;
the acquiring unit is used for acquiring the process pressure of the cluster;
the triggering unit is used for triggering a scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task;
the adjusting unit is configured to adjust the number of objects in the scrub task, which execute the current scanning task, according to the process pressure of the cluster;
and the execution unit is used for executing the scanning task according to the number of the objects, and adjusting the number of the objects for executing the scanning task in the scrub task according to the process pressure of the cluster every time the scanning task is completed, and ending the operation until all the scanning tasks included in the scrub task are completed.
Optionally, the triggering unit includes a judging subunit, an executing subunit, and a waiting subunit;
the judging subunit is configured to, when the PG is in a task execution state and the current time meets the execution time of the scrub task, judge whether the process pressure of the cluster belongs to a preset high pressure range;
the execution subunit is used for triggering a scrub state machine when the process pressure of the cluster does not belong to a preset high pressure range;
and the waiting subunit is used for triggering the scrub state machine when the process pressure of the cluster belongs to a preset high pressure range and the scrub task is waited to reach the deadline time.
Optionally, the device further comprises a pressure judging unit and a waiting unit;
the pressure judging unit is used for judging whether the process pressure of the cluster belongs to a preset high pressure range or not;
and the waiting unit is used for waiting for a preset time and then executing the step of adjusting the number of objects for executing the scanning task in the scrub task according to the process pressure of the cluster when the process pressure of the cluster belongs to a preset high pressure range.
Optionally, the adjusting unit is specifically configured to search, from a pre-established correspondence list, the number of objects that matches the process pressure of the cluster, and use the number of objects as the number of objects for executing the current scanning task in the scrub task; and the corresponding relation list records the corresponding relation between different process pressure ranges and the number of the objects.
Optionally, the system further comprises a data judging unit and a marking unit;
the data judging unit is used for judging whether the object data acquired by executing the scanning task is consistent with the object data fed back by the standby equipment;
the marking unit is configured to mark the scrub task in an error state when the object data obtained by executing the scanning task at this time is inconsistent with the object data fed back by the standby device.
The embodiment of the present invention further provides a scrub pressure adjusting device, including:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the scrub pressure adjustment method as in any one of the above.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the scrub pressure adjustment method according to any one of the foregoing embodiments are implemented.
According to the technical scheme, the process pressure of the cluster is obtained; and triggering the scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task. And after the scrub state machine is triggered, the system automatically locks the scrub task. In order to reduce the time occupied by the locking operation, the number of objects for executing the scanning task in the scrub task can be adjusted according to the process pressure of the cluster; and executing the scanning task according to the number of the objects. The number of objects to be scanned in a scrub task is often large, and multiple scanning tasks generally need to be executed, so that the number of objects to be executed in the scrub task at this time can be adjusted according to the process pressure of the cluster every time the scanning task is completed, and the operation is ended until all the scanning tasks included in the scrub task are completed. In the technical scheme, the number of objects to be scanned by the current scrub task can be automatically adjusted according to the process pressure of the cluster, and the time that the scrub task occupies a pg lock is effectively reduced under the condition that the scrub task can be smoothly executed, so that the influence of the scrub task on the front-end service is effectively reduced.
Drawings
In order to illustrate the embodiments of the present invention more clearly, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained by those skilled in the art without inventive effort.
Fig. 1 is a flowchart of a scrub pressure adjustment method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a scrub pressure adjustment device according to an embodiment of the present invention;
fig. 3 is a schematic hardware structure diagram of a scrub pressure adjustment device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without any creative work belong to the protection scope of the present invention.
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
Next, a detailed description is given of a scrub pressure adjustment method according to an embodiment of the present invention. Fig. 1 is a flowchart of a scrub pressure adjustment method according to an embodiment of the present invention, where the method includes:
s101: and acquiring the process pressure of the cluster.
After the Object-based Storage Device (OSD) is started, a scrub task is registered for each Place Group (PG).
The process pressure of the cluster may include CPU utilization, disk utilization, and the like. In an embodiment of the present invention, the OSD may scan the PG periodically to collect the process pressure of the cluster within a period time. The length of the cycle time can be set according to actual requirements, for example, the cycle time can be set to 1 minute.
It should be noted that, in order to ensure that the process pressure can accurately reflect the current pressure condition of the cluster, in the embodiment of the present invention, the process pressures of the clusters all refer to the process pressure of the cluster in the latest cycle time, and taking the cycle time as 1 minute as an example, the process pressure of the cluster in the past 1 minute may be obtained by taking the current time as a reference time.
S102: and triggering the scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task.
When the PG is in an active + clean state, that is, the PG is in a task execution state, and the current time meets the execution time of the scrub task, the OSD may trigger the scrub state machine, so as to start the operation flow of executing the scrub task. And after the scrub state machine is triggered, the system automatically locks the scrub task.
In the embodiment of the present invention, in order to reduce the influence of the scrub task on the front end service, when the PG is in the task execution state and the current time meets the execution time of the scrub task, it may be determined whether the process pressure of the cluster belongs to the preset high pressure range.
The preset high pressure range may be set to have a CPU utilization greater than 50% or a disk utilization greater than 50%.
When the process pressure of the cluster does not belong to the preset high pressure range, the OSD in the cluster can rapidly process the scrub task, and at the moment, the scrub state machine can be directly triggered.
When the process pressure of the cluster belongs to the preset high pressure range, the resource occupancy rate of the cluster is high, if the scrub state machine is triggered, the locking operation of the scrub task takes longer time due to the fact that OSD cannot process the scrub task quickly, and therefore execution of front-end services in the cluster is affected, and therefore when the process pressure of the cluster belongs to the preset high pressure range, the scrub state machine can be triggered after the scrub task reaches the cut-off time.
Each scrub task carries an expiration time at registration as long as the OSD executes the scrub task within the expiration time.
S103: and adjusting the number of objects for executing the scanning task in the scrub task according to the process pressure of the cluster.
The process pressure of the cluster directly affects the time it takes for the OSD to perform the scrub task, during which the OSD may add the pg lock. The longer the OSD takes to execute the scrub task, the longer the time it takes to occupy the pg lock, and correspondingly, the greater the impact on the front-end service in the cluster. Therefore, in the embodiment of the present invention, the number of objects in the scrub task for executing the current scanning task may be adjusted according to the process pressure of the cluster.
When the process pressure of the cluster is higher, the number of the objects of the scanning task can be properly reduced; when the process pressure of the cluster is small, the number of the objects of the scanning task can be properly increased.
In a specific implementation, the corresponding relationship between different process pressure ranges and the number of objects may be recorded in a corresponding relationship list, and after the scrub state machine is triggered, the number of objects matching the process pressure of the cluster may be searched from the pre-established corresponding relationship list, and the number of objects is used as the number of objects for executing the scanning task in the scrub task.
For example, the process pressure of the cluster may be divided into three levels, i.e., a high pressure level, a medium pressure level and a low pressure level, for example, when the CPU utilization rate or the disk utilization rate is greater than 50%, it is indicated that the process pressure of the cluster belongs to the high pressure level; when the utilization rate of the CPU or the utilization rate of the disk is less than 10%, indicating that the process pressure of the cluster belongs to low pressure; when the utilization rate of the CPU or the utilization rate of the disk is between 10% and 50%, the process pressure of the cluster belongs to the medium pressure. The number of the objects of the scanning task under high pressure can be set to be 2; the number of the objects of the scanning task at the time under the medium pressure can be set to be 5; the number of objects of the current scanning task under low pressure can be set to 10.
S104: and executing the scanning task according to the number of the objects, and adjusting the number of the objects executing the scanning task in the scrub task according to the process pressure of the cluster every time the scanning task is completed, until all the scanning tasks contained in the scrub task are completed, ending the operation.
The number of objects to be scanned in a scrub task is often large, and multiple scanning tasks generally need to be executed, so that each time a scanning task is completed, the operation can be returned to S103, the number of objects to be executed in the scrub task until the scrub task is completed is adjusted according to the cluster process pressure, and the operation is ended.
It should be noted that, in the embodiment of the present invention, the process pressure of the cluster according to which the adjustment of the number of objects is performed each time the process returns to S103 is the process pressure of the cluster in the latest cycle time.
According to the technical scheme, the process pressure of the cluster is obtained; and triggering a scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task. And after the scrub state machine is triggered, the system automatically locks the scrub task. In order to reduce the time occupied by the locking operation, the number of objects for executing the scanning task in the scrub task can be adjusted according to the process pressure of the cluster; and executing the scanning task according to the number of the objects. The number of objects to be scanned in a scrub task is often large, and multiple scanning tasks generally need to be executed, so that the number of objects to be executed in the scrub task at this time can be adjusted according to the process pressure of the cluster every time the scanning task is completed, and the operation is ended until all the scanning tasks included in the scrub task are completed. In the technical scheme, the number of objects to be scanned by the current scrub task can be automatically adjusted according to the process pressure of the cluster, and the time that the scrub task occupies a pg lock is effectively reduced under the condition that the scrub task can be smoothly executed, so that the influence of the scrub task on the front-end service is effectively reduced.
In the embodiment of the present invention, after the current scanning task is executed according to the number of the objects, the process may return to S103 to readjust the number of the objects corresponding to the next scanning task. Therefore, in the embodiment of the present invention, in order to further reduce the influence of the scrub task on the front end service, after the current scanning task is completed according to the number of objects, before the number of objects corresponding to the next scanning task is adjusted, it may be determined whether the process pressure of the cluster belongs to the preset high pressure range.
And when the process pressure of the cluster does not belong to the preset high pressure range, adjusting the number of the objects executing the scanning task in the scrub task directly according to the process pressure of the cluster.
When the process pressure of the cluster belongs to the preset high pressure range, the step of adjusting the number of the objects for executing the scanning task in the scrub task according to the process pressure of the cluster can be executed after waiting for the preset time.
The value of the preset time can be set according to actual requirements, for example, the preset time can be set to 3 seconds.
The process pressure of the cluster is evaluated every time the scanning task is executed, the next scanning task can be executed after the preset time when the process pressure of the cluster is high, the process pressure of the cluster can be effectively relieved, and therefore the influence of the execution of the scrub task on the front-end service is further reduced.
In an embodiment of the present invention, the OSDs in the cluster may include a main OSD and a standby OSD. After the main OSD executes the scanning task, the corresponding scanning task is sent to the standby OSD, and the standby OSD feeds back object data to the main OSD after the scanning task is finished. In order to ensure the consistency between the main OSD and the standby OSD data, after receiving the object data fed back by the standby OSD, the main OSD may determine whether the object data obtained by executing the current scanning task is consistent with the object data fed back by the standby device.
When the object data acquired by executing the scanning task is inconsistent with the object data fed back by the standby device, the scrub task may be marked as an error state.
By comparing the object data of the main OSD and the standby OSD, the problem of data inconsistency can be found in time. The scrub tasks are marked to be in an error state, so that a manager can conveniently and visually know which scrub task has data inconsistency, and the problem of data inconsistency is specifically solved.
Fig. 2 is a schematic structural diagram of a scrub pressure adjustment device according to an embodiment of the present invention, including an obtaining unit 21, a triggering unit 22, an adjusting unit 23, and an executing unit 24;
an obtaining unit 21, configured to obtain a process pressure of the cluster;
the triggering unit 22 is configured to trigger a scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task;
the adjusting unit 23 is configured to adjust the number of objects in the scrub task for executing the current scanning task according to the process pressure of the cluster;
and the execution unit 24 is configured to execute the scanning task according to the number of the objects, and adjust the number of the objects executing the scanning task in the scrub task according to the process pressure of the cluster every time the scanning task is completed, until all the scanning tasks included in the scrub task are completed, the operation is ended.
Optionally, the triggering unit includes a judging subunit, an executing subunit, and a waiting subunit;
the judging subunit is used for judging whether the process pressure of the cluster belongs to a preset high pressure range or not when the PG is in a task execution state and the current time meets the execution time of the scrub task;
the execution subunit is used for triggering the scrub state machine when the process pressure of the cluster does not belong to a preset high pressure range;
and the waiting subunit is used for triggering the scrub state machine when the process pressure of the cluster belongs to a preset high pressure range and the scrub task reaches the deadline time.
Optionally, the device further comprises a pressure judging unit and a waiting unit;
the pressure judging unit is used for judging whether the process pressure of the cluster belongs to a preset high pressure range or not;
and the waiting unit is used for waiting for a preset time and then executing the step of adjusting the number of the objects for executing the scanning task in the scrub task according to the process pressure of the cluster when the process pressure of the cluster belongs to the preset high pressure range.
Optionally, the adjusting unit is specifically configured to search, from a pre-established correspondence list, a number of objects that matches the process pressure of the cluster, and use the number of objects as the number of objects for executing the current scanning task in the scrub task; the corresponding relation list records the corresponding relation between different process pressure ranges and the number of the objects.
Optionally, the system further comprises a data judging unit and a marking unit;
the data judging unit is used for judging whether the object data acquired by executing the scanning task is consistent with the object data fed back by the standby equipment;
and the marking unit is used for marking the scrub task as an error state when the object data acquired by executing the scanning task is inconsistent with the object data fed back by the standby equipment.
The description of the features in the embodiment corresponding to fig. 2 may refer to the related description of the embodiment corresponding to fig. 1, and is not repeated here.
According to the technical scheme, the process pressure of the cluster is obtained; and triggering the scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task. And after the scrub state machine is triggered, the system automatically locks the scrub task. In order to reduce the time occupied by the locking operation, the number of objects for executing the scanning task in the scrub task can be adjusted according to the process pressure of the cluster; and executing the scanning task according to the number of the objects. The number of objects to be scanned in a scrub task is often large, and multiple scanning tasks generally need to be executed, so that the number of objects to be executed in the scrub task at this time can be adjusted according to the process pressure of the cluster every time the scanning task is completed, and the operation is ended until all the scanning tasks included in the scrub task are completed. In the technical scheme, the number of objects to be scanned by the current scrub task can be automatically adjusted according to the process pressure of the cluster, and the time that the scrub task occupies a pg lock is effectively reduced under the condition that the scrub task can be smoothly executed, so that the influence of the scrub task on the front-end service is effectively reduced.
Fig. 3 is a schematic hardware structure diagram of a scrub pressure adjustment device 30 according to an embodiment of the present invention, including:
a memory 31 for storing a computer program;
a processor 32 for executing a computer program to implement the steps of the scrub pressure adjustment method as described in any of the embodiments above.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the scrub pressure adjustment method according to any of the above embodiments are implemented.
The foregoing describes a scrub pressure adjustment method, apparatus and computer readable storage medium provided by the embodiments of the present invention in detail. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, without departing from the principle of the present invention, it is possible to make various improvements and modifications to the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

Claims (8)

1.一种scrub压力调整方法,其特征在于,包括:1. a scrub pressure adjustment method, is characterized in that, comprises: 获取集群的进程压力;Get the process pressure of the cluster; 依据所述集群的进程压力以及scrub任务的执行条件,触发scrub状态机;Trigger the scrub state machine according to the process pressure of the cluster and the execution conditions of the scrub task; 根据所述集群的进程压力,调整所述scrub任务中执行本次扫描任务的对象个数;According to the process pressure of the cluster, adjust the number of objects in the scrub task to perform this scan task; 依据所述对象个数执行本次扫描任务,每完成一次扫描任务则根据所述集群的进程压力,调整所述scrub任务中执行本次扫描任务的对象个数,直至所述scrub任务包含的所有扫描任务全部完成,则结束操作;Execute this scan task according to the number of objects, and adjust the number of objects to perform this scan task in the scrub task according to the process pressure of the cluster each time a scan task is completed, until all the objects included in the scrub task are When all the scanning tasks are completed, the operation ends; 所述依据所述集群的进程压力以及scrub任务的执行条件,触发scrub状态机包括:The triggering of the scrub state machine according to the process pressure of the cluster and the execution conditions of the scrub task includes: 当PG处于任务执行状态并且当前时间满足scrub任务的执行时间时,判断所述集群的进程压力是否属于预设高压力范围;When the PG is in the task execution state and the current time satisfies the execution time of the scrub task, determine whether the process pressure of the cluster belongs to the preset high pressure range; 当所述集群的进程压力不属于预设高压力范围,则触发scrub状态机;When the process pressure of the cluster does not belong to the preset high pressure range, the scrub state machine is triggered; 当所述集群的进程压力属于预设高压力范围,则等待所述scrub任务达到截止时间时则触发scrub状态机。When the process pressure of the cluster falls within the preset high pressure range, the scrub state machine is triggered when the scrub task reaches the deadline. 2.根据权利要求1所述的scrub压力调整方法,其特征在于,在所述依据所述对象个数执行本次扫描任务之后还包括:2. The method for adjusting the scrub pressure according to claim 1, characterized in that, after performing the current scanning task according to the number of objects, it further comprises: 判断所述集群的进程压力是否属于预设高压力范围;Determine whether the process pressure of the cluster belongs to a preset high pressure range; 当所述集群的进程压力属于预设高压力范围时,则等待预设时间后再执行所述根据所述集群的进程压力,调整所述scrub任务中执行本次扫描任务的对象个数的步骤。When the process pressure of the cluster falls within the preset high pressure range, wait for a preset time before executing the step of adjusting the number of objects in the scrub task to perform the scan task according to the process pressure of the cluster . 3.根据权利要求1所述的scrub压力调整方法,其特征在于,所述根据所述集群的进程压力,调整所述scrub任务中执行本次扫描任务的对象个数包括:3 . The method for adjusting the scrub pressure according to claim 1 , wherein, according to the process pressure of the cluster, adjusting the number of objects in the scrub task to perform this scan task comprises: 3 . 从预先建立的对应关系列表中查找与所述集群的进程压力相匹配的对象个数,并将所述对象个数作为所述scrub任务中执行本次扫描任务的对象个数;其中,所述对应关系列表中记载了不同进程压力范围与对象个数的对应关系。Find the number of objects matching the process pressure of the cluster from the pre-established correspondence list, and use the number of objects as the number of objects in the scrub task to perform this scan task; wherein, the The correspondence list records the correspondence between different process pressure ranges and the number of objects. 4.根据权利要求1所述的scrub压力调整方法,其特征在于,在所述依据所述对象个数执行本次扫描任务之后还包括:4. The method for adjusting the scrub pressure according to claim 1, characterized in that, after performing the current scanning task according to the number of objects, it further comprises: 判断执行本次扫描任务获取的对象数据与备用设备反馈的对象数据是否一致;Determine whether the object data obtained by executing this scan task is consistent with the object data fed back by the standby device; 当执行本次扫描任务获取的对象数据与备用设备反馈的对象数据不一致时,则将所述scrub任务标记为错误状态。When the object data obtained by executing the current scan task is inconsistent with the object data fed back by the standby device, the scrub task is marked as an error state. 5.一种scrub压力调整装置,其特征在于,包括获取单元、触发单元、调整单元和执行单元;5. A scrub pressure adjustment device, characterized in that it comprises an acquisition unit, a trigger unit, an adjustment unit and an execution unit; 所述获取单元,用于获取集群的进程压力;The obtaining unit is used to obtain the process pressure of the cluster; 所述触发单元,用于依据所述集群的进程压力以及scrub任务的执行条件,触发scrub状态机;The triggering unit is used to trigger the scrub state machine according to the process pressure of the cluster and the execution conditions of the scrub task; 所述调整单元,用于根据所述集群的进程压力,调整所述scrub任务中执行本次扫描任务的对象个数;The adjustment unit is configured to adjust the number of objects in the scrub task to perform this scan task according to the process pressure of the cluster; 所述执行单元,用于依据所述对象个数执行本次扫描任务,每完成一次扫描任务则根据所述集群的进程压力,调整所述scrub任务中执行本次扫描任务的对象个数,直至所述scrub任务包含的所有扫描任务全部完成,则结束操作;The execution unit is configured to perform this scan task according to the number of objects, and adjust the number of objects in the scrub task to perform this scan task according to the process pressure of the cluster each time a scan task is completed, until the scan task is completed. When all the scanning tasks included in the scrub task are completed, the operation ends; 所述触发单元包括判断子单元、执行子单元和等待子单元;The triggering unit includes a judging subunit, an executing subunit and a waiting subunit; 所述判断子单元,用于当PG处于任务执行状态并且当前时间满足scrub任务的执行时间时,判断所述集群的进程压力是否属于预设高压力范围;The judging subunit is used to judge whether the process pressure of the cluster belongs to the preset high pressure range when the PG is in the task execution state and the current time meets the execution time of the scrub task; 所述执行子单元,用于当所述集群的进程压力不属于预设高压力范围,则触发scrub状态机;The execution subunit is used to trigger the scrub state machine when the process pressure of the cluster does not belong to the preset high pressure range; 所述等待子单元,用于当所述集群的进程压力属于预设高压力范围,则等待所述scrub任务达到截止时间时则触发scrub状态机。The waiting subunit is configured to trigger the scrub state machine when the process pressure of the cluster belongs to a preset high pressure range and wait for the scrub task to reach the deadline. 6.根据权利要求5所述的scrub压力调整装置,其特征在于,还包括压力判断单元和等待单元;6. The scrub pressure adjusting device according to claim 5, further comprising a pressure judging unit and a waiting unit; 所述压力判断单元,用于判断所述集群的进程压力是否属于预设高压力范围;the pressure judgment unit, configured to judge whether the process pressure of the cluster belongs to a preset high pressure range; 所述等待单元,用于当所述集群的进程压力属于预设高压力范围时,则等待预设时间后再执行所述根据所述集群的进程压力,调整所述scrub任务中执行本次扫描任务的对象个数的步骤。The waiting unit is configured to, when the process pressure of the cluster belongs to a preset high pressure range, wait for a preset time before executing the current scan in adjusting the scrub task according to the process pressure of the cluster The number of steps for the number of objects in the task. 7.一种scrub压力调整装置,其特征在于,包括:7. A scrub pressure adjusting device, characterized in that, comprising: 存储器,用于存储计算机程序;memory for storing computer programs; 处理器,用于执行所述计算机程序以实现如权利要求1至4任意一项所述scrub压力调整方法的步骤。The processor is configured to execute the computer program to implement the steps of the scrub pressure adjustment method according to any one of claims 1 to 4. 8.一种计算机可读存储介质,其特征在于,所述计算机可读存储介质上存储有计算机程序,所述计算机程序被处理器执行时实现如权利要求1至4任一项所述scrub压力调整方法的步骤。8. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the scrub pressure according to any one of claims 1 to 4 is realized Steps to adjust the method.
CN202010821283.1A 2020-08-14 2020-08-14 Scrub pressure adjusting method, device and medium Active CN112000487B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010821283.1A CN112000487B (en) 2020-08-14 2020-08-14 Scrub pressure adjusting method, device and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010821283.1A CN112000487B (en) 2020-08-14 2020-08-14 Scrub pressure adjusting method, device and medium

Publications (2)

Publication Number Publication Date
CN112000487A CN112000487A (en) 2020-11-27
CN112000487B true CN112000487B (en) 2022-07-08

Family

ID=73472917

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010821283.1A Active CN112000487B (en) 2020-08-14 2020-08-14 Scrub pressure adjusting method, device and medium

Country Status (1)

Country Link
CN (1) CN112000487B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109710456A (en) * 2018-12-10 2019-05-03 新华三技术有限公司 A kind of data reconstruction method and device
CN110059068A (en) * 2019-04-11 2019-07-26 厦门网宿有限公司 Data verification method and data verification system in a kind of distributed memory system
CN111367926A (en) * 2020-02-27 2020-07-03 星辰天合(北京)数据科技有限公司 Data processing method and device for distributed system

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9672122B1 (en) * 2014-09-29 2017-06-06 Amazon Technologies, Inc. Fault tolerant distributed tasks using distributed file systems
US10089023B2 (en) * 2015-06-23 2018-10-02 Western Digital Technologies, Inc. Data management for object based storage
US20170185454A1 (en) * 2015-12-25 2017-06-29 Le Holdings (Beijing) Co., Ltd. Method and Electronic Device for Determining Resource Consumption of Task
US10635485B2 (en) * 2018-03-23 2020-04-28 Intel Corporation Devices, systems, and methods for lockless distributed object input/output

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109710456A (en) * 2018-12-10 2019-05-03 新华三技术有限公司 A kind of data reconstruction method and device
CN110059068A (en) * 2019-04-11 2019-07-26 厦门网宿有限公司 Data verification method and data verification system in a kind of distributed memory system
CN111367926A (en) * 2020-02-27 2020-07-03 星辰天合(北京)数据科技有限公司 Data processing method and device for distributed system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Scalable, Efficient, and Policy-Aware Deduplication for Primary Distributed Storage Systems;H. Fingler et al.;《2019 31st International Symposium on Computer Architecture and High Performance Computing》;20191205;第180-187页 *
基于Bluestore的分布式存储Ceph集群QoS研究;袁猛;《中国优秀硕士学位论文全文数据库-信息科技辑》;20190215;第2019年卷(第2期);全文 *

Also Published As

Publication number Publication date
CN112000487A (en) 2020-11-27

Similar Documents

Publication Publication Date Title
CN101510167B (en) A kind of method of plug-in component operation, Apparatus and system
US8639991B2 (en) Optimizing performance of an application
CN109213637B (en) Data recovery method, device and medium of distributed file system cluster node
JP2019500680A5 (en)
CN107967205A (en) A kind of memory analysis method, apparatus, system and computing device
CN110046086A (en) Expected data generation method and device and electronic equipment for test
US20070214161A1 (en) System and method for resource lock acquisition and reclamation in a network file system environment
CN112506791A (en) Application program testing method and device, computer equipment and storage medium
CN111694734A (en) Software interface checking method and device and computer equipment
CN112000487B (en) Scrub pressure adjusting method, device and medium
CN114610762A (en) Three-level response method, system, equipment and storage medium for data request
KR20190071571A (en) Apparutus and method for analyzing user process based on standard process
CN111881185A (en) Data monitoring method, device, equipment and storage medium
CN108924013B (en) Network flow accurate acquisition method and device
CN112162827A (en) A cloud platform container downgrade method, device, system and storage medium
CN112789599B (en) Information recommendation method, device, equipment and readable storage medium
CN110413607B (en) Distributed counting method, server and system
CN108279993B (en) Method and device and electronic device for realizing service degradation
CN112905602A (en) Data comparison method, computing device and computer storage medium
CN115964198B (en) Distributed flexible transaction processing method and device based on long transaction
CN111984473A (en) Memory snapshot data acquisition method and related device
CN113821414B (en) Server protection method and device, electronic equipment and storage medium
CN108549691B (en) Database session tracking and analyzing method and device
WO2018173760A1 (en) Monitoring device, monitoring method and program
US8930369B2 (en) Information processing apparatus, message classifying method and non-transitory medium for associating series of transactions

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant