Disclosure of Invention
An object of the embodiments of the present invention is to provide a method and an apparatus for regulating scrub pressure, and a computer-readable storage medium, which can reduce the influence of a scrub task on a front-end service.
To solve the above technical problem, an embodiment of the present invention provides a scrub pressure adjustment method, including:
acquiring the process pressure of a cluster;
triggering a scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task;
according to the process pressure of the cluster, adjusting the number of objects for executing the scanning task in the scrub task;
and executing the scanning task according to the number of the objects, and adjusting the number of the objects executing the scanning task in the scrub task according to the process pressure of the cluster every time the scanning task is completed until all the scanning tasks contained in the scrub task are completed, and ending the operation.
Optionally, the triggering the scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task includes:
when the PG is in a task execution state and the current time meets the execution time of a scrub task, judging whether the process pressure of the cluster belongs to a preset high pressure range;
when the process pressure of the cluster does not belong to a preset high pressure range, triggering a scrub state machine;
and when the process pressure of the cluster belongs to a preset high pressure range, triggering a scrub state machine when the scrub task reaches the cut-off time.
Optionally, after the scanning task is executed according to the number of the objects, the method further includes:
judging whether the process pressure of the cluster belongs to a preset high pressure range or not;
and when the process pressure of the cluster belongs to a preset high pressure range, waiting for a preset time and then executing the step of adjusting the number of objects for executing the scanning task in the scrub task according to the process pressure of the cluster.
Optionally, the adjusting, according to the process pressure of the cluster, the number of objects in the scrub task for executing the scanning task includes:
searching the number of objects matched with the process pressure of the cluster from a pre-established corresponding relation list, and taking the number of the objects as the number of the objects for executing the scanning task in the scrub task; wherein, the corresponding relation list records the corresponding relation between different process pressure ranges and the number of objects.
Optionally, after the scanning task is executed according to the number of the objects, the method further includes:
judging whether the object data acquired by executing the scanning task is consistent with the object data fed back by the standby equipment or not;
and when the object data acquired by executing the scanning task is inconsistent with the object data fed back by the standby equipment, marking the scrub task in an error state.
The embodiment of the invention also provides a scrub pressure adjusting device which comprises an acquisition unit, a trigger unit, an adjusting unit and an execution unit;
the acquiring unit is used for acquiring the process pressure of the cluster;
the triggering unit is used for triggering a scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task;
the adjusting unit is configured to adjust the number of objects in the scrub task, which execute the current scanning task, according to the process pressure of the cluster;
and the execution unit is used for executing the scanning task according to the number of the objects, and adjusting the number of the objects for executing the scanning task in the scrub task according to the process pressure of the cluster every time the scanning task is completed, and ending the operation until all the scanning tasks included in the scrub task are completed.
Optionally, the triggering unit includes a judging subunit, an executing subunit, and a waiting subunit;
the judging subunit is configured to, when the PG is in a task execution state and the current time meets the execution time of the scrub task, judge whether the process pressure of the cluster belongs to a preset high pressure range;
the execution subunit is used for triggering a scrub state machine when the process pressure of the cluster does not belong to a preset high pressure range;
and the waiting subunit is used for triggering the scrub state machine when the process pressure of the cluster belongs to a preset high pressure range and the scrub task is waited to reach the deadline time.
Optionally, the device further comprises a pressure judging unit and a waiting unit;
the pressure judging unit is used for judging whether the process pressure of the cluster belongs to a preset high pressure range or not;
and the waiting unit is used for waiting for a preset time and then executing the step of adjusting the number of objects for executing the scanning task in the scrub task according to the process pressure of the cluster when the process pressure of the cluster belongs to a preset high pressure range.
Optionally, the adjusting unit is specifically configured to search, from a pre-established correspondence list, the number of objects that matches the process pressure of the cluster, and use the number of objects as the number of objects for executing the current scanning task in the scrub task; and the corresponding relation list records the corresponding relation between different process pressure ranges and the number of the objects.
Optionally, the system further comprises a data judging unit and a marking unit;
the data judging unit is used for judging whether the object data acquired by executing the scanning task is consistent with the object data fed back by the standby equipment;
the marking unit is configured to mark the scrub task in an error state when the object data obtained by executing the scanning task at this time is inconsistent with the object data fed back by the standby device.
The embodiment of the present invention further provides a scrub pressure adjusting device, including:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the scrub pressure adjustment method as in any one of the above.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the scrub pressure adjustment method according to any one of the foregoing embodiments are implemented.
According to the technical scheme, the process pressure of the cluster is obtained; and triggering the scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task. And after the scrub state machine is triggered, the system automatically locks the scrub task. In order to reduce the time occupied by the locking operation, the number of objects for executing the scanning task in the scrub task can be adjusted according to the process pressure of the cluster; and executing the scanning task according to the number of the objects. The number of objects to be scanned in a scrub task is often large, and multiple scanning tasks generally need to be executed, so that the number of objects to be executed in the scrub task at this time can be adjusted according to the process pressure of the cluster every time the scanning task is completed, and the operation is ended until all the scanning tasks included in the scrub task are completed. In the technical scheme, the number of objects to be scanned by the current scrub task can be automatically adjusted according to the process pressure of the cluster, and the time that the scrub task occupies a pg lock is effectively reduced under the condition that the scrub task can be smoothly executed, so that the influence of the scrub task on the front-end service is effectively reduced.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without any creative work belong to the protection scope of the present invention.
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
Next, a detailed description is given of a scrub pressure adjustment method according to an embodiment of the present invention. Fig. 1 is a flowchart of a scrub pressure adjustment method according to an embodiment of the present invention, where the method includes:
s101: and acquiring the process pressure of the cluster.
After the Object-based Storage Device (OSD) is started, a scrub task is registered for each Place Group (PG).
The process pressure of the cluster may include CPU utilization, disk utilization, and the like. In an embodiment of the present invention, the OSD may scan the PG periodically to collect the process pressure of the cluster within a period time. The length of the cycle time can be set according to actual requirements, for example, the cycle time can be set to 1 minute.
It should be noted that, in order to ensure that the process pressure can accurately reflect the current pressure condition of the cluster, in the embodiment of the present invention, the process pressures of the clusters all refer to the process pressure of the cluster in the latest cycle time, and taking the cycle time as 1 minute as an example, the process pressure of the cluster in the past 1 minute may be obtained by taking the current time as a reference time.
S102: and triggering the scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task.
When the PG is in an active + clean state, that is, the PG is in a task execution state, and the current time meets the execution time of the scrub task, the OSD may trigger the scrub state machine, so as to start the operation flow of executing the scrub task. And after the scrub state machine is triggered, the system automatically locks the scrub task.
In the embodiment of the present invention, in order to reduce the influence of the scrub task on the front end service, when the PG is in the task execution state and the current time meets the execution time of the scrub task, it may be determined whether the process pressure of the cluster belongs to the preset high pressure range.
The preset high pressure range may be set to have a CPU utilization greater than 50% or a disk utilization greater than 50%.
When the process pressure of the cluster does not belong to the preset high pressure range, the OSD in the cluster can rapidly process the scrub task, and at the moment, the scrub state machine can be directly triggered.
When the process pressure of the cluster belongs to the preset high pressure range, the resource occupancy rate of the cluster is high, if the scrub state machine is triggered, the locking operation of the scrub task takes longer time due to the fact that OSD cannot process the scrub task quickly, and therefore execution of front-end services in the cluster is affected, and therefore when the process pressure of the cluster belongs to the preset high pressure range, the scrub state machine can be triggered after the scrub task reaches the cut-off time.
Each scrub task carries an expiration time at registration as long as the OSD executes the scrub task within the expiration time.
S103: and adjusting the number of objects for executing the scanning task in the scrub task according to the process pressure of the cluster.
The process pressure of the cluster directly affects the time it takes for the OSD to perform the scrub task, during which the OSD may add the pg lock. The longer the OSD takes to execute the scrub task, the longer the time it takes to occupy the pg lock, and correspondingly, the greater the impact on the front-end service in the cluster. Therefore, in the embodiment of the present invention, the number of objects in the scrub task for executing the current scanning task may be adjusted according to the process pressure of the cluster.
When the process pressure of the cluster is higher, the number of the objects of the scanning task can be properly reduced; when the process pressure of the cluster is small, the number of the objects of the scanning task can be properly increased.
In a specific implementation, the corresponding relationship between different process pressure ranges and the number of objects may be recorded in a corresponding relationship list, and after the scrub state machine is triggered, the number of objects matching the process pressure of the cluster may be searched from the pre-established corresponding relationship list, and the number of objects is used as the number of objects for executing the scanning task in the scrub task.
For example, the process pressure of the cluster may be divided into three levels, i.e., a high pressure level, a medium pressure level and a low pressure level, for example, when the CPU utilization rate or the disk utilization rate is greater than 50%, it is indicated that the process pressure of the cluster belongs to the high pressure level; when the utilization rate of the CPU or the utilization rate of the disk is less than 10%, indicating that the process pressure of the cluster belongs to low pressure; when the utilization rate of the CPU or the utilization rate of the disk is between 10% and 50%, the process pressure of the cluster belongs to the medium pressure. The number of the objects of the scanning task under high pressure can be set to be 2; the number of the objects of the scanning task at the time under the medium pressure can be set to be 5; the number of objects of the current scanning task under low pressure can be set to 10.
S104: and executing the scanning task according to the number of the objects, and adjusting the number of the objects executing the scanning task in the scrub task according to the process pressure of the cluster every time the scanning task is completed, until all the scanning tasks contained in the scrub task are completed, ending the operation.
The number of objects to be scanned in a scrub task is often large, and multiple scanning tasks generally need to be executed, so that each time a scanning task is completed, the operation can be returned to S103, the number of objects to be executed in the scrub task until the scrub task is completed is adjusted according to the cluster process pressure, and the operation is ended.
It should be noted that, in the embodiment of the present invention, the process pressure of the cluster according to which the adjustment of the number of objects is performed each time the process returns to S103 is the process pressure of the cluster in the latest cycle time.
According to the technical scheme, the process pressure of the cluster is obtained; and triggering a scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task. And after the scrub state machine is triggered, the system automatically locks the scrub task. In order to reduce the time occupied by the locking operation, the number of objects for executing the scanning task in the scrub task can be adjusted according to the process pressure of the cluster; and executing the scanning task according to the number of the objects. The number of objects to be scanned in a scrub task is often large, and multiple scanning tasks generally need to be executed, so that the number of objects to be executed in the scrub task at this time can be adjusted according to the process pressure of the cluster every time the scanning task is completed, and the operation is ended until all the scanning tasks included in the scrub task are completed. In the technical scheme, the number of objects to be scanned by the current scrub task can be automatically adjusted according to the process pressure of the cluster, and the time that the scrub task occupies a pg lock is effectively reduced under the condition that the scrub task can be smoothly executed, so that the influence of the scrub task on the front-end service is effectively reduced.
In the embodiment of the present invention, after the current scanning task is executed according to the number of the objects, the process may return to S103 to readjust the number of the objects corresponding to the next scanning task. Therefore, in the embodiment of the present invention, in order to further reduce the influence of the scrub task on the front end service, after the current scanning task is completed according to the number of objects, before the number of objects corresponding to the next scanning task is adjusted, it may be determined whether the process pressure of the cluster belongs to the preset high pressure range.
And when the process pressure of the cluster does not belong to the preset high pressure range, adjusting the number of the objects executing the scanning task in the scrub task directly according to the process pressure of the cluster.
When the process pressure of the cluster belongs to the preset high pressure range, the step of adjusting the number of the objects for executing the scanning task in the scrub task according to the process pressure of the cluster can be executed after waiting for the preset time.
The value of the preset time can be set according to actual requirements, for example, the preset time can be set to 3 seconds.
The process pressure of the cluster is evaluated every time the scanning task is executed, the next scanning task can be executed after the preset time when the process pressure of the cluster is high, the process pressure of the cluster can be effectively relieved, and therefore the influence of the execution of the scrub task on the front-end service is further reduced.
In an embodiment of the present invention, the OSDs in the cluster may include a main OSD and a standby OSD. After the main OSD executes the scanning task, the corresponding scanning task is sent to the standby OSD, and the standby OSD feeds back object data to the main OSD after the scanning task is finished. In order to ensure the consistency between the main OSD and the standby OSD data, after receiving the object data fed back by the standby OSD, the main OSD may determine whether the object data obtained by executing the current scanning task is consistent with the object data fed back by the standby device.
When the object data acquired by executing the scanning task is inconsistent with the object data fed back by the standby device, the scrub task may be marked as an error state.
By comparing the object data of the main OSD and the standby OSD, the problem of data inconsistency can be found in time. The scrub tasks are marked to be in an error state, so that a manager can conveniently and visually know which scrub task has data inconsistency, and the problem of data inconsistency is specifically solved.
Fig. 2 is a schematic structural diagram of a scrub pressure adjustment device according to an embodiment of the present invention, including an obtaining unit 21, a triggering unit 22, an adjusting unit 23, and an executing unit 24;
an obtaining unit 21, configured to obtain a process pressure of the cluster;
the triggering unit 22 is configured to trigger a scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task;
the adjusting unit 23 is configured to adjust the number of objects in the scrub task for executing the current scanning task according to the process pressure of the cluster;
and the execution unit 24 is configured to execute the scanning task according to the number of the objects, and adjust the number of the objects executing the scanning task in the scrub task according to the process pressure of the cluster every time the scanning task is completed, until all the scanning tasks included in the scrub task are completed, the operation is ended.
Optionally, the triggering unit includes a judging subunit, an executing subunit, and a waiting subunit;
the judging subunit is used for judging whether the process pressure of the cluster belongs to a preset high pressure range or not when the PG is in a task execution state and the current time meets the execution time of the scrub task;
the execution subunit is used for triggering the scrub state machine when the process pressure of the cluster does not belong to a preset high pressure range;
and the waiting subunit is used for triggering the scrub state machine when the process pressure of the cluster belongs to a preset high pressure range and the scrub task reaches the deadline time.
Optionally, the device further comprises a pressure judging unit and a waiting unit;
the pressure judging unit is used for judging whether the process pressure of the cluster belongs to a preset high pressure range or not;
and the waiting unit is used for waiting for a preset time and then executing the step of adjusting the number of the objects for executing the scanning task in the scrub task according to the process pressure of the cluster when the process pressure of the cluster belongs to the preset high pressure range.
Optionally, the adjusting unit is specifically configured to search, from a pre-established correspondence list, a number of objects that matches the process pressure of the cluster, and use the number of objects as the number of objects for executing the current scanning task in the scrub task; the corresponding relation list records the corresponding relation between different process pressure ranges and the number of the objects.
Optionally, the system further comprises a data judging unit and a marking unit;
the data judging unit is used for judging whether the object data acquired by executing the scanning task is consistent with the object data fed back by the standby equipment;
and the marking unit is used for marking the scrub task as an error state when the object data acquired by executing the scanning task is inconsistent with the object data fed back by the standby equipment.
The description of the features in the embodiment corresponding to fig. 2 may refer to the related description of the embodiment corresponding to fig. 1, and is not repeated here.
According to the technical scheme, the process pressure of the cluster is obtained; and triggering the scrub state machine according to the process pressure of the cluster and the execution condition of the scrub task. And after the scrub state machine is triggered, the system automatically locks the scrub task. In order to reduce the time occupied by the locking operation, the number of objects for executing the scanning task in the scrub task can be adjusted according to the process pressure of the cluster; and executing the scanning task according to the number of the objects. The number of objects to be scanned in a scrub task is often large, and multiple scanning tasks generally need to be executed, so that the number of objects to be executed in the scrub task at this time can be adjusted according to the process pressure of the cluster every time the scanning task is completed, and the operation is ended until all the scanning tasks included in the scrub task are completed. In the technical scheme, the number of objects to be scanned by the current scrub task can be automatically adjusted according to the process pressure of the cluster, and the time that the scrub task occupies a pg lock is effectively reduced under the condition that the scrub task can be smoothly executed, so that the influence of the scrub task on the front-end service is effectively reduced.
Fig. 3 is a schematic hardware structure diagram of a scrub pressure adjustment device 30 according to an embodiment of the present invention, including:
a memory 31 for storing a computer program;
a processor 32 for executing a computer program to implement the steps of the scrub pressure adjustment method as described in any of the embodiments above.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the scrub pressure adjustment method according to any of the above embodiments are implemented.
The foregoing describes a scrub pressure adjustment method, apparatus and computer readable storage medium provided by the embodiments of the present invention in detail. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, without departing from the principle of the present invention, it is possible to make various improvements and modifications to the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.