CN113836225B - Object storage multi-site synchronous alarm method, system, terminal and storage medium - Google Patents
Object storage multi-site synchronous alarm method, system, terminal and storage medium Download PDFInfo
- Publication number
- CN113836225B CN113836225B CN202111082389.5A CN202111082389A CN113836225B CN 113836225 B CN113836225 B CN 113836225B CN 202111082389 A CN202111082389 A CN 202111082389A CN 113836225 B CN113836225 B CN 113836225B
- Authority
- CN
- China
- Prior art keywords
- marks
- barrel
- fragmentation
- user storage
- bucket
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明提供一种对象存储多站点同步告警方法、系统、终端及存储介质，包括：获取目标站点的用户存储桶信息，并调取用户存储桶对应的源桶信息；分别采集用户存储桶的所有分片标记和对应源桶的所有分片标记，并对两者进行一致性比对；采集数据同步进程的当前状态，在数据同步进程的已完成状态下，如果一致性比对结果为两者不同，生成比对不一致的用户存储桶和对应源桶的告警提示；在数据同步进程的执行中状态下，则多次采集用户存储桶的所有分片标记和对应源桶的所有分片标记，根据两种标记的更新情况生成告警提示。本发明可以监控多站点之间各个桶的同步状态，使多站点功能更加完善商品化，提高了公司分布式对象存储产品的市场竞争力。
The present invention provides an object storage multi-site synchronization alarm method, system, terminal and storage medium, including: obtaining the user storage bucket information of the target site, and retrieving the source bucket information corresponding to the user storage bucket; respectively collecting all the fragmentation marks of the user storage bucket and all the fragmentation marks of the corresponding source bucket, and performing consistency comparison between the two; collecting the current state of the data synchronization process, in the completed state of the data synchronization process, if the consistency comparison result is that the two are different, generating an alarm prompt for the inconsistent user storage bucket and the corresponding source bucket; in the execution state of the data synchronization process, multiple collections of all the fragmentation marks of the user storage bucket and all the fragmentation marks of the corresponding source bucket, and generating an alarm prompt according to the update of the two marks. The present invention can monitor the synchronization status of each bucket between multiple sites, make the multi-site function more perfect and commercialized, and improve the market competitiveness of the company's distributed object storage products.
Description
Technical Field
The invention relates to the technical field of object storage, in particular to an object storage multi-site synchronous alarm method, an object storage multi-site synchronous alarm system, a terminal and a storage medium.
Background
Object store is a generic term used to describe a method of resolving and processing discrete units. Objects are no longer hierarchically structured in a hierarchy, featuring extended metadata. A Bucket (Bucket) is a carrier of objects in object storage, and can be understood as a "container" that holds objects, and there is no upper capacity limit for the "container". Objects are stored in buckets in a flattened structure, without the concept of folders and directories, and a user may choose to store objects in a single or multiple buckets. The buckets used by the user mostly correspond to one or more source buckets, when the user performs data operation on the used storage buckets, the object storage system needs to perform data synchronization, and update data of the user storage buckets are synchronously updated in the source buckets. Data synchronization is a key link for guaranteeing the security, consistency and stability of data stored by an object.
In the object storage, there is a command for checking the synchronization status of multiple sites, which is used to check whether all the cluster objects are synchronized, the command can only check whether the data of the sites are synchronized up to now, and because the statistical granularity is too large, the command always displays a status in synchronization as long as the data of the cluster is written, and the command cannot display the barrel which is abnormal and leaks synchronization and is not synchronized for a long time.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a method, a system, a terminal and a storage medium for synchronously alarming object storage and multiple sites, which are used for solving the technical problems.
In a first aspect, the present invention provides a method for synchronously alarming multiple sites in object storage, including:
acquiring user storage bucket information of a target site, and calling source bucket information corresponding to the user storage bucket;
collecting all the fragmentation marks of a user storage barrel and all the fragmentation marks of a corresponding source barrel respectively, and comparing the fragmentation marks with the source barrel according to consistency;
Collecting the current state of a data synchronization process, and generating an alarm prompt for comparing inconsistent user storage barrels and corresponding source barrels if consistency comparison results are different in the completed state of the data synchronization process;
and in the executing state of the data synchronization process, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel for multiple times, and generating an alarm prompt according to the updating conditions of the two marks.
Further, obtaining user bucket information of the target site, and calling source bucket information corresponding to the user bucket, including:
traversing a site in the object storage system as a target site;
collecting bucket information of all users on a target site and summarizing the bucket information into user bucket information;
and calling the source barrel information corresponding to each user storage barrel.
Further, all the fragment marks of the user storage barrel and all the fragment marks of the corresponding source barrel are respectively collected, and consistency comparison is carried out on the fragment marks and the fragment marks, including:
storing all the fragment marks of the user storage barrel to a first list;
storing all the fragment marks corresponding to the source barrel to a second list;
And comparing the consistency of the fragment marks in the first list and the second list according to the mapping relation between the user storage barrel and the corresponding source barrel.
Further, in the executing state of the data synchronization process, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel for multiple times, and generating an alarm prompt according to the update conditions of the two marks, including:
in the executing state of the data synchronization process, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel twice successively;
taking all the fragmentation marks of the user storage barrel collected in the previous time and all the fragmentation marks of the corresponding source barrel as first comparison data;
Taking all the fragmentation marks of the user storage barrel acquired at the next time and all the fragmentation marks of the corresponding source barrel as second comparison data;
and comparing the consistency of the second comparison data with the first comparison data, judging whether the second comparison data has an increment mark relative to the first comparison data, and if the increment mark does not exist, generating an alarm prompt of the user storage barrel and the corresponding source barrel.
Further, before comparing the second comparison data for consistency with the first comparison data, the method further comprises:
Judging whether source barrel information corresponding to a user storage barrel acquired at the next time is consistent with source barrel information acquired at the previous time or not:
if yes, consistency comparison of the second comparison data and the first comparison data is executed;
If not, after waiting for the appointed time, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel again, and acquiring the update conditions of the two marks based on the collected mark data.
In a second aspect, the present invention provides an object storage multi-site synchronous alarm system, comprising:
The information acquisition unit is used for acquiring user storage bucket information of the target site and calling source bucket information corresponding to the user storage bucket;
the mark comparison unit is used for respectively collecting all the fragment marks of the user storage barrel and all the fragment marks of the corresponding source barrel and comparing the fragment marks with the fragment marks of the corresponding source barrel;
The first processing unit is used for collecting the current state of the data synchronization process, and generating an alarm prompt for comparing inconsistent user storage barrels and corresponding source barrels if the consistency comparison results are different in the completed state of the data synchronization process;
And the second processing unit is used for acquiring all the fragmentation marks of the user storage barrel and all the fragmentation marks corresponding to the source barrel for multiple times in the execution state of the data synchronization process, and generating an alarm prompt according to the update condition of the two marks.
Further, the information acquisition unit is configured to:
traversing a site in the object storage system as a target site;
collecting bucket information of all users on a target site and summarizing the bucket information into user bucket information;
and calling the source barrel information corresponding to each user storage barrel.
Further, the tag alignment unit is configured to:
storing all the fragment marks of the user storage barrel to a first list;
storing all the fragment marks corresponding to the source barrel to a second list;
And comparing the consistency of the fragment marks in the first list and the second list according to the mapping relation between the user storage barrel and the corresponding source barrel.
Further, the second processing unit is configured to:
in the executing state of the data synchronization process, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel twice successively;
taking all the fragmentation marks of the user storage barrel collected in the previous time and all the fragmentation marks of the corresponding source barrel as first comparison data;
Taking all the fragmentation marks of the user storage barrel acquired at the next time and all the fragmentation marks of the corresponding source barrel as second comparison data;
and comparing the consistency of the second comparison data with the first comparison data, judging whether the second comparison data has an increment mark relative to the first comparison data, and if the increment mark does not exist, generating an alarm prompt of the user storage barrel and the corresponding source barrel.
Further, before comparing the consistency of the second comparison data with the first comparison data, the second processing unit is further configured to:
Judging whether source barrel information corresponding to a user storage barrel acquired at the next time is consistent with source barrel information acquired at the previous time or not:
if yes, consistency comparison of the second comparison data and the first comparison data is executed;
If not, after waiting for the appointed time, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel again, and acquiring the update conditions of the two marks based on the collected mark data.
In a third aspect, a terminal is provided, including:
A processor, a memory, wherein,
The memory is used for storing a computer program,
The processor is configured to call and run the computer program from the memory, so that the terminal performs the method of the terminal as described above.
In a fourth aspect, there is provided a computer storage medium having instructions stored therein which, when run on a computer, cause the computer to perform the method of the above aspects.
The object storage multi-site synchronous alarm method, the system, the terminal and the storage medium have the advantages that user storage bucket information of the sites stored by the object is collected, source bucket information corresponding to the user storage bucket is further called, and then according to the state of a data synchronization process in the object storage system, the fragmentation marks of the user storage bucket and the fragmentation marks of the corresponding source bucket are combined, so that the user storage bucket and the corresponding source bucket which do not realize data synchronization are detected. The invention can monitor the synchronous state of each barrel among multiple stations, and provides an alarm function for the problems of long-time asynchronism, slow synchronization and the like, so that the multiple station function is more perfect and commercialized, and the market competitiveness of the company distributed object storage product is improved.
In addition, the invention has reliable design principle, simple structure and very wide application prospect.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a schematic flow chart of a method of one embodiment of the invention.
FIG. 2 is a schematic block diagram of a system of one embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a terminal according to an embodiment of the present invention.
Detailed Description
In order to make the technical solution of the present invention better understood by those skilled in the art, the technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
The following explains key terms appearing in the present invention.
Object storage, also known as object-based storage, is a generic term used to describe a method of resolving and processing discrete units, which are referred to as objects. Just like a file, an object contains data, but unlike a file, an object no longer has a hierarchical structure in one layer structure. Each object is in the same level of a flat address space called a memory pool, and one object does not belong to the next level of another object. Both files and objects have metadata associated with the data they contain, but objects are characterized by extended metadata. Each object is assigned a unique identifier that allows a server or end user to retrieve the object without having to know the physical address of the data. This approach helps to automate and simplify data storage in a cloud computing environment. Object storage is often compared to parking in a premium restaurant. When a customer needs to stop for a substitute, he gives the key to others and changes a receipt. The customer does not know where his car is stopped, nor how many times the attendant will move his car while he is dining. In this metaphor, the unique identifier of a stored object represents the customer's receipt. This becomes a problem for the final consistency of the object store, since the object store will be spread over many nodes and the latest data is not always available. All data storage involves three elements, consistency, availability and partitioning, abbreviated as CAP. If only a write operation is performed, thereby changing an object, but someone is accessing the object from another node. Nodes may be in different physical locations because object storage supports a large geographic expansion. The new user may be reading the object but that is the old version. This is the final consistency problem of object storage, where there is no timely synchronization. This can be a problem, especially when collaboration is made with object stores, where vendors work well to ensure consistency of object stores, such as Joyent and his Manta store services, and once an object changes will not support reading old content. You have to wait, but what you read would be consistent.
The Ceph object gateway stores storage index data in the pool of index_pool, default to default.rgw.bucket.index, and sometimes users like to put multiple objects (hundreds of thousands to millions of objects) in one bucket, if the gateway management interface is not used to set a quota for the largest object in each bucket, the bucket index may suffer serious performance degradation when users put a large number of objects in the bucket. In Ceph 0.94, the storage index can be fragmented to prevent performance bottlenecks when a large number of objects are allowed in each bucket. rgw _override_bucket_index_max_ shards settings allow you to set the maximum number of slices per storage area, default to 0, which means that by default the bucket index slices are in the closed state.
FIG. 1 is a schematic flow chart of a method of one embodiment of the invention. The execution body of fig. 1 may store a multi-site synchronous alarm system for an object.
As shown in fig. 1, the method includes:
Step 110, obtaining user storage bucket information of a target site, and calling source bucket information corresponding to the user storage bucket;
Step 120, collecting all the fragment marks of the user storage barrel and all the fragment marks of the corresponding source barrel respectively, and comparing the fragment marks with the fragment marks of the corresponding source barrel;
step 130, collecting the current state of the data synchronization process, and generating an alarm prompt for comparing inconsistent user storage barrels and corresponding source barrels if the consistency comparison results are different in the completed state of the data synchronization process;
and 140, in the execution state of the data synchronization process, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks corresponding to the source barrel for multiple times, and generating an alarm prompt according to the update condition of the two marks.
In order to facilitate understanding of the present invention, the principle of the object storage multi-site synchronization alarm method according to the present invention is used to further describe the object storage multi-site synchronization alarm method according to the present invention in combination with the process of alarming object storage multi-site synchronization in the embodiment.
The embodiment obtains the source barrel of the current barrel on the current site, compares the last mark (marker) object storage system corresponding to the barrel fragments, and starts a special thread to perform the following alarm processing in the starting process. Specifically, the object storage multi-site synchronous alarm method comprises the following steps:
S1, acquiring user storage bucket information of a target site, and calling source bucket information corresponding to the user storage bucket.
Traversing a site in an object storage system as a target site, calling bucket information under all users on the target site to be summarized as user bucket information, and calling source bucket information corresponding to each user bucket.
Specifically, all barrels under all users on the current site are acquired, all source barrels of each user storage barrel are acquired, more than one source barrel of the user storage barrel is acquired, and a mapping relation of information of the user storage barrel and the corresponding source barrel is constructed.
S2, respectively collecting all the fragment marks of the user storage barrel and all the fragment marks of the corresponding source barrel, and comparing the fragment marks with the fragment marks of the corresponding source barrel.
Storing all the fragment marks of the user storage barrel into a first list, storing all the fragment marks of the corresponding source barrel into a second list, and comparing the fragment marks in the first list with the fragment marks in the second list according to the mapping relation between the user storage barrel and the corresponding source barrel.
Based on the user storage bucket information and the corresponding source bucket information acquired in the step S1, all the fragmentation marks of the user storage bucket and all the fragmentation marks of the corresponding source bucket are acquired respectively, wherein the fragmentation mark markers are mark bits of the synchronized object.
For comparison, the shard marks of all user storage barrels are saved to a first list, the shard marks of all source barrels are saved to a second list, and consistency comparison is carried out on the shard marks of the first list and the second list according to the mapping relation between the user storage barrels and the source barrels.
S3, collecting the current state of the data synchronization process, and generating an alarm prompt for comparing inconsistent user storage barrels and corresponding source barrels if consistency comparison results are different in the completed state of the data synchronization process.
Since data is continuously updated during the data synchronization process, it is necessary to determine whether data synchronization is currently being performed.
If the data synchronization process is not executed currently, if the fragmentation marks of the user storage bucket and the corresponding source bucket are consistent, the user storage bucket and the corresponding source bucket are indicated to complete data synchronization, if the fragmentation marks of the user storage bucket and the corresponding source bucket are inconsistent, the user storage bucket and the corresponding source bucket are indicated to not complete data synchronization, and a synchronization alarm prompt of the user storage bucket and the corresponding source bucket needs to be generated.
S4, in the executing state of the data synchronization process, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks corresponding to the source barrel for multiple times, and generating an alarm prompt according to the updating condition of the two marks.
And in the execution state of the data synchronization process, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel twice successively.
And judging whether the source barrel information corresponding to the user storage barrel acquired at the next time is consistent with the source barrel information acquired at the previous time, if so, executing consistency comparison of the second comparison data and the first comparison data, otherwise, waiting for a specified time period, acquiring all the fragmentation marks of the user storage barrel and all the fragmentation marks corresponding to the source barrel again, and acquiring updating conditions of the two marks based on the acquired mark data.
The method comprises the steps of collecting all the fragmentation marks of a user storage barrel and all the fragmentation marks of a corresponding source barrel in the previous time as first comparison data, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel in the next time as second comparison data, comparing the consistency of the second comparison data and the first comparison data, judging whether increment marks exist in the second comparison data relative to the first comparison data, and generating an alarm prompt of the user storage barrel and the corresponding source barrel if the increment marks do not exist.
If the data synchronization process in the object storage system is not completed and is in an executing state, the data is continuously updated, so the following method needs to be executed:
And acquiring all source barrels of the user storage barrel again on the basis that the user storage barrel information and the corresponding source barrel information are acquired once and all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel are acquired, if the corresponding source barrel of a certain user storage barrel is found to be inconsistent with the source barrel acquired last time, not comparing the current time, updating the source barrel information of the user storage barrel, updating the marker records of the user storage barrel and the source barrel, and waiting for the next comparison.
If the corresponding source barrel of the user storage barrel is consistent with the source barrel acquired last time, the acquired marker pair is compared with the acquired marker pair last time (mainly comparing whether the increment markers of the local records acquired twice are changed or not, because the markers of the source barrel fragments possibly change along with new writing), and if the markers of the two times are inconsistent, the barrel fragments are synchronous. If the two markers are consistent, the alarm is directly given.
And (3) directly landing the alarm information generated in the steps S3 and S4, and waiting for the inquiry of upper software.
The invention can monitor the synchronous state of each barrel among multiple stations, and provides an alarm function for the problems of long-time asynchronism, slow synchronization and the like, so that the multiple station function is more perfect and commercialized, and the market competitiveness of the company distributed object storage product is improved.
The specific execution flow of this embodiment is as follows:
1. all buckets under all users on the current site are acquired.
2. All source barrels of each barrel are acquired, and more than one source barrel of each barrel is acquired.
3. Comparing the markers (marks) of each barrel fragment of the current barrel and the source barrel, and if the markers are the same, synchronizing normally;
4. if the markers are different, judging whether synchronization is in progress, if the synchronization is finished, directly alarming, if the synchronization is still in progress, acquiring all source barrels of the current barrel again, if the source barrels are not consistent with the source barrel acquired last time, not comparing the current barrel, updating the source barrel information of the current barrel, updating the marker record of the current barrel and the source barrel, and waiting for the next comparison.
If the acquired marker pair is consistent with the last acquired source barrel, comparing the acquired marker pair with the last acquired marker pair (mainly comparing whether the increment markers of the local records acquired twice are changed or not, because the markers of the source barrel fragments possibly change along with new writing), and if the markers of the two times are inconsistent, indicating that the barrel fragments are synchronizing. If the two markers are consistent, the alarm is directly given.
5. The alarm information is directly dropped, and the management software is waited for inquiring and displaying.
As shown in fig. 2, the system 200 includes:
an information obtaining unit 210, configured to obtain user bucket information of a target site, and call source bucket information corresponding to a user bucket;
the tag comparison unit 220 is configured to collect all the fragment tags of the user storage bucket and all the fragment tags of the corresponding source bucket respectively, and compare the fragment tags with the fragment tags of the corresponding source bucket;
the first processing unit 230 is configured to collect a current state of the data synchronization process, and generate an alarm prompt for comparing the inconsistent user storage bucket and the corresponding source bucket if the consistency comparison result is different between the two in a completed state of the data synchronization process;
The second processing unit 240 is configured to collect all the fragmentation marks of the user storage bucket and all the fragmentation marks of the corresponding source bucket for multiple times in an executing state of the data synchronization process, and generate an alarm prompt according to update conditions of the two marks.
Alternatively, as an embodiment of the present invention, the information acquisition unit is configured to:
traversing a site in the object storage system as a target site;
collecting bucket information of all users on a target site and summarizing the bucket information into user bucket information;
and calling the source barrel information corresponding to each user storage barrel.
Alternatively, as an embodiment of the present invention, the marker alignment unit is configured to:
storing all the fragment marks of the user storage barrel to a first list;
storing all the fragment marks corresponding to the source barrel to a second list;
And comparing the consistency of the fragment marks in the first list and the second list according to the mapping relation between the user storage barrel and the corresponding source barrel.
Optionally, as an embodiment of the present invention, the second processing unit is configured to:
in the executing state of the data synchronization process, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel twice successively;
taking all the fragmentation marks of the user storage barrel collected in the previous time and all the fragmentation marks of the corresponding source barrel as first comparison data;
Taking all the fragmentation marks of the user storage barrel acquired at the next time and all the fragmentation marks of the corresponding source barrel as second comparison data;
and comparing the consistency of the second comparison data with the first comparison data, judging whether the second comparison data has an increment mark relative to the first comparison data, and if the increment mark does not exist, generating an alarm prompt of the user storage barrel and the corresponding source barrel.
Fig. 3 is a schematic structural diagram of a terminal 300 according to an embodiment of the present invention, where the terminal 300 may be used to execute the object storage multi-site synchronization alarm method according to the embodiment of the present invention.
The terminal 300 may include a processor 310, a memory 320, and a communication unit 330. The components may communicate via one or more buses, and it will be appreciated by those skilled in the art that the configuration of the server as shown in the drawings is not limiting of the invention, as it may be a bus-like structure, a star-like structure, or include more or fewer components than shown, or may be a combination of certain components or a different arrangement of components.
The memory 320 may be used to store instructions for execution by the processor 310, and the memory 320 may be implemented by any type of volatile or non-volatile memory terminal or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk. The execution of the instructions in memory 320, when executed by processor 310, enables terminal 300 to perform some or all of the steps in the method embodiments described below.
The processor 310 is a control center of the storage terminal, connects various parts of the entire electronic terminal using various interfaces and lines, and performs various functions of the electronic terminal and/or processes data by running or executing software programs and/or modules stored in the memory 320, and invoking data stored in the memory. The processor may be comprised of an integrated circuit (INTEGRATED CIRCUIT, simply referred to as an IC), for example, a single packaged IC, or may be comprised of multiple packaged ICs connected to one another for the same function or for different functions. For example, the processor 310 may include only a central processing unit (Central Processing Unit, CPU for short). In the embodiment of the invention, the CPU can be a single operation core or can comprise multiple operation cores.
And a communication unit 330 for establishing a communication channel so that the storage terminal can communicate with other terminals. Receiving user data sent by other terminals or sending the user data to other terminals.
The present invention also provides a computer storage medium in which a program may be stored, which program may include some or all of the steps in the embodiments provided by the present invention when executed. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random-access memory (random access memory RAM), or the like.
Therefore, the invention further invokes the source bucket information corresponding to the user bucket by collecting the user bucket information of the site stored by the object, and then combines the fragmentation mark of the user bucket and the fragmentation mark of the corresponding source bucket according to the state of the data synchronization process in the object storage system, thereby detecting the user bucket and the corresponding source bucket which do not realize the data synchronization. The invention can monitor the synchronous state of each barrel among the multiple stations, and provides an alarm function for the problems of long-time asynchronization, slow synchronization and the like, so that the multiple station function is more perfect and commercialized, the market competitiveness of the company distributed object storage product is improved, and the technical effects achieved by the embodiment can be seen from the description above and are not repeated here.
It will be apparent to those skilled in the art that the techniques of embodiments of the present invention may be implemented in software plus a necessary general purpose hardware platform. Based on such understanding, the technical solution in the embodiments of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium such as a U-disc, a mobile hard disc, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk or an optical disk, etc. various media capable of storing program codes, including several instructions for causing a computer terminal (which may be a personal computer, a server, or a second terminal, a network terminal, etc.) to execute all or part of the steps of the method described in the embodiments of the present invention.
The same or similar parts between the various embodiments in this specification are referred to each other. In particular, for the terminal embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and reference should be made to the description in the method embodiment for relevant points.
In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods may be implemented in other ways. For example, the system embodiments described above are merely illustrative, e.g., the division of the elements is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interface, system or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
Although the present invention has been described in detail by way of preferred embodiments with reference to the accompanying drawings, the present invention is not limited thereto. Various equivalent modifications and substitutions may be made in the embodiments of the present invention by those skilled in the art without departing from the spirit and scope of the present invention, and it is intended that all such modifications and substitutions be within the scope of the present invention/be within the scope of the present invention as defined by the appended claims. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (9)
1. An object storage multi-site synchronous alarm method, comprising the following steps:
acquiring user storage bucket information of a target site, and calling source bucket information corresponding to the user storage bucket;
collecting all the fragmentation marks of a user storage barrel and all the fragmentation marks of a corresponding source barrel respectively, and comparing the fragmentation marks with the source barrel according to consistency;
Collecting the current state of a data synchronization process, and generating an alarm prompt for comparing inconsistent user storage barrels and corresponding source barrels if consistency comparison results are different in the completed state of the data synchronization process;
in the executing state of the data synchronization process, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel for multiple times, and generating an alarm prompt according to the updating conditions of the two marks;
In the executing state of the data synchronization process, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel for multiple times, and generating an alarm prompt according to the updating conditions of the two marks, wherein the alarm prompt comprises the following steps:
in the executing state of the data synchronization process, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel twice successively;
taking all the fragmentation marks of the user storage barrel collected in the previous time and all the fragmentation marks of the corresponding source barrel as first comparison data;
Taking all the fragmentation marks of the user storage barrel acquired at the next time and all the fragmentation marks of the corresponding source barrel as second comparison data;
and comparing the consistency of the second comparison data with the first comparison data, judging whether the second comparison data has an increment mark relative to the first comparison data, and if the increment mark does not exist, generating an alarm prompt of the user storage barrel and the corresponding source barrel.
2. The method of claim 1, wherein obtaining user bucket information for the destination site and retrieving source bucket information corresponding to the user bucket comprises:
traversing a site in the object storage system as a target site;
collecting bucket information of all users on a target site and summarizing the bucket information into user bucket information;
and calling the source barrel information corresponding to each user storage barrel.
3. The method of claim 1, wherein collecting and comparing each of all the shard labels of the user bucket and the corresponding shard labels of the source bucket, respectively, comprises:
storing all the fragment marks of the user storage barrel to a first list;
storing all the fragment marks corresponding to the source barrel to a second list;
And comparing the consistency of the fragment marks in the first list and the second list according to the mapping relation between the user storage barrel and the corresponding source barrel.
4. The method of claim 1, wherein prior to comparing the second comparison data for consistency with the first comparison data, the method further comprises:
Judging whether source barrel information corresponding to a user storage barrel acquired at the next time is consistent with source barrel information acquired at the previous time or not:
if yes, consistency comparison of the second comparison data and the first comparison data is executed;
If not, after waiting for the appointed time, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel again, and acquiring the update conditions of the two marks based on the collected mark data.
5. An object storage multi-site synchronous alarm system, comprising:
The information acquisition unit is used for acquiring user storage bucket information of the target site and calling source bucket information corresponding to the user storage bucket;
the mark comparison unit is used for respectively collecting all the fragment marks of the user storage barrel and all the fragment marks of the corresponding source barrel and comparing the fragment marks with the fragment marks of the corresponding source barrel;
The first processing unit is used for collecting the current state of the data synchronization process, and generating an alarm prompt for comparing inconsistent user storage barrels and corresponding source barrels if the consistency comparison results are different in the completed state of the data synchronization process;
The second processing unit is used for collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks corresponding to the source barrel for a plurality of times in the execution state of the data synchronization process, and generating an alarm prompt according to the update condition of the two marks;
In the executing state of the data synchronization process, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel for multiple times, and generating an alarm prompt according to the updating conditions of the two marks, wherein the alarm prompt comprises the following steps:
in the executing state of the data synchronization process, collecting all the fragmentation marks of the user storage barrel and all the fragmentation marks of the corresponding source barrel twice successively;
taking all the fragmentation marks of the user storage barrel collected in the previous time and all the fragmentation marks of the corresponding source barrel as first comparison data;
Taking all the fragmentation marks of the user storage barrel acquired at the next time and all the fragmentation marks of the corresponding source barrel as second comparison data;
and comparing the consistency of the second comparison data with the first comparison data, judging whether the second comparison data has an increment mark relative to the first comparison data, and if the increment mark does not exist, generating an alarm prompt of the user storage barrel and the corresponding source barrel.
6. The system according to claim 5, wherein the information acquisition unit is configured to:
traversing a site in the object storage system as a target site;
collecting bucket information of all users on a target site and summarizing the bucket information into user bucket information;
and calling the source barrel information corresponding to each user storage barrel.
7. The system of claim 5, wherein the tag alignment unit is configured to:
storing all the fragment marks of the user storage barrel to a first list;
storing all the fragment marks corresponding to the source barrel to a second list;
And comparing the consistency of the fragment marks in the first list and the second list according to the mapping relation between the user storage barrel and the corresponding source barrel.
8. A terminal, comprising:
A processor;
a memory for storing execution instructions of the processor;
wherein the processor is configured to perform the method of any of claims 1-4.
9. A computer readable storage medium storing a computer program, which when executed by a processor implements the method of any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111082389.5A CN113836225B (en) | 2021-09-15 | 2021-09-15 | Object storage multi-site synchronous alarm method, system, terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111082389.5A CN113836225B (en) | 2021-09-15 | 2021-09-15 | Object storage multi-site synchronous alarm method, system, terminal and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113836225A CN113836225A (en) | 2021-12-24 |
| CN113836225B true CN113836225B (en) | 2025-05-16 |
Family
ID=78959628
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111082389.5A Active CN113836225B (en) | 2021-09-15 | 2021-09-15 | Object storage multi-site synchronous alarm method, system, terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113836225B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119759862A (en) * | 2024-12-18 | 2025-04-04 | 苏州元脑智能科技有限公司 | Data synchronization control method, device, equipment and medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111008185A (en) * | 2019-10-29 | 2020-04-14 | 厦门网宿有限公司 | Data sharing method, system and equipment |
| CN112052141A (en) * | 2020-09-02 | 2020-12-08 | 平安科技（深圳）有限公司 | Data fragment verification method and device, computer equipment and readable storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11023178B2 (en) * | 2018-07-24 | 2021-06-01 | Weka, Io Ltd | Implementing coherency and page cache support for a storage system spread across multiple data centers |
| US11531642B2 (en) * | 2019-11-07 | 2022-12-20 | Netapp, Inc. | Synchronous object placement for information lifecycle management |
| CN112272106B (en) * | 2020-09-28 | 2023-01-10 | 苏州浪潮智能科技有限公司 | Multi-site data synchronization abnormity warning method, device, equipment and product |
| CN113364864B (en) * | 2021-06-03 | 2022-09-30 | 上海微盟企业发展有限公司 | Server data synchronization method, system and storage medium |
-
2021
- 2021-09-15 CN CN202111082389.5A patent/CN113836225B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111008185A (en) * | 2019-10-29 | 2020-04-14 | 厦门网宿有限公司 | Data sharing method, system and equipment |
| CN112052141A (en) * | 2020-09-02 | 2020-12-08 | 平安科技（深圳）有限公司 | Data fragment verification method and device, computer equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113836225A (en) | 2021-12-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2062125B1 (en) | System and method for providing high availability data | |
| CN103607428B (en) | A kind of method and apparatus for accessing shared drive | |
| CN114490527B (en) | Metadata retrieval method, system, terminal and storage medium | |
| EP3646204A1 (en) | Systems and methods of database tenant migration | |
| CN104657435B (en) | A kind of memory management method and Network Management System using data | |
| US11599503B2 (en) | Path name cache for notifications of file changes | |
| CN106909595B (en) | A data migration method and device | |
| CN108153849A (en) | A kind of database table cutting method, device, system and medium | |
| US20200097581A1 (en) | Systems and methods of managing manifest refresh in a database | |
| US20160179435A1 (en) | Systems and methods for shadow migration progress estimation | |
| CN116225782B (en) | Method and system for processing snapshot | |
| CN113342746A (en) | File management system, file management method, electronic device, and storage medium | |
| CN113836225B (en) | Object storage multi-site synchronous alarm method, system, terminal and storage medium | |
| US11983159B2 (en) | Systems and methods for management of a log-structure | |
| CN113157487A (en) | Data recovery method and apparatus thereof | |
| CN114443598B (en) | Data writing method, device, computer equipment and storage medium | |
| CN115422137A (en) | File creation method and device, electronic equipment and computer readable storage medium | |
| CN115328859A (en) | Metadata access method, device, equipment and medium | |
| CN114168074A (en) | Mirror image storage method, device, storage medium and electronic equipment | |
| KR102214697B1 (en) | A computer program for providing space managrment for data storage in a database management system | |
| CN113742253A (en) | Storage medium management method, device, equipment and computer readable storage medium | |
| CN111209304B (en) | Data processing method, device and system | |
| KR20210013747A (en) | A computer program for providing space managrment for data storage in a database management system | |
| CN117235203B (en) | A data storage method, apparatus, electronic device, and storage medium | |
| CN104301345B (en) | The method and system of data are deleted in a kind of Cache clusters |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |