CN111541753A - Distributed storage system, method, computer device and medium for block chain data - Google Patents

Abstract

The present invention provides a distributed storage system, method, computer equipment and medium for blockchain data. The method can respond to a received file storage request and call a transaction contract in a blockchain network to store the file. Request for verification, in order to combine the verification logic in the blockchain to make data storage more secure and reliable, when the file storage request passes verification, the file to be stored is written to the target cache node, effectively improving the area The throughput of the blockchain system, in response to the file system writing the first number of data blocks and the second number of check blocks to the first stripe node, recording metadata information of the stored file, wherein the data The block and the check block are obtained by striping the file on the selected target cache node and then performing RS encoding. Since the metadata information is separately recorded, the problem of resource amplification is solved, and the recorded metadata information is guaranteed. effectiveness.

Description

Distributed storage system, method, computer equipment and medium for blockchain data

technical field

The present invention relates to the technical field of blockchain, and in particular, to a distributed storage system, method, computer equipment and medium for blockchain data.

Background technique

At present, the data storage in the block chain network usually adopts the method of directly uploading the data to the chain. All data is stored on each node of the block chain, which not only consumes a lot of storage space, but also costs more. high. With the continuous improvement of the storage level, the performance of the blockchain network will also decrease, and it cannot support the data storage of massive and large files.

In addition, because the data is stored on the blockchain nodes, it does not support permission control for file operations, and the security of the data is also low, which will also lead to serious problems of read-write amplification and storage resource amplification.

SUMMARY OF THE INVENTION

The main purpose of the present invention is to provide a distributed storage system, method, computer equipment and medium for blockchain data, which can realize distributed storage of data in the blockchain network based on cache nodes and stripe nodes, and effectively improve the block chain. The throughput and writing performance of the chain system, and because of the combination of the verification logic in the blockchain, the storage of data is more secure and reliable, which can avoid malicious tampering of the stored data.

To achieve the above object, the present invention provides a distributed storage system for blockchain data, wherein the distributed storage system for blockchain data includes a blockchain node and a file system;

The blockchain node is configured to, in response to the received file storage request, call a transaction contract in the blockchain network to verify the file storage request;

The blockchain node is further configured to send the file to be stored to the file system when the file storage request passes the verification;

the file system, configured to select a target cache node, and temporarily store the to-be-stored file to the target cache node;

The file system is further used for striping the files on the target cache node;

The file system is further configured to perform RS encoding on the striped file to obtain a first number of data blocks and a second number of check blocks, and write the data blocks and the check blocks into first strip node;

The blockchain node is also used to record the metadata information of the stored file.

According to a preferred embodiment of the present invention, the file system selecting a target cache node includes:

Determine the node that is not currently called from multiple cache nodes;

A node with the largest remaining capacity among the nodes that have not been called at present is determined as the target cache node.

According to a preferred embodiment of the present invention, each cache node includes at least one candidate node, and the file system is further configured to:

When the target cache node is abnormal, select a node from the candidate nodes of the target cache node to replace the target cache node; or

When the target cache node and the candidate nodes of the target cache node remain to serve one node, writing data to the remaining nodes is prohibited.

According to a preferred embodiment of the present invention, when the file in the target cache node is written to the stripe node, the file system is further configured to:

When it is detected that the file in the target cache node is not currently accessed, delete the file in the target cache node; or

When it is detected that the file in the target cache node is being accessed, the target cache node is locked based on the distributed lock service, and deletion of the file in the target cache node is prohibited.

According to a preferred embodiment of the present invention, the file system is further used for:

When the first stripe node is full and there are remaining data blocks and/or check blocks that have not been written, apply for a second stripe node to store the remaining data blocks and/or check blocks.

In order to achieve the above object, the present invention also provides a distributed storage method for blockchain data, which is applied to a blockchain node, the blockchain node communicates with a file system, and the distributed storage method of the blockchain data is Methods include:

In response to the received file storage request, call the transaction contract in the blockchain network to verify the file storage request;

When the file storage request passes the verification, sending the file to be stored to the file system;

In response to the file system writing the first number of data blocks and the second number of check blocks into the first stripe node, record metadata information of the stored file, wherein the data blocks and the check blocks It is obtained by striping the file on the selected target cache node and then performing RS encoding.

According to a preferred embodiment of the present invention, the distributed storage method for blockchain data further includes:

In response to the received file deletion request, call the transaction contract in the blockchain network to verify the file deletion request;

When the file deletion request passes the verification, retrieve the metadata information corresponding to the file to be deleted;

Obtain the storage location of the to-be-deleted file from the retrieved metadata information;

When detecting that the to-be-deleted file is stored in a cache node, request the file system to delete the to-be-deleted file from the detected cache node, or when detecting that the to-be-deleted file is stored in a stripe node, request The file system marks the to-be-deleted file as a deleted state in the detected stripe node;

Update the retrieved metadata information.

According to a preferred embodiment of the present invention, the distributed storage method for blockchain data further includes:

In response to the received file download request, call the transaction contract in the blockchain network to verify the file download request;

When the file download request passes the verification, retrieve the metadata information corresponding to the file to be downloaded;

Obtain the storage location of the to-be-downloaded file from the retrieved metadata information;

The file to be downloaded is read from the recalled storage location.

According to a preferred embodiment of the present invention, the distributed storage method for blockchain data further includes:

In response to the received file recovery request, obtain the non-missing data blocks and/or check blocks of the file to be recovered from the stripe node;

Perform RS decoding on the non-missing data blocks and/or check blocks to obtain the original file of the file to be restored.

In order to achieve the above purpose, the present invention also provides a distributed storage device for blockchain data, which runs on a blockchain node, the blockchain node communicates with the file system, and the distributed storage of the blockchain data The device includes:

a verification unit, used for invoking the transaction contract in the blockchain network to verify the file storage request in response to the received file storage request;

a sending unit, configured to send the file to be stored to the file system when the file storage request passes the verification;

a recording unit, configured to record the metadata information of the stored file in response to the file system writing the first number of data blocks and the second number of check blocks into the first stripe node, wherein the data blocks and The check block is obtained by performing the RS encoding after striping the file on the selected target cache node.

According to a preferred embodiment of the present invention, the verification unit is further configured to, in response to the received file deletion request, call a transaction contract in the blockchain network to verify the file deletion request;

The distributed storage device for the blockchain data also includes:

a retrieval unit, configured to retrieve the metadata information corresponding to the file to be deleted when the file deletion request passes the verification;

an obtaining unit, configured to obtain the storage location of the to-be-deleted file from the retrieved metadata information;

The requesting unit is configured to request the file system to delete the to-be-deleted file from the detected cache node when it is detected that the to-be-deleted file is stored in the cache node, or to request the file system to delete the to-be-deleted file when it is detected that the to-be-deleted file is stored in the cache node When stripping nodes, requesting the file system to mark the to-be-deleted file in the detected stripe node as a deleted state;

The update unit is used to update the retrieved metadata information.

According to a preferred embodiment of the present invention, the verification unit is further configured to, in response to the received file download request, call a transaction contract in the blockchain network to verify the file download request;

The retrieval unit is further configured to retrieve the metadata information corresponding to the file to be downloaded when the file download request passes the verification;

The obtaining unit is further configured to obtain the storage location of the to-be-downloaded file from the retrieved metadata information;

The distributed storage device for the blockchain data also includes:

A reading unit, configured to read the to-be-downloaded file from the recalled storage location.

According to a preferred embodiment of the present invention, the obtaining unit is further configured to, in response to the received file restoration request, obtain the non-missing data blocks and/or check blocks of the file to be restored from the stripe node;

The distributed storage device for the blockchain data also includes:

A decoding unit, configured to perform RS decoding on the non-missing data blocks and/or check blocks to obtain the original file of the file to be restored.

In order to achieve the above object, the present invention further provides a computer device, the computer device includes:

a memory that stores at least one instruction; and

The processor executes the instructions stored in the memory to implement the distributed storage method of the blockchain data.

In order to achieve the above object, the present invention further provides a computer-readable storage medium, on which a distributed storage program of blockchain data is stored, and the distributed storage program of blockchain data can be One or more processors execute to implement the distributed storage method of the blockchain data.

To sum up, the present invention can call the transaction contract in the blockchain network to verify the file storage request in response to the received file storage request, so as to combine the verification logic in the blockchain to store the data It is more secure and reliable. When the file storage request passes the verification, the file to be stored is written to the target cache node, which effectively improves the throughput of the blockchain system. In response to the file system, the first amount of data The block and the second number of check blocks are written into the first stripe node, and the metadata information of the stored file is recorded, wherein the data block and the check block are for striping the file on the selected target cache node. It is obtained by performing RS coding after the banding process. Since the metadata information is separately recorded, the problem of resource amplification is solved, and the validity of the recorded metadata information is guaranteed.

Description of drawings

1 is a schematic flowchart of an embodiment of the present invention;

2 is a schematic flowchart of another embodiment of the present invention;

3 is a schematic diagram of an internal structure of a computer device disclosed in an embodiment of the present invention;

FIG. 4 is a schematic diagram of functional modules of a distributed storage device for blockchain data according to the present invention.

Description of main component symbols

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are only used for For descriptive purposes, it should not be understood as indicating or implying their relative importance or implying the number of technical features indicated. Thus, features delimited with "first" and "second" may expressly or implicitly include at least one of the features and are not necessarily used to describe a specific order or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances so that the embodiments described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, system, method, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

In addition, the technical solutions between the various embodiments can be combined with each other, but must be based on the realization by those of ordinary skill in the art. When the combination of technical solutions is contradictory or cannot be realized, it should be considered that the combination of such technical solutions does not exist. , is not within the scope of protection required by the present invention.

The present invention provides a distributed storage system for blockchain data.

Referring to FIG. 1 , it is a schematic flowchart of an embodiment of the present invention. According to different requirements, the sequence of the steps in this schematic flowchart can be changed, and some steps can be omitted.

In one embodiment, the distributed storage system for blockchain data includes a blockchain node and a file system:

S10: In response to the received file storage request, the blockchain node invokes a transaction contract in the blockchain network to verify the file storage request.

In at least one embodiment of the present invention, the transaction hash is used as the verification basis.

For example, the blockchain node matches the hash value in the file storage request with the transaction hash in the blockchain network, and when the matching is successful, the blockchain node determines the The file storage request passes the verification and executes the file storage request; when the matching fails, the blockchain node determines that the file storage request fails the verification, and does not execute the file storage request.

Through the above-mentioned embodiments, data storage can be made more secure and reliable in combination with the verification logic in the blockchain.

S11, when the file storage request passes the verification, the blockchain node sends the file to be stored to the file system.

For example, the file system may be a file storage system including an operating system, the operating system is used to specify the method and data structure of files on the storage device or partition, that is, organize files on the storage device, and the operating system is responsible for managing and A software organization that stores file information may be referred to as a file management system.

S12, the file system selects a target cache node, and temporarily stores the to-be-stored file to the target cache node.

The file system may include multiple cache nodes and multiple stripe nodes for implementing a file storage function, and the target cache node is one of the multiple cache nodes.

In at least one embodiment of the present invention, the cache node (Cache Node) is used to temporarily store data in the blockchain network, and the configuration of the cache node can effectively improve the throughput of the blockchain system .

In this embodiment, since the cache node has high read and write requirements, it is preferable to equip the cache node with a solid state drive (Solid State Drive, SSD) with better read and write performance. Mechanical hard drives have faster read and write speeds, higher shock and drop resistance, low power consumption, no noise, and a wider operating temperature range.

In at least one embodiment of the present invention, the file system selecting the target cache node to store the file from the cache nodes includes:

The file system determines a node that is not currently called from a plurality of cache nodes, and determines a node with the largest remaining capacity among the nodes that is not currently called as the target cache node.

It can be understood that a cache node cannot be called at the same time, therefore, the file system first determines the node that is not currently called from the multiple cache nodes, and further selects the node with the largest remaining capacity among the nodes that are not currently called. The node is determined as the target cache node, and then the node with the best performance is selected from all the nodes that can be called for data storage.

In at least one embodiment of the present invention, each cache node includes at least one candidate node, and the file system is further configured to:

(1) When the target cache node is abnormal, the file system selects a node from the candidate nodes of the target cache node to replace the target cache node.

Through the above-mentioned embodiments, when a node fails to serve abnormally, an alternative node can be used to replace the abnormal node to continue the service, which effectively solves the single-point downtime problem of the system and improves the stability of the system.

(2) When the target cache node and the candidate nodes of the target cache node are left to serve one node, the file system prohibits writing data to the remaining nodes.

It is understandable that when the master node or standby node in a group of cache nodes goes down and only one node remains to serve, the data security of single-node storage cannot be guaranteed, so this group of cache nodes will no longer provide Write function, only provide file read support to ensure data security.

S13, the file system performs striping processing on the file on the target cache node.

Disk conflicts can occur when multiple processes access a disk at the same time. Striping is a process of automatically balancing the I/O load to multiple physical disks. Striping is to divide a continuous piece of data into many small parts and store them on different disks. This enables multiple processes to simultaneously access different parts of the data without causing disk conflicts, and when sequential access to such data is required, maximum I/O parallelism can be achieved, resulting in better performance.

S14, the file system performs RS (Reed-Solomon codes) encoding on the striped file to obtain a first number of data blocks and a second number of check blocks, and encodes the data blocks and the check blocks. The verification block is written to the first stripe node.

The first number and the second number may be configured according to actual requirements, for example, the first number may be 9, and the second number may be 3.

Further, the size of each data block and the size of each check block can also be configured according to actual requirements. For example: each data block can be 16M, and each check block is also 16M.

In at least one embodiment of the present invention, the RS coding is a forward error correction channel coding valid for polynomials generated by correcting oversampled data. When enough data is received correctly, the original data can be recovered according to the received data. Even if the received data is disturbed and distorted by many points of noise, it will not affect the data recovery. It is a low-cost data processing mode.

In this embodiment, the first stripe node may be a currently serving stripe node that is not full, and may be one of multiple stripe nodes in the file system.

Disk conflicts can occur when multiple processes access a disk at the same time. In the striping storage mode, the I/O (Input/Output, input/output) load is automatically balanced to multiple physical disks, a continuous piece of data is divided into many small parts, and the They are stored on different disks (preferably on different disks on different machines, and secondly on different disks on the same machine), so that multiple processes can access multiple different parts of the data at the same time without causing disk conflicts. , and, when sequential access to data is required, the I/O parallelism to the greatest extent can be obtained, which improves the access performance.

In addition, because the data is stored in different blocks in different locations, the possibility of simultaneous damage to the data is very low, and the undamaged data can be used for subsequent data recovery based on RS decoding, which improves the recoverability of the data.

Moreover, since there is an upper limit on the inode index of data in the system, in the striped storage mode, small files are merged into fixed-size stripes for storage, which improves write performance and avoids inode index overflow.

S15, the blockchain node records the metadata information of the stored file.

In at least one embodiment of the present invention, the file system is further used for:

When the first stripe node is full and there are remaining data blocks and/or check blocks that have not been written, apply for a second stripe node to store the remaining data blocks and/or check blocks.

It should be noted that in the striped storage mode, there is only one stripe node that is not full at the same time. When the stripe node is full, it can apply to the central controller for a second stripe node. Serve.

This embodiment can apply for a second stripe node in time when the first stripe node is full, so as to realize normal storage of data.

In at least one embodiment of the present invention, when the file in the target cache node is written to the stripe node, the file system is further configured to:

(1) When it is detected that the file in the target cache node is not currently accessed, delete the file in the target cache node.

Through the above implementation, after the file in the target cache node is successfully written to the stripe node, the file in the target cache node can be deleted in time to release the storage space of the target cache node.

(2) When it is detected that the file in the target cache node is being accessed, the target cache node is locked based on the distributed lock service, and the deletion of the file in the target cache node is prohibited.

Through the above-mentioned embodiments, it is possible to effectively avoid access abnormality caused by deleting data when data is accessed.

In at least one embodiment of the present invention, when a stripe node that is writing data is abnormal, an alarm is issued in time to avoid invalid writing of upstream data.

In this embodiment, the metadata information includes, but is not limited to, a combination of one or more of the following information:

File summary, file storage node information, file creation time, and file last read time.

Among them, the file summary is used to compare with the file summary data stored on the chain to ensure that the file has not been tampered with;

The file storage node information is used to determine the storage location of the file. For example, Cache_0_100_1234:Cache_0__100_1235, the file is located in the CacheNode0 node, and the starting point is 1234 bytes of strip 100, and the end point is 1235 bytes of strip 100; Stripe_1025_1234:; Stripe_1026:; Stripe_1027_0: Stripe_1027_65534, indicating that the file is located in In Stripe, the starting point is 1234 bytes of the 1025th storage stripe, and the end point is 64434 bytes of the 1027th storage stripe.

In this embodiment, the metadata information of the stored file may be recorded in the form of Key-Value, where Key is a path (path), and Value is the metadata information.

In addition, when storing the metadata information, an active-standby storage mode may also be adopted to avoid a single point of failure.

By recording the metadata information, it is not necessary to store the relevant information in the metadata information at the same time of data storage, thereby solving the problem of resource amplification.

Through the above embodiments, the recorded metadata information can be updated in time to ensure the validity of the recorded metadata information.

In at least one embodiment of the present invention, the blockchain node is further used for:

In response to the received file deletion request, the blockchain node invokes the transaction contract in the blockchain network to verify the file deletion request, and when the file deletion request passes the verification, the district The block chain node retrieves the metadata information corresponding to the file to be deleted. Further, the blockchain node obtains the storage location of the file to be deleted from the retrieved metadata information. When detecting that the file to be deleted is stored In the cache node, the blockchain node requests the file system to delete the to-be-deleted file from the detected cache node, or when it is detected that the to-be-deleted file is stored in the stripe node, the block The chain node requests the file system to mark the to-be-deleted file as a deleted state in the detected stripe node, and update the retrieved metadata information.

Among them, in the striped storage mode, the implementation of the delete operation is more complicated, therefore, the files to be deleted are usually only marked as deleted.

Through the above-mentioned embodiments, the file deletion operation can be performed for different storage locations after the file deletion request passes the verification.

In at least one embodiment of the present invention, the blockchain node is further used for:

In response to the received file download request, the blockchain node invokes the transaction contract in the blockchain network to verify the file download request, and when the file download request passes the verification, the district The block chain node retrieves the metadata information corresponding to the file to be downloaded, and obtains the storage location of the file to be downloaded from the retrieved metadata information. Further, the blockchain node reads from the retrieved storage location. Get the file to be downloaded.

Through the above-mentioned embodiments, the file download operation can be performed after the file download request passes the verification, so as to avoid the response to the abnormal request.

In this embodiment, it can also be verified whether the file has been tampered with.

Taking the hash value as an example, the file digest of the stored file can be obtained, and the obtained file digest can be matched with the file digest stored in the blockchain network. When the matching is successful, the stored file can be determined. Through the verification, it is further determined that the stored file has not been tampered with; when the matching fails, it is determined that the stored file has not passed the verification, and it is further determined that the stored file has a risk of being tampered with.

In at least one embodiment of the present invention, the blockchain node is further used for:

In response to the received file recovery request, the blockchain node obtains the non-missing data blocks and/or check blocks of the file to be restored from the stripe node, and performs an update on the non-missing data blocks and/or check blocks. Perform RS decoding to obtain the original file of the file to be restored.

Through the above-mentioned embodiments, automatic recovery of lost data can be realized in combination with RS decoding, and the recovery cost is low.

To sum up, the present invention can call the transaction contract in the blockchain network to verify the file storage request in response to the received file storage request, so as to combine the verification logic in the blockchain to store the data It is more secure and reliable. When the file storage request passes the verification, the file to be stored is written to the target cache node, which effectively improves the throughput of the blockchain system. In response to the file system, the first amount of data The block and the second number of check blocks are written into the first stripe node, and the metadata information of the stored file is recorded, wherein the data block and the check block are for striping the file on the selected target cache node. It is obtained by performing RS coding after the banding process. Since the metadata information is separately recorded, the problem of resource amplification is solved, and the validity of the recorded metadata information is guaranteed.

Referring to FIG. 2 , it is a schematic flowchart of another embodiment of the present invention. According to different requirements, the sequence of the steps in this schematic flowchart can be changed, and some steps can be omitted.

The distributed storage method of blockchain data is applied to one or more blockchain nodes, and the blockchain node is a kind of blockchain node that can automatically perform numerical calculation and/or information according to pre-set or stored instructions Processing equipment, the hardware equipment of the blockchain node includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a programmable gate array (Field-Programmable GateArray, FPGA), a digital processor (Digital Signal Processor, DSP), embedded devices, etc.

The blockchain node can be any electronic product that can interact with the user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network Television (Internet Protocol Television, IPTV), smart wearable devices, etc.

The blockchain nodes may also include network equipment and/or user equipment. Wherein, the network device includes, but is not limited to, a single network server, a server group formed by multiple network servers, or a cloud formed by a large number of hosts or network servers based on cloud computing (Cloud Computing).

The network where the blockchain node is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (Virtual Private Network, VPN), and the like.

S20, in response to the received file storage request, call a transaction contract in the blockchain network to verify the file storage request.

In at least one embodiment of the present invention, the transaction hash is used as the verification basis.

For example, the blockchain node matches the hash value in the file storage request with the transaction hash in the blockchain network, and when the matching is successful, the blockchain node determines the The file storage request passes the verification and executes the file storage request; when the matching fails, the blockchain node determines that the file storage request fails the verification, and does not execute the file storage request.

Through the above-mentioned embodiments, data storage can be made more secure and reliable in combination with the verification logic in the blockchain.

S21, when the file storage request passes the verification, send the file to be stored to the file system.

For example, the file system may be a file storage system including an operating system, the operating system is used to specify the method and data structure of files on the storage device or partition, that is, organize files on the storage device, and the operating system is responsible for managing and A software organization that stores file information may be referred to as a file management system.

S22, in response to the file system writing the first number of data blocks and the second number of check blocks into the first stripe node, record metadata information of the stored file, wherein the data blocks and the check blocks are The verification block is obtained by performing the RS encoding after striping the file on the selected target cache node.

In at least one embodiment of the present invention, the distributed storage method for blockchain data further includes:

In response to the received file deletion request, the blockchain node invokes the transaction contract in the blockchain network to verify the file deletion request, and when the file deletion request passes the verification, the district The block chain node retrieves the metadata information corresponding to the file to be deleted. Further, the blockchain node obtains the storage location of the file to be deleted from the retrieved metadata information. When detecting that the file to be deleted is stored In the cache node, the blockchain node requests the file system to delete the to-be-deleted file from the detected cache node, or when it is detected that the to-be-deleted file is stored in the stripe node, the block The chain node requests the file system to mark the to-be-deleted file as a deleted state in the detected stripe node, and update the retrieved metadata information.

Among them, in the striped storage mode, the implementation of the delete operation is more complicated, therefore, the files to be deleted are usually only marked as deleted.

Through the above-mentioned embodiments, the file deletion operation can be performed for different storage locations after the file deletion request passes the verification.

In at least one embodiment of the present invention, the distributed storage method for blockchain data further includes:

In response to the received file download request, the blockchain node invokes the transaction contract in the blockchain network to verify the file download request, and when the file download request passes the verification, the district The block chain node retrieves the metadata information corresponding to the file to be downloaded, and obtains the storage location of the file to be downloaded from the retrieved metadata information. Further, the blockchain node reads from the retrieved storage location. Get the file to be downloaded.

Through the above-mentioned embodiments, the file download operation can be performed after the file download request passes the verification, so as to avoid the response to the abnormal request.

In at least one embodiment of the present invention, the distributed storage method for blockchain data further includes:

In response to the received file recovery request, the blockchain node obtains the non-missing data blocks and/or check blocks of the file to be restored from the stripe node, and performs an update on the non-missing data blocks and/or check blocks. Perform RS decoding to obtain the original file of the file to be restored.

Through the above-mentioned embodiments, automatic recovery of lost data can be realized in combination with RS decoding, and the recovery cost is low.

To sum up, the present invention can call the transaction contract in the blockchain network to verify the file storage request in response to the received file storage request, so as to combine the verification logic in the blockchain to store the data It is more secure and reliable. When the file storage request passes the verification, the file to be stored is written to the target cache node, which effectively improves the throughput of the blockchain system. In response to the file system, the first amount of data The block and the second number of check blocks are written into the first stripe node, and the metadata information of the stored file is recorded, wherein the data block and the check block are for striping the file on the selected target cache node. It is obtained by performing RS coding after the banding process. Since the metadata information is separately recorded, the problem of resource amplification is solved, and the validity of the recorded metadata information is guaranteed.

Referring to FIG. 3 , in this embodiment, the computer device 1 may include a memory 12, a processor 13 and a bus, and may also include a computer program stored in the memory 12 and running on the processor 13, For example, a distributed storage program for blockchain data.

Those skilled in the art can understand that the schematic diagram is only an example of the computer device 1, and does not constitute a limitation on the computer device 1. The computer device 1 can be either a bus-type structure or a star-shaped structure. The device 1 may also include more or less other hardware or software than shown, or a different arrangement of components, for example, the computer device 1 may also include input and output devices, network access devices, and the like.

It should be noted that the computer equipment 1 is only an example. If other existing or future electronic products can be adapted to the present invention, they should also be included within the protection scope of the present invention, and are incorporated herein by reference. .

Wherein, the memory 12 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. . The memory 12 may be an internal storage unit of the computer device 1 in some embodiments, such as a removable hard disk of the computer device 1 . The memory 12 may also be an external storage device of the computer device 1 in other embodiments, such as a pluggable mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) equipped on the computer device 1 Cards, Flash Cards, etc. Further, the memory 12 may also include both an internal storage unit of the computer device 1 and an external storage device. The memory 12 can not only be used to store application software installed in the computer device 1 and various types of data, such as codes of distributed storage programs of blockchain data, etc., but also can be used to temporarily store data that has been output or will be output.

The processor 13 may be composed of integrated circuits in some embodiments, for example, may be composed of a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same function or different functions, including one or more central processing units. CPU (Central Processing Unit, CPU), microprocessor, digital processing chip, graphics processor and combination of various control chips, etc. The processor 13 is the control core (Control Unit) of the computer device 1, and uses various interfaces and lines to connect the various components of the entire computer device 1, and by running or executing the programs or modules stored in the memory 12 (such as executing distributed storage program of blockchain data, etc.), and call the data stored in the memory 12 to perform various functions of the computer device 1 and process data.

The processor 13 executes the operating system of the computer device 1 and various installed application programs. The processor 13 executes the application program to implement the steps in each of the foregoing embodiments of the distributed storage method for blockchain data, such as steps S20 , S21 , and S22 shown in FIG. 2 .

Alternatively, when the processor 13 executes the computer program, the functions of the modules/units in the above device embodiments are implemented, for example:

In response to the received file storage request, call the transaction contract in the blockchain network to verify the file storage request;

When the file storage request passes the verification, sending the file to be stored to the file system;

In response to the file system writing the first number of data blocks and the second number of check blocks into the first stripe node, record metadata information of the stored file, wherein the data blocks and the check blocks It is obtained by striping the file on the selected target cache node and then performing RS encoding.

Exemplarily, the computer program may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 12 and executed by the processor 13 to complete the present invention. invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the computer device 1 . For example, the computer program can be divided into a checking unit 111 , a sending unit 112 , a recording unit 113 , a calling unit 114 , an obtaining unit 115 , a requesting unit 116 , an updating unit 117 , a reading unit 118 , and a decoding unit 119 .

The above-mentioned integrated units implemented in the form of software functional modules may be stored in a computer-readable storage medium. The above-mentioned software function modules are stored in a storage medium, and include several instructions to cause a computer device (which may be a personal computer, a computer device, or a network device, etc.) or a processor (processor) to execute the methods described in the various embodiments of the present invention. part.

If the modules/units integrated in the computer device 1 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing relevant hardware devices through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above method embodiments can be implemented.

Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form, and the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory) .

The bus may be a peripheral component interconnect (PCI for short) bus or an extended industry standard architecture (extended industry standard architecture, EISA for short) bus, or the like. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one arrow is shown in FIG. 3, but it does not mean that there is only one bus or one type of bus. The bus is arranged to enable connection communication between the memory 12 and at least one processor 13 and the like.

Although not shown, the computer device 1 may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to the at least one processor 13 through a power management device, so as to be implemented by the power management device Charge management, discharge management, and power management functions. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and any other components. The computer device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

Further, the computer device 1 may also include a network interface, optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used in the computer device 1 Establish a communication connection with other computer equipment.

Optionally, the computer device 1 may further include a user interface, and the user interface may be a display, an input unit (such as a keyboard), and optionally, the user interface may also be a standard wired interface or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, and the like. The display, which may also be appropriately called a display screen or a display unit, is used for displaying information processed in the computer device 1 and for displaying a visual user interface.

It should be understood that the embodiments are only used for illustration, and are not limited by this structure in the scope of the patent application.

FIG. 3 only shows the computer device 1 having the components 12-13. Those skilled in the art can understand that the structure shown in FIG. 3 does not constitute a limitation on the computer device 1, and may include less than illustrated Or more components, or a combination of certain components, or a different arrangement of components.

2, the memory 12 in the computer device 1 stores multiple instructions to implement a distributed storage method for blockchain data, and the processor 13 can execute the multiple instructions to implement:

In response to the received file storage request, call the transaction contract in the blockchain network to verify the file storage request;

When the file storage request passes the verification, sending the file to be stored to the file system;

In response to the file system writing the first number of data blocks and the second number of check blocks into the first stripe node, record metadata information of the stored file, wherein the data blocks and the check blocks It is obtained by striping the file on the selected target cache node and then performing RS encoding.

Specifically, for the specific implementation method of the above-mentioned instruction by the processor 13, reference may be made to the description of the relevant steps in the embodiment corresponding to FIG. 2, and details are not described herein.

Referring to FIG. 4 , it is a schematic diagram of the functional modules of the distributed storage system for blockchain data of the present invention. The distributed storage device 11 for blockchain data includes a verification unit 111, a sending unit 112, a recording unit 113, a retrieval unit 114, an acquisition unit 115, a request unit 116, an update unit 117, a reading unit 118, and a decoding unit 119. The modules/units referred to in the present invention refer to a series of computer program segments that can be executed by the processor 13 and can perform fixed functions, and are stored in the memory 12 . In this embodiment, the functions of each module/unit will be described in detail in subsequent embodiments.

In response to the received file storage request, the verification unit 111 invokes the transaction contract in the blockchain network to verify the file storage request.

In at least one embodiment of the present invention, the transaction hash is used as the verification basis.

For example, the verification unit 111 matches the hash value in the file storage request with the transaction hash in the blockchain network, and when the matching is successful, the verification unit 111 determines that the The file storage request passes the verification, and executes the file storage request; when the matching fails, the verification unit 111 determines that the file storage request fails the verification, and does not execute the file storage request.

Through the above-mentioned embodiments, data storage can be made more secure and reliable in combination with the verification logic in the blockchain.

When the file storage request passes the verification, the sending unit 112 sends the file to be stored to the file system.

Wherein, the file system is an operating system used to specify the method and data structure of files on a storage device or partition, that is, to organize files on the storage device. The software organization responsible for managing and storing file information in the operating system is called a file management system, or file system for short.

In response to the file system writing the first number of data blocks and the second number of check blocks into the first stripe node, the recording unit 113 records the metadata information of the stored file, wherein the data blocks and the The check block is obtained by striping the file on the selected target cache node and then performing RS encoding.

In at least one embodiment of the present invention, in response to the received file deletion request, the verification unit 111 invokes a transaction contract in the blockchain network to verify the file deletion request. When the deletion request passes the verification, the retrieval unit 114 retrieves the metadata information corresponding to the file to be deleted, and further, the retrieval unit 115 obtains the storage location of the to-be-deleted file from the retrieved metadata information. When the to-be-deleted file is stored in the cache node, the requesting unit 116 requests the file system to delete the to-be-deleted file from the detected cache node, or when it is detected that the to-be-deleted file is stored in the stripe node, the The requesting unit 116 requests the file system to mark the to-be-deleted file as a deleted state in the detected stripe node, and the updating unit 117 updates the retrieved metadata information.

Among them, in the striped storage mode, the implementation of the delete operation is more complicated, therefore, the files to be deleted are usually only marked as deleted.

Through the above-mentioned embodiments, the file deletion operation can be performed for different storage locations after the file deletion request passes the verification.

In at least one embodiment of the present invention, in response to the received file download request, the verification unit 111 invokes a transaction contract in the blockchain network to verify the file download request. When the download request passes the verification, the retrieval unit 114 retrieves the metadata information corresponding to the file to be downloaded, and the retrieval unit 115 obtains the storage location of the file to be downloaded from the retrieved metadata information, and further, The reading unit 118 reads the file to be downloaded from the recalled storage location.

Through the above-mentioned embodiments, the file download operation can be performed after the file download request passes the verification, so as to avoid the response to the abnormal request.

In at least one embodiment of the present invention, in response to the received file restoration request, the obtaining unit 115 obtains the non-missing data blocks and/or check blocks of the file to be restored from the stripe node, and the decoding unit 119 obtains the Perform RS decoding on the non-missing data blocks and/or check blocks to obtain the original file of the file to be restored.

Through the above-mentioned embodiments, automatic recovery of lost data can be realized in combination with RS decoding, and the recovery cost is low.

To sum up, the present invention can call the transaction contract in the blockchain network to verify the file storage request in response to the received file storage request, so as to combine the verification logic in the blockchain to store the data It is more secure and reliable. When the file storage request passes the verification, the file to be stored is written to the target cache node, which effectively improves the throughput of the blockchain system. In response to the file system, the first amount of data The block and the second number of check blocks are written into the first stripe node, and the metadata information of the stored file is recorded, wherein the data block and the check block are for striping the file on the selected target cache node. It is obtained by performing RS coding after the banding process. Since the metadata information is separately recorded, the problem of resource amplification is solved, and the validity of the recorded metadata information is guaranteed.

In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware devices, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website, computer, slave server or data The center transmits to another website site, computer, slave server, or data center by wire (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be stored by a computer or a data storage device such as a slave server, a data center, etc. that includes one or more available media integrated. The usable media may be magnetic media (eg, floppy disks, removable hard disks, magnetic tapes), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), and the like.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above may refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware devices, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions to cause a computer device (which may be a personal computer, a slave server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a removable hard disk, a read-only memory (ROM, Read-Only Memory), a magnetic disk or an optical disk and other media that can store program codes.

It should be noted that the above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments. And the terms "comprising", "comprising" or any other variation thereof herein are intended to encompass a non-exclusive inclusion such that a process, device, article or method comprising a list of elements includes not only those elements, but also includes no explicit Other elements listed, or those inherent to such a process, apparatus, article, or method are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, apparatus, article, or method that includes the element.

The above are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields , are similarly included in the scope of patent protection of the present invention.

Claims (11)

1. A distributed storage system for blockchain data, wherein the distributed storage system for blockchain data comprises a blockchain node and a file system; The blockchain node is configured to, in response to the received file storage request, call a transaction contract in the blockchain network to verify the file storage request; The blockchain node is further configured to send the file to be stored to the file system when the file storage request passes the verification; the file system, configured to select a target cache node, and temporarily store the to-be-stored file to the target cache node; The file system is further used for striping the files on the target cache node; The file system is further configured to perform RS encoding on the striped file to obtain a first number of data blocks and a second number of check blocks, and write the data blocks and the check blocks into first strip node; The blockchain node is also used to record the metadata information of the stored file. 2. The distributed storage system for blockchain data according to claim 1, wherein the selection of the target cache node by the file system comprises: Determine the node that is not currently called from multiple cache nodes; A node with the largest remaining capacity among the nodes that have not been called at present is determined as the target cache node. 3. The distributed storage system for blockchain data according to claim 2, wherein each cache node includes at least one candidate node, and the file system is also used for: When the target cache node is abnormal, select a node from the candidate nodes of the target cache node to replace the target cache node; or When the target cache node and the candidate nodes of the target cache node remain to serve one node, writing data to the remaining nodes is prohibited. 4. The distributed storage system for blockchain data according to claim 1, wherein when the file in the target cache node is written to the stripe node, the file system is further used for: When it is detected that the file in the target cache node is not currently accessed, delete the file in the target cache node; or When it is detected that the file in the target cache node is being accessed, the target cache node is locked based on the distributed lock service, and deletion of the file in the target cache node is prohibited. 5. The distributed storage system for blockchain data according to claim 1, wherein the file system is also used for: When the first stripe node is full and there are remaining data blocks and/or check blocks that have not been written, apply for a second stripe node to store the remaining data blocks and/or check blocks. 6. A distributed storage method for blockchain data, applied to a blockchain node, wherein the blockchain node communicates with a file system, and the distributed storage method for the blockchain data comprises: In response to the received file storage request, call the transaction contract in the blockchain network to verify the file storage request; When the file storage request passes the verification, sending the file to be stored to the file system; In response to the file system writing the first number of data blocks and the second number of check blocks into the first stripe node, record metadata information of the stored file, wherein the data blocks and the check blocks It is obtained by striping the file on the selected target cache node and then performing RS encoding. 7. The distributed storage method of blockchain data according to claim 6, wherein the distributed storage method of blockchain data further comprises: In response to the received file deletion request, call the transaction contract in the blockchain network to verify the file deletion request; When the file deletion request passes the verification, retrieve the metadata information corresponding to the file to be deleted; Obtain the storage location of the to-be-deleted file from the retrieved metadata information; When detecting that the to-be-deleted file is stored in a cache node, request the file system to delete the to-be-deleted file from the detected cache node, or when detecting that the to-be-deleted file is stored in a stripe node, request The file system marks the to-be-deleted file as a deleted state in the detected stripe node; Update the retrieved metadata information. 8. The distributed storage method for blockchain data according to claim 6, wherein the distributed storage method for blockchain data further comprises: In response to the received file download request, call the transaction contract in the blockchain network to verify the file download request; When the file download request passes the verification, retrieve the metadata information corresponding to the file to be downloaded; Obtain the storage location of the to-be-downloaded file from the retrieved metadata information; The file to be downloaded is read from the recalled storage location. 9. The distributed storage method of blockchain data according to claim 6, wherein the distributed storage method of blockchain data further comprises: In response to the received file recovery request, obtain the non-missing data blocks and/or check blocks of the file to be recovered from the stripe node; Perform RS decoding on the non-missing data blocks and/or check blocks to obtain the original file of the file to be restored. 10. A computer device, characterized in that the computer device comprises: a memory that stores at least one instruction; and The processor executes the instructions stored in the memory to implement the distributed storage method for blockchain data according to any one of claims 6-9. 11. A computer-readable storage medium, wherein a distributed storage program of blockchain data is stored on the computer-readable storage medium, and the distributed storage program of the blockchain data can be stored by one or more Each processor executes to implement the distributed storage method for blockchain data according to any one of claims 6-9.

Images