WO2024055695A1 - 创建索引的方法及数据库系统 - Google Patents

创建索引的方法及数据库系统 Download PDF

Info

Publication number
WO2024055695A1
WO2024055695A1 PCT/CN2023/103975 CN2023103975W WO2024055695A1 WO 2024055695 A1 WO2024055695 A1 WO 2024055695A1 CN 2023103975 W CN2023103975 W CN 2023103975W WO 2024055695 A1 WO2024055695 A1 WO 2024055695A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
target data
storage node
computing
node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/103975
Other languages
English (en)
French (fr)
Inventor
苏斌
冯小康
祝青平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Cloud Computing Technologies Co Ltd
Original Assignee
Huawei Cloud Computing Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202211358798.8A external-priority patent/CN117743318A/zh
Application filed by Huawei Cloud Computing Technologies Co Ltd filed Critical Huawei Cloud Computing Technologies Co Ltd
Priority to EP23864434.8A priority Critical patent/EP4579478A4/en
Publication of WO2024055695A1 publication Critical patent/WO2024055695A1/zh
Priority to US19/080,392 priority patent/US20250209050A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2219—Large Object storage; Management thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2282—Tablespace storage structures; Management thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2471—Distributed queries

Definitions

  • This application relates to the field of database technology, and in particular to a method for creating an index and a database system.
  • Creating an index for data in a database is one of the most commonly used database definition semantics (data definition language, DDL) operations, and is an operation that often occurs during database system operation and maintenance. Querying through indexes can speed up querying and improve query performance. It is widely used in scenarios where users need to speed up querying.
  • the computing node when creating an index for a data table in the database, the computing node needs to read all the data of the data table from the storage node, and then create an index based on all the data of the data table.
  • This application provides a method for creating an index and a database system. This application helps improve the efficiency of index creation.
  • the technical solutions provided by this application are as follows:
  • this application provides a database system.
  • the database system includes: computing nodes and storage nodes.
  • the storage nodes store data tables, and the data tables record data with multiple rows and columns.
  • the storage node is used to determine the instruction information of the target data required to create an index.
  • the target data is part of the data in the target data table; the storage node is also used to obtain the target data from the target data table based on the instruction information.
  • providing target data to the computing node; the computing node is used to create indexes based on the target data.
  • the storage node obtains the target data required to create an index from the target data table and provides the target data to the computing node.
  • the storage node transmits the entire target data table recorded to the computing node. data, reducing the amount of data transmitted from storage nodes to computing nodes, helping to improve I/O performance between storage nodes and computing nodes.
  • the computing node is not required to perform the operation of obtaining the target data from the target data table, which reduces the computing load of the computing node and helps to improve the efficiency of index creation by the computing node and the computing performance of the computing node.
  • the computing node is further configured to receive an index creation instruction, determine instruction information based on the index creation instruction, and provide instruction information to the storage node, where the index creation instruction instructs to create an index based on the target data.
  • the computing node may determine whether to send the indication information to the storage node based on the indication information, and when it is necessary to obtain the target data through the storage node, provide the indication information to the storage node.
  • the computing node is specifically configured to provide instruction information to the storage node when the first consumption and the second consumption corresponding to the target data meet specified conditions.
  • the first consumption is for the storage node to obtain the target data based on the target data table
  • the consumption caused by providing the target data to the computing node, the second consumption is the consumption caused by the storage node providing the target data table to the computing node, and the computing node obtaining the target data based on the target data table.
  • the computing node may send the instruction information to the storage node.
  • the computing node can send instruction information to the storage node in the form of an I/O request. For example, the computing node sends a data acquisition request to the storage node, and the data acquisition request requests the storage node to provide target data indicated by the indication information to the computing node.
  • the target data is part of the data in the target data table indicated by the index creation instruction.
  • the target data may include data recorded in a specified data unit of the target data table, and the data unit may be a data row or a data column. data column.
  • the target data may be data recorded in a specified data column of the target data table.
  • the target data includes: partial data recorded in the designated data unit of the target data table.
  • this part of the data can be prefix data, intermediate data, and suffix data of the data recorded in the specified data unit. data or data of specified length.
  • the type of data recorded in the specified data unit is a string and/or a large object field
  • the target data It can be the prefix data of the data recorded in the data row corresponding to the specified data column of the target data table.
  • the storage node obtains the target data from the target data table and provides the target data to the computing node, which can further reduce storage
  • the amount of data transmitted by the node to the computing node reduces the computing load of the computing node, thereby further improving the efficiency of index creation and the computing performance of the computing node.
  • this application provides a method of creating an index.
  • This method is applied to database systems.
  • the database system includes: computing nodes and storage nodes.
  • the storage nodes store data tables.
  • the data tables record data with multiple rows and columns.
  • the method includes: the storage node determines the instruction information of the target data required to create the index, and the target data is part of the data in the target data table; the storage node obtains the target data from the target data table based on the instruction information, and provides the target data to the computing node Data; compute nodes create indexes based on target data.
  • the method before the storage node determines the instruction information of the target data required to create the index, the method also includes: the computing node receives the index creation instruction, determines the instruction information based on the index creation instruction, and provides the instruction information to the storage node, and the index is created
  • the directive instructs you to create an index based on the target data.
  • the computing node provides instruction information to the storage node, including: the computing node provides instruction information to the storage node when the first consumption and the second consumption corresponding to the target data meet specified conditions, and the first consumption is the storage node based on the target data.
  • the second consumption is the consumption caused by the storage node providing the target data table to the computing node, and the computing node obtaining the target data based on the target data table.
  • the computing node provides the indication information to the storage node, including: the computing node sends a data acquisition request to the storage node, and the data acquisition request is used to request the storage node to provide the target data indicated by the indication information to the computing node.
  • the target data includes: data recorded in the specified data unit of the target data table, and the data unit is a data row or a data column.
  • the target data includes: partial data recorded in the specified data unit of the target data table.
  • part of the data is prefix data, intermediate data, suffix data or data of a specified length of the data recorded in the specified data unit.
  • this application provides a storage node.
  • the storage node stores a data table, and the data table records data with multiple rows and columns.
  • the storage node includes: a processing module, used to determine the instruction information of the target data required to create the index, and the target data is part of the data in the target data table; the processing module is also used to obtain the target data from the target data table based on the instruction information. ;Interaction module, used to provide target data to computing nodes.
  • the interaction module is also used to receive instruction information provided by the computing node.
  • the interaction module is specifically configured to receive a data acquisition request sent by the computing node.
  • the data acquisition request is used to request the storage node to provide target data indicated by the indication information to the computing node.
  • the target data includes: data recorded in the specified data unit of the target data table, and the data unit is a data row or a data column.
  • the target data includes: partial data recorded in the specified data unit of the target data table.
  • part of the data is prefix data, intermediate data, suffix data or data of a specified length of the data recorded in the specified data unit.
  • this application provides a computing node.
  • the computing node includes: an interaction module, used to receive the target data sent by the storage node; a processing module, used to create an index based on the target data.
  • the target data is part of the data in the target data table stored in the storage node, and the target data table records data with multiple rows and columns.
  • the interaction module is also used to receive an index creation instruction, which instructs to create an index based on the target data; the processing module is also used to determine indication information based on the index creation instruction; and the interaction module is also used to provide indication information to the storage node.
  • the interaction module is specifically configured to provide instruction information to the storage node when the first consumption and the second consumption corresponding to the target data meet specified conditions.
  • the first consumption is that the storage node obtains the target data based on the target data table and provides it to the computing node.
  • the consumption caused by the node providing the target data, the second consumption is the consumption caused by the storage node providing the target data table to the computing node, and the computing node obtaining the target data based on the target data table.
  • the interaction module is specifically configured to send a data acquisition request to the storage node, and the data acquisition request is used to request the storage node to provide the target data indicated by the indication information to the computing node.
  • the target data includes: data recorded in the specified data unit of the target data table, and the data unit is a data row or a data column.
  • the target data includes: partial data recorded in the specified data unit of the target data table.
  • part of the data is prefix data, intermediate data, suffix data or data of a specified length of the data recorded in the specified data unit.
  • this application provides a method of creating an index.
  • This method applies to storage nodes.
  • the storage node stores a data table, and the data table records data with multiple rows and columns.
  • the method includes: the storage node determines the instruction information of the target data required to create the index, and the target data is part of the data in the target data table; the storage node obtains the target data from the target data table based on the instruction information, and provides the target data to the computing node data.
  • the method further includes: the storage node receives the instruction information provided by the computing node.
  • the storage node receives the instruction information provided by the computing node, including: the storage node receives a data acquisition request sent by the computing node, and the data acquisition request is used to request the storage node to provide the target data indicated by the instruction information to the computing node.
  • the target data includes: data recorded in the specified data unit of the target data table, and the data unit is a data row or a data column.
  • the target data includes: partial data recorded in the specified data unit of the target data table.
  • part of the data is prefix data, intermediate data, suffix data or data of a specified length of the data recorded in the specified data unit.
  • this application provides a method of creating an index.
  • This method applies to compute nodes.
  • the method includes: a computing node receives target data sent by a storage node, and creates an index based on the target data.
  • the target data is part of the data in the target data table stored in the storage node, and the target data table records data with multiple rows and columns.
  • the method before the computing node receives the target data sent by the storage node, the method also includes: the computing node receives the index creation instruction, determines the indication information based on the index creation instruction, and provides the instruction information to the storage node, and the index creation instruction indicates based on the target Data is indexed.
  • the computing node provides instruction information to the storage node, including: when the first consumption and the second consumption corresponding to the target data meet specified conditions, providing instruction information to the storage node.
  • the first consumption is obtained by the storage node based on the target data table.
  • the second consumption is the consumption caused by the storage node providing the target data table to the computing node, and the computing node obtaining the target data based on the target data table.
  • the computing node provides the instruction information to the storage node, including: sending a data acquisition request to the storage node.
  • the data acquisition request is used to request the storage node to provide the target data indicated by the instruction information to the computing node.
  • the target data includes: data recorded in the specified data unit of the target data table, and the data unit is a data row or a data column.
  • the target data includes: partial data recorded in the specified data unit of the target data table.
  • part of the data is prefix data, intermediate data, suffix data or data of a specified length of the data recorded in the specified data unit.
  • this application provides a computing device, including a memory and a processor.
  • the memory stores program instructions
  • the processor runs the program instructions to execute the method provided in the fifth aspect of this application and any possible implementation thereof. , or execute the method provided in the sixth aspect of this application and any possible implementation manner thereof.
  • the present application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory, and the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, So as to cause the computing device cluster to execute the method provided in the second aspect of the present application and any possible implementation manner thereof.
  • this application provides a computer program product containing instructions that, when executed by a computing device, cause the computing device to execute the method provided in the fifth aspect of this application and any possible implementation thereof, or, Execute the method provided in the sixth aspect of this application and any possible implementation manner thereof.
  • this application provides a computer program product containing instructions.
  • the computing device cluster executes the method provided in the second aspect of this application and any possible implementation manner thereof.
  • the present application provides a computer-readable storage medium.
  • the computer-readable storage medium is a non-volatile computer-readable storage medium.
  • the computer-readable storage medium includes computer program instructions. When the computer program instructions are When the computing device executes, the computing device executes the method provided in the fifth aspect of the application and any possible implementation manner thereof, or executes the method provided in the sixth aspect of the application and any possible implementation manner thereof.
  • the application provides a computer-readable storage medium.
  • the computer-readable storage medium is a non-volatile computer-readable storage medium.
  • the computer-readable storage medium includes computer program instructions. When the computer program instructions are When the computing device cluster executes, the computing device cluster executes the method provided in the second aspect of this application and any possible implementation manner thereof.
  • Figure 1 is a schematic diagram of the implementation environment involved in a method for creating an index provided by an embodiment of the present application
  • Figure 2 is a schematic structural diagram of a database system provided by an embodiment of the present application.
  • Figure 3 is a schematic structural diagram of another database system provided by an embodiment of the present application.
  • Figure 4 is a flow chart of a method for creating an index provided by an embodiment of the present application.
  • Figure 5 is a schematic structural diagram of a storage node provided by an embodiment of the present application.
  • Figure 6 is a schematic structural diagram of a computing node provided by an embodiment of the present application.
  • Figure 7 is a flow chart of a method for creating an index on a storage node provided by an embodiment of the present application.
  • Figure 8 is a flow chart of a method for creating an index on a computing node provided by an embodiment of the present application
  • Figure 9 is a schematic structural diagram of a computing device provided by an embodiment of the present application.
  • Figure 10 is a schematic structural diagram of a computing device cluster provided by an embodiment of the present application.
  • indexing through indexes can speed up querying and improve query performance. It is widely used in scenarios when users need to speed up querying.
  • the computing node when creating an index for a data table in the database, the computing node needs to read all the data of the data table from the storage node, and then create an index based on all the data of the data table.
  • the embodiments of the present application provide a method for creating an index and a database system for implementing the method.
  • the database system includes computing nodes and storage nodes.
  • a data table is stored in the storage node, and the data table records multiple rows and columns of data.
  • the storage node is used to determine the instruction information of the target data required to create the index, obtain the target data from the target data table based on the instruction information, and provide the target data to the computing node.
  • Compute nodes are used to create indexes based on target data. Among them, the target data is part of the data in the target data table.
  • the storage node obtains the target data required to create an index from the target data table and provides the target data to the computing node.
  • the storage node transmits the entire target data table recorded to the computing node. data, reducing the amount of data transmitted from storage nodes to computing nodes, helping to improve input/output (I/O) performance between storage nodes and computing nodes.
  • the computing node is not required to perform the operation of obtaining the target data from the target data table, which reduces the computing load of the computing node and helps to improve the efficiency of index creation by the computing node and the computing performance of the computing node.
  • Figure 1 is a schematic diagram of an implementation environment involved in a method for creating an index provided by an embodiment of the present application.
  • the implementation environment includes: client 01 and database system 02.
  • a data table is recorded in the database system 02.
  • Client 01 can establish a communication connection with database system 02.
  • a communication connection can be established between client 01 and database system 02 through the network.
  • the network can be a local area network, the Internet, or other networks, which are not limited in the embodiments of this application.
  • client 01 Users can interact with database system 02 through client 01.
  • the client 01 is used to send instructions to the database system 02 according to the user's instructions. For example, send an index creation command to instruct the database system to create an index.
  • a data access instruction is sent to instruct data access to the database system.
  • the client 01 can be a desktop computer, a laptop computer, a mobile phone, a smartphone, a tablet computer, a multimedia player, a smart home appliance, an artificial intelligence device, a smart wearable device, an e-reader, Smart vehicle equipment or Internet of Things equipment, etc.
  • Database system 02 is used to receive instructions sent by client 01 and perform operations indicated by the instructions.
  • the database system 02 is configured to receive an index creation instruction, and create an index based on the index creation instruction according to the index creation method provided in the embodiment of the present application.
  • the database system 02 is used to receive data access instructions and provide feedback based on the data access instructions.
  • the database system 02 can be implemented by a computing device.
  • the computing device can be a server (such as a cloud server).
  • the database system 02 can be a server cluster composed of several servers, or a cloud computing service center.
  • a large number of basic resources owned by cloud service providers are deployed in the cloud computing service center.
  • computing resources, storage resources, network resources, etc. are deployed in a cloud computing service center.
  • the cloud computing service center can utilize this large amount of basic resources to implement the index creation method provided in the embodiments of this application.
  • the cloud platform can use the resources in the cloud computing center to provide the user with the index creation method. Indexed cloud service. After users purchase the index-creating cloud service on the cloud platform, they can use the index-creating cloud service to create indexes for the data recorded in the database system to speed up the data search process.
  • the cloud platform may be a central cloud cloud platform, an edge cloud cloud platform, or a cloud platform including a central cloud and an edge cloud, which are not specifically limited in the embodiments of this application.
  • the database system 02 can also be implemented through other resource platforms besides the cloud platform, which is not specifically limited in the embodiments of this application. At this time, the database system 02 can be implemented through resources in other resource platforms and provide relevant database services to users.
  • FIG. 2 is a schematic structural diagram of a database system provided by an embodiment of the present application.
  • the database system 02 includes: a computing node 021 and a storage node 022.
  • the storage node 022 stores a data table, and the data table records data with multiple rows and columns.
  • the computing node 021 and the storage node 022 may be implemented by computing devices in a computing device cluster.
  • the computing device cluster is also configured with a storage pool, and the storage pool is used to store data tables.
  • the storage node 022 is used to determine the instruction information of the target data required to create an index, and the target data is part of the data in the target data table.
  • the storage node 022 is also used to obtain target data from the target data table based on the instruction information, and provide the target data to the computing node 021.
  • Compute node 021 is used to create indexes based on target data.
  • the computing node 021 is also configured to receive an index creation instruction, determine instruction information based on the index creation instruction, and provide instruction information to the storage node 022.
  • the index creation instruction instructs to create an index based on the target data.
  • the computing node 021 is specifically configured to provide instruction information to the storage node 022 when the first consumption and the second consumption corresponding to the target data meet specified conditions.
  • the first consumption is that the storage node 022 obtains the target data based on the target data table, And the consumption generated by providing the target data to the computing node 021.
  • the second consumption is the consumption generated by the storage node 022 providing the target data table to the computing node 021, and the computing node 021 obtaining the target data based on the target data table.
  • the computing node 021 is specifically configured to send a data acquisition request to the storage node 022, and the data acquisition request is used to request the storage node 022 to provide the target data indicated by the indication information to the computing node 021.
  • the target data includes: data recorded in the specified data unit of the target data table, and the data unit is a data row or a data column.
  • the target data includes: partial data recorded in the specified data unit of the target data table.
  • part of the data is prefix data, intermediate data, suffix data or data of a specified length of the data recorded in the specified data unit.
  • the database system 02 may also include other nodes to implement other functions or to assist in realizing the functions of the storage node 022 and the computing node 021.
  • the database system 02 may also include a management node 023, which is used to manage the storage node 022 and the computing node 021 to ensure that the database system 02 can operate normally.
  • the storage node 022, the computing node 021, and the management node 023 can all be implemented by software, or can be implemented by hardware.
  • the implementation of storage node 022 is introduced next.
  • the implementation of the computing node 021 and the management node 023 can refer to the implementation of the storage node 022.
  • storage node 022 may include code running on a computing instance.
  • the computing instance may be at least one of a physical host (computing device), a virtual machine, a container, and other computing devices. Further, the above computing device may be one or more.
  • storage node 022 may include code running on multiple hosts/virtual machines/containers. It should be noted that multiple hosts/virtual machines/containers used to run the application can be distributed in the same region (region) or in different regions. Multiple hosts/VMs/containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple geographical locations Close data center. Among them, usually a region Can include multiple AZs.
  • AZ availability zone
  • the multiple hosts/VMs/containers used to run the code can be distributed in the same virtual private cloud (VPC), or across multiple VPCs.
  • VPC virtual private cloud
  • Cross-region communication between two VPCs in the same region or between VPCs in different regions requires a communication gateway in each VPC, and the interconnection between VPCs is realized through the communication gateway.
  • the storage node 022 may include at least one computing device, such as a server.
  • the storage node 022 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD).
  • ASIC application-specific integrated circuit
  • PLD programmable logic device
  • the above-mentioned PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (field-programmable gate array, FPGA), a general array logic (generic array logic, GAL), or any combination thereof.
  • CPLD complex programmable logical device
  • FPGA field-programmable gate array
  • GAL general array logic
  • Multiple computing devices included in the storage node 022 may be distributed in the same region or in different regions. Multiple computing devices included in the storage node 022 may be distributed in the same AZ or in different AZs. Similarly, multiple computing devices included in the storage node 022 may be distributed in the same VPC or in multiple VPCs. The multiple computing devices may be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.
  • the storage node obtains the target data required to create an index from the target data table and provides the target data to the computing node.
  • the data recorded in the target data table reduces the amount of data transmitted from the storage node to the computing node and helps improve the I/O performance between the storage node and the computing node.
  • the computing node is not required to perform the operation of obtaining the target data from the target data table, which reduces the computing load of the computing node and helps to improve the efficiency of index creation by the computing node and the computing performance of the computing node.
  • Figure 4 is a flow chart of a method for creating an index through the database system provided by an embodiment of the present application. As shown in Figure 4, the method includes the following steps:
  • Step 401 The computing node receives the index creation instruction, and determines the instruction information of the target data required to create the index based on the index creation instruction.
  • the index creation instruction instructs to create an index based on the target data, and the target data is the part of the target data table indicated by the index creation instruction. data.
  • a user When a user needs to create an index for database data, he or she can perform a specified operation on the client used by the user to trigger an index creation instruction, and the index creation instruction is used to instruct the index to be created.
  • the client After the client generates the index creation instruction based on the user's operation, it will send the index creation instruction to the database, and the computing node can receive the index creation instruction.
  • Indexes are created in the database to facilitate searching of data in the database.
  • Index creation instructions indicate the data on which the index needs to be created.
  • the indication may be represented by indication information.
  • the computing node can obtain the instruction information based on the index creation instruction and obtain the instruction information of the target data required to create the index.
  • the index creation instruction can carry indication information, and the computing node can extract the indication information from the index creation instruction.
  • the index creation instruction can be ALTER TABLE t1ADD KEY(a, c).
  • the index creation instruction instructs to create an index based on data column a and data column c of data table t1, that is, the instruction information indicates data column a and data of data table t1. If the data in column c is specified, the computing node can extract the indication information from the index creation instruction.
  • the index needs to be created based on the primary key data in the data table.
  • the data indicated by the index creation instruction may or may not include primary key data.
  • the index creation instruction does not indicate the primary key data
  • the computing node receives the index creation instruction, it can also determine the primary key data required to create the index based on the data indicated by the index creation instruction, and
  • the indication information carries information indicating the primary key data.
  • the index creation instruction is ALTER TABLE t1ADD KEY(a, c). This index creation instruction instructs to create an index based on data column a and data column c of data table t1.
  • this scenario requires creating an index based on the primary key data of data table t1, and the data column d of data table t1 is used to record the primary key data of data table t1. Then the instruction information obtained by the computing node not only indicates the data column a of data table t1. and the data in data column c, also indicates the data in data column d.
  • the embodiments of the present application can also be applied to scenarios where primary key data is not required for index creation.
  • an index can be created based on some identification information without creating an index based on primary key data.
  • the embodiments of the present application can also be applied to these scenarios.
  • the target data is part of the data in the target data table indicated by the index creation instruction.
  • the target data may include data recorded in a specified data unit of the target data table, and the data unit may be a data row or a data column. data column.
  • the target data can be the data recorded in the specified data column of the target data table.
  • the target data may specifically include: partial data recorded in the specified data unit of the target data table.
  • the part of the data may be prefix data, intermediate data, suffix data or data of a specified length of the data recorded in the specified data unit.
  • the target data It can be the prefix data of the data recorded in the data row corresponding to the specified data column of the target data table.
  • the index creation instruction can be ALTER TABLE t1ADD KEY(a(10),c).
  • This index creation instruction instructs to create an index based on the prefix data of length 10 and data column c in the data row corresponding to data column a of data table t1. , that is, the instruction information indicates the prefix data with a length of 10 in the data row corresponding to data column a of data table t1 and the data in data column c.
  • the storage node obtains the target data from the target data table and provides the target data to the computing node, which can further reduce storage
  • the amount of data transmitted by the node to the computing node reduces the computing load of the computing node, thereby further improving the efficiency of index creation and the computing performance of the computing node.
  • Step 402 The computing node provides instruction information to the storage node.
  • the computing node After the computing node obtains the indication information, it can provide the indication information to the storage node, so that the storage node provides the target data indicated by the indication information to the computing node. Alternatively, after determining the indication information, the computing node may determine whether to send the indication information to the storage node based on the indication information, and when it is necessary to obtain the target data through the storage node, provide the indication information to the storage node. In one implementation, the computing node can obtain the first consumption and the second consumption corresponding to the target data, and when the first consumption and the second consumption meet specified conditions, send the indication information to the storage node.
  • the instruction information is not sent to the storage node, but the storage node is requested to provide the entire target data table recording the target data to the computing node.
  • the first consumption is the consumption caused by the storage node obtaining the target data based on the target data table and providing the target data to the computing node.
  • the second consumption is the consumption caused by the storage node providing the target data table to the computing node, and the computing node obtains the target based on the target data table. Consumption of data generation.
  • the specified conditions can be determined according to application requirements. For example, the specified condition may be that the first consumption is less than the second consumption.
  • the specified condition can be the difference between the second consumption and the first consumption. The value is greater than the specified threshold.
  • the computing node can send instruction information to the storage node in the form of an I/O request.
  • the computing node sends a data acquisition request to the storage node, and the data acquisition request requests the storage node to provide target data indicated by the indication information to the computing node.
  • the storage node can support near data processing (NDP) technology, and the computing node and the storage node can communicate through the NDP context structure.
  • NDP near data processing
  • the computing node determines the indication information it can construct an NDP context structure based on the indication information, so as to transmit the indication information through the NDP context structure.
  • the data acquisition request may include the NDP context structure and content instructing the storage node to initiate a full table scan of the target data table.
  • Step 403 The storage node determines the instruction information of the target data required to create the index.
  • the storage node After receiving the instruction information, the storage node can obtain the target data from the target data table indicated by the instruction information based on the instruction information, and provide the target data to the computing node. For example, when the data acquisition request sent by the computing node includes the NDP context structure and content instructing the storage node to initiate a full table scan of the target data table, the storage node can obtain the instruction from the NDP context structure after receiving the data acquisition request. Information, it is determined that a full table scan of the target data table is required and the target data is obtained.
  • Step 404 The storage node obtains target data from the target data table based on the instruction information.
  • the storage node After the storage node obtains the instruction information, it can perform a full table scan on the target data table indicated by the instruction information, and traverse the data rows and data columns in each relevant page of the target data table to obtain the target data.
  • the storage node can use NDP technology to obtain target data from the target data table.
  • NDP technology is generally considered for use in database selection (SELECT) and data manipulation language (DML) scenarios, but whether it can be applied in SELECT and DML scenarios also depends on the structure used in the scenario. Structured query language (SQL).
  • SQL Structured query language
  • the NDP technology can be used to obtain target data from the target data table without considering too many factors. Its usage scenario coverage is basically 100%, which can significantly Reduce the I/O in the data table reading phase. This effect is particularly obvious in scenarios where multiple users share storage and computing resources (multi-tenant scenarios on cloud platforms).
  • the operation of the storage node to obtain the target data from the target data table can be called a pruning operation of the target data table.
  • the operation of obtaining the target data from the target data table may be called a column pruning operation.
  • the operation of obtaining the target data from the target data table may be called a column content clipping operation.
  • the storage node performs a clipping operation on the target data table to obtain the target data and provides the target data to the computing node, which can reduce the amount of data transmitted from the storage node to the computing node and reduce the amount of I/O between the storage node and the computing node. Reduce the I/O overhead between storage nodes and computing nodes and achieve accurate I/O between storage nodes and computing nodes. This effect is especially obvious when obtaining target data through column content clipping.
  • Step 405 The storage node provides target data to the computing node.
  • the storage node After the storage node obtains the target data, it can store the target data in a data structure and provide the data structure to the computing node, so that the computing node obtains the target data. After the storage node obtains the target data from each page of the target data table through the clipping operation, it can provide the clipped page to the data node.
  • the clipped page can be regarded as a compressed page of the original page, but through this application
  • the compressed page obtained by the cropping operation provided in the embodiment is different from the compressed page obtained by the compression technology.
  • the data in the compressed page obtained by cropping only includes the data in the data unit specified by the indication information, while the compressed page obtained by the compression technology may also include data in other data units.
  • the compression rate of the compressed page obtained by cropping relative to the target data table can be significantly greater than the compression rate of the compressed page obtained by the compression technology.
  • Step 406 The computing node creates an index based on the target data.
  • the computing node After the computing node obtains the target data provided by the storage node, it can create an index based on the target data. For example, when the target data is the data recorded in the specified data column of the target data table, after the computing node obtains the target data, it can read the data recorded in each data column from each data row of the target data, and calculate the data based on the target data. The data recorded in each data row and each data column constitutes a sub-record and is stored independently. Then all sub-records are sorted, and the indexing operation is performed based on the sorted sub-records.
  • the index can be stored in a data structure such as a B-tree.
  • the computing node when the target data is the data recorded in the specified data column of the target data table, and the target data is presented in the form of a compressed page, after the computing node receives the compressed page sent by the storage node, it can parse the data rows of the compressed page to Get target data.
  • the computing node By obtaining the target data from the storage node, the computing node does not need to perform the operation of obtaining the target data from the target data table, which reduces the computing load of the computing node and helps to improve the efficiency of index creation by the computing node and the computing performance of the computing node.
  • data table t1 includes 24 data columns, namely data columns a, b, c,...x.
  • the string length stored in data column a and data column b is 100 bytes, and the remaining data columns store The string length is 4, and the data recorded in data column c is the primary key data.
  • the storage node can remove the data columns b, d, e,..., x in the data table t1 through column pruning, and obtain the targets recorded in the data columns a and c.
  • the storage node can remove data columns b, d, e,...,x and the 11th to 100 bytes in data column a in data table t1 through column pruning and column content pruning, and obtain data column a.
  • the prefix data with a length of 10 and the target data recorded in the data column c, for each data row, the total length of the target data is 10+4 14 bytes.
  • the width of the data table is wider, for example, the fields recorded in the data table are more complex, and the data table includes more data columns, the amount of I/O that can be reduced through the optimization methods provided by the embodiments of this application will be greater.
  • the data lines were not Header information is included. Since the data volume of header information is generally small, and column trimming or column content trimming of header information can also reduce the data volume (such as 6-8 bytes), so even if it is included in the header Information can still achieve better results through this optimization.
  • the storage node is used to determine the instruction information of the target data required to create the index, and based on the instruction information, obtain the target data from the target data table and provide it to the calculation Nodes provide target data, and compute nodes are used to create indexes based on the target data.
  • the target data is part of the data in the target data table.
  • the storage node obtains the target data required to create an index from the target data table and provides the target data to the computing node. Compared with related technologies, there is no need for the storage node to transmit the entire target data table recorded to the computing node.
  • the computing node is not required to perform the operation of obtaining the target data from the target data table, which reduces the computing load of the computing node and helps to improve the efficiency of index creation by the computing node and the computing performance of the computing node.
  • An embodiment of the present application also provides a storage node.
  • the storage node stores a data table, and the data table records data with multiple rows and columns.
  • the storage node 022 includes:
  • Processing module 0221 is used to determine the instruction information of the target data required to create the index.
  • the target data is part of the data in the target data table.
  • the processing module 0221 is also used to obtain target data from the target data table based on the instruction information.
  • Interaction module 0222 used to provide target data to computing nodes.
  • the interaction module 0222 is also used to receive indication information provided by the computing node.
  • the interaction module 0222 is specifically configured to receive a data acquisition request sent by the computing node.
  • the data acquisition request is used to request the storage node to provide the target data indicated by the indication information to the computing node.
  • the target data includes: data recorded in the specified data unit of the target data table, and the data unit is a data row or a data column.
  • the target data includes: partial data recorded in the specified data unit of the target data table.
  • part of the data is prefix data, intermediate data, suffix data or data of a specified length of the data recorded in the specified data unit.
  • computing node 021 includes:
  • the interaction module 0211 is used to receive target data sent by the storage node.
  • the target data is part of the data in the target data table stored in the storage node.
  • the target data table records multiple rows and columns of data.
  • Processing module 0212 used to create indexes based on target data.
  • the interaction module 0211 is also configured to receive an index creation instruction, which instructs to create an index based on the target data.
  • the processing module 0212 is also used to determine indication information based on the index creation instruction.
  • the interaction module 0211 is also used to provide instruction information to the storage node.
  • the interaction module 0211 is specifically configured to provide instruction information to the storage node when the first consumption and the second consumption corresponding to the target data meet the specified conditions.
  • the first consumption is that the storage node obtains the target data based on the target data table and sends it to the storage node.
  • the consumption caused by the computing node providing the target data, the second consumption is the consumption caused by the storage node providing the target data table to the computing node, and the computing node obtaining the target data based on the target data table.
  • the interaction module 0211 is specifically configured to send a data acquisition request to the storage node.
  • the data acquisition request is used to request the storage node to provide the target data indicated by the indication information to the computing node.
  • the target data includes: data recorded in the specified data unit of the target data table, and the data unit is a data row or a data column.
  • the target data includes: partial data recorded in the specified data unit of the target data table.
  • part of the data is prefix data, intermediate data, suffix data or data of a specified length of the data recorded in the specified data unit.
  • processing module 0221, the interaction module 0222, the interaction module 0211 and the processing module 0212 can all be implemented by software, or can be implemented by hardware.
  • the following takes processing module 0221 as an example to introduce the implementation of processing module 0221. Similar, pay For the implementation of mutual module 0222, interaction module 0211 and processing module 0212, please refer to the implementation of processing module 0221.
  • the processing module 0221 may include code running on a computing instance.
  • the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the above computing instance may be one or more.
  • processing module 0221 may include code running on multiple hosts/virtual machines/containers. It should be noted that multiple hosts/virtual machines/containers used to run the code can be distributed in the same region or in different regions. Furthermore, multiple hosts/virtual machines/containers used to run the code can be distributed in the same AZ or in different AZs. Each AZ includes one data center or multiple geographically close data centers. . Among them, usually a region can include multiple AZs.
  • the multiple hosts/VMs/containers used to run the code can be distributed in the same virtual private cloud (VPC), or across multiple VPCs.
  • VPC virtual private cloud
  • Cross-region communication between two VPCs in the same region and between VPCs in different regions requires a communication gateway in each VPC, and the interconnection between VPCs is realized through the communication gateway. .
  • the processing module 0221 may include at least one computing device, such as a server.
  • the processing module 0221 may also be a device implemented using ASIC, or a PLD, or the like.
  • the above-mentioned PLD can be implemented by CPLD, FPGA, GAL or any combination thereof.
  • Multiple computing devices included in the processing module 0221 may be distributed in the same region or in different regions. Multiple computing devices included in the processing module 0221 may be distributed in the same AZ or in different AZs. Similarly, multiple computing devices included in the processing module 0221 may be distributed in the same VPC or in multiple VPCs. The multiple computing devices may be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.
  • the processing module 0221 can be used to perform any steps performed by the storage node in the method of creating an index
  • the interaction module 0222 can be used to perform any steps performed by the storage node in the method of creating an index
  • Interaction Module 0211 can be used to perform any steps performed by the computing node in the method of creating an index
  • processing module 0212 can be used to perform any steps performed by the computing node in the method of creating an index.
  • the steps that the processing module 0221, the interaction module 0222, the interaction module 0211 and the processing module 0212 are responsible for can be specified as needed.
  • the processing module 0221 and the interaction module 0222 respectively implement different steps in the method of creating an index to realize all the functions of the storage node.
  • the interaction module 0211 and the processing module 0212 respectively implement different steps in the method of creating an index to realize all functions of the computing node.
  • the embodiment of this application also provides a method for creating an index.
  • This method applies to storage nodes.
  • the storage node stores a data table, and the data table records data with multiple rows and columns.
  • the method includes:
  • Step 701 The storage node determines the instruction information of the target data required to create an index.
  • the target data is part of the data in the target data table.
  • Step 702 The storage node obtains target data from the target data table based on the indication information, and provides the target data to the computing node.
  • the method before the storage node determines the instruction information of the target data required to create the index, the method also includes: step 703: the storage node receives the instruction information provided by the computing node.
  • the storage node receives the instruction information provided by the computing node, including: the storage node receives a data acquisition request sent by the computing node, and the data acquisition request is used to request the storage node to provide the target data indicated by the instruction information to the computing node.
  • the target data includes: data recorded in the specified data unit of the target data table, and the data unit is a data row or a data column.
  • the target data includes: partial data recorded in the specified data unit of the target data table.
  • part of the data is prefix data, intermediate data, suffix data or data of a specified length of the data recorded in the specified data unit.
  • the embodiment of this application also provides a method for creating an index. This method applies to compute nodes. As shown in Figure 8, the method includes:
  • Step 801 The computing node receives the target data sent by the storage node.
  • the target data is part of the data in the target data table stored in the storage node.
  • the target data table records multiple rows and columns of data.
  • Step 802 The computing node creates an index based on the target data.
  • the method before the computing node receives the target data sent by the storage node, the method also includes: step 803.
  • the computing node receives the index creation instruction, determines the indication information based on the index creation instruction, and provides the storage node with Instruction information, the index creation instruction instructs to create an index based on the target data.
  • the computing node provides instruction information to the storage node, including: when the first consumption and the second consumption corresponding to the target data meet the specified When conditions exist, instruction information is provided to the storage node.
  • the first consumption is when the storage node obtains the target data based on the target data table and provides the target data to the computing node.
  • the second consumption is when the storage node provides the target data table to the computing node. The computing node obtains the consumption of target data based on the target data table.
  • the computing node provides the instruction information to the storage node, including: sending a data acquisition request to the storage node.
  • the data acquisition request is used to request the storage node to provide the target data indicated by the instruction information to the computing node.
  • the target data includes: data recorded in the specified data unit of the target data table, and the data unit is a data row or a data column.
  • the target data includes: partial data recorded in the specified data unit of the target data table.
  • part of the data is prefix data, intermediate data, suffix data or data of a specified length of the data recorded in the specified data unit.
  • FIG. 9 is a schematic structural diagram of a computing device provided by an embodiment of the present application.
  • the computing device 900 includes a processor 901 , a memory 902 , a communication interface 903 and a bus 904 .
  • the processor 901, the memory 902, and the communication interface 903 implement communication connections between each other through the bus 904.
  • Computing device 900 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 900.
  • Processor 901 may include a general-purpose processor and/or a special-purpose hardware chip.
  • a general-purpose processor may include any one or more of a central processing unit (CPU), a microprocessor (MP), or a graphics processing unit (GPU).
  • the CPU is, for example, a single-core processor (single-CPU) or a multi-core processor (multi-CPU).
  • a dedicated hardware chip is a high-performance processing hardware module. Specialized hardware chips include digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or network processor (network processor, at least one of NP).
  • the processor 901 may also be an integrated circuit chip with signal processing capabilities. During the implementation process, part or all of the functions of the method of creating an index in this application can be completed by instructions in the form of hardware integrated logic circuits or software in the processor 901 .
  • the memory 902 is used to store computer programs, which include an operating system 902a and executable codes (ie, program instructions) 902b.
  • the memory 902 is, for example, a read-only memory or other type of static storage device that can store static information and instructions, or a random access memory or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable memory device.
  • the memory 902 is used to store outbound port queues, etc.
  • the memory 902 exists independently, for example, and is connected to the processor 901 through a bus 904 . Or the memory 902 and the processor 901 are integrated together.
  • memory 902 may include volatile memory, such as random access memory (RAM).
  • the processor 901 may also include non-volatile memory (non-volatile memory), such as read-only memory (ROM), flash memory, mechanical hard disk (hard disk drive, HDD) or solid state drive (solid state drive, SSD).
  • ROM read-only memory
  • HDD hard disk drive
  • SSD solid state drive
  • the memory 902 can store executable code.
  • the processor 901 is used to implement the function of the aforementioned storage node, thereby realizing the storage in the method of creating an index provided by the embodiment of the present application.
  • Some or all of the functionality of the node that is to say, the memory 902 stores instructions for implementing part or all of the functions of the storage node in the method of creating an index, or the processor 901 is used to implement the functions of the aforementioned computing node, thereby realizing the index creation provided by the embodiment of the present application. Compute some or all of the functionality of a node in a method.
  • the memory 902 stores instructions for implementing part or all of the functions of the computing node in the method of creating an index.
  • the memory 902 may also include operating systems and other software modules and data required for running processes.
  • the communication interface 903 uses transceiver modules such as, but not limited to, network interface cards and transceivers to implement communication with other devices or communication networks.
  • the communication interface 903 may be any one or any combination of the following devices: a network interface (such as an Ethernet interface), a wireless network card, and other devices with network access functions.
  • Bus 904 is any type of communication bus used to interconnect internal components of a computing device (eg, memory 902, processor 901, communication interface 903).
  • bus 904 may be a peripheral component interconnect (PCI) standard Bus or extended industry standard architecture (EISA) bus, etc.
  • the bus can be divided into address bus, data bus, control bus, etc. For ease of presentation, only one line is used in Figure 9, but it does not mean that there is only one bus or one type of bus.
  • Bus 904 may include a path that carries information between various components of computing device 900 (eg, memory 902, processor 901, communications interface 903).
  • the embodiment of the present application takes the interconnection of the above-mentioned devices inside the computing device through the bus 904 as an example.
  • the above-mentioned devices inside the computing device 900 may also use other connection methods besides the bus 904 to communicate with each other.
  • the above-described devices within the computing device 900 are interconnected through internal logical interfaces.
  • the above-mentioned plurality of devices can be respectively arranged on independent chips, or at least part or all of them can be arranged on the same chip. Whether each device is independently installed on different chips or integrated on one or more chips often depends on the needs of product design.
  • the embodiments of this application do not limit the specific implementation forms of the above devices.
  • the descriptions of the processes corresponding to each of the above drawings have different emphases. For parts that are not detailed in a certain process, you can refer to the relevant descriptions of other processes.
  • An embodiment of the present application also provides a computing device cluster.
  • the computing device cluster includes at least one computing device.
  • the computing device may be a server, such as a central server, an edge server, or a local server in a local data center.
  • the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.
  • the structure of at least one computing device included in the computing device cluster may refer to the computing device 900 shown in FIG. 9 .
  • the same instructions for performing the method of creating an index may be stored in the memory 902 of one or more computing devices 900 in a cluster of computing devices.
  • the memory 902 of one or more computing devices 900 in the computing device cluster may also store part of the instructions for executing the method of creating an index.
  • a combination of one or more computing devices 900 may collectively execute instructions for performing a method of creating an index.
  • the memories 902 in different computing devices 900 in the computing device cluster may store different instructions, respectively used to implement part of the functions of the method of creating an index. That is, the instructions stored in the memory 902 in different computing devices 900 can implement the functions of one or more modules in the storage node and the computing node.
  • one or more computing devices in a cluster of computing devices may be connected through a network.
  • the network can be a wide area network or a local area network, etc.
  • Figure 10 shows a possible implementation.
  • two computing devices 1000A and 1000B are connected through a network.
  • the connection to the network is made through a communication interface in each computing device.
  • computing devices 1000A and 1000B include a bus 1002, a processor 1004, a memory 1006, and a communication interface 1008.
  • Memory 1006 in computing device 1000A stores instructions that implement the functions of the storage node.
  • the memory 1006 in the computing device 1000B stores instructions for implementing the functions of the computing node.
  • connection method between the computing device clusters shown in Figure 10 can be: considering that the index creation method provided by this application requires the transmission of target data and indication information, it is considered that the functions implemented by the storage nodes are handed over to the computing device 1000A, and The functions implemented by the computing nodes are executed by the computing device 1000B.
  • computing device 1000A shown in FIG. 10 may also be performed by multiple computing devices 1000.
  • computing device 1000B may also be performed by multiple computing devices 1000 .
  • An embodiment of the present application also provides a computer-readable storage medium, which is a non-volatile computer-readable storage medium.
  • Computer-readable storage media can be any available media that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. Available media may be magnetic media (eg, floppy disk, hard disk, tape), optical media (eg, DVD), or semiconductor media (eg, solid state drive), etc.
  • the computer-readable storage medium includes instructions that instruct the computing device to execute the method for creating an index provided by embodiments of the present application.
  • An embodiment of the present application also provides a computer program product containing instructions.
  • a computer program product may be a software or program product containing instructions capable of running on a computing device or stored in any available medium.
  • the computer program product is run on at least one computing device, the computer is caused to implement the method for creating an index provided by the embodiment of the present application.
  • Embodiments of the present application also provide a chip, including a processor, configured to call and run instructions stored in the memory, so that the computing device installed with the chip executes the method for creating an index provided by the embodiment of the present application.
  • a computer program product includes one or more computer instructions.
  • the computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
  • Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means.
  • the computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. Available media can be magnetic media (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive Solid State Disk), etc.
  • Computer program codes for implementing the methods of embodiments of the present application may be written in one or more programming languages. These computer program codes may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable rule-finding device, such that the program code, when executed by the computer or other programmable rule-finding device, causes the flowcharts and/or block diagrams to be displayed. The functions/operations specified in are implemented.
  • the program code may execute entirely on the computer, partly on the computer, as a stand-alone software package, partly on the computer and partly on a remote computer or entirely on the remote computer or server.
  • the computer program code or related data may be carried by any appropriate carrier, so that the device, device or processor can perform the various processes and operations described above.
  • carriers include signals, computer-readable media, and the like.
  • signals may include electrical, optical, radio, acoustic, or other forms of propagated signals, such as carrier waves, infrared signals, and the like.
  • the disclosed systems, devices and methods can be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the modules is only a logical function division. In actual implementation, there may be other division methods.
  • multiple modules or components may be combined or may be Integrated into another system, or some features can be ignored, or not implemented.
  • the coupling or direct coupling or communication connection between each other shown or discussed may be indirect coupling or communication connection through some interfaces, devices or modules, or may be electrical, mechanical or other forms of connection.
  • the modules described as separate components may or may not be physically separated.
  • the components shown as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.
  • each functional module in each embodiment of the present application can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or software function modules.
  • first, second and other words are used to distinguish the same or similar items with basically the same functions and functions. It should be understood that the terms “first”, “second” and “nth” There is no logical or sequential dependency, and there is no limit on the number or execution order. It should also be understood that, although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, a first link may be referred to as a second link, and similarly, a second link may be referred to as a first link, without departing from the scope of various examples.
  • the size of the sequence number of each process does not mean the order of execution.
  • the execution order of each process should be determined by its function and internal logic, and should not be determined by the execution order of the embodiments of the present application.
  • the implementation process constitutes no limitation.
  • determining B based on A does not mean determining B only based on A, and B can also be determined based on A and/or other information.
  • references throughout this specification to "one embodiment,” “an embodiment,” and “a possible implementation” mean that specific features, structures, or characteristics related to the embodiment or implementation are included herein. In at least one embodiment of the application. Therefore, “in one embodiment” or “in an embodiment” or “a possible implementation” appearing in various places throughout this specification do not necessarily refer to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments.
  • the information including but not limited to user equipment information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • signals involved in this application All are authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.
  • the instruction information and target data involved in this application were obtained with full authorization.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Fuzzy Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本申请公开了一种创建索引的方法及数据库系统，属于数据库技术领域。该数据库系统包括：计算节点和存储节点，存储节点存储有数据表，数据表记载有多行和多列数据。在该数据库系统中，存储节点用于确定创建索引所需的目标数据的指示信息，目标数据为目标数据表中的部分数据；存储节点还用于基于指示信息，从目标数据表中获取目标数据，并向计算节点提供目标数据；计算节点用于基于目标数据创建索引。本申请有助于提高创建索引的效率。

Description

创建索引的方法及数据库系统
本申请要求于2022年09月15日提交的申请号为202211120347.0、发明名称为“创建索引的方法及装置”的中国专利申请的优先权，以及于2022年11月01日提交的申请号为202211358798.8、发明名称为“创建索引的方法及数据库系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及数据库技术领域，特别涉及一种创建索引的方法及数据库系统。
背景技术
为数据库中的数据创建索引，是一种最常用的数据库定义语义(data definition language，DDL)操作，是数据库系统运维过程中经常会发生的操作。通过索引进行查询能够加快查询速度，带来查询性能上的提升，其广泛使用于当用户需要加速查询的场景。
在目前的数据库系统中，在为数据库中的数据表创建索引时，计算节点需要从存储节点中读取数据表的全部数据，然后根据数据表的全部数据创建索引。
但是，目前创建索引的效率较低。
发明内容
本申请提供了一种创建索引的方法及数据库系统。本申请有助于提高创建索引的效率。本申请提供的技术方案如下：
第一方面，本申请提供了一种数据库系统。该数据库系统包括：计算节点和存储节点，存储节点存储有数据表，数据表记载有多行和多列数据。在该数据库系统中，存储节点用于确定创建索引所需的目标数据的指示信息，目标数据为目标数据表中的部分数据；存储节点还用于基于指示信息，从目标数据表中获取目标数据，并向计算节点提供目标数据；计算节点用于基于目标数据创建索引。
在该数据库系统中，由存储节点从目标数据表中获取创建索引所需的目标数据，并向计算节点提供目标数据，相较于相关技术，无需存储节点向计算节点传输整个目标数据表记载的数据，减少了存储节点向计算节点传输的数据量，有助于提高存储节点和计算节点之间的I/O性能。并且，也不需要计算节点执行从目标数据表中获取目标数据的操作，降低了计算节点的计算负载，有助于提高计算节点创建索引的效率和计算节点的计算性能。
在一种实现方式中，计算节点还用于接收索引创建指令，基于索引创建指令确定指示信息，并向存储节点提供指示信息，索引创建指令指示基于目标数据创建索引。
可选的，计算节点在确定指示信息后，可以基于指示信息确定是否向存储节点发送指示信息，在有必要通过存储节点获取目标数据时，向存储节点提供该指示信息。在一种实现方式中，计算节点具体用于在目标数据对应的第一消耗和第二消耗满足指定条件时，向存储节点提供指示信息，第一消耗为存储节点基于目标数据表获取目标数据，并向计算节点提供目标数据产生的消耗，第二消耗为存储节点向计算节点提供目标数据表，由计算节点基于目标数据表获取目标数据产生的消耗。
在本申请中，计算节点向存储节点发送指示信息的实现方式可以有多种。在一种可实现方式中，计算节点可以以I/O请求的方式，向存储节点发送指示信息。例如，计算节点向存储节点发送数据获取请求，该数据获取请求请求存储节点向计算节点提供指示信息指示的目标数据。
在本申请中，目标数据为索引创建指令指示的目标数据表中的部分数据。在一种实现方式中，当数据表采用数据行和数据列对具有不同属性的数据进行区分时，目标数据可以包括目标数据表的指定数据单元中记载的数据，该数据单元可以为数据行或数据列。例如，目标数据可以为目标数据表的指定数据列中记载的数据。
进一步地，目标数据包括：目标数据表的指定数据单元中记载的部分数据。
在一种可实现方式中，该部分数据可以为指定数据单元中记载的数据的前缀数据、中间数据、后缀数 据或指定长度的数据。例如，当指定数据单元中记载的数据的类型为字符串和/或大对象类字段时，在创建索引时，可以仅基于字符串和/或大对象类字段的前缀数据创建索引，则目标数据可以为目标数据表的指定数据列对应的数据行中记载的数据的前缀数据。
当目标数据为索引创建指令指示的目标数据表中指定数据单元中记载的部分数据时，通过存储节点从目标数据表中获取该目标数据，并向计算节点提供该目标数据，能够进一步地减少存储节点向计算节点传输的数据量，降低计算节点的计算负载，从而进一步提高创建索引的效率和计算节点的计算性能。
第二方面，本申请提供了一种创建索引的方法。该方法应用于数据库系统。数据库系统包括：计算节点和存储节点，存储节点存储有数据表，数据表记载有多行和多列数据。该方法包括：存储节点确定创建索引所需的目标数据的指示信息，目标数据为目标数据表中的部分数据；存储节点基于指示信息，从目标数据表中获取目标数据，并向计算节点提供目标数据；计算节点基于目标数据创建索引。
可选的，在存储节点确定创建索引所需的目标数据的指示信息之前，该方法还包括：计算节点接收索引创建指令，基于索引创建指令确定指示信息，并向存储节点提供指示信息，索引创建指令指示基于目标数据创建索引。
可选的，计算节点向存储节点提供指示信息，包括：计算节点在目标数据对应的第一消耗和第二消耗满足指定条件时，向存储节点提供指示信息，第一消耗为存储节点基于目标数据表获取目标数据，并向计算节点提供目标数据产生的消耗，第二消耗为存储节点向计算节点提供目标数据表，由计算节点基于目标数据表获取目标数据产生的消耗。
可选的，计算节点向存储节点提供指示信息，包括：计算节点向存储节点发送数据获取请求，数据获取请求用于请求存储节点向计算节点提供指示信息指示的目标数据。
可选的，目标数据包括：目标数据表的指定数据单元中记载的数据，数据单元为数据行或数据列。
可选的，目标数据包括：目标数据表的指定数据单元中记载的部分数据。
可选的，部分数据为指定数据单元中记载的数据的前缀数据、中间数据、后缀数据或指定长度的数据。
可选的，指定数据单元中记载的数据的类型为字符串和/或大对象类字段。
第三方面，本申请提供了一种存储节点。存储节点存储有数据表，数据表记载有多行和多列数据。该存储节点包括：处理模块，用于确定创建索引所需的目标数据的指示信息，目标数据为目标数据表中的部分数据；处理模块还用于基于指示信息，从目标数据表中获取目标数据；交互模块，用于向计算节点提供目标数据。
可选的，交互模块还用于接收计算节点提供的指示信息。
可选的，交互模块具体用于接收计算节点发送的数据获取请求，数据获取请求用于请求存储节点向计算节点提供指示信息指示的目标数据。
可选的，目标数据包括：目标数据表的指定数据单元中记载的数据，数据单元为数据行或数据列。
可选的，目标数据包括：目标数据表的指定数据单元中记载的部分数据。
可选的，部分数据为指定数据单元中记载的数据的前缀数据、中间数据、后缀数据或指定长度的数据。
可选的，指定数据单元中记载的数据的类型为字符串和/或大对象类字段。
第四方面，本申请提供了一种计算节点。计算节点包括：交互模块，用于接收存储节点发送的目标数据；处理模块，用于基于目标数据创建索引。该目标数据为存储节点中存储的目标数据表中的部分数据，目标数据表记载有多行和多列数据。
可选的，交互模块还用于接收索引创建指令，索引创建指令指示基于目标数据创建索引；处理模块还用于基于索引创建指令确定指示信息；交互模块还用于向存储节点提供指示信息。
可选的，交互模块具体用于在目标数据对应的第一消耗和第二消耗满足指定条件时，向存储节点提供指示信息，第一消耗为存储节点基于目标数据表获取目标数据，并向计算节点提供目标数据产生的消耗，第二消耗为存储节点向计算节点提供目标数据表，由计算节点基于目标数据表获取目标数据产生的消耗。
可选的，交互模块具体用于向存储节点发送数据获取请求，数据获取请求用于请求存储节点向计算节点提供指示信息指示的目标数据。
可选的，目标数据包括：目标数据表的指定数据单元中记载的数据，数据单元为数据行或数据列。
可选的，目标数据包括：目标数据表的指定数据单元中记载的部分数据。
可选的，部分数据为指定数据单元中记载的数据的前缀数据、中间数据、后缀数据或指定长度的数据。
可选的，指定数据单元中记载的数据的类型为字符串和/或大对象类字段。
第五方面，本申请提供了一种创建索引的方法。该方法应用于存储节点。存储节点存储有数据表，数据表记载有多行和多列数据。该方法包括：存储节点确定创建索引所需的目标数据的指示信息，目标数据为目标数据表中的部分数据；存储节点基于指示信息，从目标数据表中获取目标数据，并向计算节点提供目标数据。
可选的，在存储节点确定创建索引所需的目标数据的指示信息之前，该方法还包括：存储节点接收计算节点提供的指示信息。
可选的，存储节点接收计算节点提供的指示信息，包括：存储节点接收计算节点发送的数据获取请求，数据获取请求用于请求存储节点向计算节点提供指示信息指示的目标数据。
可选的，目标数据包括：目标数据表的指定数据单元中记载的数据，数据单元为数据行或数据列。
可选的，目标数据包括：目标数据表的指定数据单元中记载的部分数据。
可选的，部分数据为指定数据单元中记载的数据的前缀数据、中间数据、后缀数据或指定长度的数据。
可选的，指定数据单元中记载的数据的类型为字符串和/或大对象类字段。
第六方面，本申请提供了一种创建索引的方法。该方法应用于计算节点。该方法包括：计算节点接收存储节点发送的目标数据，并基于目标数据创建索引。该目标数据为存储节点中存储的目标数据表中的部分数据，目标数据表记载有多行和多列数据。
可选的，在计算节点接收存储节点发送的目标数据之前，该方法还包括：计算节点接收索引创建指令，基于索引创建指令确定指示信息，并向存储节点提供指示信息，索引创建指令指示基于目标数据创建索引。
可选的，计算节点向存储节点提供指示信息，包括：在目标数据对应的第一消耗和第二消耗满足指定条件时，向存储节点提供指示信息，第一消耗为存储节点基于目标数据表获取目标数据，并向计算节点提供目标数据产生的消耗，第二消耗为存储节点向计算节点提供目标数据表，由计算节点基于目标数据表获取目标数据产生的消耗。
可选的，计算节点向存储节点提供指示信息，包括：向存储节点发送数据获取请求，数据获取请求用于请求存储节点向计算节点提供指示信息指示的目标数据。
可选的，目标数据包括：目标数据表的指定数据单元中记载的数据，数据单元为数据行或数据列。
可选的，目标数据包括：目标数据表的指定数据单元中记载的部分数据。
可选的，部分数据为指定数据单元中记载的数据的前缀数据、中间数据、后缀数据或指定长度的数据。
可选的，指定数据单元中记载的数据的类型为字符串和/或大对象类字段。
第七方面，本申请提供了一种计算设备，包括存储器和处理器，存储器存储有程序指令，处理器运行程序指令以执行本申请第五方面以及其任一种可能的实现方式中提供的方法，或者，执行本申请第六方面以及其任一种可能的实现方式中提供的方法。
第八方面，本申请提供了一种计算设备集群，包括至少一个计算设备，每个计算设备包括处理器和存储器，至少一个计算设备的处理器用于执行至少一个计算设备的存储器中存储的指令，以使得计算设备集群执行本申请第二方面以及其任一种可能的实现方式中提供的方法。
第九方面，本申请提供了一种包含指令的计算机程序产品，当指令被计算设备运行时，使得计算设备执行本申请第五方面以及其任一种可能的实现方式中提供的方法，或者，执行本申请第六方面以及其任一种可能的实现方式中提供的方法。
第十方面，本申请提供了一种包含指令的计算机程序产品，当指令被计算设备集群运行时，使得计算设备集群执行本申请第二方面以及其任一种可能的实现方式中提供的方法。
第十一方面，本申请提供了一种计算机可读存储介质，该计算机可读存储介质为非易失性计算机可读存储介质，该计算机可读存储介质包括计算机程序指令，当计算机程序指令由计算设备执行时，计算设备执行本申请第五方面以及其任一种可能的实现方式中提供的方法，或者，执行本申请第六方面以及其任一种可能的实现方式中提供的方法。
第十二方面，本申请提供了一种计算机可读存储介质，该计算机可读存储介质为非易失性计算机可读存储介质，该计算机可读存储介质包括计算机程序指令，当计算机程序指令由计算设备集群执行时，计算设备集群执行本申请第二方面以及其任一种可能的实现方式中提供的方法。
附图说明
图1是本申请实施例提供的一种创建索引的方法涉及的实施环境的示意图；
图2是本申请实施例提供的一种数据库系统的结构示意图；
图3是本申请实施例提供的另一种数据库系统的结构示意图；
图4是本申请实施例提供的一种创建索引的方法的流程图；
图5是本申请实施例提供的一种存储节点的结构示意图；
图6是本申请实施例提供的一种计算节点的结构示意图；
图7是本申请实施例提供的一种存储节点实现创建索引的方法的流程图；
图8是本申请实施例提供的一种计算节点实现创建索引的方法的流程图；
图9是本申请实施例提供的一种计算设备的结构示意图；
图10是本申请实施例提供的一种计算设备集群的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
为数据库中的数据创建索引，是一种最常用的数据库定义语义操作，是数据库系统运维过程中经常会发生的操作。通过索引进行查询能够加快查询速度，带来查询性能上的提升，其广泛使用于当用户需要加速查询的场景。
在目前的数据库系统中，在为数据库中的数据表创建索引时，计算节点需要从存储节点中读取数据表的全部数据，然后根据数据表的全部数据创建索引。
但是，目前创建索引的效率较低。
本申请实施例提供了一种创建索引的方法和用于实现该方法的数据库系统。该数据库系统包括计算节点和存储节点。存储节点中存储有数据表，数据表记载有多行和多列数据。在该数据库系统中，存储节点用于确定创建索引所需的目标数据的指示信息，基于指示信息，从目标数据表中获取目标数据，并向计算节点提供目标数据。计算节点用于基于目标数据创建索引。其中，目标数据为目标数据表中的部分数据。
在该数据库系统中，由存储节点从目标数据表中获取创建索引所需的目标数据，并向计算节点提供目标数据，相较于相关技术，无需存储节点向计算节点传输整个目标数据表记载的数据，减少了存储节点向计算节点传输的数据量，有助于提高存储节点和计算节点之间的输入/输出(input/output，I/O)性能。并且，也不需要计算节点执行从目标数据表中获取目标数据的操作，降低了计算节点的计算负载，有助于提高计算节点创建索引的效率和计算节点的计算性能。
图1是本申请实施例提供的一种创建索引的方法涉及的实施环境的示意图。如图1所示，该实施环境包括：客户端01和数据库系统02。数据库系统02中记载有数据表。客户端01能够与数据库系统02建立通信连接。例如，客户端01与数据库系统02之间可以通过网络建立通信连接。可选的，该网络可以为局域网，也可以为互联网，还可以为其它网络，本申请实施例不作限定。
用户可以通过客户端01与数据库系统02交互。在一种实现方式中，客户端01用于按照用户的指示向数据库系统02发送指令。例如，发送索引创建指令，以指示数据库系统创建索引。又例如，发送数据访问指令，以指示对数据库系统进行数据访问。
在一种可实现方式中，客户端01可以为台式计算机、膝上型计算机、移动电话、智能手机、平板电脑、多媒体播放器、智能家电、人工智能设备、智能可穿戴设备、电子阅读器、智能车载设备或物联网设备等。
数据库系统02用于接收客户端01发送的指令，并执行指令指示的操作。例如，数据库系统02用于接收索引创建指令，按照本申请实施例提供的创建索引的方法，基于索引创建指令创建索引。例如，数据库系统02用于接收数据访问指令，基于数据访问指令进行反馈。
在一种可实现方式中，数据库系统02可以通过计算设备实现。并且，该计算设备可以为服务器(如云服务器)。通常地，数据库系统02可以由若干台服务器组成的服务器集群，或者是一个云计算服务中心实 现。其中，云计算服务中心中部署有云服务提供商拥有的大量基础资源。例如云计算服务中心中部署有计算资源、存储资源和网络资源等。云计算服务中心可以利用该大量基础资源，实现本申请实施例提供的创建索引的方法。
当数据库系统02通过云计算服务中心实现时。用户可以通过客户端01访问云平台，并通过云平台使用数据库系统提供的数据库服务。此时，本申请实施例提供的创建索引的方法实现的功能，可以由云服务提供商在云平台抽象成一种创建索引的云服务，云平台能够利用云计算中心中的资源向用户提供该创建索引的云服务。用户在云平台购买该创建索引的云服务后，能够通过该创建索引的云服务为数据库系统记录的数据创建索引，以加速对数据的查找过程。可选地，云平台可以是中心云的云平台、边缘云的云平台或包括中心云和边缘云的云平台，本申请实施例对其不做具体限定。
需要说明的是，在图1所示的实施环境中，数据库系统02也可以通过除云平台外的其他资源平台实现，本申请实施例对其不做具体限定。此时，数据库系统02可以通过其他资源平台中的资源实现，并向用户提供相关的数据库服务。
应当理解的是，以上内容是对本申请实施例提供的创建索引的方法的应用场景的示例性说明，并不构成对于该创建索引的方法的应用场景的限定，本领域普通技术人员可知，随着业务需求的改变，其应用场景可以根据应用需求进行调整，本申请实施例对其不做一一列举。
图2是本申请实施例提供的一种数据库系统的结构示意图。如图2所示，该数据库系统02包括：计算节点021和存储节点022。存储节点022存储有数据表，数据表记载有多行和多列数据。在一种实现方式中，计算节点021和存储节点022可以通过计算设备集群中的计算设备实现。并且，为了对数据进行存储，该计算设备集群还配置有存储池，该存储池用于存储数据表。
在本申请实施例提供的数据库系统02中，存储节点022用于确定创建索引所需的目标数据的指示信息，目标数据为目标数据表中的部分数据。存储节点022还用于基于指示信息，从目标数据表中获取目标数据，并向计算节点021提供目标数据。计算节点021用于基于目标数据创建索引。
可选的，计算节点021还用于接收索引创建指令，基于索引创建指令确定指示信息，并向存储节点022提供指示信息，索引创建指令指示基于目标数据创建索引。
可选的，计算节点021具体用于在目标数据对应的第一消耗和第二消耗满足指定条件时，向存储节点022提供指示信息，第一消耗为存储节点022基于目标数据表获取目标数据，并向计算节点021提供目标数据产生的消耗，第二消耗为存储节点022向计算节点021提供目标数据表，由计算节点021基于目标数据表获取目标数据产生的消耗。
可选的，计算节点021具体用于向存储节点022发送数据获取请求，数据获取请求用于请求存储节点022向计算节点021提供指示信息指示的目标数据。
可选的，目标数据包括：目标数据表的指定数据单元中记载的数据，数据单元为数据行或数据列。
可选的，目标数据包括：目标数据表的指定数据单元中记载的部分数据。
可选的，部分数据为指定数据单元中记载的数据的前缀数据、中间数据、后缀数据或指定长度的数据。
可选的，指定数据单元中记载的数据的类型为字符串和/或大对象类字段。
需要说明的是，根据应用需求，该数据库系统02还以包括其他节点，以实现其他功能或者辅助实现存储节点022和计算节点021的功能。例如，如图3所示，数据库系统02还可以包括管理节点023，该管理节点023用于对存储节点022和计算节点021进行管理，以保证数据库系统02能够正常运行。
其中，存储节点022、计算节点021和管理节点023均可以通过软件实现，或者可以通过硬件实现。示例性的，接下来介绍存储节点022的实现方式。类似的，计算节点021和管理节点023的实现方式可以参考存储节点022的实现方式。
作为软件功能单元的一种举例，存储节点022可以包括运行在计算实例上的代码。其中，计算实例可以是物理主机(计算设备)、虚拟机、容器等计算设备中的至少一种。进一步地，上述计算设备可以是一台或者多台。例如，存储节点022可以包括运行在多个主机/虚拟机/容器上的代码。需要说明的是，用于运行该应用程序的多个主机/虚拟机/容器可以分布在相同的区域(region)中，也可以分布在不同的region中。用于运行该代码的多个主机/虚拟机/容器可以分布在相同的可用区(availability zone，AZ)中，也可以分布在不同的AZ中，每个AZ包括一个数据中心或多个地理位置相近的数据中心。其中，通常一个region 可以包括多个AZ。
同样，用于运行该代码的多个主机/虚拟机/容器可以分布在同一个虚拟私有云(virtual private cloud，VPC)中，也可以分布在多个VPC中。其中，通常一个VPC设置在一个region内。同一region内两个VPC之间，以及不同region的VPC之间跨区通信需在每个VPC内设置通信网关，经通信网关实现VPC之间的互连。
作为硬件功能单元的一种举例，存储节点022可以包括至少一个计算设备，如服务器等。或者，存储节点022也可以是利用专用集成电路(application-specific integrated circuit，ASIC)实现、或可编程逻辑器件(programmable logic device，PLD)实现的设备等。其中，上述PLD可以是复杂程序逻辑器件(complex programmable logical device，CPLD)、现场可编程门阵列(field-programmable gate array，FPGA)、通用阵列逻辑(generic array logic，GAL)或其任意组合实现。
存储节点022包括的多个计算设备可以分布在相同的region中，也可以分布在不同的region中。存储节点022包括的多个计算设备可以分布在相同的AZ中，也可以分布在不同的AZ中。同样，存储节点022包括的多个计算设备可以分布在同一个VPC中，也可以分布在多个VPC中。其中，多个计算设备可以是服务器、ASIC、PLD、CPLD、FPGA和GAL等计算设备的任意组合。
综上所述，在该数据库系统中，由存储节点从目标数据表中获取创建索引所需的目标数据，并向计算节点提供目标数据，相较于相关技术，无需存储节点向计算节点传输整个目标数据表记载的数据，减少了存储节点向计算节点传输的数据量，有助于提高存储节点和计算节点之间的I/O性能。并且，也不需要计算节点执行从目标数据表中获取目标数据的操作，降低了计算节点的计算负载，有助于提高计算节点创建索引的效率和计算节点的计算性能。
为便于理解，在下面内容中会对本申请实施例提供的创建索引的方法的实现过程进行说明。而此处为描述的方便和简洁，未在该数据库系统的相关内容对创建索引的方法的实现过程做过多说明，该数据库系统实现创建索引的方法的实现过程，可以下面方法实施例中的对应内容，在此不再赘述。
下面以图2所示的数据库系统为例，对通过数据库系统创建索引的实现过程进行说明。图4是本申请实施例提供的一种通过该数据库系统创建索引的方法的流程图。如图4所示，该方法包括以下步骤：
步骤401、计算节点接收索引创建指令，基于索引创建指令确定创建索引所需的目标数据的指示信息，索引创建指令指示基于目标数据创建索引，目标数据为索引创建指令指示的目标数据表中的部分数据。
用户需要为数据库的数据创建索引时，可以在用户使用的客户端上执行指定操作，以触发索引创建指令，通过该索引创建指令指示创建索引。客户端基于用户的操作生成索引创建指令后，会向数据库发送索引创建指令，计算节点可以接收到索引创建指令。
在数据库中创建索引是为了方便对数据库中的数据进行查找。索引创建指令会指示需要基于哪些数据创建索引。该指示可以通过指示信息表示。计算节点可以基于索引创建指令获取该指示信息，得到创建索引所需的目标数据的指示信息。在一种实现方式中，索引创建指令可以携带有指示信息，则计算节点可以从索引创建指令中提取该指示信息。例如，索引创建指令可以为ALTER TABLE t1ADD KEY(a，c)，该索引创建指令指示基于数据表t1的数据列a和数据列c创建索引，即指示信息指示数据表t1的数据列a和数据列c中的数据，则计算节点可以从索引创建指令中提取该指示信息。
需要说明的是，在一些场景中，索引需要基于数据表中的主键数据创建，索引创建指令指示的数据可能包括主键数据，也可能不包括主键数据。当索引需要基于数据表中的主键数据创建，且索引创建指令没有指示主键数据时，计算节点接收到索引创建指令后，还可以根据索引创建指令指示的数据确定创建索引所需的主键数据，并在指示信息中携带指示该主键数据的信息。例如，仍以上面的例子为例，索引创建指令为ALTER TABLE t1ADD KEY(a，c)，该索引创建指令指示基于数据表t1的数据列a和数据列c创建索引。然而，该场景需要基于数据表t1的主键数据创建索引，且数据表t1的数据列d用于记载该数据表t1的主键数据，则计算节点获取的指示信息不仅指示数据表t1的数据列a和数据列c中的数据，还指示数据列d中的数据。
还需要说明的是，本申请实施例也可以适用于创建索引不需要主键数据的场景。例如，在一些场景中，可以根据一些标识信息创建索引，而不需要根据主键数据创建索引，此时，本申请实施例也可以适用于这些场景。
在本申请实施例中，目标数据为索引创建指令指示的目标数据表中的部分数据。在一种实现方式中，当数据表采用数据行和数据列对具有不同属性的数据进行区分时，目标数据可以包括目标数据表的指定数据单元中记载的数据，该数据单元可以为数据行或数据列。例如，请参考前面两个例子，目标数据可以为目标数据表的指定数据列中记载的数据。
进一步地，目标数据具体可以包括：目标数据表的指定数据单元中记载的部分数据。在一种可实现方式中，该部分数据可以为指定数据单元中记载的数据的前缀数据、中间数据、后缀数据或指定长度的数据。例如，当指定数据单元中记载的数据的类型为字符串和/或大对象类字段时，在创建索引时，可以仅基于字符串和/或大对象类字段的前缀数据创建索引，则目标数据可以为目标数据表的指定数据列对应的数据行中记载的数据的前缀数据。例如，索引创建指令可以为ALTER TABLE t1ADD KEY(a(10)，c)，该索引创建指令指示基于数据表t1的数据列a对应的数据行中长度为10的前缀数据和数据列c创建索引，即指示信息指示数据表t1的数据列a对应的数据行中长度为10的前缀数据和数据列c中的数据。
当目标数据为索引创建指令指示的目标数据表中指定数据单元中记载的部分数据时，通过存储节点从目标数据表中获取该目标数据，并向计算节点提供该目标数据，能够进一步地减少存储节点向计算节点传输的数据量，降低计算节点的计算负载，从而进一步提高创建索引的效率和计算节点的计算性能。
步骤402、计算节点向存储节点提供指示信息。
计算节点获取指示信息后，即可向存储节点提供该指示信息，以便于存储节点向计算节点提供指示信息指示的目标数据。或者，计算节点在确定指示信息后，可以基于指示信息确定是否向存储节点发送指示信息，在有必要通过存储节点获取目标数据时，向存储节点提供该指示信息。在一种实现方式中，计算节点可以获取目标数据对应的第一消耗和第二消耗，在第一消耗和第二消耗满足指定条件时，向存储节点发送指示信息。在第一消耗和第二消耗不满足指定条件时，不向存储节点发送指示信息，而请求存储节点向计算节点提供记载有目标数据的整个目标数据表。其中，第一消耗为存储节点基于目标数据表获取目标数据，并向计算节点提供目标数据产生的消耗，第二消耗为存储节点向计算节点提供目标数据表，由计算节点基于目标数据表获取目标数据产生的消耗。其中，指定条件可以根据应用需求确定。例如，指定条件可以为第一消耗小于第二消耗。又例如，考虑到若由计算节点执行从目标数据表中获取目标数据的操作会占用计算节点的计算资源，为保证计算节点的计算性能，该指定条件可以为第二消耗与第一消耗的差值大于指定的阈值。
在本申请实施例中，计算节点向存储节点发送指示信息的实现方式可以有多种。在一种可实现方式中，计算节点可以以I/O请求的方式，向存储节点发送指示信息。例如，计算节点向存储节点发送数据获取请求，该数据获取请求请求存储节点向计算节点提供指示信息指示的目标数据。在一种实现场景中，存储节点可以支持近数据处理(near data processing，NDP)技术，计算节点和存储节点之间可以通过NDP上下文结构进行通信。则计算节点确定指示信息后，可以基于指示信息构造NDP上下文结构，以通过该NDP上下文结构传递指示信息。此时，数据获取请求可以包括NDP上下文结构和指示存储节点发起对目标数据表进行全表扫描的内容。
步骤403、存储节点确定创建索引所需的目标数据的指示信息。
存储节点接收到指示信息后，即可基于该指示信息从该指示信息指示的目标数据表中获取目标数据，并向计算节点提供该目标数据。例如，当计算节点发送的数据获取请求包括NDP上下文结构和指示存储节点发起对目标数据表进行全表扫描的内容时，存储节点接收到该数据获取请求后，可以从NDP上下文结构中获取到指示信息，确定需要对目标数据表进行全表扫描并获取目标数据。
步骤404、存储节点基于指示信息，从目标数据表中获取目标数据。
存储节点获取到指示信息后，可以对指示信息指示的目标数据表进行全表扫描，遍历目标数据表的每个相关页面中的数据行和数据列，以获取到目标数据。例如，存储节点可以采用NDP技术从目标数据表中获取目标数据。通常来说，NDP技术一般考虑在数据库的选择(SELECT)及数据操纵语言(data manipulation language，DML)场景中使用，但其能否应用在SELECT和DML场景中还决定于该场景中使用的结构化查询语言(structured query language，SQL)。而在本申请实施例中，只要存储节点能够支持NDP技术，就能够使用该NDP技术从目标数据表中获取目标数据，无需考虑过多因素，其使用场景覆盖率基本上是100％，能够大幅消减数据表读取阶段的I/O，该效果在多用户共享存储资源和计算资源的场景(云平台的多租户场景)下表现尤其明显。
其中，存储节点从目标数据表中获取目标数据的操作，可称为对目标数据表的裁剪操作。例如，当目标数据为目标数据表的指定数据列中记载的数据时，从目标数据表中获取目标数据的操作可称为列裁剪操作。当目标数据为目标数据表的数据行中记载的数据的前缀数据时，从目标数据表中获取目标数据的操作可称为列内容裁剪操作。通过存储节点对目标数据表执行裁剪操作获取目标数据，并向计算节点提供该目标数据，能够减少存储节点向计算节点传输的数据量，减小存储节点和计算节点之间的I/O量，减小存储节点和计算节点之间的I/O开销，实现存储节点和计算节点之间的精准I/O。该效果在通过列内容裁剪获取目标数据时表现的尤其明显。
步骤405、存储节点向计算节点提供目标数据。
存储节点获取目标数据后，可将目标数据存储在数据结构中，并向计算节点提供该数据结构，使得计算节点获取该目标数据。存储节点通过裁剪操作从目标数据表的每个页面中获取完目标数据后，即可向数据节点提供经过裁剪的页面，该经过裁剪的页面可以看成是原页面的压缩页面，但是通过本申请实施例提供的裁剪操作获取的压缩页面与通过压缩技术得到的压缩页面又不同。例如，裁剪得到的压缩页面中的数据仅包括指示信息指定的数据单元中的数据，而压缩技术得到的压缩页面还可能包括其他数据单元中的数据。又例如，裁剪得到的压缩页面相对于目标数据表的压缩率能够明显大于压缩技术得到的压缩页面的压缩率。
步骤406、计算节点基于目标数据创建索引。
计算节点获取到存储节点提供的目标数据后，即可基于目标数据创建索引。例如，当目标数据为目标数据表的指定数据列中记载的数据时，计算节点在获取到目标数据后，可以从目标数据的每个数据行中读取每个数据列记载的数据，并基于每个数据行和每个数据列记载的数据构成一条子记录进行独立存储，然后对所有子记录进行排序，并基于排序后的子记录执行建立索引的操作。在一种实现方式中，索引可以保存在B树等数据结构中。并且，当目标数据为目标数据表的指定数据列中记载的数据，且目标数据以压缩页面的形式呈现时，计算节点接收到存储节点发送的压缩页面后，可以解析压缩页面的数据行，以得到目标数据。计算节点通过从存储节点获取目标数据，无需计算节点执行从目标数据表中获取目标数据的操作，降低了计算节点的计算负载，有助于提高计算节点创建索引的效率和计算节点的计算性能。
下面以一个示例为例，对该创建索引的方法产生的收益进行说明：
假设数据表t1包括24个数据列，分别为数据列a,b,c,......x，数据列a和数据列b存储的字符串长度为100字节，其余数据列存储的字符串长度为4，且数据列c记载的数据为主键数据。索引创建指令ALTER TABLE t1ADD KEY(a(10)，c)，即索引创建指令指示基于数据表t1的数据列a中长度为10的前缀数据和数据列c创建索引。
如果按照相关技术，计算节点需要从存储节点读取整个数据表t1，对一每个数据行，其需要读取的字节数大约为100+100+4×22＝288个字节。在该过程中，计算节点为了获取数据表t1的数据列a中长度为10的前缀数据和数据列c，需要额外读取数据列b,d,e,…x和数据列a中第11至100个字节。当需要读取的数据行较多时，如需要读取10亿个数据行记载的数据时，IO量至少会达到288×1000000000＝288G。
当采用本申请实施例提供的创建索引的方法时，存储节点可以通过列裁剪，在数据表t1中去掉数据列b,d,e,……,x，获取到数据列a和c记载的目标数据，对一每个数据行，该目标数据的总长度为100+4＝104个字节。当需要读取的数据行较多时，如需要读取10亿个数据行记载的数据时，IO量至少会达到104×1000000000＝104G。
甚至，存储节点可以通过列裁剪和列内容裁剪，在数据表t1中去掉数据列b,d,e,……,x和数据列a中第11至100个字节，获取到数据列a中长度为10的前缀数据和数据列c记载的目标数据，对一每个数据行，该目标数据的总长度为10+4＝14个字节。当需要读取的数据行较多时，如需要读取10亿个数据行记载的数据时，IO量至少会达到14×1000000000＝14G。
根据以上内容可以看到，通过使用列裁剪的优化，每个数据行中被存储节点传输的数据量大小，可以从原本的288个字节，减少到104个字节，其I/O消减量可以达到(288-104)/288＝63.89％。通过使用列裁剪和列内容裁剪的优化，每个数据行中被存储节点传输的数据量大小，可以从原本的288个字节，减少到14个字节，其I/O消减量可以达到甚至(288-14)/288＝95.14％。
并且，当数据表的宽度更宽，比如数据表中记载的字段更加复杂，数据表包括的数据列更多时，通过本申请实施例提供的优化方式能够消减的I/O量将会更大。需要说明的是，在上述对比中没有将数据行的 头部信息计入，由于头部信息的数据量一般较小，且对头部信息进行列裁剪或列内容裁剪也能够减少数据量(如6-8个字节)，因此即使计入头部信息通过该优化仍然能够达到较好的效果。
综上所述，在本申请实施例提供的创建索引的方法中，存储节点用于确定创建索引所需的目标数据的指示信息，基于指示信息，从目标数据表中获取目标数据，并向计算节点提供目标数据，计算节点用于基于目标数据创建索引。其中，目标数据为目标数据表中的部分数据。在该数据库系统中，由存储节点从目标数据表中获取创建索引所需的目标数据，并向计算节点提供目标数据，相较于相关技术，无需存储节点向计算节点传输整个目标数据表记载的数据，减少了存储节点向计算节点传输的数据量，有助于提高存储节点和计算节点之间的I/O性能。并且，也不需要计算节点执行从目标数据表中获取目标数据的操作，降低了计算节点的计算负载，有助于提高计算节点创建索引的效率和计算节点的计算性能。
并且，在多用户共享存储资源和计算资源的场景下，由于多用户共用存储节点和计算节点，其中部分用户创建索引的操作会直接影响到其它用户的IO体验和计算性能，因此本申请实施例提供的优化方式的效果在这种共享场景下被放大得更明显。
需要说明的是，本申请实施例提供的创建索引的方法的步骤先后顺序可以进行适当调整，步骤也可以根据情况进行相应增减。任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化的方法，都应涵盖在本申请的保护范围之内，因此不再赘述。
本申请实施例还提供了一种存储节点。存储节点存储有数据表，数据表记载有多行和多列数据。如图5所示，该存储节点022包括：
处理模块0221，用于确定创建索引所需的目标数据的指示信息，目标数据为目标数据表中的部分数据。
处理模块0221还用于基于指示信息，从目标数据表中获取目标数据。
交互模块0222，用于向计算节点提供目标数据。
可选的，交互模块0222还用于接收计算节点提供的指示信息。
可选的，交互模块0222具体用于接收计算节点发送的数据获取请求，数据获取请求用于请求存储节点向计算节点提供指示信息指示的目标数据。
可选的，目标数据包括：目标数据表的指定数据单元中记载的数据，数据单元为数据行或数据列。
可选的，目标数据包括：目标数据表的指定数据单元中记载的部分数据。
可选的，部分数据为指定数据单元中记载的数据的前缀数据、中间数据、后缀数据或指定长度的数据。
可选的，指定数据单元中记载的数据的类型为字符串和/或大对象类字段。
本申请实施例还提供了一种计算节点。如图6所示，计算节点021包括：
交互模块0211，用于接收存储节点发送的目标数据，该目标数据为存储节点中存储的目标数据表中的部分数据，目标数据表记载有多行和多列数据。
处理模块0212，用于基于目标数据创建索引。
可选的，交互模块0211还用于接收索引创建指令，索引创建指令指示基于目标数据创建索引。处理模块0212还用于基于索引创建指令确定指示信息。交互模块0211还用于向存储节点提供指示信息。
可选的，交互模块0211具体用于在目标数据对应的第一消耗和第二消耗满足指定条件时，向存储节点提供指示信息，第一消耗为存储节点基于目标数据表获取目标数据，并向计算节点提供目标数据产生的消耗，第二消耗为存储节点向计算节点提供目标数据表，由计算节点基于目标数据表获取目标数据产生的消耗。
可选的，交互模块0211具体用于向存储节点发送数据获取请求，数据获取请求用于请求存储节点向计算节点提供指示信息指示的目标数据。
可选的，目标数据包括：目标数据表的指定数据单元中记载的数据，数据单元为数据行或数据列。
可选的，目标数据包括：目标数据表的指定数据单元中记载的部分数据。
可选的，部分数据为指定数据单元中记载的数据的前缀数据、中间数据、后缀数据或指定长度的数据。
可选的，指定数据单元中记载的数据的类型为字符串和/或大对象类字段。
其中，处理模块0221、交互模块0222、交互模块0211和处理模块0212均可以通过软件实现，或者可以通过硬件实现。示例性的，接下来以处理模块0221为例，介绍处理模块0221的实现方式。类似的，交 互模块0222、交互模块0211和处理模块0212的实现方式可以参考处理模块0221的实现方式。
模块作为软件功能单元的一种举例，处理模块0221可以包括运行在计算实例上的代码。其中，计算实例可以包括物理主机(计算设备)、虚拟机、容器中的至少一种。进一步地，上述计算实例可以是一台或者多台。例如，处理模块0221可以包括运行在多个主机/虚拟机/容器上的代码。需要说明的是，用于运行该代码的多个主机/虚拟机/容器可以分布在相同的region中，也可以分布在不同的region中。进一步地，用于运行该代码的多个主机/虚拟机/容器可以分布在相同的AZ中，也可以分布在不同的AZ中，每个AZ包括一个数据中心或多个地理位置相近的数据中心。其中，通常一个region可以包括多个AZ。
同样，用于运行该代码的多个主机/虚拟机/容器可以分布在同一个虚拟私有云(virtual private cloud，VPC)中，也可以分布在多个VPC中。其中，通常一个VPC设置在一个region内，同一region内两个VPC之间，以及不同region的VPC之间跨区通信需在每个VPC内设置通信网关，经通信网关实现VPC之间的互连。
模块作为硬件功能单元的一种举例，处理模块0221可以包括至少一个计算设备，如服务器等。或者，处理模块0221也可以是利用ASIC实现、或PLD实现的设备等。其中，上述PLD可以是CPLD、FPGA、GAL或其任意组合实现。
处理模块0221包括的多个计算设备可以分布在相同的region中，也可以分布在不同的region中。处理模块0221包括的多个计算设备可以分布在相同的AZ中，也可以分布在不同的AZ中。同样，处理模块0221包括的多个计算设备可以分布在同一个VPC中，也可以分布在多个VPC中。其中，多个计算设备可以是服务器、ASIC、PLD、CPLD、FPGA和GAL等计算设备的任意组合。
需要说明的是，在其他实施例中，处理模块0221可以用于执行创建索引的方法中存储节点执行的任意步骤，交互模块0222可以用于执行创建索引的方法中存储节点执行的任意步骤，交互模块0211可以用于执行创建索引的方法中计算节点执行的任意步骤，处理模块0212可以用于执行创建索引的方法中计算节点执行的任意步骤。处理模块0221、交互模块0222、交互模块0211和处理模块0212负责实现的步骤可根据需要指定，通过处理模块0221和交互模块0222分别实现创建索引的方法中不同的步骤来实现存储节点的全部功能，通过交互模块0211和处理模块0212分别实现创建索引的方法中不同的步骤来实现计算节点的全部功能。
本申请实施例还提供了一种创建索引的方法。该方法应用于存储节点。存储节点存储有数据表，数据表记载有多行和多列数据。如图7所示，该方法包括：
步骤701、存储节点确定创建索引所需的目标数据的指示信息，目标数据为目标数据表中的部分数据。
步骤702、存储节点基于指示信息，从目标数据表中获取目标数据，并向计算节点提供目标数据。
可选的，如图7所示，在存储节点确定创建索引所需的目标数据的指示信息之前，该方法还包括：步骤703、存储节点接收计算节点提供的指示信息。
可选的，存储节点接收计算节点提供的指示信息，包括：存储节点接收计算节点发送的数据获取请求，数据获取请求用于请求存储节点向计算节点提供指示信息指示的目标数据。
可选的，目标数据包括：目标数据表的指定数据单元中记载的数据，数据单元为数据行或数据列。
可选的，目标数据包括：目标数据表的指定数据单元中记载的部分数据。
可选的，部分数据为指定数据单元中记载的数据的前缀数据、中间数据、后缀数据或指定长度的数据。
可选的，指定数据单元中记载的数据的类型为字符串和/或大对象类字段。
本申请实施例还提供了一种创建索引的方法。该方法应用于计算节点。如图8所示，该方法包括：
步骤801、计算节点接收存储节点发送的目标数据，该目标数据为存储节点中存储的目标数据表中的部分数据，目标数据表记载有多行和多列数据。
步骤802、计算节点基于目标数据创建索引。
可选的，如图8所示，在计算节点接收存储节点发送的目标数据之前，该方法还包括：步骤803、计算节点接收索引创建指令，基于索引创建指令确定指示信息，并向存储节点提供指示信息，索引创建指令指示基于目标数据创建索引。
可选的，计算节点向存储节点提供指示信息，包括：在目标数据对应的第一消耗和第二消耗满足指定 条件时，向存储节点提供指示信息，第一消耗为存储节点基于目标数据表获取目标数据，并向计算节点提供目标数据产生的消耗，第二消耗为存储节点向计算节点提供目标数据表，由计算节点基于目标数据表获取目标数据产生的消耗。
可选的，计算节点向存储节点提供指示信息，包括：向存储节点发送数据获取请求，数据获取请求用于请求存储节点向计算节点提供指示信息指示的目标数据。
可选的，目标数据包括：目标数据表的指定数据单元中记载的数据，数据单元为数据行或数据列。
可选的，目标数据包括：目标数据表的指定数据单元中记载的部分数据。
可选的，部分数据为指定数据单元中记载的数据的前缀数据、中间数据、后缀数据或指定长度的数据。
可选的，指定数据单元中记载的数据的类型为字符串和/或大对象类字段。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的节点和模块的具体工作过程，可以参考前述方法实施例中的对应内容，在此不再赘述。
本申请实施例提供了一种计算设备。该计算设备用于实现本申请实施例提供的创建索引的方法中存储节点或计算节点的部分或全部功能。图9是本申请实施例提供的一种计算设备的结构示意图。如图9所示，该计算设备900包括处理器901、存储器902、通信接口903和总线904。其中，处理器901、存储器902、通信接口903通过总线904实现彼此之间的通信连接。计算设备900可以是服务器或终端设备。应理解，本申请不限定计算设备900中的处理器、存储器的个数。
处理器901可以包括通用处理器和/或专用硬件芯片。通用处理器可以包括：中央处理器(central processing unit，CPU)、微处理器(micro processor，MP)或图形处理器(graphics processing unit，GPU)等处理器中的任意一种或多种。CPU例如是一个单核处理器(single-CPU)，又如是一个多核处理器(multi-CPU)。专用硬件芯片是一个高性能处理的硬件模块。专用硬件芯片包括数字信号处理器(digital signal processor，DSP)、专用集成电路(application-specific integrated circuit，ASIC)、现场可编程逻辑门阵列(field-programmable gate array，FPGA)或者网络处理器(network processer，NP)中的至少一项。处理器901还可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，本申请的创建索引的方法的部分或全部功能，可以通过处理器901中的硬件的集成逻辑电路或者软件形式的指令完成。
存储器902用于存储计算机程序，计算机程序包括操作系统902a和可执行代码(即程序指令)902b。存储器902例如是只读存储器或可存储静态信息和指令的其它类型的静态存储设备，又如是随机存取存储器或者可存储信息和指令的其它类型的动态存储设备，又如是电可擦可编程只读存储器、只读光盘或其它光盘存储、光碟存储(包括压缩光碟、激光碟、光碟、数字通用光碟、蓝光光碟等)、磁盘存储介质或者其它磁存储设备，或者是能够用于携带或存储具有指令或数据结构形式的期望的可执行代码并能够由计算机存取的任何其它介质，但不限于此。例如存储器902用于存放出端口队列等。存储器902例如是独立存在，并通过总线904与处理器901相连接。或者存储器902和处理器901集成在一起。例如，存储器902可以包括易失性存储器(volatile memory)，例如随机存取存储器(random access memory，RAM)。处理器901还可以包括非易失性存储器(non-volatile memory)，例如只读存储器(read-only memory，ROM)，快闪存储器，机械硬盘(hard disk drive，HDD)或固态硬盘(solid state drive，SSD)。
存储器902可以存储可执行代码，当存储器902中存储的可执行代码被处理器901执行时，处理器901用于实现前述存储节点的功能，从而实现本申请实施例提供的创建索引的方法中存储节点的部分或全部功能。也即是，存储器902上存有用于实现创建索引的方法中存储节点的部分或全部功能的指令，或者，处理器901用于实现前述计算节点的功能，从而实现本申请实施例提供的创建索引的方法中计算节点的部分或全部功能。也即是，存储器902上存有用于实现创建索引的方法中计算节点的部分或全部功能的指令。处理器901执行该过程的实现方式请相应参考前述实施例中的相关描述。存储器902中还可以包括操作系统等其他运行进程所需的软件模块和数据等。
通信接口903使用例如但不限于网络接口卡、收发器一类的收发模块，来实现与其他设备或通信网络之间的通信。例如，通信接口903可以是以下器件的任一种或任一种组合：网络接口(如以太网接口)、无线网卡等具有网络接入功能的器件。
总线904是任何类型的，用于实现计算设备的内部器件(例如，存储器902、处理器901、通信接口903)互连的通信总线。例如，总线904可以是外设部件互连标准(peripheral component interconnect，PCI) 总线或扩展工业标准结构(extended industry standard architecture，EISA)总线等。总线可以分为地址总线、数据总线、控制总线等。为便于表示，图9中仅用一条线表示，但并不表示仅有一根总线或一种类型的总线。总线904可包括在计算设备900各个部件(例如，存储器902、处理器901、通信接口903)之间传送信息的通路。本申请实施例以计算设备内部的上述器件通过总线904互连为例说明，可选地，计算设备900内部的上述器件还可以采用除了总线904之外的其他连接方式彼此通信连接。例如，计算设备900内部的上述器件通过内部的逻辑接口互连。
需要说明的是，上述多个器件可以分别设置在彼此独立的芯片上，也可以至少部分的或者全部的设置在同一块芯片上。将各个器件独立设置在不同的芯片上，还是整合设置在一个或者多个芯片上，往往取决于产品设计的需要。本申请实施例对上述器件的具体实现形式不做限定。且上述各个附图对应的流程的描述各有侧重，某个流程中没有详述的部分，可以参见其他流程的相关描述。
本申请实施例还提供了一种计算设备集群。该计算设备集群包括至少一台计算设备。该计算设备可以是服务器，例如是中心服务器、边缘服务器，或者是本地数据中心中的本地服务器。在一些实施例中，计算设备也可以是台式机、笔记本电脑或者智能手机等终端设备。
可选地，计算设备集群包括的至少一个计算设备的结构可参见图9示出的计算设备900。计算设备集群中的一个或多个计算设备900中的存储器902中可以存有相同的用于执行创建索引的方法的指令。
在一些可能的实现方式中，该计算设备集群中的一个或多个计算设备900的存储器902中也可以分别存有用于执行创建索引的方法的部分指令。换言之，一个或多个计算设备900的组合可以共同执行用于执行创建索引的方法的指令。
需要说明的是，计算设备集群中的不同的计算设备900中的存储器902可以存储不同的指令，分别用于实现创建索引的方法的部分功能。也即，不同的计算设备900中的存储器902存储的指令可以实现存储节点和计算节点中的一个或多个模块的功能。
在一些可能的实现方式中，计算设备集群中的一个或多个计算设备可以通过网络连接。其中，该网络可以是广域网或局域网等等。图10示出了一种可能的实现方式。如图10所示，两个计算设备1000A和1000B之间通过网络进行连接。具体地，通过各个计算设备中的通信接口与该网络进行连接。在这一类可能的实现方式中，计算设备1000A和1000B包括总线1002、处理器1004、存储器1006和通信接口1008。计算设备1000A中的存储器1006中存有实现存储节点的功能的指令。同时，计算设备1000B中的存储器1006中存有实现计算节点的功能的指令。
图10所示的计算设备集群之间的连接方式可以是考虑到本申请提供的创建索引的方法需要传输目标数据和指示信息的需求，因此考虑将存储节点实现的功能交由计算设备1000A，将计算节点实现的功能交由计算设备1000B执行。
应理解，图10中示出的计算设备1000A的功能也可以由多个计算设备1000完成。同样，计算设备1000B的功能也可以由多个计算设备1000完成。
本申请实施例还提供了一种计算机可读存储介质，该计算机可读存储介质为非易失性计算机可读存储介质。计算机可读存储介质可以是计算设备能够存储的任何可用介质或者是包含一个或多个可用介质的数据中心等数据存储设备。可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘)等。该计算机可读存储介质包括指令，该指令指示计算设备执行本申请实施例提供的创建索引的方法。
本申请实施例还提供了一种包含指令的计算机程序产品。计算机程序产品可以是包含指令的，能够运行在计算设备上或被储存在任何可用介质中的软件或程序产品。当计算机程序产品在至少一个计算设备上运行时，使得计算机实现本申请实施例提供的创建索引的方法。
本申请实施例还提供了一种芯片，包括处理器，用于从存储器中调用并运行存储器中存储的指令，使得安装有芯片的计算设备执行本申请实施例提供的创建索引的方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行计算机程序指令时，全部或部分地产生按照本申请提供的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘Solid State Disk)等。
为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各实施例的步骤及组成。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。本领域普通技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
用于实现本申请实施例的方法的计算机程序代码可以用一种或多种编程语言编写。这些计算机程序代码可以提供给通用计算机、专用计算机或其他可编程的规则查找装置的处理器，使得程序代码在被计算机或其他可编程的规则查找装置执行的时候，引起在流程图和/或框图中规定的功能/操作被实施。程序代码可以完全在计算机上、部分在计算机上、作为独立的软件包、部分在计算机上且部分在远程计算机上或完全在远程计算机或服务器上执行。
在本申请实施例的上下文中，计算机程序代码或者相关数据可以由任意适当载体承载，以使得设备、装置或者处理器能够执行上文描述的各种处理和操作。载体的示例包括信号、计算机可读介质等等。信号的示例可以包括电、光、无线电、声音或其它形式的传播信号，诸如载波、红外信号等。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，上述描述的系统、设备和模块的具体工作过程，可以参见前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、设备和方法，可以通过其它的方式实现。例如，以上所描述的设备实施例仅仅是示意性的，例如，该模块的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个模块或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口、设备或模块的间接耦合或通信连接，也可以是电的，机械的或其它的形式连接。
该作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理模块，即可以位于一个地方，或者也可以分布到多个网络模块上。可以根据实际的需要选择其中的部分或者全部模块来实现本申请实施例方案的目的。
另外，在本申请各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以是两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
本申请中术语“第一”“第二”等字样用于对作用和功能基本相同的相同项或相似项进行区分，应理解，“第一”、“第二”、“第n”之间不具有逻辑或时序上的依赖关系，也不对数量和执行顺序进行限定。还应理解，尽管以下描述使用术语第一、第二等来描述各种元素，但这些元素不应受术语的限制。这些术语只是用于将一元素与另一元素区别分开。例如，在不脱离各种示例的范围的情况下，第一链路可以被称为第二链路，并且类似地，第二链路可以被称为第一链路。
还应理解，在本申请的各个实施例中，各个过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
本申请中术语“至少一个”的含义是指一个或多个，本申请中术语“多个”的含义是指两个或两个以上，例如，多个第二报文是指两个或两个以上的第二报文。本文中术语“系统”和“网络”经常可互换使用。
应理解，在本文中对各种示例的描述中所使用的术语只是为了描述特定示例，而并非旨在进行限制。如在对各种示例的描述和所附权利要求书中所使用的那样，单数形式“一个(“a”，“an”)”和“该” 旨在也包括复数形式，除非上下文另外明确地指示。
还应理解，术语“包括”(也称“includes”、“including”、“comprises”和/或“comprising”)当在本说明书中使用时指定存在所陈述的特征、整数、步骤、操作、元素、和/或部件，但是并不排除存在或添加一个或多个其他特征、整数、步骤、操作、元素、部件、和/或其分组。
还应理解，根据上下文，短语“若确定...”或“若检测到[所陈述的条件或事件]”可被解释为意指“在确定...时”或“响应于确定...”或“在检测到[所陈述的条件或事件]时”或“响应于检测到[所陈述的条件或事件]”。
应理解，根据A确定B并不意味着仅仅根据A确定B，还可以根据A和/或其它信息确定B。
还应理解，说明书通篇中提到的“一个实施例”、“一实施例”、“一种可能的实现方式”意味着与实施例或实现方式有关的特定特征、结构或特性包括在本申请的至少一个实施例中。因此，在整个说明书各处出现的“在一个实施例中”或“在一实施例中”、“一种可能的实现方式”未必一定指相同的实施例。此外，这些特定的特征、结构或特性可以任意适合的方式结合在一个或多个实施例中。
需要说明的是，本申请所涉及的信息(包括但不限于用户设备信息、用户个人信息等)、数据(包括但不限于用于分析的数据、存储的数据、展示的数据等)以及信号，均为经用户授权或者经过各方充分授权的，且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准。例如，本申请中涉及到的指示信息和目标数据等都是在充分授权的情况下获取的。
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的保护范围。

Claims (19)

  1. 一种数据库系统，其特征在于，所述数据库系统包括：计算节点和存储节点，所述存储节点存储有数据表，所述数据表记载有多行和多列数据；
    所述存储节点用于确定创建索引所需的目标数据的指示信息，所述目标数据为目标数据表中的部分数据；
    所述存储节点还用于基于所述指示信息，从所述目标数据表中获取所述目标数据，并向所述计算节点提供所述目标数据；
    所述计算节点用于基于所述目标数据创建索引。
  2. 如权利要求1所述的系统，其特征在于，
    所述计算节点还用于接收索引创建指令，基于所述索引创建指令确定所述指示信息，并向所述存储节点提供所述指示信息，所述索引创建指令指示基于所述目标数据创建索引。
  3. 如权利要求2所述的系统，其特征在于，
    所述计算节点具体用于在所述目标数据对应的第一消耗和第二消耗满足指定条件时，向所述存储节点提供所述指示信息，所述第一消耗为所述存储节点基于所述目标数据表获取所述目标数据，并向所述计算节点提供所述目标数据产生的消耗，所述第二消耗为所述存储节点向所述计算节点提供所述目标数据表，由所述计算节点基于所述目标数据表获取所述目标数据产生的消耗。
  4. 如权利要求2或3所述的系统，其特征在于，
    所述计算节点具体用于向所述存储节点发送数据获取请求，所述数据获取请求用于请求所述存储节点向所述计算节点提供所述指示信息指示的所述目标数据。
  5. 如权利要求1至4任一所述的系统，其特征在于，所述目标数据包括：所述目标数据表的指定数据单元中记载的数据，所述数据单元为数据行或数据列。
  6. 如权利要求5所述的系统，其特征在于，所述目标数据包括：所述目标数据表的指定数据单元中记载的部分数据。
  7. 如权利要求6所述的系统，其特征在于，所述部分数据为所述指定数据单元中记载的数据的前缀数据、中间数据、后缀数据或指定长度的数据。
  8. 如权利要求6或7所述的系统，其特征在于，所述指定数据单元中记载的数据的类型为字符串和/或大对象类字段。
  9. 一种创建索引的方法，其特征在于，该方法应用于数据库系统，所述数据库系统包括：计算节点和存储节点，所述存储节点存储有数据表，所述数据表记载有多行和多列数据，所述方法包括：
    所述存储节点确定创建索引所需的目标数据的指示信息，所述目标数据为目标数据表中的部分数据；
    所述存储节点基于所述指示信息，从所述目标数据表中获取所述目标数据，并向所述计算节点提供所述目标数据；
    所述计算节点基于所述目标数据创建索引。
  10. 如权利要求9所述的方法，其特征在于，在所述存储节点确定创建索引所需的目标数据的指示信息之前，所述方法还包括：
    所述计算节点接收索引创建指令，基于所述索引创建指令确定所述指示信息，并向所述存储节点提供 所述指示信息，所述索引创建指令指示基于所述目标数据创建索引。
  11. 如权利要求10所述的方法，其特征在于，所述计算节点向所述存储节点提供所述指示信息，包括：
    所述计算节点在所述目标数据对应的第一消耗和第二消耗满足指定条件时，向所述存储节点提供所述指示信息，所述第一消耗为所述存储节点基于所述目标数据表获取所述目标数据，并向所述计算节点提供所述目标数据产生的消耗，所述第二消耗为所述存储节点向所述计算节点提供所述目标数据表，由所述计算节点基于所述目标数据表获取所述目标数据产生的消耗。
  12. 如权利要求10或11所述的方法，其特征在于，所述计算节点向所述存储节点提供所述指示信息，包括：
    所述计算节点向所述存储节点发送数据获取请求，所述数据获取请求用于请求所述存储节点向所述计算节点提供所述指示信息指示的所述目标数据。
  13. 如权利要求9至12任一所述的方法，其特征在于，所述目标数据包括：所述目标数据表的指定数据单元中记载的数据，所述数据单元为数据行或数据列。
  14. 如权利要求13所述的方法，其特征在于，所述目标数据包括：所述目标数据表的指定数据单元中记载的部分数据。
  15. 如权利要求14所述的方法，其特征在于，所述部分数据为所述指定数据单元中记载的数据的前缀数据、中间数据、后缀数据或指定长度的数据。
  16. 如权利要求14或15所述的方法，其特征在于，所述指定数据单元中记载的数据的类型为字符串和/或大对象类字段。
  17. 一种计算设备集群，其特征在于，包括至少一个计算设备，每个计算设备包括处理器和存储器，所述至少一个计算设备的处理器用于执行所述至少一个计算设备的存储器中存储的指令，以使得所述计算设备集群执行如权利要求9至16任一项所述的方法。
  18. 一种包含指令的计算机程序产品，其特征在于，当所述指令被计算设备集群运行时，使得所述计算设备集群执行如权利要求9至16任一项所述的方法。
  19. 一种计算机可读存储介质，其特征在于，包括计算机程序指令，当所述计算机程序指令由计算设备集群执行时，所述计算设备集群执行如权利要求9至16任一项所述的方法。
PCT/CN2023/103975 2022-09-15 2023-06-29 创建索引的方法及数据库系统 Ceased WO2024055695A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP23864434.8A EP4579478A4 (en) 2022-09-15 2023-06-29 INDEX CREATION METHOD AND DATABASE SYSTEM
US19/080,392 US20250209050A1 (en) 2022-09-15 2025-03-14 Index Creation Method and Database System

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202211120347 2022-09-15
CN202211120347.0 2022-09-15
CN202211358798.8A CN117743318A (zh) 2022-09-15 2022-11-01 创建索引的方法及数据库系统
CN202211358798.8 2022-11-01

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US19/080,392 Continuation US20250209050A1 (en) 2022-09-15 2025-03-14 Index Creation Method and Database System

Publications (1)

Publication Number Publication Date
WO2024055695A1 true WO2024055695A1 (zh) 2024-03-21

Family

ID=90274159

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/103975 Ceased WO2024055695A1 (zh) 2022-09-15 2023-06-29 创建索引的方法及数据库系统

Country Status (3)

Country Link
US (1) US20250209050A1 (zh)
EP (1) EP4579478A4 (zh)
WO (1) WO2024055695A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118511167A (zh) * 2022-03-02 2024-08-16 华为技术有限公司 数据库实体、始发数据库实体以及在数据库实体和始发数据库实体中使用的方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110399375A (zh) * 2019-07-24 2019-11-01 南京云白信息科技有限公司 一种数据表索引创建方法及装置
CN114090589A (zh) * 2022-01-20 2022-02-25 苏州浪潮智能科技有限公司 基于HBase的数据查询方法、装置和计算机设备
US11327937B1 (en) * 2015-09-18 2022-05-10 Amazon Technologies, Inc. Determining indexing progress for a table in a distributed data store

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5108252B2 (ja) * 2006-04-27 2012-12-26 株式会社日立製作所 インデクス更新方法及びそのシステム
US8510290B1 (en) * 2008-12-30 2013-08-13 Teradata Us, Inc. Index selection in a multi-system database management system
JP6717152B2 (ja) * 2016-10-06 2020-07-01 富士通株式会社 インデックス生成プログラム、インデックス生成装置、インデックス生成方法、検索プログラム、検索装置および検索方法
JP6871504B2 (ja) * 2016-12-27 2021-05-12 富士通株式会社 情報処理装置、データロードプログラム及びデータロード方法
CN108287835B (zh) * 2017-01-09 2022-06-21 腾讯科技（深圳）有限公司 一种数据清理方法及装置
US20200233882A1 (en) * 2019-01-18 2020-07-23 Huawei Technologies Co., Ltd. Bucketizing data into buckets for processing by code modules

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11327937B1 (en) * 2015-09-18 2022-05-10 Amazon Technologies, Inc. Determining indexing progress for a table in a distributed data store
CN110399375A (zh) * 2019-07-24 2019-11-01 南京云白信息科技有限公司 一种数据表索引创建方法及装置
CN114090589A (zh) * 2022-01-20 2022-02-25 苏州浪潮智能科技有限公司 基于HBase的数据查询方法、装置和计算机设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4579478A4

Also Published As

Publication number Publication date
US20250209050A1 (en) 2025-06-26
EP4579478A1 (en) 2025-07-02
EP4579478A4 (en) 2025-12-17

Similar Documents

Publication Publication Date Title
US12153575B2 (en) Database query splitting
CN110019080B (zh) 数据访问方法和装置
WO2023273544A1 (zh) 日志文件的存储方法、装置、设备和存储介质
US11310316B2 (en) Methods, devices and computer program products for storing and accessing data
CN113839977A (zh) 消息推送方法、装置、计算机设备及存储介质
US10866960B2 (en) Dynamic execution of ETL jobs without metadata repository
CN113688160B (zh) 数据处理方法、处理装置、电子设备及存储介质
US20220300503A1 (en) Querying distributed databases
US9367620B2 (en) Local context search results improvements
CN112583941A (zh) 一种支持接入多电力终端的方法、单元节点及电力物联网
CN107480205A (zh) 一种进行数据分区的方法和装置
US20250209050A1 (en) Index Creation Method and Database System
CN103902614B (zh) 一种数据处理方法、设备和系统
WO2024055715A1 (zh) 大数据集群部署方案的确定方法、装置、集群和存储介质
CN114637969A (zh) 目标对象的鉴权方法及装置
US20170364293A1 (en) Method and apparatus for data processing
US10628416B2 (en) Enhanced database query processing
CN111444017A (zh) 多媒体数据处理方法、装置、系统、电子设备及存储介质
CN109213815B (zh) 控制执行次数的方法、装置、服务器终端以及可读介质
CN117743318A (zh) 创建索引的方法及数据库系统
CN116185530A (zh) 一种页面分享方法、装置及介质
CN115687340A (zh) 业务查询方法、装置、设备及存储介质
US10503731B2 (en) Efficient analysis of distinct aggregations
US20260003894A1 (en) Data processing method and apparatus
US12367070B1 (en) Method for managing storage instances in a multi cluster environment and a system thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23864434

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023864434

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2023864434

Country of ref document: EP

Effective date: 20250324

NENP Non-entry into the national phase

Ref country code: DE

WWP Wipo information: published in national office

Ref document number: 2023864434

Country of ref document: EP