CN113553372B

CN113553372B - Method, device, computing equipment and medium for writing to database

Info

Publication number: CN113553372B
Application number: CN202010341339.3A
Authority: CN
Inventors: 石朝阳
Original assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Current assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Priority date: 2020-04-26
Filing date: 2020-04-26
Publication date: 2024-10-18
Anticipated expiration: 2040-04-26
Also published as: CN113553372A

Abstract

The present disclosure provides a method for writing to a database, comprising: obtaining user data generated within a preset time window; determining the data volume of the user data; determining first data and/or second data from the user data according to the determined data volume; and performing a preset write operation on the database for the first data and/or the second data, wherein the preset write operation includes: writing the first data to the database in a synchronous manner; and/or writing the second data to the database in an asynchronous manner. The present disclosure also provides a writing device, a computing device, and a medium for a database.

Description

Writing method and device for database, computing equipment and medium

Technical Field

The present disclosure relates to the field of computer technology, and more particularly, to a writing method, apparatus, computing device, and medium for a database.

Background

Existing server systems have an upper traffic limit at design time, and when the upper traffic limit is exceeded, the writing operation to the database cannot be performed normally. For example, for a system with an upper design flow limit of 10 ten thousand transactions/second, when it is required to process concurrent data of 500 ten thousand transactions/second, the system crashes, so that the operation of writing into the database cannot be normally performed.

The related art adopts a method of re-writing a new version system which can adapt to higher traffic and updating an old version server system to the new version system to solve the above problems, but this method is time-consuming and labor-consuming.

Disclosure of Invention

In view of this, the present disclosure provides a writing method, apparatus, computing device and medium for a database.

One aspect of the present disclosure provides a writing method for a database, including: acquiring user data generated in a preset time window; determining the data quantity of user data; determining first data and/or second data from the user data according to the determined data amount; and performing a preset write operation on the database for the first data and/or the second data, wherein the preset write operation comprises: writing the first data to the database in a synchronized manner; and/or writing the second data to the database in an asynchronous manner.

According to an embodiment of the present disclosure, the first data and/or the second data are determined from the user data according to the determined data amount, including at least one of: under the condition that the determined data quantity is less than or equal to a first threshold value, taking all user data as first data; dividing user data into first data and second data according to a preset proportion under the condition that the determined data quantity is larger than a first threshold value and smaller than or equal to a second threshold value; and in the case that the determined data amount is greater than the second threshold, taking all the user data as second data.

According to an embodiment of the present disclosure, writing first data to a database in a synchronized manner includes: after the first data is obtained from the user data, the first data is directly written into the database.

According to an embodiment of the present disclosure, writing second data to a database in an asynchronous manner includes: after the second data is obtained from the user data, the second data is cached in the message queue, and after the preset delay time is elapsed, the second data cached in the message queue is written into the database.

According to an embodiment of the present disclosure, the above method further includes: determining a data writing speed of writing second data into the database in the process of writing the second data cached in the message queue into the database; and configuring at least one writing thread corresponding to the preset writing operation into a dormant state under the condition that the data writing speed exceeds a speed threshold.

Another aspect of the present disclosure provides a writing apparatus for a database, including: the acquisition module is used for acquiring user data generated in a preset time window; a data amount determining module for determining the data amount of the user data; the data judging module is used for determining first data and/or second data from the user data according to the determined data quantity; and a writing module for executing a preset writing operation on the database for the first data and/or the second data, wherein the writing module comprises: the synchronous writing sub-module is used for writing the first data into the database in a synchronous mode; and/or an asynchronous writing sub-module for writing the second data to the database in an asynchronous manner.

According to an embodiment of the present disclosure, the data determination module includes at least one of: the first determining submodule is used for taking all user data as first data when the determined data quantity is smaller than or equal to a first threshold value; the second determining submodule is used for dividing the user data into first data and second data according to a preset proportion under the condition that the determined data quantity is larger than a first threshold value and smaller than or equal to a second threshold value; and a third determining sub-module for taking all the user data as the second data in case the determined data amount is larger than the second threshold.

According to an embodiment of the present disclosure, the synchronous writing sub-module is specifically configured to: after the first data is obtained from the user data, the first data is directly written into the database.

According to an embodiment of the present disclosure, the asynchronous writing sub-module is specifically configured to: after the second data is obtained from the user data, the second data is cached in the message queue, and after the preset delay time is elapsed, the second data cached in the message queue is written into the database.

According to an embodiment of the present disclosure, in the process of writing the second data buffered in the message queue into the database, the apparatus further includes: the writing speed determining module is used for determining the data writing speed of writing the second data into the database; and the dormancy module is used for configuring at least one writing thread corresponding to the preset writing operation into a dormancy state under the condition that the data writing speed exceeds a speed threshold value.

Another aspect of the present disclosure provides a computing device comprising: one or more processors; and a storage means for storing one or more programs, which when executed by the one or more processors cause the one or more processors to implement the methods as described above.

Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed, are configured to implement a method as described above.

Another aspect of the present disclosure provides a computer program comprising computer executable instructions which when executed are for implementing a method as described above.

According to the embodiment of the disclosure, by monitoring the data size of the user data, determining the first data and/or the second data from the user data according to the data size of the user data, writing the first data into the database in a synchronous manner, and/or writing the second data into the database in an asynchronous manner, the user data is written into the database, and the corresponding writing manner is automatically switched according to the data size, so that the system can still normally operate when the writing data size is larger under the condition that the original system is not rewritten.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments thereof with reference to the accompanying drawings in which:

FIG. 1 schematically illustrates an exemplary system architecture in which a method and apparatus for writing to a database may be applied, in accordance with an embodiment of the present disclosure;

FIG. 2A schematically illustrates a flow chart of a method of writing to a database in accordance with an embodiment of the present disclosure;

FIG. 2B schematically illustrates a flow chart for determining first data and/or second data from user data based on a determined amount of data, according to an embodiment of the disclosure;

FIG. 2C schematically illustrates a flow chart of a method of writing to a database according to another embodiment of the present disclosure;

FIG. 3 schematically illustrates a flow chart of a method of writing to a database in accordance with an embodiment of the disclosure;

FIG. 4A schematically illustrates a block diagram of a writing apparatus for a database, according to an embodiment of the disclosure;

FIG. 4B schematically illustrates a block diagram of a data determination module according to an embodiment of the disclosure;

FIG. 5 schematically illustrates a block diagram of a writing apparatus for a database according to another embodiment of the present disclosure; and

Fig. 6 schematically illustrates a block diagram of a computer system suitable for implementing the above-described writing method for a database, according to an embodiment of the present disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.

Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a convention should be interpreted in accordance with the meaning of one of skill in the art having generally understood the convention (e.g., "a system having at least one of A, B and C" would include, but not be limited to, systems having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a formulation similar to at least one of "A, B or C, etc." is used, in general such a formulation should be interpreted in accordance with the ordinary understanding of one skilled in the art (e.g. "a system with at least one of A, B or C" would include but not be limited to systems with a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

The embodiment of the disclosure provides a writing method for a database and a device capable of applying the method. The method comprises the steps of obtaining user data generated in a preset time window; determining the data quantity of user data; determining first data and/or second data from the user data according to the determined data amount; and performing a preset write operation on the database for the first data and/or the second data, wherein the preset write operation comprises: writing the first data to the database in a synchronized manner; and/or writing the second data to the database in an asynchronous manner.

FIG. 1 schematically illustrates an exemplary system architecture 100 in which the writing methods and apparatuses for a database may be applied, according to embodiments of the present disclosure. It should be noted that fig. 1 is only an example of a system architecture to which embodiments of the present disclosure may be applied to assist those skilled in the art in understanding the technical content of the present disclosure, but does not mean that embodiments of the present disclosure may not be used in other devices, systems, environments, or scenarios.

As shown in fig. 1, a system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the terminal devices 101, 102, 103.

The terminal devices 101, 102, 103 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.

The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by users using the terminal devices 101, 102, 103. The background management server is provided with a database. The background management server may store the received data such as the user request in the database, then analyze the data, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.

It should be noted that the writing method for a database provided by the embodiments of the present disclosure may be generally performed by the server 105. Accordingly, the writing device for the database provided by the embodiments of the present disclosure may be generally disposed in the server 105. The writing method for a database provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the writing means for a database provided by the embodiments of the present disclosure may also be provided in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Fig. 2A schematically illustrates a flow chart of a method of writing to a database according to an embodiment of the disclosure.

As shown in fig. 2A, the method includes operations S210 to S240.

More specifically, in operation S210, user data generated within a preset time window is acquired.

According to an embodiment of the present disclosure, the size of the preset time window may be set according to actual needs. The size of the preset time window may be set to 1 second, for example. It should be noted that, the present application does not specifically limit the size of the preset time window, and the size of the preset time window may be set arbitrarily in the practical application process.

According to embodiments of the present disclosure, the user data may be a request sent by a user through a terminal device, including, for example, a request to acquire data in a server, a request to submit data to the server for processing, and the like.

Then, in operation S220, the data amount of the user data is determined.

According to an embodiment of the present disclosure, the data amount of the user data may be determined by previously setting a sliding window traffic counter. By way of example, the data amount of user data may be characterized by a Query Per Second (QPS) or throughput (Transactions Per Second, TPS) parameter.

In operation S230, first data and/or second data are determined from the user data according to the determined data amount.

Fig. 2B schematically illustrates a flow chart of determining first data and/or second data from user data according to a determined amount of data according to an embodiment of the present disclosure.

As shown in fig. 2B, operation S230 may include operations S231 to S233, for example. More specifically, in the case where the determined data amount is less than or equal to the first threshold, operation S231 is performed. In case the determined data amount is greater than the second threshold, operation S232 is performed. In case the determined data amount is greater than the first threshold value and less than or equal to the second threshold value, operation S233 is performed.

In operation S231, all user data are used as first data.

In operation S232, all of the user data are taken as second data.

In operation S233, the user data is divided into first data and second data in a preset ratio.

According to an embodiment of the present disclosure, the preset ratio may be preconfigured, for example, the preset ratio is configured to be 1:1, that is, 50% of the data in the user data is divided into the first data, and the other 50% of the data in the user data is divided into the second data. It should be noted that, for the configuration of the preset proportion, a fixed configuration mode may be adopted, so that a certain specific proportion is always maintained in a specific application process, or a dynamic configuration mode may be adopted, so that corresponding changes may be performed as required in the specific application process.

In operation S240, a preset write operation is performed on the database with respect to the first data and/or the second data.

The preset writing operation comprises the following steps: writing the first data to the database in a synchronized manner; and/or writing the second data to the database in an asynchronous manner.

According to an embodiment of the present disclosure, if the user data includes first data, the first data in the user data is written into the database in a synchronous manner. And if the second data is contained in the user data, writing the second data in the user data into the database in an asynchronous mode.

Wherein writing the first data in the user data to the database in a synchronized manner may comprise, for example, directly writing the first data to the database after obtaining the first data from the user data. By the synchronous mode, each unit data in the first data can be synchronously written into the database, and the writing speed is higher.

In addition, writing the second data in the user data into the database in an asynchronous manner may include, for example, buffering the second data in the message queue after obtaining the second data from the user data, and writing the second data buffered in the message queue into the database after a predetermined delay time has elapsed. Through the asynchronous mode, each unit data in the second data can be written into the database in batches so as to reduce the data volume of single writing, so that the data volume of single writing does not exceed the maximum writing volume which can be born by the database, the overlarge pressure of the database is avoided, and the stability of data writing is ensured.

Fig. 2C schematically illustrates a flow chart of a method of writing to a database according to another embodiment of the present disclosure.

As shown in fig. 2C, the method may further include operations S250 to S260 in addition to operations S210 to S240 in writing the second data buffered in the message queue to the database.

In operation S250, a data writing speed of writing the second data into the database is determined.

According to embodiments of the present disclosure, a speed monitoring program may be provided to monitor the data write speed of the write database.

In operation S260, at least one write thread corresponding to a preset write operation is configured to be in a sleep state in case that the data writing speed exceeds the speed threshold.

According to embodiments of the present disclosure, the speed threshold may be determined from the maximum write speed that the database may withstand. When the data writing speed exceeds the speed threshold, the writing speed is reduced to a safe value by configuring part of writing threads into a dormant state, so that the collapse of the warehouse-in interface is avoided.

The method illustrated in fig. 2A-2C is further described below with reference to fig. 3 in conjunction with the exemplary embodiment.

Fig. 3 schematically illustrates a flow chart of a writing method for a database according to an embodiment of the disclosure.

As shown in fig. 3, the server obtains user data by user traffic binning. A window sliding traffic calculator in the core service is then invoked to determine the traffic size of the user data, i.e., the amount of seconds of invocation. In this embodiment, the first threshold is configured to 10000 (transactions/second), and the second threshold is configured to 100000 (transactions/second). If the calling quantity is more than 0 and less than 10000, taking all user data as first data, directly calling a warehouse-in interface to write the user data into a database. If 10000 < s-class call quantity is less than 100000, 50% of user data is used as first data, a warehouse-in interface is directly called for warehouse-in, the other 50% is used as second data, the second data is pressed into an MQ (message queue) pipeline, and then the second data is consumed by a worker at the back end (namely a consumption end), wherein the worker consumption refers to the operation of data warehouse-in through a task set at a fixed time. If the second call volume is more than 100000, all user data are used as second data, pressed into the MQ pipeline, and consumed by a worker at the back end.

In the process of consuming through the workbench of the back end, the consumption speed can be monitored, namely the speed of writing into the database. And determining whether the consumption speeds match by determining whether the current consumption speed is less than a speed threshold. When the consumption speed is greater than the speed threshold, namely the consumption speed is not matched, the part of the writing thread is transferred into the dormant state through the thread () method so as to adjust the consumption speed. And when the consumption speed is less than or equal to the speed threshold, namely the consumption speed is matched, calling a downstream warehousing interface and writing the subsequent data into a database.

According to the embodiment of the disclosure, the core service utilizes the peak clipping and valley filling characteristics of the MQ to press the instantaneous flow into the MQ pipeline, and then the instantaneous flow is consumed by the worker, and the consumption speed is controlled in the process of the consumption by the worker, so that the normal operation of the service is ensured.

Fig. 4A schematically shows a block diagram of a writing device for a database according to an embodiment of the present disclosure.

As shown in fig. 4A, the apparatus 400 includes an acquisition module 410, a data amount determination module 420, a data determination module 430, and a writing module 440.

The acquisition module 410, for example, performs operation S210 described above with reference to fig. 2A, for acquiring user data generated within a preset time window.

The data amount determining module 420, for example, performs operation S220 described above with reference to fig. 2A, for determining the data amount of the user data.

The data judging module 430, for example, performs operation S230 described above with reference to fig. 2A, for determining the first data and/or the second data from the user data according to the determined data amount.

The writing module 440, for example, performs operation S240 described above with reference to fig. 2A, for performing a preset writing operation on the database with respect to the first data and/or the second data.

Wherein the writing module 440 includes: a synchronous writing sub-module 441, configured to write the first data into the database in a synchronous manner; and/or an asynchronous write sub-module 442 for writing the second data to the database in an asynchronous manner.

Fig. 4B schematically illustrates a block diagram of a data determination module according to an embodiment of the disclosure.

As shown in fig. 4B, the data determination module 430 may include at least one of a first determination sub-module 431, a second determination sub-module 432, and a third determination sub-module 433 according to an embodiment of the present disclosure:

wherein the first determining sub-module 431, for example, performs operation S231 described above with reference to fig. 2B, for taking all the user data as the first data in the case where the determined data amount is less than or equal to the first threshold.

The second determining sub-module 432, for example, performs operation S232 described above with reference to fig. 2B, for dividing the user data into the first data and the second data at a preset ratio in the case where the determined data amount is greater than the first threshold and less than or equal to the second threshold.

The third determination submodule 433, for example, performs operation S233 described above with reference to fig. 2B, for taking all the user data as the second data in the case where the determined data amount is greater than the second threshold.

According to an embodiment of the present disclosure, the synchronous writing sub-module 441 is specifically configured to: after the first data is obtained from the user data, the first data is directly written into the database.

According to an embodiment of the present disclosure, the asynchronous write sub-module 442 is specifically configured to: after the second data is obtained from the user data, the second data is cached in the message queue, and after the preset delay time is elapsed, the second data cached in the message queue is written into the database.

Fig. 5 schematically shows a block diagram of a writing device for a database according to another embodiment of the present disclosure.

As shown in fig. 5, according to an embodiment of the present disclosure, the writing apparatus 500 for a database includes a writing speed determining module 550 and a sleep module 560 in addition to the acquiring module 410, the data amount determining module 420, the data judging module 430, and the writing module 440 in writing the second data buffered in the message queue into the database.

The writing speed determining module 550, for example, performs operation S250 described above with reference to fig. 2C, for determining a data writing speed of writing the second data into the database.

The sleep module 560, for example, performs operation S260 described above with reference to fig. 2C, for configuring at least one write thread corresponding to a preset write operation to a sleep state in the case where the data write speed exceeds the speed threshold.

Any number of modules, sub-modules, units, sub-units, or at least some of the functionality of any number of the sub-units according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented as split into multiple modules. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system-on-chip, a system-on-substrate, a system-on-package, an Application Specific Integrated Circuit (ASIC), or in any other reasonable manner of hardware or firmware that integrates or encapsulates the circuit, or in any one of or a suitable combination of three of software, hardware, and firmware. Or one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be at least partially implemented as computer program modules, which, when executed, may perform the corresponding functions.

For example, the acquisition module 410, the data amount determination module 420, the data determination module 430, the write module 440, the write speed determination module 550, and the sleep module 560. May be combined in one module or any one of the modules may be split into a plurality of modules. Or at least some of the functionality of one or more of the modules may be combined with, and implemented in, at least some of the functionality of other modules. According to embodiments of the present disclosure, at least one of the acquisition module 410, the data amount determination module 420, the data determination module 430, the write module 440, the write speed determination module 550, and the sleep module 560 may be implemented, at least in part, as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system-on-chip, a system-on-substrate, a system-on-package, an Application Specific Integrated Circuit (ASIC), or as hardware or firmware in any other reasonable manner of integrating or packaging the circuitry, or as any one of or a suitable combination of any of the three. Or at least one of the acquisition module 410, the data amount determination module 420, the data determination module 430, the writing module 440, the writing speed determination module 550, and the sleep module 560 may be at least partially implemented as a computer program module, which when executed, may perform the corresponding functions.

Fig. 6 schematically illustrates a block diagram of a computer system suitable for implementing the above-described writing method for a database, according to an embodiment of the present disclosure. The computer system illustrated in fig. 6 is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present disclosure.

As shown in fig. 6, a computer system 600 according to an embodiment of the present disclosure includes a processor 601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. The processor 601 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. Processor 601 may also include on-board memory for caching purposes. The processor 601 may comprise a single processing unit or a plurality of processing units for performing different actions of the method flows according to embodiments of the disclosure.

In the RAM 603, various programs and data required for the operation of the system 600 are stored. The processor 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The processor 601 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 602 and/or the RAM 603. Note that the program may be stored in one or more memories other than the ROM 602 and the RAM 603. The processor 601 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in the one or more memories.

According to an embodiment of the present disclosure, the system 600 may further include an input/output (I/O) interface 605, the input/output (I/O) interface 605 also being connected to the bus 604. The system 600 may also include one or more of the following components connected to the I/O interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to the I/O interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608.

According to embodiments of the present disclosure, the method flow according to embodiments of the present disclosure may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 601. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium described above carries one or more programs that, when executed, implement a writing method for a database according to an embodiment of the present disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 602 and/or RAM 603 and/or one or more memories other than ROM 602 and RAM 603 described above.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be provided in a variety of combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.

The embodiments of the present disclosure are described above. These examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.

Claims

1.A method of writing to a database, comprising:

acquiring user data generated in a preset time window;

determining a data amount of the user data;

Determining first data and/or second data from the user data according to the determined data amount; and

Performing a preset write operation to the database for the first data and/or the second data,

Wherein the preset write operation includes:

Writing the first data to the database in a synchronized manner; and/or

Writing the second data to the database in an asynchronous manner;

Said asynchronously writing said second data to said database, comprising: after the second data is obtained from the user data, the second data is cached in a message queue, and after a preset delay time, the second data cached in the message queue is written into the database;

Determining a data writing speed of writing second data cached in the message queue into the database in the process of writing the second data into the database; and under the condition that the data writing speed exceeds a speed threshold value, configuring at least one writing thread corresponding to the preset writing operation into a dormant state.

2. The method of claim 1, wherein the determining the first data and/or the second data from the user data based on the determined amount of data comprises at least one of:

taking all the user data as first data under the condition that the determined data amount is less than or equal to a first threshold value;

Dividing the user data into first data and second data according to a preset proportion under the condition that the determined data quantity is larger than the first threshold value and smaller than or equal to a second threshold value; and

And taking all the user data as second data when the determined data amount is larger than the second threshold value.

3. The method of claim 1, wherein the writing the first data to the database in a synchronized manner comprises:

And after the first data is obtained from the user data, directly writing the first data into the database.

4. A writing apparatus for a database, comprising:

The acquisition module is used for acquiring user data generated in a preset time window;

a data amount determining module for determining a data amount of the user data;

The data judging module is used for determining first data and/or second data from the user data according to the determined data quantity; and

A writing module for executing preset writing operation to the database aiming at the first data and/or the second data,

Wherein, the write module includes:

a synchronous writing sub-module, configured to write the first data into the database in a synchronous manner; and/or

An asynchronous writing sub-module, configured to asynchronously write the second data into the database;

Wherein, the asynchronous writing submodule is further used for: after the second data is obtained from the user data, the second data is cached in a message queue, and after a preset delay time, the second data cached in the message queue is written into the database;

During writing of the second data buffered in the message queue to the database,

The writing speed determining module is used for determining the data writing speed of writing the second data into the database; and

And the dormancy module is used for configuring at least one writing thread corresponding to the preset writing operation into a dormant state under the condition that the data writing speed exceeds a speed threshold value.

5. The apparatus of claim 4, wherein the data determination module comprises at least one of:

A first determining submodule, configured to take all the user data as first data when the determined data amount is less than or equal to a first threshold value;

The second determining submodule is used for dividing the user data into first data and second data according to a preset proportion under the condition that the determined data quantity is larger than the first threshold value and smaller than or equal to a second threshold value; and

And a third determining sub-module, configured to take all the user data as second data when the determined data amount is greater than the second threshold.

6. The apparatus of claim 4, wherein the synchronous writing sub-module is specifically configured to:

7. A computing device, comprising:

One or more processors;

Storage means for storing one or more programs,

Wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1 to 3.

8. A computer readable storage medium having stored thereon executable instructions which when executed by a processor cause the processor to implement the method of any of claims 1 to 3.