HK40046413B - Method and system for processing data, computer readable medium and electronic apparatus - Google Patents

Description

Data processing methods, systems, computer-readable media, and electronic devices

Technical Field

This application belongs to the field of computer technology, and specifically relates to a data processing method, system, computer-readable medium, and electronic device.

Background Technology

In high-concurrency systems, there is a phenomenon called "cache breakdown," which occurs when a large number of requests simultaneously query one or more cached data. If the cached data happens to expire, a large number of requests will directly access the database, causing a surge in database requests and resulting in query failures in both the database and the cache.

Even if a large number of requests simultaneously query the same cached data, and only a small number of requests directly access the database when the cache expires, preventing the database from crashing, other requests still need to wait for the database to be read and the cached data to be updated before responding to other requests. This results in a long response time for data queries and poor system responsiveness.

It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art.

Summary of the Invention

The purpose of this application is to provide a data processing method, system, computer-readable medium, and electronic device that at least to some extent overcomes the technical problems existing in related technologies, such as cache breakdown and poor data query responsiveness.

Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

According to one aspect of the embodiments of this application, a data processing method is provided, the method comprising:

In response to a data query request, the target cached data that matches the data query request is retrieved from the cache;

Obtain the timeout period of the target cached data and determine whether the timeout period has expired, wherein the timeout period is less than the expiration time of the target cached data;

If the timeout period has not expired, the target cached data is returned to the requester of the data query request;

If the timeout period has expired, the timeout period of the target cached data is extended, and the target database data that matches the data query request is searched in the database.

According to one aspect of the embodiments of this application, a data processing system is provided, the system comprising:

The cache lookup module is configured to, in response to a data query request, search the cache for target cache data that matches the data query request.

The time acquisition module is configured to acquire the timeout time of the target cached data and determine whether the timeout time has expired, wherein the timeout time is less than the expiration time of the target cached data;

The first execution module is configured to return the target cached data to the requester of the data query request if the timeout period has not expired.

The second execution module is configured to extend the timeout period of the target cached data if the timeout period has expired, and to search for target database data that matches the data query request in the database.

In some embodiments of this application, based on the above technical solutions, the cache lookup module includes:

The first keyword acquisition unit is configured to acquire the query keywords carried in the data query request;

The first keyword matching unit is configured to search for a target keyword that matches the query keyword in the cache, and determine whether the expiration time of the target keyword has expired;

The cached data lookup unit is configured to determine the cached data associated with the target keyword as the target cached data that matches the data query request if the expiration time of the target keyword has not expired.

In some embodiments of this application, based on the above technical solutions, the cache lookup module further includes:

The execution unit is configured to search for database data associated with the target keyword in the database if the expiration time of the target keyword has expired.

The cache update unit is configured to update the cache data associated with the target keyword based on the retrieved database data;

The target cache data acquisition unit is configured to determine the updated cache data as the target cache data that matches the data query request.

In some embodiments of this application, based on the above technical solutions, the second execution module includes:

The first delay unit is configured to determine the request time for initiating the data query request based on the request timestamp carried in the data query request;

The second delay unit is configured to extend the timeout period of the target cached data to a target time point between the request time and the expiration time of the target cached data.

In some embodiments of this application, based on the above technical solutions, the second delay unit includes:

The time difference acquisition subunit is configured to acquire the time difference between the request time and the timeout of the target cached data.

The target time point selection subunit is configured to select the target time point located between the request time and the expiration time of the target cached data based on the time difference;

The delay subunit is configured to extend the timeout period of the target cached data to the target time point.

In some embodiments of this application, based on the above technical solutions, the second execution module further includes:

The second keyword acquisition unit is configured to acquire the query keywords carried in the data query request;

The second keyword matching unit is configured to search the database for target keywords that match the query keywords;

The database data lookup unit is configured to identify database data associated with the target keyword as target database data that matches the data query request.

In some embodiments of this application, based on the above technical solutions, the second execution module further includes:

The locking unit is configured to initiate a locking request to acquire the thread lock through the current thread that initiated the data query request;

The sleep unit is configured to put the current thread into a sleep state if the current thread fails to acquire the lock.

The running unit is configured to keep the current thread running if the current thread successfully acquires the lock.

In some embodiments of this application, based on the above technical solutions, the sleep unit includes:

The monitoring subunit is configured to monitor the current thread to determine whether the wake-up condition of the current thread is met;

The wake-up subunit is configured to set the current thread to a running state when the wake-up condition of the current thread is met, so as to re-initiate the data query request through the current thread.

In some embodiments of this application, based on the above technical solutions, the monitoring subunit includes:

The time monitoring subunit is configured to monitor the sleep time of the current thread;

The first execution subunit is configured to determine that the wake-up condition of the current thread is met if the sleep time is greater than or equal to a preset time threshold.

The second execution subunit is configured to determine that the wake-up condition of the current thread is not met if the sleep time is less than the time threshold.

In some embodiments of this application, based on the above technical solutions, the monitoring subunit includes:

The state monitoring subunit is configured to monitor the current state of the thread lock that the current thread is requesting to acquire;

The third execution subunit is configured to determine that the wake-up condition of the current thread is met if the thread lock is in a released state.

The fourth execution subunit is configured to determine that the wake-up condition of the current thread is not met if the thread lock is in a locked state.

In some embodiments of this application, based on the above technical solutions, the thread lock is a mutex lock.

In some embodiments of this application, based on the above technical solutions, the second execution module further includes:

The cache update unit is configured to update the target cache data in the cache according to the target database data;

The time reset unit is configured to reset the timeout and expiration time of the target cached data;

The data return unit is configured to release the thread lock and return the updated target cached data to the requester of the data query request.

According to one aspect of the embodiments of this application, a computer-readable medium is provided, on which a computer program is stored, which, when executed by a processor, implements the data processing method as described in the above technical solutions.

According to one aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform a data processing method as described above by executing the executable instructions.

According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data processing method as described in the above technical solutions.

In the technical solution provided in this application embodiment, by returning the target cached data to the requester of the data query request before the timeout period of the target cached data expires, extending the timeout period of the target cached data after the timeout period has expired, and searching for target database data that matches the data query request in the database, the target cached data whose timeout period has expired but whose expiration time has not expired can be made available again, and the target cached data whose timeout period has expired can be updated. This enables frequently queried cached data to remain valid for a long time, avoiding cache breakdown and query failure problems that occur when a large number of queries simultaneously query the same cached data and the cached data just expires. It also avoids the waiting time required for querying the database after cache expiration, thereby improving the system response speed.

It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application.

Attached Figure Description

The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

Figure 1 schematically illustrates an exemplary system architecture block diagram applying the technical solution of this application.

Figure 2 schematically illustrates the steps of the data processing method provided in the embodiments of this application.

Figure 3 schematically illustrates the exemplary processing steps of the data query request of the technical solution of this application under the exemplary architecture.

Figure 4 schematically illustrates part of the steps in an embodiment of this application for finding target cached data that matches a data query request in the cache.

Figure 5 schematically illustrates part of the steps in an embodiment of this application for finding target cached data that matches a data query request in the cache.

Figure 6 schematically illustrates the steps for extending the timeout period of the target cached data in an embodiment of this application.

Figure 7 schematically illustrates the steps in an embodiment of this application to extend the timeout of the target cached data to a target time point between the request time and the expiration time of the target cached data.

Figure 8 schematically illustrates the steps of the statistical content triggering operation in an embodiment of this application.

Figure 9 schematically illustrates the steps in an embodiment of this application before searching for target database data that matches the data query request.

Figure 10 schematically illustrates the steps after configuring the current thread to a sleep state in an embodiment of this application.

Figure 11 schematically illustrates the steps in an embodiment of this application to monitor the current thread to determine whether the wake-up conditions of the current thread are met.

Figure 12 schematically illustrates the steps in an embodiment of this application to monitor the current thread to determine whether the wake-up conditions of the current thread are met.

Figure 13 schematically illustrates the steps in an embodiment of this application after searching for target database data that matches the data query request in the database.

Figure 14 schematically shows a structural block diagram of the data processing system 1400 provided in an embodiment of this application.

Figure 15 schematically illustrates a computer system architecture block diagram suitable for implementing electronic devices according to embodiments of the present application.

Detailed Implementation

Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and/or processor devices and/or microcontroller devices.

The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations/steps, nor do they necessarily have to be performed in the described order. For example, some operations/steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

Figure 1 schematically illustrates an exemplary system architecture block diagram applying the technical solution of this application.

As shown in Figure 1, the system architecture 100 may include terminal devices 110, a network 120, and a server 130. Terminal devices 110 may include various electronic devices such as smartphones, tablets, laptops, desktop computers, and in-vehicle devices. Server 130 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Network 120 may be a communication medium of various connection types capable of providing a communication link between terminal devices 110 and server 130, such as a wired communication link or a wireless communication link.

Depending on the implementation requirements, the system architecture in this application embodiment can have any number of terminal devices, networks, and servers. For example, server 130 can be a server group composed of multiple server devices. In addition, the technical solutions provided in this application embodiment can be applied to terminal device 110, or to server 130, or can be implemented jointly by terminal device 110 and server 130. This application does not impose any special limitations on this.

For example, server 130 may store financial payment agreements for querying, and terminal device 110 may initiate a query request to query the financial payment agreements on server 130. When a user entrusts WeChat Pay to make a deduction, at the preset deduction time, WeChat Pay will send a data query request to query the financial payment agreement signed by the user on the server to deduct the fee. When a large number of deduction requests are generated at the same time, a large number of data query requests will be generated to query the financial payment agreement signed by the user on the server, resulting in high-concurrency queries. A large number of queries are also querying cached data. When the cached data happens to expire, a large number of requests will directly access the database, impacting the database and causing query failures in both the database and the cache. To avoid impacting the database and causing query failures in the database and cache, the data processing method provided in this application can be executed on the terminal device 110 or server 130. This method returns the target cached data to the requester of the data query request before the timeout expires, extends the timeout of the target cached data if it has expired, and searches the database for target database data matching the data query request to update the cache. This allows target cached data whose timeout has expired but whose expiration time has not yet expired to become available again, and updates the expired target cached data. This ensures that frequently queried cached data remains valid for a long time, preventing cache breakdown and query failures that occur when a large number of queries simultaneously query the same cached data and that data has just expired. This guarantees cache availability, protects the database, improves business stability and database availability, and avoids request waiting caused by the time required to query the database after cache expiration, thereby improving system response speed.

The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, in-vehicle device, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

Cloud technology is a collective term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied based on the cloud computing business model. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will all require robust system support, which can only be achieved through cloud computing.

Cloud computing is a computing model that distributes computing tasks across a resource pool consisting of a large number of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, the resources in the "cloud" are infinitely scalable, readily available, on-demand, expandable, and pay-as-you-go.

As a provider of fundamental cloud computing capabilities, a cloud resource pool (referred to as a cloud platform, generally called an IaaS (Infrastructure as a Service) platform) is established. Various types of virtual resources are deployed in the resource pool for external customers to choose from. The cloud resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.

Based on logical function, a PaaS (Platform as a Service) layer can be deployed on top of the IaaS (Infrastructure as a Service) layer, and a SaaS (Software as a Service) layer can be deployed on top of the PaaS layer. Alternatively, SaaS can be directly deployed on top of IaaS. PaaS is the platform for running software, such as databases and web containers. SaaS refers to various types of business software, such as web portals and bulk SMS senders. Generally speaking, SaaS and PaaS are upper layers compared to IaaS.

Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology and distributed storage file systems to bring together a large number of storage devices of various types in the network (storage devices are also called storage nodes) to work together through application software or application interfaces to provide data storage and business access functions to the outside world.

Currently, the storage method in storage systems is as follows: Logical volumes are created, and during creation, physical storage space is allocated to each logical volume. This physical storage space may consist of a single storage device or the disks of several storage devices. Clients store data on a logical volume, which means storing the data on the file system. The file system divides the data into many parts, each part being an object. Each object contains not only the data but also additional information such as a data identifier (ID, ID entity). The file system writes each object to the physical storage space of that logical volume, and it records the storage location information of each object. Therefore, when a client requests access to data, the file system can allow the client to access the data based on the storage location information of each object.

The process by which a storage system allocates physical storage space to a logical volume is as follows: the physical storage space is pre-divided into stripes according to the capacity estimate of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of Redundant Array of Independent Disks (RAID). A logical volume can be understood as a stripe, thus allocating physical storage space to the logical volume.

A database, simply put, can be viewed as an electronic filing cabinet—a place to store electronic files, where users can perform operations such as adding, querying, updating, and deleting data. A "database" is a collection of data stored together in a certain way, capable of being shared by multiple users, with minimal redundancy, and independent of application programs.

A Database Management System (DBMS) is a computer software system designed to manage databases, generally possessing basic functions such as storage, retrieval, security, and backup. DBMSs can be classified according to the database model they support, such as relational or XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters or mobile devices; or according to the query language used, such as SQL (Structured Query Language) or XQuery; or according to performance priorities, such as maximum scale or maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, supporting multiple query languages simultaneously.

The system involved in the embodiments of the present invention can be a distributed system formed by connecting clients and multiple nodes (any form of computing device in the network, such as servers and user terminals) through network communication. As disclosed in the data processing method or data processing system of this application, multiple servers can form a blockchain, and the servers are nodes on the blockchain.

Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and cryptographic algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying platform, a platform product service layer, and an application service layer.

The underlying blockchain platform can include processing modules such as user management, basic services, and smart contracts. The user management module is responsible for managing the identity information of all blockchain participants, including maintaining public and private key generation (account management), key management, and maintaining the correspondence between user real identities and blockchain addresses (access management). Under authorization, it also monitors and audits transactions of certain real identities and provides risk control rule configuration (risk control audit). The basic services module is deployed on all blockchain node devices to verify the validity of business requests. After consensus is reached on valid requests, they are recorded in storage. For a new business request, the basic services first perform interface adaptation parsing and authentication (interface adaptation), then encrypt the business information using a consensus algorithm (consensus management), and finally transmit it completely and consistently to the shared ledger (network communication) for recording and storage. The smart contract module is responsible for contract registration, issuance, triggering, and execution. Developers can define contract logic using a programming language and publish it to the blockchain (contract registration). Based on the contract terms, execution is triggered by calling keys or other events to complete the contract logic. It also provides functions for contract upgrades and cancellations.

The platform's product service layer provides the basic capabilities and implementation frameworks for typical applications. Developers can leverage these basic capabilities, along with the specific characteristics of their business needs, to implement blockchain-based business logic. The application service layer provides blockchain-based application services to business stakeholders.

The data processing method provided in this application will be described in detail below with reference to specific implementation methods.

Figure 2 shows a flowchart of the data processing method according to certain embodiments of this application. The executing entity of this data processing method can be a terminal device or a server, etc., and this application does not set any limitations. As shown in Figure 2, the information processing method mainly includes the following steps S210 to S240:

S210. In response to a data query request, locate the target cached data that matches the data query request in the cache.

Frequently accessed data can be copied to a fast storage location closer to the application for caching, which can improve system performance and scalability. One such cache is Memcached, a distributed high-speed caching system that caches database queries, reducing the number of database accesses and improving the response speed of web pages or application servers.

Figure 3 schematically illustrates the visualization effect of content display information being triggered and displayed in the main content display body. As shown in Figure 3, the requesting end can generate a large number of high-concurrency requests, including concurrent query request 1 and concurrent query request 2. Concurrent query request 1 and concurrent query request 2 correspond to the same target cached data. In response to requests 1 and 2 in this high-concurrency request, the cache searches for the target cached data that matches the data query request in the cache.

S220. Obtain the timeout of the target cached data and determine whether the timeout has expired. The timeout is less than the expiration time of the target cached data.

After finding the target cached data that matches the data query request, the timeout period (timeout1) of the target cached data is obtained, and it is determined whether timeout1 has expired. Timeout1 is less than the expiration time (timeout2) of the target cached data. If the timeout timestamp of cached data has expired, the timeout is older than the current system time, indicating that the cached data is outdated and needs to be updated by querying the database. Cached data whose timeout1 has expired but whose expiration time (timeout2) has not expired is still stored in the cache and can be read, but the data is older. Cached data whose expiration time (timeout2) has expired is no longer stored in the cache and cannot be read.

Specifically, the current system time can be obtained and compared with the timeout period (timeout1) to determine whether timeout1 has expired. If the current system time is after the timeout period (timeout1), then timeout1 is considered expired. If the current system time is before the timeout period (timeout1) or the two times are the same, then timeout1 is considered not expired.

Similarly, the current system time can be obtained and compared with the time of expiration time 2 to determine whether expiration time 2 has expired. If the current system time is after the time of expiration time 2, then expiration time 2 is considered to have expired. If the current system time is before the time of expiration time 2, or if the current system time and the time of expiration time 2 are the same, then expiration time 2 is considered not to have expired.

S230. If the timeout period has not expired, return the target cached data to the requester of the data query request.

If the timeout period 1 has not expired, the target cached data is still stored in the cache and does not need to be updated. At this time, the target cached data is directly returned to the requester of the data query request, and the system response speed is relatively fast.

S240. If the timeout period has expired, extend the timeout period of the target cached data and search the database for target database data that matches the data query request.

When timeout1 expires, the timeout of the target cached data is extended by timeout1. The system then searches the database for target database data that matches the data query request, updates the cache with this new target cached data, and overwrites or deletes the original target cached data whose timeout1 has expired. As shown in Figure 3, if the timeout1 of the target cached data matching the concurrent query service's data query request 1 has expired, timeout1 is extended and used as the new timeout for the target cached data matching data query request 1. This ensures that when concurrent query service request 2, carrying the same query keyword as request 1, arrives at the cache after data query request 1, the timeout1 for request 2 to find and retrieve the target cached data has not expired, and timeout2, being after timeout1, has also not expired. Therefore, request 2 can directly hit the cache and return the cached data, thus improving the system's response speed.

In a specific example, the timeout period (timeout1) can be set to 10 minutes after the cached data is generated, and the expiration period (timeout2) can be set to 20 minutes after the cached data is generated. When the timeout period (timeout1) of the target cached data has expired, the timeout period (timeout1) of the target cached data is extended. For example, the timeout period (timeout1) of the target cached data is extended from 10 minutes after the cached data is generated to 15 minutes after the cached data is generated. In this case, the extended timeout period (timeout1) is still less than the expiration period (timeout2), meaning that the time of the extended timeout period (timeout1) is still before the time of the expiration period (timeout2).

The pseudocode for setting a timeout or expiration time can be shown below:

The following describes some of the steps in the above embodiments in detail with reference to Figures 4 to 11.

Figure 4 schematically illustrates part of the steps in the embodiment of this application for finding target cache data that matches the data query request in the cache. As shown in Figure 4, based on the above embodiment, step S210, finding target cache data that matches the data query request in the cache, can further include the following steps S410 to S430:

S410. Obtain the query keywords carried in the data query request;

S420. Locate the target keyword that matches the query keyword in the cache and determine whether the expiration time of the target keyword has expired;

S430. If the expiration time of the target keyword has not expired, the cached data associated with the target keyword will be determined as the target cached data that matches the data query request.

Caching can store data in a key-value format. A query request carries a query keyword, which can be used to search the cache for a matching target keyword and determine if the target keyword's expiration time has expired. If the target keyword's expiration time has not expired, the cached data associated with that target keyword is identified as the target cached data matching the data query request. The expiration time of the target keyword can be the same as the expiration time of the corresponding target cached data. For example, both the expiration time of the target keyword and the expiration time of the corresponding target cached data can be `timeout2`.

Figure 5 schematically illustrates part of the steps in the embodiment of this application for finding target cache data that matches the data query request in the cache. As shown in Figure 5, based on the above embodiment, step S210, finding target cache data that matches the data query request in the cache, may further include the following steps S510 to S530:

S510. If the expiration time of the target keyword has expired, search the database for database data associated with the target keyword;

S520. Update the cached data associated with the target keyword based on the retrieved database data;

S530. Determine the updated cached data as the target cached data that matches the data query request.

If the timeout period 2 for the target keyword has expired, it means that the target cache data corresponding to the target keyword is no longer stored in the cache, or the target cache data stored in the cache has expired and is unavailable. In this case, the database associated with the target keyword is searched, and the cache data associated with the target keyword is updated based on the searched database data. The updated cache data is then identified as the target cache data that matches the data query request, and this target cache data is returned to the requester who issued the data query request.

It is understandable that cache storage space is limited. When the cache space is full, in order to ensure stable service while effectively improving the hit rate, an expiration time can be set for the cache data. After the cache data expires, the cache data can be cleaned up, which can ensure the accuracy of the cache data while cleaning up expired cache, thus improving the cache hit rate.

Figure 6 schematically illustrates the steps for extending the timeout period of the target cached data in an embodiment of this application. As shown in Figure 6, based on the above embodiment, extending the timeout period of the target cached data in step S240 may further include the following steps S610 to S620:

S610. Determine the request time of initiating the data query request based on the request timestamp carried in the data query request;

S620. Extend the timeout period of the target cached data to the target time point between the request time and the expiration time of the target cached data.

It's understandable that determining the request time based on the request timestamp carried in the data query request, and extending the timeout of the target cached data to a target time point between the request time and the expiration time of the target cached data, ensures that the target cache timeout does not expire for a period after the data query request is initiated. This period could be 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, etc. Within this period, the data query request has sufficient time to complete the search in the database and return the target database data matching the data query request to the cache, and update the target cached data in the cache according to the target database data. At this time, the generation time of the cached data is also updated along with the update of the target cached data in the cache, updating to the current system time. Simultaneously, the timeout and expiration time of the target cached data are also reset, for example, resetting the timeout of the target cached data to 10 minutes after the cached data was generated, and resetting the expiration time of the target cached data to 20 minutes after the cached data was generated. Therefore, it can be guaranteed that during the data query request 1's database query, the timeout of the target cached data does not expire, allowing for direct cache hits and returns, which helps improve system response speed. Furthermore, after data query request 1 completes the database query, it can update the target cache data in the cache based on the target database data, and reset the timeout and expiration time of the target cache data, which helps to improve the accuracy of the data in the cache.

Figure 7 schematically illustrates the steps of extending the timeout of the target cached data to a target time point between the request time and the expiration time of the target cached data in an embodiment of this application. As shown in Figure 7, based on the above embodiment, step S620, extending the timeout of the target cached data to a target time point between the request time and the expiration time of the target cached data, may further include the following steps S710 to S730:

S710. Obtain the time difference between the request time and the timeout of the target cached data;

S720. Select the target time point between the request time and the expiration time of the target cached data based on the time difference;

S730. Extend the timeout period for the target cached data to the target time point.

The specific process of extending the timeout of the target cached data to a target time point between the request time and the expiration time of the target cached data can be as follows: obtain the time difference between the request time of the data query request and the timeout time of the target cached data, and select a target time point between the request time and the expiration time of the target cached data based on the time difference. When the time difference between the request time of the data query request and the timeout time of the target cached data is large, the target time point is selected as a time point closer to the expiration time of the target cached data.

Figure 8 schematically illustrates the steps of searching for target database data that matches the data query request in the database according to an embodiment of this application. As shown in Figure 8, based on the above embodiment, step S240, searching for target database data that matches the data query request in the database, may further include the following steps S810 to S830:

S810. Obtain the query keywords carried in the data query request;

S820. Search the database for target keywords that match the query keywords;

S830. Identify the database data associated with the target keyword as the target database data that matches the data query request.

Databases can also store data in key-value pairs. Data query requests carry query keywords, which can be used to search the database for target keywords that match the query keywords. During the database query process, the database data associated with the target keywords is identified as the target database data that matches the data query request. Both databases and caches store data in key-value pairs, which facilitates data retrieval and allows the target database data to easily update the cached data.

Figure 9 schematically illustrates the steps in this embodiment before searching for target database data that matches the data query request in the database. As shown in Figure 9, based on the above embodiment, before searching for target database data that matches the data query request in the database in step S240, the following steps S910 to S930 may be further included:

S910. The current thread that initiated the data query request initiates a lock acquisition request to obtain the thread lock;

S920. If the current thread fails to acquire the lock, then configure the current thread to a sleeping state;

S930. If the current thread successfully acquires the lock, then keep the current thread running.

Before searching for target database data that matches the query request, the current thread initiating the query request requests a lock acquisition request. Threads that successfully acquire the lock remain running, while threads that fail to acquire the lock are put into a sleep state. This limits the number of database query requests, preventing a large number of requests from directly accessing the database and avoiding database overload. Only requests that successfully acquire the thread lock can query the database; other requests must enter a sleep state and wait.

Figure 10 schematically illustrates the process flow after configuring the current thread to a sleep state in an embodiment of this application. As shown in Figure 10, based on the above embodiment, after configuring the current thread to a sleep state in step S920, the following steps S1010 to S1020 may be further included:

S1010. Monitor the current thread to determine if the wake-up condition for the current thread is met;

S1020. When the wake-up condition of the current thread is met, configure the current thread to run so that the data query request can be re-initiated through the current thread.

After configuring the current thread to a sleeping state, the current thread can be monitored to determine whether the wake-up conditions are met so that the thread can be woken up again. When the wake-up conditions are met, the current thread is configured to run, allowing it to re-initiate the data query request and continue completing the data query.

Figure 11 schematically illustrates the steps of monitoring the current thread to determine whether the wake-up condition of the current thread is met in an embodiment of this application. As shown in Figure 11, based on the above embodiment, the monitoring of the current thread in step S1010 to determine whether the wake-up condition of the current thread is met may further include the following steps S1110 to S1130:

S1110. Monitor the current thread's sleep time;

S1120. If the sleep time is greater than or equal to the preset time threshold, then the wake-up condition of the current thread is determined to be met;

S1130. If the sleep time is less than the time threshold, then it is determined that the wake-up condition of the current thread is not met.

The current thread is monitored to determine if its wake-up condition is met. Specifically, the current thread's sleep time can be monitored. If the sleep time is greater than or equal to a preset time threshold, the wake-up condition is met; if the sleep time is less than the time threshold, the wake-up condition is not met. The time threshold can be set slightly longer than the time required for a data query request to retrieve data from the database and update the cache. For example, if the time required for a data query request to retrieve data from the database and update the cache is typically 8ms to 10ms, the time threshold can be set to 12ms. Before the sleep time reaches the time threshold, the data query request has already completed its search in the database and returned the target database data matching the query request to the cache. The target cache data is then updated based on the target database data, and the timeout and expiration time of the target cache data are reset. Thus, after the current thread completes its sleep and wakes up, it can re-initiate the data query request to hit the cache and return the corresponding target cache data to the requester.

Figure 12 schematically illustrates the steps of monitoring the current thread to determine whether the wake-up condition of the current thread is met in an embodiment of this application. As shown in Figure 12, based on the above embodiment, the monitoring of the current thread in step S1010 to determine whether the wake-up condition of the current thread is met can further include the following steps S1210 to S1230:

S1210. Monitor the current state of the thread lock that the current thread is requesting to acquire;

S1220. If the thread lock is in a released state, then the wake-up condition of the current thread is determined to be met;

S1230. If the thread lock is in a locked state, then it is determined that the wake-up condition of the current thread is not met.

The system monitors the current thread to determine if its wake-up condition is met. Specifically, it monitors the current state of the thread lock the current thread is requesting. If the thread lock is in a released state, the wake-up condition is met; if the thread lock is in a locked state, the wake-up condition is not met. The thread lock is released only when a data query successfully retrieves data from the database and updates the target cache data based on the target database data. At this point, the wake-up condition is met, the current thread is woken up, and it re-initiates the data query to hit the cache, returning the corresponding target cache data to the requester. Therefore, when the thread lock is released, the current thread is woken up to re-initiate the data query to hit the cache; when the thread lock is locked, the current thread is not woken up and continues to sleep, waiting for the thread to be released. This ensures that the thread lock is woken up when the data query successfully retrieves data from the database and updates the target cache data based on the target database data, minimizing the waiting time of sleeping threads and improving system response speed.

In some implementations, thread locks can be mutexes. A mutex consists of a mutex, which a thread acquires before accessing a shared resource and releases afterward. Once a mutex is locked, any other thread attempting to acquire it again will be blocked until the current thread releases the mutex. If multiple threads are blocked when the mutex is released, all blocked threads on that mutex become runnable. The first runnable thread can acquire the mutex, and other threads will see that the mutex is still locked and must wait for it to become available again. In this approach, only one thread can execute a database query at a time, preventing a large number of requests from directly accessing the database, thus avoiding the impact of high concurrency and improving system stability. Furthermore, for high concurrency requests, only one request can access the database to query data and update the cache at any given time, avoiding multiple database accesses and saving system resources.

Figure 13 schematically illustrates the steps following the search for target database data matching the data query request in the database in this embodiment of the application. As shown in Figure 13, based on the above embodiment, after searching for target database data matching the data query request in the database in step S240, the following steps S1310 to S1330 may be further included:

S1310. Update the target cache data in the cache according to the target database data;

S1320. Reset the timeout and expiration time of the target cached data;

S1330. Release the thread lock and return the updated target cached data to the requester of the data query request.

After retrieving the target database data that matches the data query request, the target cache data in the cache can be updated based on the target database data. The timeout and expiration time of the target cache data are then reset. The thread lock is released, and the updated target cache data is returned to the requester of the data query request, thus realizing the system's response to the data query requester. This allows the retrieved database data to be updated in the cache after each database query, thereby updating the cached data. Furthermore, for frequently accessed cached data, the likelihood of it being accessed within the period between the timeout and expiration time is higher. Therefore, for frequently accessed cached data, when the timeout expires but the expiration time has not, there is a greater chance that the target cache data in the cache will be updated based on the corresponding target database data, and the timeout and expiration time of the target cache data will be reset. This allows the target cache data to be updated and its expiration time reset before it expires, preventing the frequently accessed target cache data from becoming invalid. This improves the system's response speed to data query requests carrying hot keywords and avoids the system being impacted by high-concurrency requests.

In related technologies, to protect database security, even if a large number of requests do not arrive at the backend database simultaneously, at the moment the cache expires, a large number of requests will sleep because they cannot acquire the mutex lock. Before the request that acquires the thread lock returns the database query result, all requests querying the same key need to wait. Only after waiting for a period of time and retrying can the data be read. The client does not receive a timely response, thereby reducing the response speed of the query service and causing the billing or information retrieval process to be sluggish. When the data processing method provided in this application is executed on the terminal device 110 or the server 130, the target cache data whose timeout period has expired but whose expiration time has not expired can be made available again. This avoids blocking new requests with the same keywords during the database query process. Instead, it allows the new request to directly hit the cache and return the data immediately. Furthermore, the data of the target cache data whose timeout period has expired is updated and the timeout and expiration time are reset. This enables frequently queried cache data to remain valid for a long time and achieve automatic renewal. This ensures the continuous availability of the cache, improves the cache hit rate, protects the database, improves the stability of system business, and avoids the waiting time caused by the time required to query the database after the cache expires, thereby improving the system response speed.

It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and/or one step may be broken down into multiple steps.

The following describes an apparatus embodiment of this application, which can be used to execute the data processing method described in the above embodiments of this application. Figure 14 schematically shows a structural block diagram of a data processing system 1400 provided in an embodiment of this application. As shown in Figure 14, the data processing system 1400 may include:

The cache lookup module 1410 is configured to, in response to a data query request, search the cache for target cache data that matches the data query request.

The time acquisition module 1420 is configured to acquire the timeout period of the target cached data and determine whether the timeout period has expired. The timeout period is less than the expiration time of the target cached data.

The first execution module 1430 is configured to return the target cached data to the requester of the data query request if the timeout period has not expired.

The second execution module 1440 is configured to extend the timeout period of the target cached data if the timeout period has expired, and search for target database data that matches the data query request in the database.

In some embodiments of this application, based on the above technical solutions, the cache lookup module 1410 includes:

The first keyword acquisition unit is configured to acquire the query keywords carried in the data query request;

The first keyword matching unit is configured to search for the target keyword that matches the query keyword in the cache and determine whether the expiration time of the target keyword has expired.

The cached data lookup unit is configured to determine the cached data associated with the target keyword as the target cached data that matches the data query request if the expiration time of the target keyword has not expired.

In some embodiments of this application, based on the above technical solutions, the cache lookup module 1410 further includes:

The execution unit is configured to search for database data associated with the target keyword if the expiration time of the target keyword has expired.

The cache update unit is configured to update the cache data associated with the target keyword based on the retrieved database data;

The target cache data acquisition unit is configured to determine the updated cache data as the target cache data that matches the data query request.

In some embodiments of this application, based on the above technical solutions, the second execution module 1440 includes:

The first delay unit is configured to determine the request time for initiating the data query request based on the request timestamp carried in the data query request;

The second delay unit is configured to extend the timeout period of the target cached data to a target time point between the request time and the expiration time of the target cached data.

In some embodiments of this application, based on the above technical solutions, the second delay unit includes:

The time difference acquisition subunit is configured to acquire the time difference between the request time and the timeout of the target cached data.

The target time point selection subunit is configured to select a target time point between the request time and the expiration time of the target cached data based on the time difference.

The delay subunit is configured to extend the timeout period of the target cached data to the target time point.

In some embodiments of this application, based on the above technical solutions, the second execution module 1440 further includes:

The second keyword acquisition unit is configured to acquire the query keywords carried in the data query request;

The second keyword matching unit is configured to search the database for target keywords that match the query keywords;

The database lookup unit is configured to identify database data associated with a target keyword as the target database data that matches the data query request.

In some embodiments of this application, based on the above technical solutions, the second execution module 1440 further includes:

The locking unit is configured to initiate a locking request to acquire the thread lock through the current thread that initiated the data query request;

The sleep unit is configured to put the current thread into a sleep state if the current thread fails to acquire the lock.

The execution unit is configured to keep the current thread running if the current thread successfully acquires the lock.

In some embodiments of this application, based on the above technical solutions, the sleep unit includes:

The monitoring subunit is configured to monitor the current thread to determine whether the wake-up condition of the current thread is met;

The wake-up subunit is configured to set the current thread to a running state when the wake-up condition of the current thread is met, so that the data query request can be re-initiated through the current thread.

In some embodiments of this application, based on the above technical solutions, the monitoring subunit includes:

The time monitoring subunit is configured to monitor the sleep time of the current thread;

The first execution subunit is configured to determine that the wake-up condition of the current thread is met if the sleep time is greater than or equal to a preset time threshold.

The second execution subunit is configured to determine that the wake-up condition of the current thread is not met if the sleep time is less than the time threshold.

In some embodiments of this application, based on the above technical solutions, the monitoring subunit includes:

The state monitoring subunit is configured to monitor the current state of the thread lock that the current thread is requesting to acquire;

The third execution subunit is configured to determine if the current thread wake-up condition is met if the thread lock is in a released state.

The fourth execution subunit is configured to determine that the wake-up condition for the current thread is not met if the thread lock is in a locked state.

In some embodiments of this application, based on the above technical solutions, the thread lock is a mutex lock.

In some embodiments of this application, based on the above technical solutions, the second execution module 1440 further includes:

The cache update unit is configured to update the target cache data in the cache based on the target database data.

The time reset unit is configured to reset the timeout and expiration time of the target cached data.

The data return unit is configured to release the thread lock and return the updated target cached data to the requester of the data query request.

The specific details of the data processing systems provided in the various embodiments of this application have been described in detail in the corresponding method embodiments, and will not be repeated here.

Figure 15 schematically illustrates a computer system architecture block diagram for implementing an electronic device according to an embodiment of this application.

It should be noted that the computer system 1500 of the electronic device shown in Figure 15 is only an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

As shown in Figure 15, the computer system 1500 includes a central processing unit (CPU) 1501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1502 or programs loaded from storage section 1508 into random access memory (RAM). The RAM 1503 also stores various programs and data required for system operation. The CPU 1501, ROM 1502, and RAM 1503 are interconnected via a bus 1504. An input/output interface 1505 (I/O interface) is also connected to the bus 1504.

The following components are connected to the input/output interface 1505: an input section 1506 including a keyboard, mouse, etc.; an output section 1507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1508 including a hard disk, etc.; and a communication section 1509 including a network interface card such as a local area network card, modem, etc. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the input/output interface 1505 as needed. A removable medium 1511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1510 as needed so that computer programs read from it can be installed into the storage section 1508 as needed.

Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1509, and/or installed from removable medium 1511. When the computer program is executed by central processing unit 1501, it performs various functions defined in the system of this application.

It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims (13)

1. A data processing method, characterized in that it includes: In response to a data query request, the target cached data that matches the data query request is retrieved from the cache; Obtain the timeout period of the target cached data and determine whether the timeout period has expired, wherein the timeout period is less than the expiration time of the target cached data; If the timeout period has not expired, the target cached data is returned to the requester of the data query request; If the timeout period has expired, a lock acquisition request to acquire the thread lock is initiated by the current thread that initiated the data query request. If the current thread fails to acquire the lock, then the current thread is configured to sleep. If the current thread successfully acquires the lock, then the current thread remains running. Extend the timeout period of the target cached data so that other data query requests can hit and return the target cached data; Search the database for target database data that matches the data query request; Update the target cache data in the cache according to the target database data; Reset the timeout and expiration time of the target cached data; Release the thread lock and return the updated target cached data to the requester of the data query request. 2. The data processing method according to claim 1, characterized in that, the step of searching for target cached data matching the data query request in the cache includes: Obtain the query keywords carried in the data query request; Search the cache for a target keyword that matches the query keyword, and determine whether the expiration time of the target keyword has expired; If the expiration time of the target keyword has not expired, the cached data associated with the target keyword will be determined as the target cached data that matches the data query request. 3. The data processing method according to claim 2, characterized in that, the step of searching for target cached data matching the data query request in the cache further includes: If the expiration time of the target keyword has expired, then search the database for database data associated with the target keyword; Update the cached data associated with the target keyword based on the retrieved database data; The updated cached data is identified as the target cached data that matches the data query request. 4. The data processing method according to claim 1, wherein extending the timeout period of the target cached data includes: The request time when the data query request was initiated is determined based on the request timestamp carried in the data query request; Extend the timeout period of the target cached data to a target time point between the request time and the expiration time of the target cached data. 5. The data processing method according to claim 4, characterized in that extending the timeout of the target cached data to a target time point located between the request time and the expiration time of the target cached data includes: Obtain the time difference between the request time and the timeout of the target cached data; The target time point is selected based on the time difference, which is located between the request time and the expiration time of the target cached data. Extend the timeout period of the target cached data to the target time point. 6. The data processing method according to claim 1, characterized in that, the step of searching for target database data matching the data query request in the database includes: Obtain the query keywords carried in the data query request; Search the database for target keywords that match the query keywords; The database data associated with the target keyword is identified as the target database data that matches the data query request. 7. The data processing method according to claim 1, characterized in that, after configuring the current thread to a sleep state, the method further includes: Monitor the current thread to determine whether the wake-up condition for the current thread is met; When the wake-up condition of the current thread is met, the current thread is configured to run, so that the data query request can be re-initiated through the current thread. 8. The data processing method according to claim 7, characterized in that, monitoring the current thread to determine whether the wake-up condition of the current thread is met includes: Monitor the sleep time of the current thread; If the sleep time is greater than or equal to a preset time threshold, then the wake-up condition of the current thread is determined to be met. If the sleep time is less than the time threshold, then it is determined that the wake-up condition of the current thread is not met. 9. The data processing method according to claim 7, characterized in that, monitoring the current thread to determine whether the wake-up condition of the current thread is met includes: Monitor the current state of the thread lock that the current thread is requesting to acquire; If the thread lock is in a released state, then the wake-up condition of the current thread is determined to be met; If the thread lock is in a locked state, it is determined that the wake-up condition for the current thread is not met. 10. The data processing method according to claim 1, wherein the thread lock is a mutex lock. 11. A data processing system, characterized in that it comprises: The cache lookup module is configured to, in response to a data query request, search the cache for target cache data that matches the data query request. The time acquisition module is configured to acquire the timeout time of the target cached data and determine whether the timeout time has expired, wherein the timeout time is less than the expiration time of the target cached data; The first execution module is configured to return the target cached data to the requester of the data query request if the timeout period has not expired. The second execution module is configured to extend the timeout period of the target cached data if the timeout period has expired, so that other data query requests can hit and return the target cached data, and search for target database data that matches the data query request in the database. The second execution module also includes: The locking unit is configured to initiate a locking request to acquire the thread lock through the current thread that initiated the data query request; The sleep unit is configured to put the current thread into a sleep state if the current thread fails to acquire the lock. The execution unit is configured to keep the current thread running if the current thread successfully acquires the lock. The cache update unit is configured to update the target cache data in the cache according to the target database data; The time reset unit is configured to reset the timeout and expiration time of the target cached data; The data return unit is configured to release the thread lock and return the updated target cached data to the requester of the data query request. 12. A computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method according to any one of claims 1 to 10. 13. An electronic device, characterized in that it comprises: Processor; and Memory for storing the executable instructions of the processor; The processor is configured to execute the data processing method of any one of claims 1 to 10 by executing the executable instructions.