CN113778867A

CN113778867A - Test data generation method and device

Info

Publication number: CN113778867A
Application number: CN202111031365.7A
Authority: CN
Inventors: 李亦辰; 刘迎; 赵月; 周亮; 谭野墨; 刘哲; 梁福德; 韩娜; 李鹤; 丁艳丽; 赵琳
Original assignee: Bank of China Ltd
Current assignee: Bank of China Ltd
Priority date: 2021-09-03
Filing date: 2021-09-03
Publication date: 2021-12-10

Abstract

The application provides a test data generation method and device, relates to the technical field of data processing, and can generate test data which are in line with actual production in batches. The method comprises the following steps: determining first data, wherein the first data comprises multiple types of data; determining a test data dictionary according to the first data; determining a first characteristic of the first data, the first characteristic comprising at least one of: the method comprises the following steps of (1) frequency of characteristic values, cluster analysis results, data correlation, association rules and mutual exclusion rules; determining the type of the test data; the type of the test data is any one or more of a plurality of types; generating the test data according to the test data dictionary, the type of the test data, and the first feature. The embodiment of the application is used in the process of generating the test data.

Description

Test data generation method and device

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a test data generation method and apparatus.

Background

Data testing is a very important link in system design. The system can detect the problems in the system to be detected and whether the functions realized by the system to be detected are complete or not in time, thereby avoiding unnecessary loss caused by the problems when the system is actually used.

In the data testing link, whether the required test data can be generated or not often determines the quality of the data testing environment. The following methods are generally used in the prior art to generate test data: mode 1, a tester manually prepares test data. However, the quality of the test data generated in this way depends on the experience of the tester, and the quality of the test data generation is not stable. Meanwhile, the method cannot generate a large amount of test data, and the test is possibly incomplete. And 2, designing a generating script by a tester according to the format required by the data, and randomly generating test data by the generating script. Although the data generated by the method meets the format requirement of the data, the difference between the data generated by the method and the data generated by the method in actual production is too large, and the test quality is influenced.

Disclosure of Invention

The application provides a test data generation method and device, which can generate test data conforming to actual production in batch.

In order to achieve the purpose, the technical scheme is as follows:

in a first aspect, the present application provides a test data generating method, including: determining first data, wherein the first data comprises multiple types of data; determining a test data dictionary according to the first data; determining a first characteristic of the first data, the first characteristic comprising at least one of: the method comprises the following steps of (1) frequency of characteristic values, cluster analysis results, data correlation, association rules and mutual exclusion rules; determining the type of the test data; the type of the test data is any one or more of a plurality of types; generating the test data according to the test data dictionary, the type of the test data, and the first feature.

Based on the technical scheme, the test data generation device determines the test data dictionary and the first characteristic of the first data by determining the first data, so that the test data generation device can generate the test data which accords with the first characteristic from the test data dictionary according to the type of the test data which needs to be generated. In this way, since the test data dictionary is created from the first data, it is ensured that the data information of the generated test data coincides with the first data. Because the generated test data conforms to the first characteristic, the generated test data is ensured to be closer to the actual situation. Meanwhile, the embodiment of the application can realize the automatic generation of the test data through the technical scheme, and greatly improve the test data generation efficiency.

With reference to the first aspect, in a possible implementation manner, the method further includes: determining a second characteristic according to the type of the test data to be generated and the first characteristic; the second feature includes all or a part of the first feature; and generating data satisfying the second feature and having the same type as the test data according to the test data dictionary.

With reference to the first aspect, in a possible implementation manner, the method further includes: acquiring original data; performing data preprocessing on the original data to generate first data; the data pre-processing includes at least one of: filling missing data in the original data; redundant information in the original data is deleted.

With reference to the first aspect, in a possible implementation manner, the method further includes: in the case where the first feature includes a feature value frequency, determining the first feature of the first data includes: and calculating the occurrence frequency of each type of data in the first data, and determining the frequency of the characteristic value.

With reference to the first aspect, in a possible implementation manner, the method further includes: where the first characteristic comprises a data dependency, determining the first characteristic of the first data comprises: and calculating the correlation among the data in the first data, and determining the data correlation.

With reference to the first aspect, in a possible implementation manner, the method further includes: in the case where the first feature comprises a cluster analysis result, determining the first feature of the first data comprises: and performing cluster analysis on the first data according to the LDA topic model to determine a cluster analysis result.

With reference to the first aspect, in a possible implementation manner, the method further includes: where the first characteristic comprises an association rule, determining the first characteristic of the first data comprises: and analyzing the first data through an association rule function to determine an association rule.

With reference to the first aspect, in a possible implementation manner, the method further includes: where the first characteristic comprises a mutual exclusion rule, determining the first characteristic of the first data comprises: receiving input mutual exclusion rule request information; and determining the mutual exclusion rule according to the mutual exclusion rule request information.

In a second aspect, the present application provides a test data generating apparatus, the apparatus comprising a processing unit; a processing unit for determining first data, the first data including a plurality of types of data; the processing unit is further used for determining a test data dictionary according to the first data; a processing unit further configured to determine a first characteristic of the first data, the first characteristic including at least one of: the method comprises the following steps of (1) frequency of characteristic values, cluster analysis results, data correlation, association rules and mutual exclusion rules; the processing unit is also used for determining the type of the test data; the type of the test data is any one or more of a plurality of types; and the processing unit is also used for generating the test data according to the test data dictionary, the type of the test data and the first characteristic.

With reference to the second aspect, in a possible implementation manner, the processing unit is further configured to: determining a second characteristic according to the type of the test data to be generated and the first characteristic; the second feature includes all or a part of the first feature; and generating data satisfying the second feature and having the same type as the test data according to the test data dictionary.

With reference to the second aspect, in a possible implementation manner, the apparatus further includes a communication unit; a communication unit for acquiring original data; the processing unit is used for carrying out data preprocessing on the original data to generate first data; the data pre-processing includes at least one of: filling missing data in the original data; redundant information in the original data is deleted.

With reference to the second aspect, in a possible implementation manner, the processing unit is further configured to: and under the condition that the first characteristic comprises characteristic value frequency, calculating the frequency of each type of data in the first data, and determining the characteristic value frequency.

With reference to the second aspect, in a possible implementation manner, the processing unit is further configured to: in the case where the first feature includes data correlation, correlation between data in the first data is calculated, and the data correlation is determined.

With reference to the second aspect, in a possible implementation manner, the processing unit is further configured to: and under the condition that the first characteristic comprises a cluster analysis result, performing cluster analysis on the first data according to the LDA topic model, and determining the cluster analysis result.

With reference to the second aspect, in a possible implementation manner, the processing unit is further configured to: in the case where the first characteristic includes an association rule, the first data is analyzed by an association rule function to determine the association rule.

With reference to the second aspect, in a possible implementation manner, the apparatus further includes a communication unit; the communication unit is used for receiving input mutual exclusion rule request information under the condition that the first characteristic comprises a mutual exclusion rule; the processing unit is further configured to determine the mutual exclusion rule according to the mutual exclusion rule request information.

In a third aspect, the present application provides a test data generating apparatus, including: a processor and a communication interface; the communication interface is coupled to a processor for executing a computer program or instructions for implementing the test data generation method as described in the first aspect and any one of the possible implementations of the first aspect.

In a fourth aspect, the present application provides a computer-readable storage medium having stored therein instructions that, when executed on a terminal, cause the terminal to perform the test data generation method as described in the first aspect and any one of the possible implementations of the first aspect.

In a fifth aspect, embodiments of the present application provide a computer program product containing instructions that, when run on a test data generation apparatus, cause the test data generation apparatus to perform the test data generation method as described in the first aspect and any one of the possible implementations of the first aspect.

In a sixth aspect, an embodiment of the present application provides a chip, where the chip includes a processor and a communication interface, where the communication interface is coupled to the processor, and the processor is configured to execute a computer program or instructions to implement the test data generation method described in the first aspect and any possible implementation manner of the first aspect.

In particular, the chip provided in the embodiments of the present application further includes a memory for storing a computer program or instructions.

It should be noted that all or part of the above computer instructions may be stored on the first computer readable storage medium. The first computer readable storage medium may be packaged with or separately from a processor of the apparatus, which is not limited in this application.

In a seventh aspect, the present application provides a test data generating system, including: a test data generation apparatus for performing the test data generation method as described in the second aspect and any one of the possible implementations of the second aspect.

For the description of the second aspect to the seventh aspect in the present invention, reference may be made to the detailed description of the first aspect; moreover, the beneficial effects described in the second to seventh aspects may refer to the beneficial effect analysis of the first aspect, and are not described herein again.

In the present application, the names of the above-mentioned test data generating apparatuses do not limit the devices or functional modules themselves, and in actual implementation, the devices or functional modules may appear by other names. Insofar as the functions of the respective devices or functional blocks are similar to those of the present invention, they are within the scope of the claims of the present invention and their equivalents.

These and other aspects of the invention will be more readily apparent from the following description.

Drawings

Fig. 1 is a schematic structural diagram of a test data generating apparatus according to an embodiment of the present disclosure;

fig. 2 is a flowchart of a test data generation method according to an embodiment of the present disclosure;

fig. 3 is a schematic flowchart of another test data generation method according to an embodiment of the present disclosure;

fig. 4 is a schematic flowchart of another test data generation method according to an embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of a test data generating apparatus according to an embodiment of the present disclosure;

fig. 6 is a schematic structural diagram of another test data generation apparatus according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a chip according to an embodiment of the present application.

Detailed Description

The following describes in detail a test data generation method and apparatus provided in the embodiments of the present application with reference to the accompanying drawings.

The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone.

The terms "first" and "second" and the like in the description and drawings of the present application are used for distinguishing different objects or for distinguishing different processes for the same object, and are not used for describing a specific order of the objects.

Furthermore, the terms "including" and "having," and any variations thereof, as referred to in the description of the present application, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements but may alternatively include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described herein as "exemplary" or "e.g.," is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion.

In the description of the present application, the meaning of "a plurality" means two or more unless otherwise specified.

Hereinafter, terms related to the embodiments of the present application are explained for the convenience of the reader.

(1) Data dictionary (data dictionary)

A data dictionary, also referred to as a data dictionary, is a collection of descriptions of data objects or items in a data model.

The data dictionary typically includes: data item, data structure, data flow, data storage and processing. Wherein the data item is the smallest component unit of data, several data items may compose a data structure. The data dictionary describes the logical content of the data stream and data store by the definition of data items and data structures.

A data dictionary is a common analysis tool for data analysis, which is used to query other data associated with a certain type of data.

(2) Association rule (association rules)

Association rules are an important technique in the field of data mining to reflect the interdependency and association of one data with other data. By extracting and analyzing association rules between data, associations between valuable data items can be mined from a large amount of data.

An item set is a collection of data items in a set of data. A1 item set refers to a collection of data items, a2 item set refers to a collection of two data items, and so on. The commonly used evaluation criteria of the association rules of the item set are support and confidence.

The support degree refers to the number of times that a plurality of associated data appear in the data set, which accounts for the weight of the total data set.

Confidence is the probability that one data appears after another, also referred to as the conditional probability of the data.

(3) Apriori algorithm

The Apriori algorithm is used to extract association rules for data. The algorithm is based on the following principle:

1. if an item set is a frequent item set, then all of its subsets are frequent item sets.

2. If a collection is not a frequent item set, then all of its parent sets are not frequent item sets.

The frequent item set refers to an item set of which the support degree of the item set is greater than or equal to a preset support degree.

The Apriori algorithm can determine a frequent item set by calculating the support degree among data, and determine an association rule with strong association from the frequent item set through confidence.

(4) LDA (latent Dirichlet allocation) topic model

The LDA topic model is a bag-of-words model, and topics of each document in a document set can be given in a probability distribution mode through the LDA topic model, so that topic clustering and text classification are performed according to topic distribution. Therefore, through the LDA topic model, the distribution probability of a plurality of topics under one document and the word distribution under each topic can be obtained.

(5) Mutual exclusion rule

Mutual exclusion rules refer to conflicting relationships that multiple data types in test data have in a test system.

For example, in a system for verifying identity document information, data of other data types that are input non-identity document information may not be recognizable. Thus, under this system, identity document information is contradictory to non-identity document information. This contradictory relationship is a mutual exclusion rule.

As shown in fig. 1, which is a schematic structural diagram of a test data generating apparatus according to an embodiment of the present application, the test data generating apparatus 50 includes: the system comprises a data acquisition module 501, a data preprocessing module 502, a characteristic value processing module 503, an association rule extraction module 504, a mutual exclusion rule module 505 and a test data generation module 506.

The data collection module 501 is used for collecting production data generated in a production environment. An example, the production data includes at least one of: data messages, imported files, packets (packets) transferred between systems, and the like. The data collecting module 501 is further configured to send the collected data information to the data preprocessing module 502.

The data preprocessing module 502 is configured to receive the acquired data information sent by the data acquisition module 501, and preprocess the data information acquired by the data acquisition module 501. An example, the pre-processing method includes at least one of: identifying a data type of the data information, supplementing missing values in the data information, and deleting redundant information in the data information.

After the data preprocessing module 502 preprocesses the data information, the data preprocessing module 502 is further configured to send the preprocessed data information to the characteristic value processing module 503 and the association rule extracting module 504, respectively. Meanwhile, the data preprocessing module 502 is further configured to create a test data dictionary according to the preprocessed data information, and send the test data dictionary to the test data generating module 506.

The eigenvalue processing module 503 is configured to receive the preprocessed data information sent by the data preprocessing module 502. Then, the eigenvalue processing module 503 extracts eigenvalues in the preprocessed data information, and performs cluster analysis and frequency analysis on the eigenvalues to obtain at least one of the following characteristics of the data information: eigenvalue frequency, cluster analysis results, and data correlation. The feature value processing module 503 is further configured to send at least one of the following features of the data information to the test data generating module 506: eigenvalue frequency, cluster analysis results, and data correlation.

The association rule extracting module 504 is configured to receive the preprocessed data information sent by the data preprocessing module 502, extract an association rule in the preprocessed data information, and send the extracted association rule to the test data generating module 506.

The mutual exclusion rule module 505 is configured to generate a mutual exclusion rule, and receive the mutual exclusion rule request information sent by the test data generating module 506. The mutual exclusion rule module 505 sends the corresponding mutual exclusion rule to the test data generation module 506 according to the mutual exclusion rule request information.

The test data generating module 506 is configured to receive the test data dictionary sent by the data preprocessing module 502, receive the feature information such as the feature value frequency, the cluster analysis result, and the data correlation sent by the feature value processing module 503, and receive the association rule sent by the association rule extracting module 504. The test data generating module 506 is further configured to send mutual exclusion rule request information to the mutual exclusion rule module 505 and receive the mutual exclusion rule sent by the mutual exclusion rule module 505. The test data generation module 506 is also used to determine the type of test data. Thereafter, the test data generation module 506 generates test data based on the test data dictionary, the type of test data, and the above-described characteristics.

In different application scenarios, the data acquisition module 501, the data preprocessing module 502, the feature value processing module 503, the association rule extraction module 504, the mutual exclusion rule module 505, and the test data generation module 506 may be deployed in different devices included in the test data generation apparatus 50, or may be integrated in the same device included in the test data generation apparatus 50, which is not specifically limited in this application.

When the data acquisition module 501, the data preprocessing module 502, the feature value processing module 503, the association rule extraction module 504, the mutual exclusion rule module 505, and the test data generation module 506 are integrated in the same device included in the test data generation apparatus 50, the communication mode among the data acquisition module 501, the data preprocessing module 502, the feature value processing module 503, the association rule extraction module 504, the mutual exclusion rule module 505, and the test data generation module 506 is communication among internal modules of the device. In this case, the communication flow between the modules is the same as that in the case where the modules are independent of each other.

In the embodiment of the present application, the execution subject is the test data generating device 50, and the test data generating device 50 may be an electronic device having the functions of the above modules, a Central Processing Unit (CPU) in the electronic device, a control module for generating test data in the electronic device, or an application client for generating test data in the electronic device. In the embodiment of the present application, the test data generation apparatus 50 is taken as an example to describe the test data generation method provided in the embodiment of the present application.

In the prior art, test data is typically generated in the following manner: mode 1, a tester manually prepares test data. However, this method depends on the experience of the tester, and cannot ensure the quality of the generated test data, and meanwhile, manually preparing the test data may take a lot of time for the tester, and the test data cannot be generated in batch, which may result in incomplete test. And 2, designing a generating script by a tester according to the format required by the data, and randomly generating test data by the generating script. Although the data generated by the method meets the format requirement of the data, the difference between the data generated by the method and the data generated by the actual production is too large, the test quality is seriously influenced, and the requirement of normal test cannot be met.

In order to solve the problems that the generation efficiency of test data is low and the generated test data is not in line with the actual production condition in the prior art, the application provides a test data generation method.

As shown in fig. 2, a flowchart of a test data generating method provided in an embodiment of the present application is shown, where the method includes the following steps:

s101, the test data generation device determines first data.

Wherein the first data includes a plurality of types of data.

Alternatively, the first data may be obtained by preprocessing production data generated in the production environment by the test data generation device.

It should be noted that the test data may classify the first data from multiple dimensions.

For example, the first data may be classified according to the purpose of the test data, such as dividing the data into correct value test data and error value test data. The correct value test data can be used for testing whether the system can generate a corresponding output result according to the correct value test data; the error value test data may be used to test whether the system can identify the error data and generate an error report. The first data may also be classified according to the transaction type of the test data, such as a stock transaction type, an option transaction type, and the like. The first data may also be classified according to the business domain of the test data, for example, the data may be classified into a fund business direction, a loan business direction, and the like.

The data may also be classified according to other dimensions in the embodiments of the present application, which is not specifically limited in this application.

In a possible implementation manner, step S101 may be specifically executed by a data preprocessing module included in the test data generation apparatus. The data preprocessing module determines first data and sends the first data to the characteristic value processing module and the association rule extraction module. Correspondingly, the characteristic value processing module and the association rule extraction module receive the first data from the data preprocessing module.

S102, the test data generation device determines a test data dictionary according to the first data.

In addition, when the first data is obtained by preprocessing the production data generated in the production environment by the test data generation device, the test data generation device determines a test data dictionary according to the first data, so that the data in the first data can be normalized, and the test data generation device can generate the structured test data by calling the test data dictionary in the subsequent step. Meanwhile, the test data dictionary in the embodiment of the application is generated based on the first data, so that each item of data in the test data dictionary is determined by the data in the first data, and the test data dictionary can reflect the authenticity of the production data.

In a possible implementation manner, step S102 may be specifically executed by a data preprocessing module included in the test data generation apparatus. The data preprocessing module determines a test data dictionary according to the first data and sends the test data dictionary to the test data generating module. Accordingly, the test data generation module receives the test data dictionary from the data preprocessing module.

S103, the test data generation device determines a first characteristic of the first data.

Wherein the first feature comprises at least one of: the characteristic value frequency, the cluster analysis result, the data correlation, the association rule and the mutual exclusion rule.

It should be noted that the first feature may represent features of different data in the same data group in the first data, may represent features between different data groups in the first data, and may also represent features of different data in different data groups in the first data.

Illustratively, the first data includes a data set a and a data set B. The data group A comprises data A1, A2, A3, A4 and A5. Data set B includes data B1, B2, B3, B4, B5. The first feature may characterize the features between data a1, a2, A3, a4, a5, between data sets a and B, and between data a1, A3, B1, B2.

Note that the frequency of the feature value is used to indicate the frequency of occurrence of the feature value in the first data. The cluster analysis result is used to represent an analysis result of dividing each data in the first data into different classes or clusters. Data correlation is used to indicate some relationship or regularity that exists between data in the first data. The association rule is used to determine a set of frequent items in the first data. The mutual exclusion rule is used for representing the mutual exclusion rule of the data in the first data.

The first feature in the embodiment of the present application may further include another feature for representing a relationship of the first data, and the present application is not particularly limited to this.

In a possible implementation manner, step S103 may be specifically executed by a feature value processing module, an association rule extracting module, and a mutual exclusion rule module included in the test data generating apparatus. And the characteristic value processing module sends the determined characteristic value frequency, the cluster analysis result and the data correlation to the test data generation module. And the association rule extraction module sends the determined association rule to the test data generation module. The mutual exclusion rule module receives input mutual exclusion rule instruction information to determine a mutual exclusion rule, and responds to the mutual exclusion rule request information sent by the test data generation module to send the corresponding mutual exclusion rule to the test data generation module. Correspondingly, the test data generation module receives the characteristic value frequency, the cluster analysis result and the data correlation from the characteristic value processing module. The test data generation module receives the association rule from the association rule extraction module. The test data generation module sends the mutual exclusion rule request information to the mutual exclusion rule module and receives the mutual exclusion rule from the mutual exclusion rule module.

And S104, determining the type of the test data by the test data generation device.

Wherein the type of the test data is any one or more of the above-mentioned types.

For example, in the case of classifying according to usage, the type of the test data may be a correct value type of test data (i.e., correct value test data); or the type of test data may also be an error value type of test data (i.e., error value test data).

For example, in the case of classifying by transaction type, the type of test data may be a stock transaction type of test data; or the type of test data may also be option transaction type test data.

For example, in the case of classifying by business field, the type of test data may be test data of a fund business direction; or the type of test data may also be test data for the direction of the loan transaction.

Alternatively, the test data generating means may determine the type of the test data by receiving instruction information input by a user.

In a possible implementation manner, step S104 may specifically determine the type of the test data by a test data generation module included in the test data generation apparatus.

In the embodiment of the present application, the execution sequence of the steps S102 to S104 is not limited. The test data generation device may perform the above steps in combination in any order, and the test data generation device may also perform the above steps simultaneously.

And S105, generating test data by the test data generating device according to the test data dictionary, the type of the test data and the first characteristic.

It should be noted that, since the test data dictionary is created based on the first data, the data in the test data dictionary corresponds to the data in the first data. At the same time, the first feature can characterize a data feature in the first data. Therefore, through step S105, the test data generation apparatus can generate the test data according to the dimension corresponding to the test data type, so that the generated test data conforms to the first feature in the first data.

In a possible implementation manner, in step S105, the test data generation module included in the test data generation apparatus may specifically generate the test data according to the test data dictionary, the type of the test data, and the first feature.

Illustratively, taking the type of the test data as correct value data as an example, the test data generation device determines the type of the test data as correct value data, and generates the test data from the test data dictionary according to the feature value frequency, the cluster analysis result, the correlation analysis result, and the association rule.

Illustratively, taking the type of the test data as the error value data as an example, the test data generation device determines the type of the test data as the error value data, and generates the test data from the test data dictionary according to the feature value frequency, the cluster analysis result, the correlation analysis result, the association rule, and the at least one mutual exclusion rule.

Illustratively, taking the type of the test data as a stock transaction type as an example, the test data generating device determines the type of the test data as a stock transaction type, and generates the test data from the test data dictionary according to the feature value frequency, the cluster analysis result, the correlation analysis result and the association rule.

Illustratively, taking the type of the test data as the fund traffic direction as an example, the test data generation device determines the type of the test data as the fund traffic direction, and generates the test data from the test data dictionary according to the eigenvalue frequency, the cluster analysis result, the correlation analysis result and the association rule.

Optionally, the test data generating device may further generate a corresponding transaction instruction and a corresponding message according to the type of the test data.

Optionally, the test data generation device may further set the generation number of the test data.

The process of generating test data by the test data generating apparatus in S101 to S105 is described in detail above.

Hereinafter, a process of determining the first data by the test data generating device is specifically described with reference to the step S101, specifically, with reference to fig. 2 and as shown in fig. 3, fig. 3 is a schematic flow chart of a test data generating method provided in the embodiment of the present application. The step S101 can be specifically realized by the following steps S1011 to S1012:

and S1011, the test data generation device acquires original data.

Wherein the raw data is production data generated in an actual production environment.

Optionally, the test data generating device may obtain the original data through a packet transmitted between the message, the file, and the system.

For example, in the case of a transaction instruction sent by a transaction system, the content of the obtained original data may include an escrow account, a transaction type, a transaction location, a transaction date, a security account number, a security clearing date, a security delivery date, a delivery security code, a delivery price, a delivery amount, a payment fund account number, a payment fund account name, a payment fund issuer, a collection fund account number, a collection fund account name, a collection fund issuer, and the like in the transaction instruction.

In a possible implementation manner, in step S1011, the data acquisition module included in the test data generating apparatus may specifically acquire the raw data, and send the acquired raw data to the data preprocessing module. Correspondingly, the data preprocessing module receives the raw data acquired by the data acquisition module.

S1012, the test data generation device performs data preprocessing on the original data to generate first data.

Optionally, the method for performing data preprocessing on the original data by the test data generation device includes filling missing data in the original data by the test data generation device.

Specifically, the test data generation device may be supplemented by calculating an average value of data types corresponding to missing data.

Optionally, the method for performing data preprocessing on the original data by the test data generation device further includes deleting redundant information in the original data by the test data generation device.

Illustratively, the redundant information may be header information in a data packet. The header information is a fixed format of the data message, and cannot reflect the data content in the actual production data.

Optionally, the method for performing data preprocessing on the original data by the test data generation device further includes deleting or modifying abnormal data in the original data by the test data generation device.

Optionally, the method for performing data preprocessing on the original data by the test data generation device further includes sequencing the original data by the test data generation device according to a certain standard, so as to facilitate subsequent calling.

In a possible implementation manner, in step S1012, the raw data may be subjected to data preprocessing by a data preprocessing module included in the test data generating apparatus to obtain the first data.

Through the technical scheme, the test data generation device preprocesses the original data generated by the actual production environment to generate the first data, so that the generated first data better accords with the actual data, and the quality of the test data is improved.

The procedure of generating the first data by the test data generating apparatus in S1011 to S1012 has been specifically described above.

In the following, with reference to the step S103, a process of determining the first feature of the first data by the test data generating device is described in detail, and in the following modes one to five, the test data generating device may generate the feature value frequency according to mode one, generate the data correlation according to mode two, generate the cluster analysis result according to mode three, generate the association rule according to mode four, and generate the mutual exclusion rule according to mode five. Modes one to five are described in detail below, respectively:

the first method is as follows:

the test data generation device calculates the frequency of occurrence of each type of data in the first data and determines the frequency of the characteristic value.

For example, the test data generating means first determines a feature value whose frequency is to be calculated, for example, "8/31/2021" on the transaction date, and traverses all data of the first data whose data type is the transaction date, thereby obtaining the frequency of the feature value "8/31/2021".

The second method comprises the following steps:

the test data generation device calculates the correlation between the data in the first data and determines the data correlation of the first data.

It should be noted that data correlation is often used to characterize some correlation relationship of multiple data types, such as positive correlation, negative correlation, etc.

Illustratively, the test data generating means calculates the correlation of "owner age" and "owner deposit". And carrying out polynomial fitting on the specific numerical information of the 'owner age' and the 'owner deposit' in the first data by calling, so as to obtain a functional relation between the 'owner age' and the 'owner deposit', and further judge whether the 'owner age' and the 'owner deposit' have correlation.

The third method comprises the following steps:

and the test data generation device performs cluster analysis on the first data according to the LDA topic model to determine a cluster analysis result.

Specifically, the test data generation device may determine probability distribution of each data in the first data according to the LDA topic model, thereby determining a clustering relation of each data, that is, a clustering analysis result.

It should be noted that, after the test data generation device determines the cluster analysis result through the LDA topic model, the test data that is the same as the cluster analysis result in the first data may be generated through the LDA topic model according to the determined cluster analysis result.

The method is as follows:

the test data generation device analyzes the first data through the association rule function to determine the association rule.

Specifically, the association rule function includes Apriori algorithm.

The following describes in detail the process of determining the association rule by the test data generating apparatus, taking Apriori algorithm as an example:

illustratively, the first data is shown in table 1 below:

TABLE 1 first data Chart

ID	Sex	Line of opening an account	Number of stock trades	Number of fund transactions
					1	Woman	A	0	1
2	For male	A	1	0
					3	For male	A	1	1
4	For male	B	1	1

And setting the preset support degree to be 0.5, and extracting the association rule of the first data.

Calculating the support degree of the 1 item set in the first data, and obtaining: { sex: male has a support of 0.75; { sex: female has a support of 0.25; { open account row: a } has a support degree of 0.75; { open account row: b } has a support degree of 0.25; { number of stock trades: 0} is 0.25; { number of stock trades: 1} has a support degree of 0.75; { number of fund transactions: 1} has a support degree of 0.75; { number of fund transactions: 0} is 0.25.

Deleting 1 item set, wherein the item set is lower than the preset support degree, and obtaining a frequent 1 item set: { sex: male }, { open account row: a }, { stock exchange number: 1}, { number of fund transactions: 1}.

Calculating the support of 2 item sets in the frequent 1 item set can obtain: { sex: male, open an account and walk: a } has a support degree of 0.5; { sex: male and stock trading times: 1} has a support degree of 0.75; { sex: male and fund transaction frequency: 1} has a support degree of 0.5; { open account row: A. the number of stock trades: 1} has a support degree of 0.5; { open account row: A. number of fund transactions: 1} has a support degree of 0.5; { number of stock trades: 1. number of fund transactions: 1} the support was 0.5.

Because the support degrees in the 2 item sets are all greater than or equal to the preset support degree, the frequent 2 item sets are obtained: { sex: male, open an account and walk: a }, { sex: male and stock trading times: 1}, { sex: male and fund transaction frequency: 1}, { open account row: A. the number of stock trades: 1}, { open account row: A. number of fund transactions: 1}, { stock exchange number: 1. number of fund transactions: 1}.

Calculating the support of 3 item sets in the frequent 2 item sets can obtain: { sex: male, open an account and walk: A. the number of stock trades: 1} has a support degree of 0.5; { sex: male, open an account and walk: A. number of fund transactions: 1} has a support degree of 0.25; { sex: male and stock trading times: 1. number of fund transactions: 1} has a support degree of 0.5; { open account row: A. the number of stock trades: 1. number of fund transactions: 1} the support was 0.25.

Deleting 3 item sets, wherein the item sets are lower than the preset support degree, and obtaining 3 frequent item sets: { sex: male, open an account and walk: A. the number of stock trades: 1}, { sex: male and stock trading times: 1. number of fund transactions: 1}.

Since the 4 sets include subsets that belong to a non-frequent set of items, it is determined that the association rule in the first data includes two frequent 3 sets, which are { gender: male, open an account and walk: A. the number of stock trades: 1} and { gender: male and stock trading times: 1. number of fund transactions: 1}.

And meanwhile, parameter information such as the support degree and the confidence degree of the frequent 3 item sets can be obtained.

In a possible implementation manner, in the above scheme, the association rule extraction module included in the test data generation device may extract the association rule of the first data, and send the association rule to the test data generation module. Correspondingly, the test data generation module receives the association rule from the association rule extraction module.

The fifth mode is as follows:

the test data generation device determines the mutual exclusion rule by receiving the input mutual exclusion rule instruction information.

Alternatively, the mutual exclusion rule may be a mutual exclusion of the data types in the first data.

For example, a test system for verifying identity card information cannot verify other certificate information. When the test system receives test data "6354125895," it is verified that the test data is not identification card information, a verification failure is indicated and an error report is generated. In order to test whether the system can recognize other certificate information and generate a corresponding error report, the test data generation device may determine a rule that the identity certificate information and the other certificate information are mutually exclusive as the first feature of the first data.

In a possible implementation manner, the above-mentioned scheme may be implemented by a mutual exclusion rule module included in the test data generation apparatus receiving input mutual exclusion rule instruction information, and determining the mutual exclusion rule.

Through the technical scheme, the test data generation device can acquire the characteristic information among the data in the various characterization first data, so that the test data can be generated according to the characteristic information.

With reference to the step S105, a process of generating test data by the test data generating apparatus is described in detail, specifically, with reference to fig. 2 and as shown in fig. 4, fig. 4 is a schematic flow chart of a test data generating method provided in this embodiment of the present application. The step S105 can be specifically realized by the following steps S1051 to S1052:

s1051, the test data generating device determines a second characteristic according to the type of the test data to be generated and the first characteristic.

Wherein the second feature comprises all or a portion of the first feature.

It should be noted that the test data generation device selects the required relevant feature information, that is, the second feature, from the first feature according to the type of the test data required to be generated, so that the test data generation device can generate the required test data according to different dimensions.

In a possible implementation manner, the above scheme may determine the second characteristic according to the type of the test data to be generated and the first characteristic by a test data generation module included in the test data generation apparatus.

Illustratively, taking the type of the test data as the correct value test data as an example, the test data generation device determines the type of the test data as the correct value data, determines the frequency of the characteristic value, the cluster analysis result, the data correlation, the association rule and the mutual exclusion rule in the first characteristic, and determines the second characteristic as the frequency of the characteristic value, the cluster analysis result, the correlation analysis result and the association rule.

Optionally, the test data generating module may further determine, according to the type of the test data to be generated, whether it is necessary to send a mutual exclusion rule request message to the mutual exclusion rule module to request the mutual exclusion rule module to send the corresponding mutual exclusion rule.

And S1052, the test data generating device generates data which satisfies the second characteristic and is the same as the type of the test data as the test data according to the test data dictionary.

Specifically, with reference to the above example, step S1052 is described in detail:

illustratively, the feature value frequency and association rule are selected as the second feature. The test data generation means may generate the test data having the same frequency and the same association rule as the feature values in the first data, based on the test data dictionary.

For example, the first data includes 50 pieces of data, in which the frequency of the feature value "age of account holder" of 18 is 12.

The test data generating means may generate 2000 pieces of test data in which a characteristic value "age of account holder" in 480 pieces of data is set to 18.

As another example, the association rule in the first data is { gender: male and stock trading times: 1. number of fund transactions: 1} was 0.5, { sex: male and stock trading times: 1} the support was 0.75.

The test data generating apparatus may set "sex" to male, "number of stock exchanges" to 1, and "number of fund exchanges" to 1 in 1000 pieces of the 2000 pieces of test data. Meanwhile, "sex" under 500 pieces of test data among the 1000 pieces of data was selected as male, "number of stock exchanges" was set as 1, "number of fund exchanges" was not 1.

The specific embodiments of the setting method for other features are similar to the above-mentioned schemes, and are not described in detail in this application.

Through the technical scheme, the test data generation device can select the required second characteristics from the first characteristics according to the data types of the test data and aiming at different dimensions, and then generates the test data meeting the second characteristics, so that the generated test data can be more fit with the test requirements.

In the embodiment of the present application, the test data generating apparatus may be divided into the functional modules or the functional units according to the method example, for example, each functional module or functional unit may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The integrated module may be implemented in a form of hardware, or may be implemented in a form of a software functional module or a functional unit. The division of the modules or units in the embodiment of the present application is schematic, and is only a logic function division, and there may be another division manner in actual implementation.

As shown in fig. 5, a schematic structural diagram of a test data generating apparatus provided in an embodiment of the present application is shown, where the apparatus includes:

a processing unit 201 for determining the first data.

Wherein the first data includes a plurality of types of data.

The processing unit 201 is further configured to determine a test data dictionary according to the first data.

The processing unit 201 is further configured to determine a first characteristic of the first data.

The processing unit 201 is further configured to determine a type of the test data.

Wherein the type of the test data is any one or more of the plurality of types.

The processing unit 201 is further configured to generate the test data according to the test data dictionary, the type of the test data, and the first feature.

Optionally, the processing unit 201 is further configured to: determining a second characteristic according to the type of the test data to be generated and the first characteristic; the second feature includes all or a part of the first feature; and generating data satisfying the second feature and having the same type as the test data according to the test data dictionary.

Optionally, the apparatus further comprises a communication unit 202; a communication unit 202 for acquiring raw data; a processing unit 201, configured to perform data preprocessing on original data to generate first data; the data pre-processing includes at least one of: filling missing data in the original data; redundant information in the original data is deleted.

Optionally, the processing unit 201 is further configured to: and under the condition that the first characteristic comprises characteristic value frequency, calculating the frequency of each type of data in the first data, and determining the characteristic value frequency.

Optionally, the processing unit 201 is further configured to: in the case where the first feature includes data correlation, correlation between data in the first data is calculated, and the data correlation is determined.

Optionally, the processing unit 201 is further configured to: and under the condition that the first characteristic comprises a cluster analysis result, performing cluster analysis on the first data according to the LDA topic model, and determining the cluster analysis result.

Optionally, the processing unit 201 is further configured to: in the case where the first characteristic includes an association rule, the first data is analyzed by an association rule function to determine the association rule.

Optionally, the apparatus further comprises a communication unit 202; the communication unit 202 is configured to receive input mutual exclusion rule request information in a case where the first feature includes a mutual exclusion rule; the processing unit 201 is further configured to determine the mutual exclusion rule according to the mutual exclusion rule request information.

When implemented by hardware, the communication unit 202 in the embodiment of the present application may be integrated on a communication interface, and the processing unit 201 may be integrated on a processor. The specific implementation is shown in fig. 6.

Fig. 6 shows a schematic diagram of still another possible structure of the test data generation apparatus according to the above embodiment. The test data generation apparatus includes: a processor 302 and a communication interface 303. The processor 302 is used to control and manage the actions of the test data generation apparatus, e.g., to perform the steps performed by the processing unit 201 described above, and/or other processes for performing the techniques described herein. The communication interface 303 is used to support communication between the test data generation apparatus and other network entities, for example, to perform the steps performed by the communication unit 202. The test data generating device may further comprise a memory 301 and a bus 304, the memory 301 being arranged to store program codes and data of the test data generating device.

The memory 301 may be a memory in the test data generation apparatus, and the like, and the memory may include a volatile memory, such as a random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, a hard disk, or a solid state disk; the memory may also comprise a combination of memories of the kind described above.

The processor 302 may be implemented or performed with various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor may be a central processing unit, general purpose processor, digital signal processor, application specific integrated circuit, field programmable gate array or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor may also be a combination of computing functions, e.g., comprising one or more microprocessors, DSPs, and microprocessors, among others.

The bus 304 may be an Extended Industry Standard Architecture (EISA) bus or the like. The bus 304 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 6, but this is not intended to represent only one bus or type of bus.

Fig. 7 is a schematic structural diagram of a chip 170 according to an embodiment of the present disclosure. Chip 170 includes one or more (including two) processors 1710 and a communication interface 1730.

Optionally, the chip 170 further includes a memory 1740, where the memory 1740 may include both read-only memory and random access memory, and provides operational instructions and data to the processor 1710. A portion of memory 1540 may also include non-volatile random access memory (NVRAM).

In some embodiments, memory 1740 stores elements, execution modules, or data structures, or a subset thereof, or an expanded set thereof.

In the embodiment of the present application, the corresponding operation is performed by calling an operation instruction stored in the memory 1740 (the operation instruction may be stored in an operating system).

The processor 1710 may implement or execute various illustrative logical blocks, units, and circuits described in connection with the disclosure herein. The processor may be a central processing unit, general purpose processor, digital signal processor, application specific integrated circuit, field programmable gate array or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, units, and circuits described in connection with the disclosure. The processor may also be a combination of computing functions, e.g., comprising one or more microprocessors, DSPs, and microprocessors, among others.

Memory 1740 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, a hard disk, or a solid state disk; the memory may also comprise a combination of memories of the kind described above.

Bus 1720 may be an Extended Industry Standard Architecture (EISA) bus or the like. Bus 1720 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one line is shown in FIG. 7, but it is not intended that there be only one bus or one type of bus.

Through the above description of the embodiments, it is clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the above described functions. For the specific working processes of the system, the apparatus and the unit described above, reference may be made to the corresponding processes in the foregoing method embodiments, and details are not described here again.

The present application provides a computer program product containing instructions, which when run on a computer, causes the computer to execute the test data generation method in the above method embodiments.

The embodiment of the present application further provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the instructions are run on a computer, the computer is caused to execute the test data generation method in the method flow shown in the above method embodiment.

The computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM), a register, a hard disk, an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, any suitable combination of the above, or any other form of computer readable storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuit (ASIC). In embodiments of the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

Embodiments of the present invention provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform a test data generation method as described in figures 2 to 4.

Since the test data generating apparatus, the computer-readable storage medium, and the computer program product in the embodiments of the present invention may be applied to the method described above, the technical effects obtained by the method may also refer to the method embodiments described above, and the details of the embodiments of the present invention are not repeated herein.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The above is only an embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. a test data generation method, is characterized in that, described method comprises:

determining first data, the first data including multiple types of data;

According to the first data, determine a test data dictionary;

determining a first feature of the first data, where the first feature includes at least one of the following features: feature value frequency, cluster analysis result, data correlation, association rule, and mutual exclusion rule;

Determine the type of the test data; the type of the test data is any one or more of the multiple types;

The test data is generated according to the test data dictionary, the type of the test data, and the first characteristic.

2. The method according to claim 1, wherein generating the test data according to the test data dictionary, the type of the test data to be generated, and the first feature, comprises:

According to the type of the test data to be generated and the first feature, determine a second feature; the second feature includes all or part of the first features;

Based on the test data dictionary, data satisfying the second characteristic and of the same type as the test data is generated as the test data.

3. The method according to claim 1, wherein the determining the first data comprises:

get raw data;

Perform data preprocessing on the original data to generate the first data; the data preprocessing includes at least one of the following:

filling the missing data in the original data;

Redundant information in the original data is deleted.

4. The method according to any one of claims 1-3, wherein, when the first feature includes the frequency of the feature value, the determining the first feature of the first data comprises:

Calculate the frequency of occurrence of each type of data in the first data, and determine the frequency of the feature value;

In the case where the first feature includes the data correlation, the determining the first feature of the first data includes:

calculating the correlation between data in the first data, and determining the data correlation;

When the first feature includes the cluster analysis result, the determining the first feature of the first data includes:

Cluster analysis is performed on the first data according to the LDA topic model, and the cluster analysis result is determined.

5. The method according to any one of claims 1-3, wherein, when the first feature includes the association rule, the determining the first feature of the first data comprises:

The first data is analyzed by an association rule function to determine the association rule.

6. A device for generating test data, comprising: a processing unit;

the processing unit, configured to determine first data, where the first data includes multiple types of data;

The processing unit is further configured to determine a test data dictionary according to the first data;

The processing unit is further configured to determine a first feature of the first data, where the first feature includes at least one of the following features: feature value frequency, cluster analysis result, data correlation, association rule, mutual exclusion rule ;

The processing unit is further configured to determine the type of the test data; the type of the test data is any one or more of the multiple types;

The processing unit is further configured to generate the test data according to the test data dictionary, the type of the test data, and the first feature.

7. The apparatus according to claim 6, wherein the processing unit is further configured to:

8. The apparatus according to claim 6, wherein the apparatus further comprises a communication unit;

the communication unit, used to obtain original data;

the processing unit, configured to perform data preprocessing on the original data to generate the first data;

The data preprocessing includes at least one of the following:

filling the missing data in the original data;

Redundant information in the original data is deleted.

9. A test data generating device, characterized in that it comprises: a processor and a communication interface; the communication interface is coupled with the processor, and the processor is used for running a computer program or instruction to realize the method as claimed in claim 1- The test data generation method described in any one of 5.

10. A computer-readable storage medium having instructions stored in the computer-readable storage medium, wherein when a computer executes the instruction, the computer executes the test described in any one of the preceding claims 1-5 Data generation method.