CN113297358B

CN113297358B - Data processing method, device, server and computer-readable storage medium

Info

Publication number: CN113297358B
Application number: CN202010749883.1A
Authority: CN
Inventors: 李燕群; 徐扬; 贾冰鑫
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-07-30
Filing date: 2020-07-30
Publication date: 2025-04-18
Anticipated expiration: 2040-07-30
Also published as: CN113297358A

Abstract

This specification provides a data processing method, device, server and computer-readable storage medium. In one embodiment, based on the above data processing method, before implementation, multiple basic problems in the scene can be pre-determined, and corresponding atomic models can be constructed for the basic problems, and corresponding data packets can be configured; during implementation, the target problem in the target scene can be first determined according to the specific data processing involved in the target scene, and then the target atomic model can be matched from the above-mentioned pre-constructed multiple atomic models according to the target problem, and then the model can be trained on the basis of the target atomic model according to the target scene and the pre-configured data packet to obtain a target model suitable for data processing in the target scene. In this way, it is possible to efficiently train and obtain a suitable target model for data processing in multiple target scenes for multiple types of data processing.

Description

Data processing method, device, server and computer readable storage medium

Technical Field

The present disclosure relates to the field of internet technologies, and in particular, to a data processing method, a data processing device, a server, and a computer readable storage medium.

Background

For different data processing in different application scenarios, it is often necessary to separately construct and train corresponding processing models from scratch for each type of data processing in each application scenario, respectively, to perform a certain data processing in that scenario.

Therefore, there is a need for a method that can efficiently train to obtain an applicable target model for multiple types of data processing in multiple target scenarios.

Disclosure of Invention

The specification provides a data processing method, a data processing device, a server and a computer readable storage medium, so that an applicable target model can be efficiently trained and obtained according to various types of data processing in various target scenes, and training efficiency of the target model is improved.

The data processing method, device, server and computer readable storage medium provided in the present specification are implemented as follows:

The data processing method comprises the steps of determining a target problem in a target scene, determining a target atomic model matched with the target problem from a plurality of atomic models according to the target problem, and obtaining a target data packet corresponding to the target atomic model, wherein the atomic model comprises a preset unit model corresponding to a basic problem in a basic scene, the atomic model is further respectively provided with a corresponding data packet, and the target atomic model is trained according to the target scene and the target data packet to obtain the target model, wherein the target model is used for data processing in the target scene.

The data processing method comprises the steps of determining a target problem according to a target scene, generating and sending a corresponding atomic model selection request to a server according to the target problem, wherein the atomic model selection request is used for requesting the server to determine a target atomic model matched with the target problem from a plurality of atomic models, the atomic model comprises a preset unit model corresponding to a basic problem in a basic scene, the atomic model is further respectively provided with a corresponding data packet, the target atomic model fed back by the server and a target data packet corresponding to the target atomic model are received, and training the target atomic model according to the target scene and the target data packet to obtain the target model.

A data processing method comprises the steps of receiving and responding to a processing request of a merchant, determining that a target scene is a customer service group reply scene, determining whether a target problem is a text message issued by a customer in the customer service group and needs to be replied manually, generating and sending a corresponding model training request to a server according to the target problem, wherein the model training request carries the target problem and the target scene, the server is used for responding to the model training request, determining a target atomic model matched with the target problem from a plurality of atomic models and a target data packet corresponding to the target atomic model, and training the target atomic model according to the target scene and the target data packet to obtain the target model.

The data processing device comprises a first determining module, a second determining module and a training module, wherein the first determining module is used for determining a target problem in a target scene, the second determining module is used for determining a target atomic model matched with the target problem from a plurality of atomic models according to the target problem and obtaining a target data packet corresponding to the target atomic model, the atomic model comprises a preset unit model corresponding to a basic problem in a basic scene, the atomic model is further respectively provided with a corresponding data packet, and the training module is used for training the target atomic model according to the target scene and the target data packet to obtain the target model, wherein the target model is used for data processing in the target scene.

The server comprises a processor and a memory for storing executable instructions of the processor, wherein the processor is used for determining target problems in a target scene when executing the instructions, determining a target atomic model matched with the target problems from a plurality of atomic models according to the target problems, and acquiring target data packets corresponding to the target atomic model, wherein the atomic model comprises preset unit models corresponding to basic problems in a basic scene, the atomic model is further respectively provided with corresponding data packets, and training the target atomic model according to the target scene and the target data packets to obtain the target model, wherein the target model is used for data processing in the target scene.

A computer readable storage medium is provided, wherein the computer instructions are stored, the instructions are executed to determine target problems in a target scene, a target atomic model matched with the target problems is determined from a plurality of atomic models according to the target problems, and a target data packet corresponding to the target atomic model is acquired, the atomic model comprises a preset unit model corresponding to a basic problem in a basic scene, the atomic model is further respectively provided with a corresponding data packet, the target atomic model is trained according to the target scene and the target data packet to obtain a target model, and the target model is used for data processing in the target scene.

According to the data processing method, the device, the server and the computer readable storage medium, before implementation, a plurality of basic problems are split from a basic scene with larger coverage, corresponding atomic models are built for the basic problems in the basic scene respectively, corresponding data packages are configured for the atomic models, during implementation, target problems in the target scene can be determined firstly according to data processing in the target scene, then the target problems are matched to a proper target atomic model from the pre-built plurality of atomic models according to the target problems, a pre-configured target data package corresponding to the target atomic model is obtained, model training related to the target scene can be conducted on the basis of the target atomic model according to the target scene and the target data package, and the target model suitable for specific data processing in the target scene is obtained, so that the data processing in the target scene can be conducted by using the model. Therefore, the method can efficiently train and obtain the applicable target model aiming at various types of data processing in various target scenes, and the training efficiency of the target model is improved.

Drawings

In order to more clearly illustrate the embodiments of the present disclosure, the drawings that are required for the embodiments will be briefly described below, in which the drawings are only some of the embodiments described in the present disclosure, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of one embodiment of a system configuration composition to which the data processing method provided in the embodiments of the present specification is applied;

FIG. 2 is a schematic diagram of one embodiment of a data processing method provided by embodiments of the present specification, in one example scenario;

FIG. 3 is a schematic diagram of one embodiment of a data processing method provided by embodiments of the present specification, in one example scenario;

FIG. 4 is a schematic diagram of one embodiment of a data processing method provided by embodiments of the present specification, in one example scenario;

FIG. 5 is a schematic diagram of one embodiment of a data processing method provided by embodiments of the present specification, in one example of a scenario;

FIG. 6 is a schematic diagram of one embodiment of a data processing method provided by embodiments of the present specification, in one example of a scenario;

FIG. 7 is a flow chart of a data processing method according to one embodiment of the present disclosure;

FIG. 8 is a flow chart of a data processing method according to another embodiment of the present disclosure;

FIG. 9 is a schematic diagram of the structural composition of a server provided in one embodiment of the present disclosure;

fig. 10 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present specification.

Detailed Description

In order to make the technical solutions in the present specification better understood by those skilled in the art, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only some embodiments of the present specification, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.

The embodiment of the specification provides a data processing method which can be applied to a system comprising a server and terminal equipment. Reference may be made in particular to fig. 1. Different terminal devices can be connected with the server in a wired or wireless mode so as to carry out specific data interaction.

Before implementation, the server may split a plurality of basic questions from a basic scene (for example, a scene with a relatively wide coverage area) in advance, and construct corresponding atomic models for the plurality of basic questions, so as to obtain a plurality of atomic models. Meanwhile, data packets for using the atomic models are respectively configured for the plurality of atomic models.

In practice, the data processing system is faced with different types of data processing involved in different target scenes, a plurality of terminal devices (e.g., a terminal device 1 terminal device 2. To request the server to train a target model applicable to the target scene.

The server may first receive and respond to each model training request to determine a target problem in each target scene faced by each terminal device. Further, the server may determine, from the plurality of atomic models, an atomic model matching the target problem as a target atomic model according to the target problem, and acquire a data packet corresponding to the target atomic model as a target data packet. Then according to the target scene and the target data packet, model training is carried out on the basis of the target atomic model, so that a target model suitable for specific data processing in a target scene faced by each terminal device can be quickly and efficiently trained (for example, a target model 1 object model 2.

The server may issue the plurality of trained object models to the network, or send the plurality of trained object models to corresponding terminal devices, respectively.

Therefore, the plurality of terminal devices can conveniently and efficiently acquire and use the target model to process specific data in the corresponding target scene.

In this embodiment, the server may specifically include a server that is applied to a network platform side and is responsible for data processing in a background that can implement functions such as data transmission and data processing. Specifically, the server may be, for example, an electronic device having data operation, storage function and network interaction function. Or the server may be a software program running in the electronic device that provides support for data processing, storage, and network interactions. In the present embodiment, the number of servers is not particularly limited. The server may be one server, several servers, or a server cluster formed by several servers.

In this embodiment, the terminal device may specifically include a front-end device applied to a user side and capable of implementing functions such as data acquisition and data transmission. Specifically, the terminal device may be, for example, a desktop computer, a tablet computer, a notebook computer, a smart phone, etc. Or the terminal device may be a software application capable of running in the electronic device described above. For example, it may be an application running on a computer, or the like.

In a specific scenario example, referring to fig. 2, the data processing method provided in the embodiment of the present disclosure may be applied to receive and respond to a model training request of a user, train and feed back to a corresponding target model of the user, so that the user may efficiently acquire and use the target model to perform specific data processing in the target scenario.

In this scenario example, the server may specifically be a server of a network platform that provides model training services. Specifically, the server may be connected to the first terminal device and the second terminal device. The first terminal device may be specifically disposed on a side of an algorithm end technician responsible for atomic model construction. The second terminal device may be specifically disposed on the front-end technician or client (hereinafter, referred to as user) side that needs to build or use a specific model.

The atomic model may specifically include a unit model that is built in advance and corresponds to a basic problem in a basic scene. In particular, the above atomic model may be understood as a minimum model structure that includes semantics, has been trained in advance, and can solve a corresponding basic problem.

The basic scene may specifically include a scene with a wider coverage range (for example, a coverage range greater than a preset coverage range threshold), and has good applicability and referential performance for various specific scenes. The basic problem can be specifically understood as a data processing problem which is obtained by abstracting and summarizing related data processing involved in a basic scene, has certain representativeness and coverage and has general characteristics for processing a problem belonging to a certain type. Typically one basic problem can be applied to target problems in multiple different target scenarios at the same time.

It should be noted that the atomic model is different from the basic model or the initial model (e.g., there is no scene combination or no training of the initial two-classification model). The basic model or the initial model usually does not contain any semantics, and when the model is trained based on the basic model or the initial model, a new model needs to be built and trained from scratch by combining scenes. The atomic model is a model structure which is built to contain a certain semantic and can solve a basic problem in a scene after learning and training the basic problem in the basic scene to a certain extent in advance. Of course, the atomic model described above is not equivalent to a true model either. Although the above atomic model already contains a certain semantic, more attention is paid to the algorithm type, model structure and other factors adopted for basic problems in the corresponding basic scene.

In this scenario example, before implementation, referring to fig. 3, a technician at the algorithm end may pre-split a plurality of basic questions from the basic scenario based on the basic scenario, and then construct a plurality of corresponding atomic models for the plurality of basic questions.

Taking a customer service scenario as an example, a technician can abstract a plurality of related problems according to data processing frequently occurring in the customer service scenario (which can be understood as a basic scenario covering, for example, customer service scenarios of a plurality of shops of different commodity types) by using the first terminal device, and then perform data processing such as classification, induction and the like on the plurality of related problems, so that a plurality of basic problems can be split from the customer service scenario. For example, it is determined whether the text information sent by the client contains a question of a sensitive word, whether the text information sent by the client needs to be manually replied to a question, or whether the text information sent by the client belongs to a question of an automatic reply type, and so on.

Through the processing, the first terminal device can abstract various concrete data processing frequently occurring in the customer service scene into a plurality of corresponding basic problems. One of the basic questions may correspond to one or more specific data processes in the customer service scenario.

Further, the first terminal device may obtain, by accessing a history database of the network platform, history data related to the customer service scenario as first type sample data. For example, chat records in customer service groups of different commodity type stores of a certain shopping site historically may be acquired as the first type of text data.

Then, the first terminal device may construct an atomic model corresponding to each of the plurality of basic questions according to the first type of sample data, and may solve the basic questions to a certain extent.

In the following, an example of constructing an atomic model No. 1 for the basic problem a is given, which illustrates how the first terminal device specifically constructs an atomic model. The construction method of the atomic model for other basic problems can be implemented by referring to the following construction method of the atomic model No. 1 for the basic problem a, which is not described in detail in this specification.

In implementation, the first terminal device may first screen out an algorithm component suitable for solving the basic problem from existing algorithm components according to the basic problem a.

Specifically, the first terminal device may determine, according to the basic problem a, an activity flow for solving the basic problem a in combination with historically relevant processing experience. And determining the data processing event related to the activity flow and the execution sequence of the data processing event according to the activity flow. And then, according to the data processing events and the execution sequence of the data processing events, corresponding algorithm components can be selected and constructed.

For example, the basic question a is a question for judging whether the text information sent by the client needs to be manually replied, and the basic question a is analyzed in combination with processing experience, so that the basic question a can be further determined as a sort of question, that is, whether the type of the text information sent by the client needs to be manually replied or not. Further, from the existing algorithm components, for example, two-class algorithm components can be screened out and used as algorithm components for constructing the corresponding atomic model.

Of course, the manner in which the above-listed algorithmic components of an atomic model are constructed is merely a schematic illustration. In specific implementation, other suitable manners may be adopted to construct corresponding algorithm components according to specific situations and processing requirements. For example, a new algorithm may be separately designed as a corresponding algorithm component for the basic problem a by the first terminal device.

At the same time of constructing the algorithm component, the first terminal device can also perform standardization processing on the first type of sample data according to a preset standardization rule. The preset standardized rule may specifically include a standard format of data for constructing an atomic model. For example, the first terminal device may convert the data format of the first type of sample data into a unified preset data format defined by a preset standardization rule according to the preset standardization rule, and use the unified preset data format as the standardized first type of sample data.

And then, according to the basic problem A, carrying out corresponding labeling on the standardized first type sample data to obtain labeled first type sample data. And the first type of sample data after the labeling can be used for training the algorithm component to obtain a model which meets the requirements and can be used for solving the basic problem A, and the model is used as an atomic model No. 1.

The atomic model 1 is obtained by training according to the standardized first-type sample data. Therefore, the atomic model No.1 has better universality. Therefore, the atomic model 1 can be directly trained for the second time conveniently by using other data in a unified data format obtained based on the preset standardized rule processing.

It should be noted that, the above-mentioned process of constructing an atomic model is relatively focused on the selection and training of the algorithm level. Therefore, the obtained atomic model is often provided with an algorithm structure capable of better solving basic problems. But the atomic model described above may have relatively little attention to the details of the scene during construction. Therefore, if the above atomic model is directly applied to specific data processing in a certain target scene, the effect is often not ideal.

In this scenario example, the first terminal device constructs the atomic model No.1, and also configures a corresponding data packet for using the atomic model No.1 for the atomic model.

Specifically, the first terminal device may acquire and store the standardized first type sample data used when the atomic model No.1 is trained, and/or the standardized first type sample data that is not used but is suitable for the atomic model training or evaluation is used as the sample set. The first terminal device may also store and establish a corresponding feature set according to data generated in the process of training the atomic model No.1 or using model parameters, weight values of the model parameters, and the like.

Meanwhile, the first terminal device may also establish a training script file (for example, a training rule, a processing manner of training sample data in a training process, a model parameter used in a training process, etc.) for training the number 1 atomic model, an evaluation script file (for example, an evaluation rule, an evaluation index used in an evaluation process, etc.) for evaluating the number 1 atomic model, and deployment environment information (for example, CPU parameters, MEM parameters, etc. relied on in running the number 1 atomic model) for deploying the running environment of the number 1 atomic model according to training record data (for example, a training log, etc.) of the number 1 atomic model, model characteristics of the number 1 atomic model, a memory environment of the number 1 atomic model, and data such as program codes used in training the number 1 atomic model.

In addition, the first terminal device may further store an algorithm component of the atomic model No. 1, and/or further expand other associated algorithm components used in subsequent training of the atomic model No. 1 to build an algorithm component library of the atomic model No. 1.

Furthermore, one or more of the algorithm component library, the training script file, the evaluation script file, the deployment environment information, the sample set and the feature set of the atomic model No. 1 obtained in the above manner may be combined and packaged to obtain a data packet corresponding to the atomic model No. 1. And establishing a corresponding relation between the atomic model No. 1 and the data packet. For example, an identity for indicating atomic model number 1 may be set on the data packet. The data packet corresponding to the atomic model number 1 can be found by retrieving the identity. Thus configuring a corresponding data packet for the atomic model number 1.

According to the above manner, the first terminal device may construct a plurality of atom models respectively corresponding to a plurality of basic questions in the basic scene, and configure corresponding data packets for the plurality of atom models respectively. The atomic model and the data packet may then be sent to a server.

The server receives and stores the plurality of atomic models and the data packets in a database of the network platform.

In this scenario example, when a user needs to train a suitable model for specific data processing in a specific target scenario, the user may interact with the server through the second terminal device, and initiate a model training request, so as to be able to efficiently train to obtain a target model suitable for data processing in the target scenario, and further be able to use the target model to perform data processing in the target scenario.

Specifically, for example, merchant A is the merchant of a clothing store on a shopping site. The merchant A owns and manages the customer service group of the clothing store. In the customer service group, the customer can issue some text information to interact with the merchant A.

For example, the customer may ask the merchant first whether the purchased clothing can be mailed by issuing text information in the customer service group, or ask the merchant first about the size of the clothing, or reflect the presence of flaws in the received clothing to the merchant first, ask the merchant first to provide a corresponding solution, and so on.

Many text messages appear daily in the customer service group. Some text information can be directly replied by using a preset reply text of the merchant A without reminding the merchant A, and other text information can be manually replied by reminding the merchant A.

Currently, a merchant A hopes to train to obtain a target model which can automatically detect and identify text information (namely specific data processing in a target scene) which needs manual answer of the merchant A in customer service groups of clothing stores of the merchant A. The target model can be used for automatically identifying text information requiring manual reply in the customer service group so as to carry out targeted reply processing, and the workload of the merchant A is reduced.

In the implementation, the merchant A can firstly determine the corresponding target problem through the second terminal equipment according to the target scene, namely judging whether text information in a customer service group of the clothing store is text information which needs to be replied manually. Further, a corresponding model training request may be generated based on the target problem. The model training request may carry the target problem. And the second terminal equipment sends the model training request to the server.

Referring to fig. 4, the server receives and determines a target problem based on the model training request. Further, the server may search a database storing a plurality of atomic models based on the target problem, and find the basic problem B that is the same as the target problem or has a high degree of similarity as the matching problem. And searching a number 2 atomic model corresponding to the matched basic problem B from the database to serve as a target atomic model, and searching a pre-configured data packet corresponding to the number 2 atomic model to serve as a target data packet according to the identity information of the number 2 atomic model.

Further, the server may deploy a model operating environment suitable for the atomic model No. 2 according to the deployment environment information in the target data packet. Meanwhile, the server can also generate and send a model training setting interface about the atomic model No. 2 to the second terminal device. The second terminal device may display the model training setting interface to the merchant first.

The model training setting interface may specifically include setting items of a plurality of model training setting parameters. The model training setting parameters may specifically include parameter data for user-defined setting, which affects model training. The user can customize the model training scheme meeting the own requirements by setting corresponding model training setting parameters in the model training setting interface.

Specifically, the model training set parameters may include one or more of the following types of training modes, types of training periods, training targets, model parameters, and the like.

For example, a user may customize whether the model training process is incremental training or full training by setting the type of training mode according to a specific target scenario. For another example, the user may customize whether the model training process is a single training or a periodic training by setting the type of training period at the model training setup interface. If the user selects periodic training, a periodic training setting frame can be further displayed in the model training setting interface, and the user can flexibly set the training task period of the periodic training, the starting time and the ending time of each period and the like through the setting frame according to specific situations. For example, the user can also customize the end conditions of model training by setting training targets, or flexibly adjust model structures, model constraints, training features, etc. in the model training process by setting model parameters. Of course, it should be noted that the model training set parameters listed above are only illustrative. In specific implementation, other types of parameters besides the parameters listed above can be introduced as model training setting parameters according to specific situations and processing requirements. The present specification is not limited to this.

In addition, the model training setting interface may further include a sample data importing interface. The user can flexibly access the sample data related to the target scene owned by the user through the importing interface according to the need to participate in the training of the model together. Therefore, sample data which is better in effectiveness and closer to a target scene and owned by a user can be flexibly imported through the import interface to participate in training of the model, and a better training effect is obtained.

In this scenario example, the merchant first may set the type of the training mode to be a full quantity through the model training setting interface displayed by the second terminal device, the type of the training period is periodic training, and the period of the training task is 9:00 to 12:00 per monday. Meanwhile, the merchant A can also access the sample data to the store customer service group through the model training setting interface, so that the chat record of the last week in the customer service group can be periodically acquired as sample data (recorded as second type sample data) every week, and the model is regularly trained and updated by utilizing the sample data, so that the accuracy of the model is continuously improved.

Correspondingly, the server can receive model training setting parameters set by the merchant A through a model training setting interface displayed by the second terminal equipment, and second-class sample data provided by the merchant A. Furthermore, the server can train the number 2 atomic model in the constructed running environment according to the model training setting parameters, the second type sample data and the pre-configured data packet to obtain the target model meeting the requirements of the commercial tenant A.

Of course, the merchant A may also do nothing in the model training setup interface. Correspondingly, the server can perform model training on the number 2 atomic model according to preset model training setting parameters as default parameters.

In the implementation process, the server can also determine the target characteristics according to the target problems in the target scene. Of course, the model training set parameters set by the merchant A can be combined, and the features determined by the server can be further screened to obtain more accurate target features. Meanwhile, the server can perform standardization processing on the second type sample data provided by the merchant A according to the sample set in the data packet, unify the data format of the newly accessed second type sample data, and obtain the standardized second type sample data. And combining the normalized second type sample data with the normalized first type sample data extracted from the sample set to obtain training sample data for training the model. And performing marking treatment on the training sample data according to the target characteristics to obtain the marked training sample data.

Further, the server can perform model training on the number 2 atomic model by using the training sample data after marking processing in an operation environment according to the training script file in the data packet, so as to obtain the trained number 2 atomic model.

It should be noted that, the training process of the model is different from the construction process of the atomic model, and focuses more on details of the target scene, and actual requirements, use experience and the like when the user uses the model to process specific data in the target scene. Therefore, the trained atomic model No. 2 obtained through training in the mode is closer to specific data processing in the target scene than the untrained atomic model No. 2. For example, in addition to higher accuracy in practical use, the method further shows higher processing efficiency, relatively better use experience, and can more meet the actual demands of front-end users or technicians.

After training is finished, the server shown in fig. 5 may further acquire sample data for evaluation from the sample set, and/or extract sample data for evaluation from the normalized second sample data as evaluation sample data. And according to the evaluation script file in the data packet, carrying out specific evaluation on the trained No. 2 atomic model by using the evaluation sample data, and acquiring corresponding evaluation index parameters to obtain a corresponding evaluation result. And generating a corresponding evaluation report according to the evaluation result. The evaluation report may specifically include evaluation index parameters such as coverage rate, accuracy, and the like.

Of course, before the evaluation, the server may also receive the model evaluation setting parameters set by the merchant A in a user-defined manner through the second terminal device. Correspondingly, the server can evaluate the trained number 2 atomic model by combining the model evaluation setting parameters to obtain a corresponding evaluation report.

The server can determine whether the current trained atomic model No. 2 meets the preset requirement according to the evaluation index parameters in the evaluation report. For example, whether the trained atomic model number 2 meets the preset requirements may be determined by determining whether the coverage of the trained atomic model number 2 is greater than a preset coverage threshold, and/or determining whether the accuracy of the trained atomic model number 2 is greater than a preset accuracy threshold, and the like.

If the currently trained atomic model number 2 meets the preset requirement, training can be finished, and the currently trained atomic model number 2 is determined to be a target model for delivery to the merchant A.

If the currently trained atomic model number 2 does not meet the preset requirement in the above manner, the secondary model training can be continued on the basis of the currently trained atomic model number 2 in the above manner until the trained model finally meets the preset requirement.

The target model is obtained by taking a pre-prepared model number 2 atomic model which contains semantics and can solve corresponding basic problems as a model basis and utilizing a pre-configured data packet to perform model training in combination with data and parameters which are subsequently input by a merchant. Therefore, the server does not need to find and construct proper algorithm components again from scratch, and does not need to regenerate and configure relevant data such as deployment environment information, training script files, evaluation script files and the like of the model, so that the complexity of model training and the data processing capacity of the server are reduced, the target model suitable for data processing in a target scene facing a merchant A can be efficiently trained and obtained, and the training efficiency of the target model is improved.

In addition, based on the training mode, the server performs secondary training on the basis of the atomic model to acquire the model. Therefore, the data processing capacity of the server in model training can be effectively reduced, the server can be capable of simultaneously accessing and responding to a plurality of model training requests, a plurality of matched target atomic models and corresponding data packets are determined from a plurality of preset atomic models, and training of a plurality of models can be simultaneously carried out according to the plurality of target atomic models and the corresponding data packets, so that a plurality of target models corresponding to data processing of different target scenes are obtained.

In this scenario example, after the server trains according to the above manner to obtain the target model meeting the preset requirements and suitable for data processing in the target scenario of the merchant first, referring to fig. 6, the server may further perform corresponding configuration processing on the target model, and then provide the configured target model to the second terminal device, so that the merchant first may access and use the target model more conveniently and directly, thereby further simplifying the operation of the merchant first and improving the use experience of the merchant first.

Specifically, the server may configure the input interface of the corresponding model, the output interface of the model, the model operating environment and related protocols for the target model, and so on, to obtain the configured target model. And issuing the configured target model to a network platform, and simultaneously feeding back to the second terminal equipment for calling links of the configured target model.

The merchant A can directly call the configured target model based on the call link through the second terminal equipment. Furthermore, customer service groups of clothing stores can be conveniently accessed to the input interface of the configured model, and the second terminal equipment is connected with the output interface of the model.

In this way, every preset time interval (for example, 5 minutes), the target model can automatically acquire text information (namely, target data) issued by a customer to be identified and processed, which appears in the customer service group within the last 5 minutes, through the model input interface, and the running model performs identification processing on the text information to determine whether text information which needs to remind a merchant A to answer manually exists, so that a corresponding identification result is obtained and is output as the model.

And the second terminal equipment can determine whether text information which needs manual reply of the merchant A exists in the current customer service group according to the identification result output by the model. Under the condition that the text information needing to be manually replied by the merchant A is determined to exist, the prompting information or the prompting sound can be generated to remind the merchant A that the text information sent by the customer in the current customer service group needs to be manually replied by the merchant A, so that the merchant A does not need to consume a large amount of time and energy, judges and identifies the text information issued by a large amount of customers, the workload of the merchant A is effectively reduced, and meanwhile, the merchant A can be ensured to timely discover and reply the text information needing to be manually replied by the customer in the customer service group.

Referring to fig. 7, an embodiment of the present disclosure provides a data processing method. The method is particularly applied to the server side. In particular implementations, the method may include the following.

And S701, determining a target problem in a target scene.

In some embodiments, the target scenario may be a specific data processing scenario of interest to a front-end user or technician. Specifically, the target scene may be a reply scene of a certain shop customer service group, a processing scene of order data of a certain shopping website, an identification scene of transaction risk on a certain transaction platform, and the like. Of course, the above listed target scenarios are only one illustrative illustration. In specific implementation, the target scene may also include other types of scenes according to specific situations and processing requirements. The present specification is not limited to this.

In some embodiments, the target problem may be specifically abstracting and summarizing related data processing involved in the target scene of interest according to the specific requirements of the front-end user or the technician, and the determined one or more data processing problems.

For example, in a reply scenario of a certain shop customer service group, a front-end technician needs to construct a model capable of automatically identifying text information which is issued by a customer in the shop customer service group and needs to be manually replied by the customer service. At this time, the target problem aiming at the target scene can be determined as whether the type of the text information issued by the clients in the client service group of the shop is the type of the client service manual reply or the type of the client service manual reply is not required.

In some embodiments, the server may determine, according to the model training requirements of the user or technician at the technical end and the related data processing involved in the target scenario, a data processing problem in the target scenario as a target problem to be solved by the model to be trained.

Or the second terminal equipment of the user or the technician arranged at the technical end can sort and analyze related data processing related to the target scene according to the requirements of the user or the technician at the technical end, and after the corresponding target problem is determined, the corresponding target problem is regenerated and the model training request carrying the target problem is sent to the server. The server receives the model training request, and obtains target problems and the like carried by the model training request through data analysis.

S702, determining a target atomic model matched with the target problem from a plurality of atomic models according to the target problem, and acquiring a target data packet corresponding to the target atomic model, wherein the atomic model comprises a preset unit model corresponding to a basic problem in a basic scene, and the atomic models are respectively configured with corresponding data packets.

In some embodiments, the atomic model may specifically include a pre-built unit model corresponding to a basic problem in the basic scene. In particular, the above atomic model may be understood as a minimum model structure that includes semantics, has been trained in advance, and can solve a corresponding basic problem.

In some embodiments, the foregoing data packet may be specifically understood as a pre-configured data packet corresponding to an atomic model, where the data packet includes data related to using the atomic model. Specifically, the data packet may include one or more of an algorithm component library of the target atomic model, a training script file of the target atomic model, an evaluation script file of the target atomic model, deployment environment information of the target atomic model, a sample set of the target atomic model, a feature set of the target atomic model, and the like. Of course, it should be noted that the data in the above-listed data packets is only a schematic illustration. In implementation, the data packet may further include other types of data according to the specific situation and usage rules of the atomic model. The present specification is not limited to this.

In some embodiments, before implementation, a technician at the algorithm end may split the basic scene into a plurality of basic questions through the first terminal device. And respectively constructing corresponding atomic models aiming at the plurality of basic problems to obtain a plurality of atomic models, and respectively configuring the plurality of atomic models and using the data packet of the atomic model. And establishing a corresponding relation between the atomic model and the data packet. For example, an identity for indicating the corresponding atomic model is set on the data packet.

In some embodiments, in implementation, the basic problem with the same or higher similarity may be found as the basic problem of matching according to the target problem, and an atomic model corresponding to the basic problem of matching is found from a plurality of atomic models and is used as the target atomic model. And finding out the pre-configured data packet corresponding to the target atomic model as a target data packet according to the corresponding relation between the atomic model and the data packet.

In some embodiments, the target data packet specifically may include at least one of an algorithm component library of a target atomic model, a training script file of the target atomic model, an evaluation script file of the target atomic model, deployment environment information of the target atomic model, a sample set of the target atomic model, a feature set of the target atomic model, and the like.

In some embodiments, in the case that the data processing involved in the target scene is complex, when the implementation is performed, multiple matching basic questions can be found simultaneously according to the target questions in the target scene. At this time, the target problem may be a combination including a plurality of basic problems.

In this case, a plurality of atomic models corresponding to a plurality of basic questions included in the target question, respectively, can be found as target atomic models based on the target question. Subsequently, the target atomic model including the plurality of atomic models can be trained to obtain a corresponding target model.

S703, training the target atomic model according to the target scene and the target data packet to obtain a target model, wherein the target model is used for data processing in the target scene.

In some embodiments, the above-mentioned object model may be specifically understood as a complete model structure that is trained and can be used to perform specific data processing in the object scene, and can effectively solve specific object problems in the object scene.

For example, for a reply scene of a certain shop customer service group, the target model may be a model that can be better applied to the scene, and accurately identify text information issued by a customer who needs to reply manually from the shop customer service group.

In some embodiments, the training the target atomic model according to the target scene and the target data packet to obtain the target model may include obtaining data in the target scene as second type sample data, performing normalization processing on the second type sample data according to the target data packet to obtain normalized second type sample data, and training the target atomic model according to the target data packet and the normalized second type sample data.

In some embodiments, in implementation, the standardized first type of sample data contained in the sample set in the target data packet may be used as a reference, or the second type of sample data may be subjected to corresponding standardized processing according to a processing rule about the sample data contained in the target data packet, so that a data format of the newly accessed second type of sample data is unified into a data format matched with the target atomic model, so that the target atomic model may be trained by using the standardized second type of sample data more efficiently, training efficiency in subsequent training may be improved, and errors generated due to non-uniformity of the sample data in subsequent training may be reduced.

In some embodiments, the training the target atomic model according to the target data packet and the normalized second type sample data may include extracting normalized first type sample data from the sample set according to the target data packet and the target data packet, combining the normalized first type sample data and the normalized second type sample data to obtain training sample data, and training the target atomic model according to the target data packet by using the training sample data. Therefore, the data characteristics of the first type sample data for constructing the atomic model and the second type sample data close to the target scene can be comprehensively utilized, and the target model with better use effect can be obtained through training.

In some embodiments, in implementation, the normalized first type of sample data may be used alone, or the normalized second type of sample data may be used alone as training sample data to train the target atomic model to obtain the corresponding target model according to specific situations.

In some embodiments, the training the target atomic model according to the target data packet by using the training sample data may further include performing a marking process on the training sample data according to the target scene to obtain marked training sample data, and training the target atomic model according to a training script file in the target data packet by using the marked training sample data.

In some embodiments, before specific training, corresponding operation resources (such as CPU resources and the like) may be allocated to the target atomic model according to the environment deployment information of the target atomic model in the target data packet, and then a matched operation environment is deployed for the target atomic model based on the operation resources, so that the target atomic model can be efficiently trained in the deployed operation environment directly.

In some embodiments, according to algorithm components based on some different target atomic models, the training sample data may be directly input to the target atomic models for model training without performing marking processing on the training sample data.

In some embodiments, in order to further meet the personalized requirements of the front-end user or the technician, the method may further include receiving model training setting parameters, and training the target atomic model according to the training script file and the model training setting parameters and using the training sample data after the marking process.

In some embodiments, the model training setting parameters may specifically include parameter data for user-defined settings to influence model training.

In some embodiments, the model training set parameters may specifically include at least one of a type of training mode, a type of training period, a training target, model parameters, and the like. Of course, it should be noted that the model training set parameters listed above are only illustrative. In specific implementation, the model training set parameters can also contain other types of parameter data according to specific situations and processing requirements. The present specification is not limited to this.

In some embodiments, the server may present the model training setup interface to a front end technician or user via the second terminal device when embodied. The front-end technician or user can set corresponding model training setting parameters through the model training setting interface so as to customize a model training scheme meeting the self requirements. The server can receive the model training setting parameters through the model training setting interface, and can train the atomic model according to the requirements of front-end technicians or users by combining the model training setting parameters, so as to further meet the diversified training requirements of the users.

In some embodiments, after training the target atomic model according to the training script file in the target data packet by using the training sample data after the marking processing, the method may further include performing model evaluation on the trained target atomic model according to the evaluation script file in the target data packet, determining whether the trained target atomic model meets a preset requirement according to the evaluation result, and determining the trained target atomic model as a target model if the trained target atomic model meets the preset requirement.

In some embodiments, the evaluation report may specifically include evaluation index parameters such as coverage rate, accuracy, and the like. Correspondingly, determining whether the trained target atomic model meets the preset requirement according to the evaluation result, and determining the evaluation index parameter related to the trained target atomic model according to the evaluation report when the method is implemented; comparing a preset evaluation index parameter threshold with the evaluation index parameters of the trained target atomic model to obtain a corresponding comparison result, and determining whether the trained target atomic model meets preset requirements according to the comparison result.

For example, when it is determined that the coverage rate of the trained target atomic model is greater than a preset coverage rate threshold and the accuracy is greater than a preset accuracy threshold, determining that the currently trained target atomic model meets the preset requirements, stopping model training at this time, and determining the currently trained target atomic model as the target model.

In some embodiments, after determining whether the trained target atomic model meets the preset requirement according to the evaluation result, the method may further include training the trained target atomic model according to a training script file in the target data packet when it is determined that the trained target atomic model does not meet the preset requirement.

For example, when it is determined that the coverage rate of the trained target atomic model is less than or equal to a preset coverage rate threshold or the accuracy is less than or equal to a preset accuracy threshold, it is determined that the currently trained target atomic model does not meet the preset requirement, and then the model training is not stopped, but the training process is repeated based on the currently trained target atomic model, and the model training is continued until the trained target atomic model meets the preset requirement.

In some embodiments, the performing model evaluation on the trained target atomic model according to the evaluation script file in the target data packet may include receiving a model evaluation setting parameter, and performing model evaluation on the trained target atomic model according to the evaluation script file in the target data packet and the model evaluation setting parameter.

In some embodiments, the model evaluation setting parameters may specifically include parameter data for user-defined settings to affect the model evaluation.

In some embodiments, the model evaluation setting parameters may specifically include at least one of a type of an evaluation mode, an evaluation index parameter, an evaluation period, and the like. Of course, the above-listed model evaluation setting parameters are only one illustrative example. In specific implementation, the model evaluation setting parameters can also include other types of parameter data according to specific situations and processing requirements. The present specification is not limited to this.

In some embodiments, the server may present the model evaluation setting interface to the front-end technician or user via the second terminal device during implementation. The front-end technician or user can set corresponding model evaluation setting parameters through the model evaluation setting interface so as to customize a model evaluation scheme meeting the own requirements. The server can receive the model evaluation setting parameters through the model evaluation setting interface, and can subsequently evaluate the atomic model according to the requirements of front-end technicians or users by combining the model evaluation setting parameters, so as to meet the diversified evaluation requirements of the users.

In some embodiments, after the target atomic model is trained in the above manner to obtain a target model corresponding to a preset requirement, when the method is implemented, the method may further include configuring the target model according to the target scene to obtain a configured target model, where the configuring includes at least one of configuration of an input interface of the model, configuration of an output interface of the model, configuration of an operating environment of the model, and the like. And releasing the configured target model.

In some embodiments, the configuration process listed above is only a schematic illustration. In specific implementation, the configuration process may further include other types of configuration processes according to specific situations and processing requirements. For example, the configuration of the data processing protocol involved in the model run, the configuration of the model run parameters, etc. The present specification is not limited to this.

In some embodiments, after the configured target model is released, the method may further include receiving and responding to a call request of a user for the target model, performing data processing on target data related to a target scene input by the user by using the target model to obtain a corresponding target data processing result, and feeding back the target data processing result.

In some embodiments, the server may issue the configured object model onto the network or send the configured object model to the second terminal device in the above manner. Therefore, the front-end user or the technician can directly call the configured target model to process the data in the target scene, the operation of the front-end technician or the user is simplified, and the use experience of the user is improved.

In some embodiments, the target problem may specifically include a combination of multiple basic problems for a more complex target scenario. Accordingly, the target atomic model may include a plurality of atomic models corresponding to the plurality of basic problems described above.

In some embodiments, the method may further include obtaining a plurality of target atom models and a plurality of corresponding target data packets according to the target problem, wherein each target atom model in the plurality of target atom models is matched with one basic problem in the plurality of basic problems, training the plurality of target atom models according to the target scene and the plurality of target data packets respectively to obtain a plurality of sub-models, and combining the plurality of sub-models to obtain a target model.

In some embodiments, before implementation, a plurality of corresponding atomic models may be pre-built according to a plurality of basic questions split from a basic scene by a first terminal device disposed on a technician side of an algorithm end.

In some embodiments, the atomic model can be specifically built by splitting a plurality of basic questions from a basic scene, wherein the basic scene comprises a scene with coverage larger than a preset coverage threshold, acquiring first-type sample data from the basic scene, and building an atomic model corresponding to the plurality of basic questions in the basic scene according to the first-type sample data.

In some embodiments, the basic scene may be the same scene as the target scene, or a scene similar to or related to the target scene.

In some embodiments, the first type of sample data may be data collected in a basic scene.

In some embodiments, the establishing an atomic model corresponding to each of the plurality of basic questions in the basic scene may include establishing a corresponding algorithm component according to the basic questions, performing standardization processing on the first type sample data according to a preset standardization rule to obtain standardized first type sample data, and establishing an atomic model corresponding to the basic questions according to the standardized first type sample data and the algorithm component.

In some embodiments, the first terminal device may determine, according to the basic problem, an activity flow for solving the basic problem by combining experience, determine, according to the activity flow, a data processing event related to the activity flow and an execution sequence of the data processing event, and construct a corresponding algorithm component according to the data processing event and the execution sequence of the data processing event. The algorithm component may specifically be an existing algorithm component, or may be an algorithm component that is separately recreated by the first terminal device according to the basic problem, or the like.

In some embodiments, after the atomic model corresponding to the basic problem is built, the method may further include configuring a data packet corresponding to the atomic model according to the basic problem and the atomic model when the method is implemented.

In some embodiments, during the process of constructing an atomic model, relevant data used in constructing the atomic model may be acquired and saved to configure a data packet corresponding to the atomic model.

Specifically, for example, first-type sample data used in the process of building an atomic model, model features extracted in the process of building the atomic model, deployment environment parameters in the process of building the atomic model, training script files and evaluation script files used in the process of building the atomic model, and related data such as an initial algorithm component based on which the atomic model is built can be obtained to configure a corresponding data packet.

According to the data processing method, a plurality of basic problems are split from a basic scene with larger coverage area in advance before implementation, corresponding atomic models are built for each basic problem in the basic scene, corresponding data packages are configured for each atomic model, during implementation, the target problem in the target scene can be determined for data processing in the specific target scene, then the target problem is matched to the proper target atomic model from the plurality of pre-built atomic models according to the target problem, the pre-configured target data packages corresponding to the target atomic model are acquired, and model training related to the target scene can be performed on the basis of the target atomic model according to the target scene and the target data packages, so that the target model suitable for specific data processing in the target scene is obtained, and the data processing in the target scene is performed by using the model. Therefore, the method can efficiently train and obtain the applicable target model aiming at various types of data processing in various target scenes, and the training efficiency of the target model is improved.

Referring to fig. 8, the embodiment of the present disclosure further provides a data processing method. The method is particularly applied to the side of the second terminal equipment. The method, when embodied, may include the following.

S801, determining a target problem according to a target scene.

S802, generating and sending a corresponding atomic model selection request to a server according to the target problem, wherein the atomic model selection request is used for requesting the server to determine a target atomic model matched with the target problem from a plurality of atomic models, the atomic model comprises preset unit models corresponding to the basic problem in a basic scene, and the atomic models are respectively configured with corresponding data packets.

S803, receiving the target atomic model fed back by the server and a target data packet corresponding to the target atomic model.

S804, training the target atomic model according to the target scene and the target data packet to obtain a target model.

In some embodiments, in the above manner, the server may be prepared in advance with an atomic model corresponding to a plurality of basic questions in the basic scene, respectively, and a corresponding data packet. When the front-end user or the technician needs to train the target model aiming at the target problem in a specific target scene, an atomic model selection request carrying the target problem can be sent to the server through second terminal equipment arranged on one side of the front-end user or the technician. The server may receive and respond to the atomic model selection request, find, for the second terminal device, a target atomic model matching the target problem from a plurality of pre-constructed atomic models, and send a corresponding target data packet to the second terminal device. Furthermore, the second terminal device can train the target atomic model locally according to the target scene and the target data packet so as to obtain a required target model.

In some embodiments, after obtaining the target model, the method may further include obtaining target data in a target scene and performing data processing on the target data by using the target model.

In some embodiments, the target scene may specifically include a customer service group reply scene. Correspondingly, the target data can specifically include text information sent by clients in the customer service group. Of course, the above listed target scenarios and target data are only one illustrative example. In specific implementation, the target scene may further include other types of scenes (e.g. a web page message customer service answering scene, a mail question customer service answering scene, etc.), and correspondingly, the target data may further include other types of data, according to specific situations and processing requirements. The present specification is not limited to this.

In some embodiments, the data processing of the target data by using the target model may include identifying text information sent by the client by using the target model to determine whether the text information is target text information that needs to be replied manually, generating reply prompt information if the text information is determined to be target text information, and sending the reply prompt information to customer service in the customer service group.

According to the method, the second terminal equipment can determine the target problem according to specific data processing related to the target scene, find the matched target atomic model and the target data packet from the preset atomic model according to the target problem, train the target atomic model according to the target data packet on the basis of the target atomic model so as to obtain the required target model, reduce the complexity of acquiring the target model and improve the training efficiency of the target model.

The embodiment of the specification also provides another data processing method. The method is particularly applied to one side of the first terminal equipment. The method, when embodied, may include the following.

S1, splitting a plurality of basic problems from basic scenes, wherein the basic scenes comprise scenes with coverage areas larger than a preset coverage area threshold value.

S2, acquiring first-type sample data from the basic scene.

And S3, establishing atomic models respectively corresponding to the plurality of basic problems in the basic scene according to the first type of sample data, and configuring corresponding data packets for the atomic models.

And S4, sending the atomic model and the corresponding data packet to a server so that the target model suitable for data processing in the target scene can be obtained by using the corresponding data packet in a high-efficiency training way based on the atomic model.

By the method, the corresponding multiple atomic models and the data packets can be constructed in advance based on the basic problems in the basic scene, so that the workload of subsequent training of the target model is simplified, and the subsequent training of the target model is more convenient and efficient.

The embodiment of the specification also provides another data processing method. The data processing method can be particularly applied to the second terminal equipment arranged at one side of the merchant. The method, when embodied, may include the following.

S1, receiving and responding to a processing request of a merchant, determining that a target scene is a customer service group reply scene, and determining that a target problem is whether text information issued by a customer in the customer service group needs manual reply or not.

S2, generating and sending a corresponding model training request to a server according to the target problem, wherein the model training request carries the target problem and the target scene, the server is used for determining a target atomic model matched with the target problem and a target data packet corresponding to the target atomic model from a plurality of atomic models in response to the model training request, and the server is also used for training the target atomic model according to the target scene and the target data packet so as to obtain the target model.

In one embodiment, the method can be implemented by receiving a target model fed back by a server, wherein the target model is used for judging whether text information issued by a customer in a customer service group needs manual reply or not and prompting the merchant of the text information needing manual reply.

In this embodiment, the second terminal device may acquire and operate the target model, so that text information that needs to be manually replied in the customer service group of the merchant may be timely found through the target model, and the merchant may be timely prompted to reply.

The embodiment of the specification also provides a server, which comprises a processor and a memory for storing executable instructions of the processor, wherein the processor can be used for determining a target problem in a target scene according to the instructions, determining a target atomic model matched with the target problem from a plurality of atomic models according to the target problem, and acquiring a target data packet corresponding to the target atomic model, wherein the atomic model comprises a preset unit model corresponding to a basic problem in a basic scene, the atomic models are further respectively configured with corresponding data packets, and training the target atomic model according to the target scene and the target data packet to obtain the target model, wherein the target model is used for data processing in the target scene.

In order to more accurately complete the above instructions, referring to fig. 9, another specific server is provided in this embodiment of the present disclosure, where the server includes a network communication port 901, a processor 902, and a memory 903, where the foregoing structures are connected by an internal cable, so that each structure may perform specific data interaction.

The network communication port 901 may be specifically configured to receive a model training request sent by the second terminal device.

The processor 902 is specifically configured to respond to a model training request, determine a target problem in a target scene, determine a target atomic model matching the target problem from a plurality of atomic models according to the target problem, and acquire a target data packet corresponding to the target atomic model, where the atomic model includes a preset unit model corresponding to a basic problem in a basic scene, and the atomic model is further configured with corresponding data packets, and train the target atomic model according to the target scene and the target data packet to obtain a target model, where the target model is used for performing data processing in the target scene.

The memory 903 may be used to store a corresponding program of instructions.

In this embodiment, the network communication port 901 may be a virtual port that binds with different communication protocols, so that different data may be sent or received. For example, the network communication port may be a port responsible for performing web data communication, a port responsible for performing FTP data communication, or a port responsible for performing mail data communication. The network communication port may also be an entity's communication interface or a communication chip. For example, it may be a wireless mobile network communication chip such as GSM, CDMA, etc., it may also be a Wifi chip, it may also be a bluetooth chip.

In this embodiment, the processor 902 may be implemented in any suitable manner. For example, a processor may take the form of, for example, a microprocessor or processor, and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application SPECIFIC INTEGRATED Circuits (ASICs), programmable logic controllers, and embedded microcontrollers, among others. The description is not intended to be limiting.

In this embodiment, the memory 903 may include multiple levels, and in a digital system, the memory may be any memory as long as binary data can be stored, in an integrated circuit, a circuit with a storage function without a physical form, such as a RAM, a FIFO, etc., and in a system, a storage device with a physical form, such as a memory bank, a TF card, etc.

The embodiment of the specification also provides a computer storage medium based on the data processing method, wherein the computer storage medium stores computer program instructions, the computer program instructions are implemented when executed, the computer program instructions are used for determining target problems in a target scene, determining target atomic models matched with the target problems from a plurality of atomic models according to the target problems, and acquiring target data packets corresponding to the target atomic models, the atomic models comprise preset unit models corresponding to basic problems in a basic scene, the atomic models are further respectively configured with corresponding data packets, and training the target atomic models according to the target scene and the target data packets to obtain target models, wherein the target models are used for data processing in the target scene.

In the present embodiment, the storage medium includes, but is not limited to, a random access Memory (Random Access Memory, RAM), a Read-Only Memory (ROM), a Cache (Cache), a hard disk (HARD DISK DRIVE, HDD), or a Memory Card (Memory Card). The memory may be used to store computer program instructions. The network communication unit may be an interface for performing network connection communication, which is set in accordance with a standard prescribed by a communication protocol.

In this embodiment, the functions and effects of the program instructions stored in the computer storage medium may be explained in comparison with other embodiments, and are not described herein.

Referring to fig. 10, on a software level, the embodiment of the present disclosure further provides a data processing apparatus, which may specifically include the following structural modules.

The first determining module 1001 may be specifically configured to determine a target problem in a target scene.

The second determining module 1002 may be specifically configured to determine, according to the target problem, a target atomic model that matches the target problem from a plurality of atomic models, and obtain a target data packet corresponding to the target atomic model, where the atomic model includes a preset unit model corresponding to a basic problem in a basic scene, and the atomic models are further configured with corresponding data packets respectively.

The training module 1003 may specifically be configured to train the target atomic model according to the target scene and the target data packet, so as to obtain a target model, where the target model is used for performing data processing in the target scene.

It should be noted that, the units, devices, or modules described in the above embodiments may be implemented by a computer chip or entity, or may be implemented by a product having a certain function. For convenience of description, the above devices are described as being functionally divided into various modules, respectively. Of course, when the present description is implemented, the functions of each module may be implemented in the same piece or pieces of software and/or hardware, or a module that implements the same function may be implemented by a plurality of sub-modules or a combination of sub-units, or the like. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

From the above, the data processing device provided in the embodiments of the present disclosure may perform model training on the basis of the target atomic model according to the target scene and the pre-configured data packet, to obtain the target model suitable for specific data processing of the target scene. Therefore, the method can efficiently train to obtain the applicable target model aiming at various types of data processing in various target scenes, improves the training efficiency of the target model, and meets the model training requirements of user diversification. In addition, the data processing amount at one side of the platform server can be reduced, so that the platform server can be simultaneously accessed and responded to a plurality of model training requests based on the existing data processing performance, and a plurality of different target models can be simultaneously trained for a plurality of different users.

Although the present description provides method operational steps as described in the examples or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps recited in the embodiments is merely one way of performing the order of steps and does not represent a unique order of execution. When implemented by an apparatus or client product in practice, the methods illustrated in the embodiments or figures may be performed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even in a distributed data processing environment). The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, it is not excluded that additional identical or equivalent elements may be present in a process, method, article, or apparatus that comprises a described element. The terms first, second, etc. are used to denote a name, but not any particular order.

Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller can be regarded as a hardware component, and means for implementing various functions included therein can also be regarded as a structure within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.

The description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

From the above description of embodiments, it will be apparent to those skilled in the art that the present description may be implemented in software plus a necessary general hardware platform. Based on such understanding, the technical solutions of the present specification may be embodied essentially in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and include several instructions to cause a computer device (which may be a personal computer, a mobile terminal, a server, or a network device, etc.) to perform the methods described in the various embodiments or portions of the embodiments of the present specification.

Various embodiments in this specification are described in a progressive manner, and identical or similar parts are all provided for each embodiment, each embodiment focusing on differences from other embodiments. The specification is operational with numerous general purpose or special purpose computer system environments or configurations. Such as a personal computer, a server computer, a hand-held or portable device, a tablet device, a multiprocessor system, a microprocessor-based system, a set top box, a programmable electronic device, a network PC, a minicomputer, a mainframe computer, a distributed computing environment that includes any of the above systems or devices, and the like.

Although the present specification has been described by way of example, it will be appreciated by those skilled in the art that there are many variations and modifications to the specification without departing from the spirit of the specification, and it is intended that the appended claims encompass such variations and modifications as do not depart from the spirit of the specification.

Claims

1. A data processing method, comprising:

Identify the target problem in the target scenario;

According to the target problem, a target atomic model matching the target problem is determined from a plurality of atomic models, and a target data packet corresponding to the target atomic model is acquired; wherein the atomic model includes a preset unit model corresponding to a basic problem in a basic scenario, and the atomic models are also respectively configured with corresponding data packets;

The target atomic model is trained according to the target scene and the target data packet to obtain a target model; wherein the target model is used for data processing in the target scene; specifically comprising: acquiring second-category sample data related to the target scene, and performing standardization processing to obtain standardized second-category sample data; combining the standardized second-category sample data with the standardized first-category sample data extracted based on the target data packet to train the target atomic model.

2. According to the method described in claim 1, the target data packet includes at least one of the following: an algorithm component library of the target atomic model, a training script file of the target atomic model, an evaluation script file of the target atomic model, deployment environment information of the target atomic model, a sample set of the target atomic model, and a feature set of the target atomic model.

3. The method according to claim 2, wherein the atomic model is established in the following manner:

Separating a plurality of basic problems from a basic scenario; wherein the basic scenario includes a scenario where the coverage is greater than a preset coverage threshold;

Acquire first type of sample data from the basic scene;

According to the first type of sample data, atomic models corresponding to the multiple basic problems in the basic scenario are established.

4. The method according to claim 3, establishing atomic models corresponding to the multiple basic problems in the basic scenario according to the first type of sample data, comprising:

According to the basic problems, build corresponding algorithm components;

According to a preset standardization rule, the first category of sample data is standardized to obtain standardized first category sample data;

An atomic model corresponding to the basic problem is established based on the standardized first-category sample data and the algorithm component.

5. The method according to claim 4, after establishing the atomic model corresponding to the basic problem, the method further comprises:

According to the basic problem and the atomic model, a data packet corresponding to the atomic model is configured.

6. The method according to claim 1, after obtaining the target model, further comprising:

According to the target scenario, the target model is configured to obtain a configured target model; wherein the configuration process includes at least one of the following: configuration of the input interface of the model, configuration of the output interface of the model, and configuration of the operating environment of the model;

Publish the configured target model.

7. The method according to claim 2, wherein the training of the target atomic model according to the target scene and the target data packet comprises:

Acquire the data in the target scene as the second type of sample data;

According to the target data packet, the second type of sample data is standardized to obtain standardized second type of sample data;

The target atomic model is trained according to the target data packet and the standardized second-category sample data.

8. The method according to claim 7, wherein training the target atomic model according to the target data packet and the standardized second type sample data comprises:

According to the target data packet, extracting standardized first-category sample data from the sample set;

Combining the standardized first-category sample data and the standardized second-category sample data to obtain training sample data;

The target atomic model is trained according to the target data packet and using the training sample data.

9. The method according to claim 8, wherein the training of the target atomic model using the training sample data according to the target data packet comprises:

According to the target scenario, the training sample data is labeled to obtain labeled training sample data;

According to the training script file in the target data packet, the target atomic model is trained using the labeled training sample data.

10. The method according to claim 9, further comprising:

Receive model training setting parameters;

According to the training script file and the model training setting parameters, the target atomic model is trained using the labeled training sample data.

11. According to the method of claim 10, the model training setting parameters include at least one of the following: the type of training method, the type of training cycle, the training target, and the model parameters.

12. The method according to claim 9, after training the target atomic model using the labeled training sample data according to the training script file in the target data packet, the method further comprises:

According to the evaluation script file in the target data packet, a model evaluation is performed on the trained target atomic model to determine an evaluation result for the trained target atomic model;

Determining whether the trained target atomic model meets preset requirements according to the evaluation results;

When it is determined that the trained target atomic model meets the preset requirements, the trained target atomic model is determined as the target model.

13. The method according to claim 12, after determining whether the trained target atomic model meets the preset requirements according to the evaluation results, the method further comprises:

When it is determined that the trained target atomic model does not meet the preset requirements, the trained target atomic model is trained according to the training script file in the target data packet.

14. The method according to claim 12, wherein the step of performing model evaluation on the trained target atomic model according to the evaluation script file in the target data packet comprises:

Receive model evaluation setting parameters;

Model evaluation is performed on the trained target atomic model according to the evaluation script file in the target data packet and the model evaluation setting parameters.

15. The method according to claim 6, after publishing the configured target model, the method further comprises:

Receive and respond to a user's call request for a target model, and use the target model to process target data related to a target scene input by the user to obtain a corresponding target data processing result;

Feedback the target data processing result.

The method according to claim 1 , wherein the target question comprises a combination of a plurality of basic questions.

17. The method according to claim 16, further comprising:

According to the target problem, a plurality of target atomic models and a corresponding plurality of target data packets are obtained; wherein each target atomic model in the plurality of target atomic models is matched with one basic problem in the plurality of basic problems respectively;

According to the target scene and the multiple target data packets, respectively train the multiple target atomic models to obtain multiple sub-models;

The multiple sub-models are combined to obtain a target model.

18. A data processing method, comprising:

Determine the target problem based on the target scenario;

According to the target problem, a corresponding atomic model selection request is generated and sent to the server; wherein the atomic model selection request is used to request the server to determine a target atomic model matching the target problem from multiple atomic models, the atomic model includes a preset unit model corresponding to the basic problem in the basic scene, and the atomic model is also respectively configured with a corresponding data packet;

receiving the target atomic model fed back by the server and a target data packet corresponding to the target atomic model;

The target atomic model is trained according to the target scene and the target data packet to obtain the target model, specifically comprising: acquiring second-category sample data related to the target scene and performing standardization processing to obtain standardized second-category sample data; combining the standardized second-category sample data with the standardized first-category sample data extracted based on the target data packet to train the target atomic model.

19. The method according to claim 18, after obtaining the target model, the method further comprises:

Obtain target data in the target scene;

The target model is used to perform data processing on the target data.

20. The method according to claim 19, wherein the target scenario comprises a customer service group reply scenario, and the target data comprises text messages sent by customers in the customer service group.

21. The method according to claim 20, wherein the step of processing the target data using the target model comprises:

Using the target model to perform recognition processing on the text message sent by the customer to determine whether the text message is a target text message that requires a manual reply;

In the case where it is determined that the text information is the target text information, generating a reply prompt information;

The reply prompt information is sent to the customer service staff in the customer service group.

22. A data processing method, comprising:

Receive and respond to the merchant's processing request, determine that the target scenario is the customer service group reply scenario, and the target question is to determine whether the text message posted by the customer in the customer service group requires a manual reply;

According to the target problem, a corresponding model training request is generated and sent to the server; wherein the model training request carries the target problem and the target scenario; the server is used to respond to the model training request, and determine a target atomic model that matches the target problem from multiple atomic models, and a target data packet corresponding to the target atomic model; the server is also used to train the target atomic model according to the target scenario and the target data packet to obtain the target model; the server specifically obtains the second category sample data related to the target scenario, and performs standardization processing to obtain the standardized second category sample data; the standardized second category sample data is combined with the standardized first category sample data extracted based on the target data packet to train the target atomic model.

23. The method according to claim 22, further comprising:

Receive the target model fed back by the server; wherein the target model is used to determine whether the text message posted by the customer in the customer service group requires a manual reply, and to prompt the merchant with the text message that requires a manual reply.

24. A data processing device, comprising:

A first determination module is used to determine a target problem in a target scenario;

A second determination module is used to determine a target atomic model matching the target problem from a plurality of atomic models according to the target problem, and obtain a target data packet corresponding to the target atomic model; wherein the atomic model includes a preset unit model corresponding to the basic problem in the basic scene, and the atomic models are also respectively configured with corresponding data packets;

A training module, used for training the target atomic model according to the target scene and the target data packet to obtain a target model; wherein the target model is used for data processing in the target scene;

The training module is specifically used to obtain second-category sample data related to the target scene, and perform standardization processing to obtain standardized second-category sample data; the standardized second-category sample data is combined with the standardized first-category sample data extracted based on the target data packet to train the target atomic model.

25. A server, comprising a processor and a memory for storing processor-executable instructions, wherein the processor implements the steps of the method according to any one of claims 1 to 17 when executing the instructions.

26. A computer-readable storage medium having computer instructions stored thereon, wherein the instructions, when executed, implement the steps of the method according to any one of claims 1 to 17.