CN116383141A - File management method, device and system - Google Patents

Abstract

The application provides a file management method, a device and a system, which relate to the technical field of finance, and the method is applied to a file management platform; the file management platform is in butt joint with a plurality of channels through a plurality of front-end servers; the method comprises the following steps: acquiring channel file data from a plurality of front-end servers at regular time; storing channel file data; analyzing the channel file data; and sending the parsed data to a downstream system. The method and the system can collect and analyze the content aiming at the file data of different channels, and provide real-time data for each downstream service system.

Description

File management method, device and system

technical field

The present application relates to the field of financial technology, in particular to a file management method, device and system.

Background technique

The development of fund custody-related businesses requires valuation, accounting and reconciliation for each product and each trading day. In this process, it is necessary to receive daily transaction flow and data landing documents such as market data through various channels (such as: Shenzhen Securities Connect, China Securities Regulatory Commission, China Securities Securities, China Bond, and data sent by various managers, etc.). The landing file formats sent by different channels and platforms vary greatly, including txt, Excel, dbf and picture files. The acquisition of the content of these files has brought great trouble to each user system.

In the prior art, receiving and parsing files by each system and saving relevant files by itself will cause file processing confusion, which cannot be managed uniformly and will cause redundant file processing during use, resulting in unnecessary waste of resources.

Contents of the invention

The purpose of this application is to provide a document management method, device and system, which can collect and analyze document data from different channels, and provide real-time data for various downstream business systems.

In the first aspect, the embodiment of the present application provides a file management method, which is applied to a file management platform; the file management platform connects to multiple channels through multiple front-end servers; the method includes: regularly obtaining channel file data from multiple front-end servers ; Store the channel file data; analyze the channel file data; send the analyzed data to the downstream system.

In a preferred implementation manner of the present application, the above-mentioned step of storing channel file data includes: storing channel file data according to file receiving time and preset path format.

In a preferred embodiment of the present application, the above method further includes: if detecting target channel file data whose storage time exceeds a preset time threshold, compressing the target channel file data into a data packet and storing the data packet.

In a preferred embodiment of the present application, the above-mentioned step of analyzing the channel file data includes: judging whether the channel file data is in image format; if so, performing optical character recognition on the channel file data to obtain the parsed data; if No, perform preset text analysis on the channel file data to obtain the analyzed data.

In a preferred embodiment of the present application, the above method further includes: regularly monitoring whether the specified channel file data arrives on time; if not, sending a reminder message that the file has not arrived on time to the supervisor terminal corresponding to the specified channel file data.

In a preferred embodiment of the present application, the above method also includes: regularly obtaining the running status information corresponding to all front-end servers; judging whether there is a front-end server failure according to the running status information; if so, sending the server to the relevant staff terminal Fault reminder information.

In a preferred embodiment of the present application, the above-mentioned multiple channels include mail terminals, platforms or enterprise systems for generating financial-related file data.

In the second aspect, the embodiment of the present application also provides a file management device, which is applied to a file management platform; the file management platform connects to multiple channels through multiple front-end servers; The front-end server acquires channel file data; the data storage module is used to store the channel file data; the data analysis module is used to analyze the channel file data; the data sending module is used to send the analyzed data to the downstream system.

In the third aspect, the embodiment of the present application also provides a file management system. The system includes: a file management platform and multiple front-end servers; the file management platform connects to multiple channels through multiple front-end servers; the file management platform is used to execute such as The method described in the first aspect.

In the fourth aspect, the embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are invoked and executed by the processor, the computer-executable instructions prompt the processor to Implement the method described in the first aspect above.

In the file management method, device and system provided in the embodiments of the present application, the method is applied to the file management platform; the file management platform connects to multiple channels through multiple front-end servers; the method includes: regularly obtaining channel file data from multiple front-end servers ; Store the channel file data; analyze the channel file data; send the analyzed data to the downstream system. The embodiment of the present application can collect and analyze the content of file data from different channels, and provide real-time data for various downstream business systems.

Description of drawings

In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the accompanying drawings that need to be used in the description of the specific embodiments or prior art. Obviously, the accompanying drawings in the following description The figures show some implementations of the present application, and those skilled in the art can obtain other figures based on these figures without any creative effort.

FIG. 1 is an architecture diagram of a file management system provided by an embodiment of the present application;

FIG. 2 is a flow chart of a file management method provided in an embodiment of the present application;

FIG. 3 is a structural block diagram of a file management device provided in an embodiment of the present application;

FIG. 4 is a structural block diagram of a file management system provided by an embodiment of the present application;

FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions of the present application will be clearly and completely described below in conjunction with the embodiments. Apparently, the described embodiments are part of the embodiments of the present application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

At present, the acquisition or analysis of files from certain channels requires manual acquisition or manual download, which not only consumes a lot of manpower, but also leads to messy maintenance of data by multiple people, and it is difficult to ensure that files will not be lost. Greatly limited the development of business. In terms of file parsing, multiple downstream systems of various files must perform their own parsing work, which greatly slows down the business processing capabilities of downstream systems, and multiple downstream systems receiving and parsing the same file will lead to work redundancy.

Based on this, the embodiments of the present application provide a file management method, device, and system, which can collect and analyze file data from different channels, and provide real-time data for various downstream business systems. To facilitate the understanding of this embodiment, a file management method disclosed in the embodiment of the present application is first introduced in detail.

A file management method provided in the embodiment of the present application is applied to a file management platform; the file management platform is connected to multiple channels through multiple front-end servers; referring to the system architecture diagram shown in Figure 1, the multiple channels include Mail terminals, platforms, or enterprise systems used to generate financial-related document data, such as Shenzhen Securities Connect, Mail, Exchange, China Securities, China Securities Regulatory Commission, China Bond, Caihui, etc. Each channel has a different receiving method and is located in a different network area. Each receiving file server is deployed for different channels and network areas. These servers are collectively called the front-end processor, which is the above-mentioned front-end server. The multiple channels are explained below:

1) Shenzhen Zhengtong: full name Shenzhen Zhengtong Information Technology Co., Ltd., responsible for financial data transmission. It mainly receives data from various fund managers.

2) Email: Some administrators send files in the form of email attachments, and need to process emails in batches to obtain business data.

3) Exchanges: Receive data through their own gateways of the three major exchanges.

4) China Securities: China Securities Index Co., Ltd.

5) Zhongdeng: China Securities Depository and Clearing Corporation Limited.

6) China Bond: China Central Depository and Clearing Co., Ltd.

7) Caihui: Shanghai Great Wisdom Caihui Data Technology Co., Ltd.

8) Others (SFTP&Saturn): SFTP is a file transfer protocol, and Saturn is an object asynchronous transfer queue. Referring to the flowchart of the file management method shown in Figure 2, the method specifically includes the following steps:

Step S202, regularly acquire channel file data from multiple front-end servers.

Step S204, storing the channel file data; synchronizing the files on the front-end server to the file management platform through the preset storage configuration. Storage configuration refers to the path mapping relationship from the file structure of the front-end server to the file storage server of the file management platform.

Step S206, analyzing the channel file data.

Here, regardless of the format of the channel file data, it can be automatically parsed without manual participation. For example, for the image format type, it can be parsed by optical character recognition technology.

Step S208, sending the analyzed data to the downstream system.

The file management method provided by the embodiment of the present application is applied to a file management platform; the file management platform connects to multiple channels through multiple front-end servers; the method includes: regularly obtaining channel file data from multiple front-end servers; Store; analyze the channel file data; send the analyzed data to the downstream system. The embodiment of the present application can collect and analyze the content of file data from different channels, and provide real-time data for various downstream business systems.

The embodiment of the present application also provides another file management method, which is implemented on the basis of the above-mentioned embodiments; this embodiment focuses on describing the processes of file storage, parsing, and monitoring.

In a preferred implementation manner of the present application, the above-mentioned step of storing channel file data includes: storing channel file data according to file receiving time and preset path format.

In a preferred embodiment of the present application, the above method further includes: if detecting target channel file data whose storage time exceeds a preset time threshold, compressing the target channel file data into a data packet and storing the data packet.

In a preferred embodiment of the present application, the above-mentioned step of analyzing the channel file data includes: judging whether the channel file data is in image format; if so, performing optical character recognition on the channel file data to obtain the parsed data; if No, perform preset text analysis on the channel file data to obtain the analyzed data.

OCR Optical Character Recognition: The system automatically parses the text in pictures, PDFs and other results that cannot be read directly, and extracts the information in such files through OCR technology for use in downstream systems.

In this embodiment, for a variety of Excel file templates, different parsing configurations (supporting functions such as addition, subtraction, multiplication, and division, and string splicing and cutting) are automatically matched and parsed into different template data as data entities. The business types corresponding to different data entities have achieved horizontal configuration and expansion of business.

For PDF files, the OCR technology can automatically adapt to extract the information required by the file templates corresponding to the hosting business through a large number of machine learning and corrections. When adding similar business type data templates, it can reduce the problems caused by different templates. The workload of re-development and analysis can also be expanded horizontally by business templates.

The above parsing process can be realized through the fuzzy matching configuration of the full path of the file. For example, if you want to parse the file in the /a folder and send it to system a, a configuration like this will be added to the database, and the system will automatically parse according to this configuration and Send the analysis result to system a.

In a preferred embodiment of the present application, the above method further includes: regularly monitoring whether the specified channel file data arrives on time; if not, sending a reminder message that the file has not arrived on time to the supervisor terminal corresponding to the specified channel file data. If the file does not arrive within the specified time, the relevant person in charge will be reminded by SMS or email.

In a preferred embodiment of the present application, the above method also includes: regularly obtaining the running status information corresponding to all front-end servers; judging whether there is a front-end server failure according to the running status information; if so, sending the server to the relevant staff terminal Fault reminder information.

In this embodiment, the system involves many front-end servers with various applications. The system monitors the normal status of the applications of each machine, and if it is abnormal, it will send an alarm by email or short message.

The file management method provided in the embodiment of the present application can realize the following functions:

1. Files from different sources have a file storage rule for saving files.

2. Ensure the real-time performance of the file arrival and push business system.

3. Support the content analysis of landing documents in various formats, and ensure accuracy and timeliness.

4. Monitoring whether key documents arrive on time.

5. Business continuity for system upgrade and maintenance.

This program has the following advantages:

1. It can manage the rapidly increasing data in batches and in an orderly manner, reduce the cost of a large amount of manual maintenance data, and improve business processing efficiency.

2. OCR (Optical Character Recognition) character recognition technology is implemented internally, which reduces the error rate and operational complexity caused by manual inspection by business personnel, and improves the business efficiency of various downstream systems.

3. Unified analysis and processing of files reduces system coupling and reusability in terms of business group system architecture.

4. The complete dimension of the monitoring system can detect and solve problems in time, ensuring normal business continuity and real-time performance.

5. Data files are managed in a unified manner and accessed according to rights to ensure data security requirements and avoid file loss or leakage.

Based on the above method embodiment, the embodiment of the present application also provides a file management device, which is applied to a file management platform; the file management platform connects to multiple channels through multiple front-end servers; as shown in Figure 3, the device includes: Obtaining module 32, is used for regularly obtaining channel file data from a plurality of front-end servers; Data storage module 34, is used for storing channel file data; Data analysis module 36, is used for analyzing channel file data; Data sending module 38 , used to send the parsed data to downstream systems.

In a preferred embodiment of the present application, the above-mentioned data storage module 34 is configured to store channel file data according to file receiving time and preset path format.

In a preferred embodiment of the present application, the data storage module 34 is configured to compress the target channel file data into data packets and store the data packets if the target channel file data whose storage time exceeds a preset time threshold is detected.

In a preferred embodiment of the present application, the above-mentioned data analysis module 36 is used to judge whether the channel file data is in image format; if yes, perform optical character recognition on the channel file data to obtain the parsed data; The file data is analyzed by preset text, and the analyzed data is obtained.

In a preferred embodiment of the present application, the above-mentioned device also includes a monitoring module, which is used to regularly monitor whether the specified channel file data arrives on time; if not, send a reminder message that the file has not arrived on time to the supervisor terminal corresponding to the specified channel file data .

In a preferred embodiment of the present application, the above-mentioned monitoring module is used to regularly obtain the running status information corresponding to all front-end servers; judge whether there is a front-end server failure according to the running status information; if so, send a message to the relevant staff terminal Server failure reminder message.

In a preferred embodiment of the present application, the above multiple channels include multiple of the following: Shenzhen Securities Connect, mail, stock exchange, China Securities, China Securities Regulatory Commission, China Bond, and Caihui.

The implementation principles and technical effects of the devices provided in the embodiments of the present application are the same as those of the aforementioned method embodiments. For brief description, for the parts not mentioned in the device embodiments, reference may be made to the corresponding content in the aforementioned method embodiments.

Based on the above-mentioned method embodiment, the embodiment of the present application also provides a file management system, as shown in FIG. 4 , the system includes: a file management platform 42 and a plurality of front-end servers 44; 44 docking multiple channels; the file management platform 42 is used to perform the following method steps:

Regularly obtain channel file data from multiple front-end servers; store channel file data; analyze channel file data; and send the analyzed data to downstream systems.

In a preferred implementation manner of the present application, the above-mentioned step of storing channel file data includes: storing channel file data according to file receiving time and preset path format.

In a preferred embodiment of the present application, the above method further includes: if detecting target channel file data whose storage time exceeds a preset time threshold, compressing the target channel file data into a data packet and storing the data packet.

In a preferred embodiment of the present application, the above-mentioned step of analyzing the channel file data includes: judging whether the channel file data is in image format; if so, performing optical character recognition on the channel file data to obtain the parsed data; if No, perform preset text analysis on the channel file data to obtain the analyzed data.

In a preferred embodiment of the present application, the above method further includes: regularly monitoring whether the specified channel file data arrives on time; if not, sending a reminder message that the file has not arrived on time to the supervisor terminal corresponding to the specified channel file data.

In a preferred embodiment of the present application, the above method also includes: regularly obtaining the running status information corresponding to all front-end servers; judging whether there is a front-end server failure according to the running status information; if so, sending the server to the relevant staff terminal Fault reminder information.

In a preferred embodiment of the present application, the above multiple channels include multiple of the following: Shenzhen Securities Connect, mail, stock exchange, China Securities, China Securities Regulatory Commission, China Bond, and Caihui.

The implementation principles and technical effects of the system provided by the embodiment of the present application are the same as those of the foregoing method embodiments. For brief description, for the parts not mentioned in the system embodiments, reference may be made to the corresponding content in the foregoing method embodiments.

The embodiment of the present application also provides an electronic device. As shown in FIG. Executing computer-executable instructions, the processor 51 executes the computer-executable instructions to implement the above method.

In the embodiment shown in FIG. 5 , the electronic device further includes a bus 52 and a communication interface 53 , wherein the processor 51 , the communication interface 53 and the memory 50 are connected through the bus 52 .

Wherein, the memory 50 may include a high-speed random access memory (RAM, Random Access Memory), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 53 (which may be wired or wireless), and the Internet, wide area network, local network, metropolitan area network, etc. can be used. The bus 52 may be an ISA (Industry Standard Architecture, industry standard architecture) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus, or an EISA (Extended Industry Standard Architecture, extended industry standard architecture) bus, etc. The bus 52 can be divided into address bus, data bus, control bus and so on. For ease of representation, only one double-headed arrow is used in FIG. 5 , but it does not mean that there is only one bus or one type of bus.

The processor 51 may be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 51 or instructions in the form of software. Above-mentioned processor 51 can be general-purpose processor, comprises central processing unit (Central Processing Unit, be called for short CPU), network processor (Network Processor, be called for short NP) etc.; Can also be digital signal processor (Digital Signal Processor, be called for short DSP) , Application Specific Integrated Circuit (ASIC for short), Field Programmable Gate Array (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register. The storage medium is located in the memory, and the processor 51 reads the information in the memory, and completes the steps of the methods in the foregoing embodiments in combination with its hardware.

The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are invoked and executed by a processor, the computer-executable instructions prompt the processor to Implement the following methods:

Regularly obtain channel file data from multiple front-end servers; store channel file data; analyze channel file data; and send the analyzed data to downstream systems.

In a preferred implementation manner of the present application, the above-mentioned step of storing channel file data includes: storing channel file data according to file receiving time and preset path format.

In a preferred embodiment of the present application, the above method further includes: if detecting target channel file data whose storage time exceeds a preset time threshold, compressing the target channel file data into a data packet and storing the data packet.

In a preferred embodiment of the present application, the above-mentioned step of analyzing the channel file data includes: judging whether the channel file data is in image format; if so, performing optical character recognition on the channel file data to obtain the parsed data; if No, perform preset text analysis on the channel file data to obtain the analyzed data.

In a preferred embodiment of the present application, the above method further includes: regularly monitoring whether the specified channel file data arrives on time; if not, sending a reminder message that the file has not arrived on time to the supervisor terminal corresponding to the specified channel file data.

In a preferred embodiment of the present application, the above method also includes: regularly obtaining the running status information corresponding to all front-end servers; judging whether there is a front-end server failure according to the running status information; if so, sending the server to the relevant staff terminal Fault reminder information.

In a preferred embodiment of the present application, the above multiple channels include multiple of the following: Shenzhen Securities Connect, mail, stock exchange, China Securities, China Securities Regulatory Commission, China Bond, and Caihui.

The computer program product of the method, device, and electronic equipment provided in the embodiments of the present application includes a computer-readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the methods described in the foregoing method embodiments, For specific implementation, reference may be made to the method embodiments, which will not be repeated here.

Relative steps, numerical expressions and values of components and steps set forth in these embodiments do not limit the scope of the present application unless specifically stated otherwise.

If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes. .

In the description of this application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" etc. The indicated orientation or positional relationship is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the application and simplification of the description, rather than indicating or implying that the referred device or element must have a specific orientation, use a specific orientation construction and operation, therefore should not be construed as limiting the application. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and should not be construed as indicating or implying relative importance.

Finally, it should be noted that: the above-described embodiments are only specific implementations of the application, used to illustrate the technical solutions of the application, rather than limiting it, and the scope of protection of the application is not limited thereto, although referring to the aforementioned The embodiment has described this application in detail, and those of ordinary skill in the art should understand that any person familiar with this technical field can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in this application Changes can be easily imagined, or equivalent replacements can be made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be covered by this application. within the scope of protection. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims (10)

1. A file management method, characterized in that, the method is applied to a file management platform; the file management platform is connected to multiple channels through a plurality of front-end servers; the method includes: Regularly obtain channel file data from multiple front-end servers; storing the channel file data; Analyzing the channel file data; Send the parsed data to downstream systems. 2. The method according to claim 1, wherein the step of storing the channel file data comprises: The channel file data is stored according to the file receiving time and the preset path format. 3. The method according to claim 2, wherein the method further comprises: If the target channel file data whose storage time exceeds the preset time threshold is detected, the target channel file data is compressed into a data packet, and the data packet is stored. 4. The method according to claim 1, wherein the step of parsing the channel file data comprises: Judging whether the channel file data is in image format; If so, performing optical character recognition on the channel file data to obtain the parsed data; If not, perform preset text parsing on the channel file data to obtain parsed data. 5. The method according to claim 1, wherein the method further comprises: Regularly monitor whether the specified channel file data arrives on time; If not, send a reminder message that the file has not arrived on time to the supervisor terminal corresponding to the specified channel file data. 6. The method according to claim 1, further comprising: Regularly obtain the running status information corresponding to all front-end servers; judging whether there is a failure of the front-end server according to the operation state information; If so, send a server failure reminder message to the relevant staff terminal. 7. The method of claim 1, wherein the plurality of channels includes a mail terminal, a platform, or an enterprise system for generating financial-related document data. 8. A file management device, characterized in that the device is applied to a file management platform; the file management platform connects to multiple channels through multiple front-end servers; the device includes: The data acquisition module is used to regularly acquire channel file data from multiple front-end servers; A data storage module, configured to store the channel file data; A data parsing module, configured to parse the channel file data; The data sending module is used to send the parsed data to a downstream system. 9. A file management system, characterized in that, the system includes: a file management platform and a plurality of front-end servers; the file management platform connects to multiple channels through a plurality of the front-end servers; the file management platform For performing the method as described in any one of claims 1-7. 10. A computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are invoked and executed by a processor, the computer-executable instructions prompt the processor Realize the method described in any one of claims 1 to 7.

Images