CN112069425A - Log management method and device, electronic equipment and readable storage medium - Google Patents
Log management method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN112069425A CN112069425A CN202010780439.6A CN202010780439A CN112069425A CN 112069425 A CN112069425 A CN 112069425A CN 202010780439 A CN202010780439 A CN 202010780439A CN 112069425 A CN112069425 A CN 112069425A
- Authority
- CN
- China
- Prior art keywords
- log
- information
- log information
- identity
- application system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/955—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
- G06F16/9566—URL specific, e.g. using aliases, detecting broken or misspelled links
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
本申请提供一种日志管理方法、装置、电子设备及可读存储介质。方法可以包括:通过预先配置的登录日志采集适配规则,获取登录应用系统的登录操作的得到的第一日志信息,第一日志信息包括账户信息、与账户信息对应的第一用户终端的第一身份标识;通过预先配置的访问日志采集适配规则,获取访问应用系统的访问操作的得到的第二日志信息;当第二日志信息中存在与第一身份标识相同的第二身份标识时,将第一身份标识对应的账户信息与第二身份标识对应的日志关联,得到目标日志信息,能够改善因采用Session/Cookie机制保存账户信息时,无法从日志获取到账户信息而导致获取的日志存在信息缺失的问题。
The present application provides a log management method, apparatus, electronic device and readable storage medium. The method may include: acquiring first log information obtained from a login operation of logging in to an application system through a preconfigured login log collection and adaptation rule, where the first log information includes account information and a first log of the first user terminal corresponding to the account information. identity identifier; obtain the second log information obtained by accessing the access operation of the application system through the preconfigured access log collection and adaptation rules; when there is a second identity identifier identical to the first identity identifier in the second log information, the The account information corresponding to the first identity identifier is associated with the log corresponding to the second identity identifier, and the target log information is obtained, which can improve the log existence information obtained because the account information cannot be obtained from the log when the session/cookie mechanism is used to save the account information. missing question.
Description
技术领域technical field
本发明涉及计算机日志处理技术领域,具体而言,涉及一种日志管理方法、装置、电子设备及可读存储介质。The present invention relates to the technical field of computer log processing, and in particular, to a log management method, apparatus, electronic device and readable storage medium.
背景技术Background technique
在浏览器/服务器架构,以及使用公钥基础设施(Public Key Infrastructure,PKI)/数字证书模式登录的业务应用系统中,针对发生在账户密码模式登录的应用系统上的应用访问行为,由于普遍采用的是Session/Cookie机制来保存账户信息,所保存的用户信息不具备可读性,也就无法从日志获取到账户信息。在账户密码模式登录的应用系统中,获取的日志存在信息缺失的问题。In the browser/server architecture and the business application system that uses the public key infrastructure (Public Key Infrastructure, PKI)/digital certificate mode to log in, for the application access behavior that occurs on the application system logged in in the account password mode, due to the common use of The Session/Cookie mechanism is used to save account information. The saved user information is not readable, and account information cannot be obtained from the log. In the application system logged in in the account password mode, there is a problem of missing information in the obtained logs.
发明内容SUMMARY OF THE INVENTION
本申请提供一种日志管理方法、装置、电子设备及可读存储介质,能够改善在账户密码模式登录的应用系统中,获取的日志存在信息缺失的问题。The present application provides a log management method, device, electronic device and readable storage medium, which can improve the problem of missing information in the obtained log in an application system logged in in an account password mode.
为了实现上述目的,本申请实施例所提供的技术方案如下所示:In order to achieve the above purpose, the technical solutions provided by the embodiments of the present application are as follows:
第一方面,本申请实施例提供一种日志管理方法,所述方法包括:In a first aspect, an embodiment of the present application provides a log management method, the method includes:
通过预先配置的登录日志采集适配规则,获取登录应用系统的登录操作的得到的第一日志信息,所述第一日志信息包括账户信息、与所述账户信息对应的第一用户终端的第一身份标识;Obtain the first log information obtained from the login operation of logging in to the application system by using the preconfigured login log collection and adaptation rules, where the first log information includes account information and the first log information of the first user terminal corresponding to the account information. identification;
通过预先配置的访问日志采集适配规则,获取访问所述应用系统的访问操作的得到的第二日志信息,所述第二日志信息包括第二用户终端的第二身份标识;Acquire second log information obtained from an access operation of accessing the application system through a preconfigured access log collection and adaptation rule, where the second log information includes the second identity of the second user terminal;
当所述第二日志信息中存在与所述第一身份标识相同的第二身份标识时,将所述第一身份标识对应的账户信息与所述第二身份标识对应的日志关联,得到目标日志信息。When there is a second identity identifier that is the same as the first identity identifier in the second log information, associate the account information corresponding to the first identity identifier with the log corresponding to the second identity identifier to obtain a target log information.
在上述的实施方式中,通过预先配置的登录日志采集适配规则,可以获取到登录应用系统中产生的第一日志信息,第一日志信息中通常可以包括账户信息。然后结合访问日志采集适配规则获取的第二日志信息,可以在账户密码模式登录的应用系统中获取到账户信息,改善因采用Session/Cookie机制保存账户信息时,无法从日志获取到账户信息而导致获取的日志存在信息缺失的问题。In the above-mentioned embodiment, the first log information generated in the login application system can be obtained through the preconfigured log-in log collection and adaptation rules, and the first log information may generally include account information. Then, in combination with the second log information obtained from the access log collection and adaptation rules, account information can be obtained in the application system logged in in the account password mode. There is a problem of missing information in the obtained logs.
结合第一方面,在一些可选的实施方式中,通过预先配置的登录日志采集适配规则,获取登录应用系统的登录操作的得到的第一日志信息,包括:With reference to the first aspect, in some optional implementation manners, the first log information obtained from the login operation of logging in to the application system is obtained through preconfigured login log collection and adaptation rules, including:
通过所述登录日志采集适配规则,确定表征登录所述应用系统的第一URL信息;Determine the first URL information representing the login to the application system by collecting the adaptation rules through the login log;
从所述第一URL信息中获取包括所述账户信息的日志信息;Obtain log information including the account information from the first URL information;
在所述日志信息中添加第一标识,得到所述第一日志信息,所述第一标识用于表征所述第一日志信息为登录日志。A first identifier is added to the log information to obtain the first log information, where the first identifier is used to represent that the first log information is a login log.
在上述的实施方式中,通过获取表征登录应用系统的URL信息,便可以从URL信息中解析得到账户信息。In the above embodiment, by acquiring the URL information representing the login application system, the account information can be obtained by parsing the URL information.
结合第一方面,在一些可选的实施方式中,通过预先配置的访问日志采集适配规则,获取访问所述应用系统的访问操作的得到的第二日志信息,包括:With reference to the first aspect, in some optional implementation manners, the second log information obtained from the access operation of accessing the application system is obtained through preconfigured access log collection and adaptation rules, including:
通过所述访问日志采集适配规则,确定表征访问所述应用系统的第二URL信息;Through the access log collection and adaptation rules, determine the second URL information representing the access to the application system;
从所述第二URL信息中获取日志信息;Obtain log information from the second URL information;
在所述日志信息中添加第二标识,得到所述第二日志信息,所述第二标识用于表征所述第二日志信息为访问日志。A second identifier is added to the log information to obtain the second log information, where the second identifier is used to represent that the second log information is an access log.
结合第一方面,在一些可选的实施方式中,所述方法还包括:In conjunction with the first aspect, in some optional embodiments, the method further includes:
根据预设清洗分类策略,对所述目标日志信息进行清洗分类,得到清洗分类后的日志信息。According to a preset cleaning and classification strategy, cleaning and classifying the target log information is performed to obtain cleaned and classified log information.
在上述的实施方式中,通过预设清洗分类策略便于用户选择相应类型的日志信息,有利于减少日志的处理量。In the above-mentioned embodiment, the preset cleaning and classification strategy facilitates the user to select the corresponding type of log information, which is beneficial to reduce the processing amount of the log.
结合第一方面,在一些可选的实施方式中,根据预设清洗分类策略,对所述目标日志信息进行清洗分类,得到清洗分类后的日志信息,包括:With reference to the first aspect, in some optional implementations, according to a preset cleaning and classification strategy, the target log information is cleaned and classified, and the cleaned and classified log information is obtained, including:
从所述目标日志信息中保留指定数据类型的数据,删除非所述指定数据类型的数据,得到清洗分类后的日志信息。The data of the specified data type is retained from the target log information, and the data not of the specified data type is deleted to obtain cleaned and classified log information.
在上述的实施方式中,有利于用户根据实际需求灵活地保留相应类型的日志数据,从而有利于减少日志的处理量。In the above-mentioned embodiments, it is beneficial for the user to flexibly retain the corresponding types of log data according to actual needs, thereby helping to reduce the processing volume of the log.
结合第一方面,在一些可选的实施方式中,所述方法还包括:In conjunction with the first aspect, in some optional embodiments, the method further includes:
通过显示模块输出展示所述清洗分类后的日志信息。The log information after cleaning and classification is displayed by outputting the display module.
结合第一方面,在一些可选的实施方式中,所述方法还包括:In conjunction with the first aspect, in some optional embodiments, the method further includes:
通过键值数据库存储所述第一日志信息、所述第二日志信息,在所述键值数据库中,键为所述用户终端的身份标识,值为账户信息。The first log information and the second log information are stored in a key-value database, where the key is the identity of the user terminal and the value is account information.
第二方面,本申请实施例还提供一种日志管理装置,所述装置包括:In a second aspect, an embodiment of the present application further provides a log management device, the device comprising:
第一获取单元,用于通过预先配置的登录日志采集适配规则,获取登录应用系统的登录操作的得到的第一日志信息,所述第一日志信息包括账户信息、与所述账户信息对应的第一用户终端的第一身份标识;The first obtaining unit is configured to obtain the first log information obtained from the login operation of logging in to the application system through the preconfigured login log collection and adaptation rules, where the first log information includes account information, and the first log information corresponding to the account information. the first identity of the first user terminal;
第二获取单元,通过预先配置的访问日志采集适配规则,获取访问所述应用系统的访问操作的得到的第二日志信息,所述第二日志信息包括第二用户终端的第二身份标识;The second obtaining unit obtains second log information obtained by accessing the access operation of the application system through a preconfigured access log collection and adaptation rule, where the second log information includes the second identity of the second user terminal;
日志处理单元,用于当所述第二日志信息中存在与所述第一身份标识相同的第二身份标识时,将所述第一身份标识对应的账户信息与所述第二身份标识对应的日志关联,得到目标日志信息。The log processing unit is configured to, when the second log information has a second identity that is the same as the first identity, compare the account information corresponding to the first identity with the account information corresponding to the second identity. Log association to get the target log information.
第三方面,本申请实施例还提供一种电子设备,所述电子设备包括相互耦合的存储器、处理器,所述存储器内存储计算机程序,当所述计算机程序被所述处理器执行时,使得所述电子设备执行上述的方法。In a third aspect, embodiments of the present application further provide an electronic device, the electronic device includes a memory and a processor coupled to each other, and a computer program is stored in the memory, and when the computer program is executed by the processor, the The electronic device performs the above-described method.
第四方面,本申请实施例还提供一种计算机可读存储介质,所述可读存储介质中存储有计算机程序,当所述计算机程序在计算机上运行时,使得所述计算机执行上述的方法。In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored in the readable storage medium, and when the computer program runs on a computer, the computer is made to execute the above method.
附图说明Description of drawings
为了更清楚地说明本申请实施例的技术方案,下面将对实施例中所需要使用的附图作简单地介绍。应当理解,以下附图仅示出了本申请的某些实施例,因此不应被看作是对范围的限定,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他相关的附图。In order to illustrate the technical solutions of the embodiments of the present application more clearly, the accompanying drawings required in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation of the scope. Other related figures are obtained from these figures.
图1为本申请实施例提供的电子设备与应用系统的通信连接示意图。FIG. 1 is a schematic diagram of a communication connection between an electronic device and an application system according to an embodiment of the present application.
图2为本申请实施例提供的电子设备的结构示意图。FIG. 2 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
图3为本申请实施例提供的日志管理方法的流程示意图。FIG. 3 is a schematic flowchart of a log management method provided by an embodiment of the present application.
图4为本申请实施例提供的日志管理装置的功能框图。FIG. 4 is a functional block diagram of a log management apparatus provided by an embodiment of the present application.
图标:10-电子设备;11-处理模块;12-存储模块;13-通信模块;100-日志管理装置;110-第一获取单元;120-第二获取单元;130-日志处理单元。Icons: 10-electronic equipment; 11-processing module; 12-storage module; 13-communication module; 100-log management device; 110-first acquisition unit; 120-second acquisition unit; 130-log processing unit.
具体实施方式Detailed ways
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行描述。需要说明的是,术语“第一”、“第二”等仅用于区分描述,而不能理解为指示或暗示相对重要性。The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. It should be noted that the terms "first", "second" and the like are only used for distinguishing descriptions, and cannot be understood as indicating or implying relative importance.
下面结合附图,对本申请实施例作详细说明。在不冲突的情况下,下述的实施例及实施例中的特征可以相互组合。The embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and features in the embodiments may be combined with each other without conflict.
请参照图1,本申请实施例提供一种电子设备10。电子设备10可以通过网络与应用系统建立通信连接,以进行数据交互。例如,电子设备10可以从应用系统获取用户登录、访问应用系统的日志数据。Referring to FIG. 1 , an embodiment of the present application provides an
可理解地,本申请实施例提供的电子设备10,可以针对采用浏览器/服务器(Browser/Server,B/S)架构、使用PKI/数字证书模式登录的应用系统上发生的应用登录与访问,获取到包括账户信息的日志信息。能够改善因应用系统采用Session/Cookie机制保存账户信息时,无法从日志获取到账户信息而导致获取的日志存在信息缺失的问题。Understandably, the
电子设备10可以是但不限于智能手机、个人电脑(Personal Computer,PC)、平板电脑、个人数字助理(Personal Digital Assistant,PDA)、服务器等。The
应用系统可以通过网络与用户终端建立通信连接。用户终端可以通过网络访问应用系统中的相应资源。应用系统可理解为配置有应用/业务服务的服务器。应用系统可以根据不同的业务场景、应用场景进行配置。例如,应用系统为用于提供供视频节目类资源的服务器,或者为用于提供购物平台的服务器。The application system can establish a communication connection with the user terminal through the network. The user terminal can access corresponding resources in the application system through the network. An application system can be understood as a server configured with application/business services. The application system can be configured according to different business scenarios and application scenarios. For example, the application system is a server for providing resources for video programs, or a server for providing a shopping platform.
用户终端可以是,但不限于,智能手机、个人电脑(Personal Computer,PC)、平板电脑等。The user terminal may be, but not limited to, a smart phone, a personal computer (Personal Computer, PC), a tablet computer, and the like.
请参照图2,在本实施例中,电子设备10可以包括处理模块11、存储模块12、通信模块13以及日志管理装置100,处理模块11、通信模块13、存储模块12以及日志管理装置100各个元件之间直接或间接地电性连接,以实现数据的传输或交互。例如,这些元件相互之间可通过一条或多条通讯总线或信号线实现电性连接。Referring to FIG. 2 , in this embodiment, the
可理解地,电子设备10的存储模块12内存储有计算机程序,当计算机程序被处理模块11执行时,可以使得电子设备10能够执行下述日志管理方法中的各步骤。Understandably, a computer program is stored in the
请参照图3,本申请实施例还提供一种日志管理方法,可以应用于上述的电子设备10中,由电子设备10执行或实现方法中的各步骤,能够改善在账户密码模式登录的应用系统中,获取的日志存在信息缺失的问题。在本实施例中,方法可以包括以下步骤:Referring to FIG. 3 , an embodiment of the present application also provides a log management method, which can be applied to the above-mentioned
步骤S210,通过预先配置的登录日志采集适配规则,获取登录应用系统的登录操作的得到的第一日志信息,所述第一日志信息包括账户信息、与所述账户信息对应的第一用户终端的第一身份标识;Step S210: Obtain first log information obtained from a login operation of logging in to the application system through a preconfigured login log collection and adaptation rule, where the first log information includes account information and a first user terminal corresponding to the account information 's primary identity;
步骤S220,通过预先配置的访问日志采集适配规则,获取访问所述应用系统的访问操作的得到的第二日志信息,所述第二日志信息包括第二用户终端的第二身份标识;Step S220, obtaining second log information obtained by accessing the access operation of the application system through a preconfigured access log collection and adaptation rule, where the second log information includes the second identity of the second user terminal;
步骤S230,当所述第二日志信息中存在与所述第一身份标识相同的第二身份标识时,将所述第一身份标识对应的账户信息与所述第二身份标识对应的日志关联,得到目标日志信息。Step S230, when the second log information has the same second identity as the first identity, associate the account information corresponding to the first identity with the log corresponding to the second identity, Get the target log information.
在本实施例中,通过预先配置的登录日志采集适配规则,可以获取到登录应用系统中产生的第一日志信息,第一日志信息中通常可以包括账户信息。然后结合访问日志采集适配规则获取的第二日志信息,可以在账户密码模式登录的应用系统中获取到账户信息,改善因采用Session/Cookie机制保存账户信息时,无法从日志获取到账户信息而导致获取的日志存在信息缺失的问题。In this embodiment, the first log information generated in the login application system can be acquired through the preconfigured login log collection and adaptation rules, and the first log information may generally include account information. Then, in combination with the second log information obtained from the access log collection and adaptation rules, account information can be obtained in the application system logged in in the account password mode. There is a problem of missing information in the obtained logs.
下面将对方法中的各步骤进行详细阐述,如下:Each step in the method will be described in detail below:
在步骤S210中,电子设备10预先配置有登录日志采集适配规则。其中,登录日志采集适配规则可以根据实际情况进行设置,可以用于获取登录应用系统的相关信息。例如,登录日志采集适配规则可以为正则表达式适配规则,可以通过正则表达式获取登录应用系统的登录操作的日志。In step S210, the
例如,电子设备10可以通过正则表达式适配规则可以获取表征登录应用系统的统一资源定位系统(Uniform Resource Locator,URL)信息。可理解地,当用户终端采用浏览器/服务器(Browser/Server,B/S)架构,或者使用PKI/数字证书模式登录的应用系统时,用户终端需要向应用系统发送表征登录应用系统的URL信息。该URL信息中通常包括账户信息及用户终端的身份标识,该身份标识即为第一身份标识。比如,通过正则表达式适配规则可以判断URL信息中是否存在表征账户信息的字段,若URL信息中存在表征账户信息的字段,则该URL信息即为表征登录应用系统的URL信息,同时,电子设备10还可以从该URL信息中获取到登录操作中的账户信息。其中,第一身份标识可以根据实际情况进行确定,用于标识用户终端的唯一身份,不同的用户终端的身份标识不相同。例如,第一身份标识可以是但不限于终端设备的MAC地址、IP地址等,这里不作具体限定。For example, the
在本实施例中,步骤S210可以包括:通过所述登录日志采集适配规则,确定表征登录所述应用系统的第一URL信息;从所述第一URL信息中获取包括所述账户信息的日志信息;在所述日志信息中添加第一标识,得到所述第一日志信息,所述第一标识用于表征所述第一日志信息为登录日志。In this embodiment, step S210 may include: by collecting and adapting rules for the login log, determining first URL information representing logging in to the application system; obtaining a log including the account information from the first URL information information; adding a first identifier to the log information to obtain the first log information, where the first identifier is used to represent that the first log information is a login log.
可理解地,登录日志采集适配规则可以区分登录应用系统的URL信息和访问应用系统的URL信息。登录应用系统的URL信息即为第一URL信息,访问应用系统的URL信息即为下述的第二URL信息。Understandably, the login log collection and adaptation rules can distinguish the URL information for logging in to the application system and the URL information for accessing the application system. The URL information for logging in to the application system is the first URL information, and the URL information for accessing the application system is the following second URL information.
例如,电子设备10从第一URL信息中获取到包括账户信息的日志信息后,可以自动在该日志信息中添加表示该日志信息为登录操作的日志的标识。该标识即为第一标识,可以根据实际情况进行设置,例如,可以为数字或字符等,用于与访问操作的日志信息进行区分,以便于后续对登录日志、访问日志进行分析处理。For example, after acquiring log information including account information from the first URL information, the
在步骤S220中,访问日志采集适配规则的实现原理与登录日志采集适配规则的实现原理相类似。访问日志采集适配规则可以根据实际情况进行设置,例如,可以是用于检测访问操作的正则表达式适配规则。为便于区分,登录日志采集适配规则的正则表达式适配规则可以称为第一正则表达式适配规则,访问日志采集适配规则的正则表达式适配规则可以称为第二正则表达式适配规则。第二正则表达式适配规则可以检测用户终端访问应用系统时发送的URL信息或其他信息。其中,该URL信息即为下述的第二URL信息。第一日志信息可以称为登录日志,第二日志信息可以称为访问日志。In step S220, the realization principle of the access log collection adaptation rule is similar to that of the login log collection adaptation rule. The access log collection adaptation rule can be set according to the actual situation, for example, it can be a regular expression adaptation rule used to detect access operations. For ease of distinction, the regular expression adaptation rule of the login log collection adaptation rule may be referred to as the first regular expression adaptation rule, and the regular expression adaptation rule of the access log collection adaptation rule may be referred to as the second regular expression. Adaptation rules. The second regular expression adaptation rule may detect URL information or other information sent by the user terminal when accessing the application system. The URL information is the following second URL information. The first log information may be referred to as a login log, and the second log information may be referred to as an access log.
步骤S220可以包括:通过所述访问日志采集适配规则,确定表征访问所述应用系统的第二URL信息;从所述第二URL信息中获取日志信息;在所述日志信息中添加第二标识,得到所述第二日志信息,所述第二标识用于表征所述第二日志信息为访问日志。Step S220 may include: determining second URL information representing access to the application system through the access log collection and adaptation rules; obtaining log information from the second URL information; adding a second identifier to the log information to obtain the second log information, where the second identifier is used to represent that the second log information is an access log.
在本实施例中,第二正则表达式适配规则可以通过正则表达式,对URL信息中的相应字段进行匹配检测。例如,可以用于检测匹配URL信息中用户终端的MAC地址、IP地址,以及表征访问操作的字段。其中,表征访问操作的字段可以根据实际情况进行设置,这里不作具体限定。In this embodiment, the second regular expression adaptation rule may perform matching detection on a corresponding field in the URL information through a regular expression. For example, it can be used to detect the MAC address, IP address of the user terminal in the matching URL information, and fields that characterize the access operation. The fields representing the access operation can be set according to the actual situation, which is not specifically limited here.
在本实施例中,第二正则表达式适配规则还可以检测URL信息中是否存在表征账户信息的字段。若检测到URL信息中不存在表征账户信息的字段,则确定该URL信息为访问操作对应的URL信息。电子设备10可以通过第二正则表达式适配规则,从该URL信息中解析得到终端设备的第二身份标识等第二日志信息。In this embodiment, the second regular expression adaptation rule may also detect whether there is a field representing account information in the URL information. If it is detected that there is no field representing account information in the URL information, it is determined that the URL information is the URL information corresponding to the access operation. The
步骤S230,电子设备10在获取到第一日志信息及第二日志信息后,便可以对日志信息进行关联处理。可理解地,由于第二日志信息为用户终端在访问应用系统中得到的访问日志,这类日志中通常不存在账户信息。而第一日志信息、第二日志信息中通常可以包括用户终端的身份标识信息。若第一日志信息、第二日志信息中,存在相同的用户终端的身份标识信息,则表示该第二日志信息为第一日志信息中的账户执行相应操作得到的日志信息。此时,便可以将第一日志信息中的账户信息与第二身份标识对应的日志关联,或者将账户信息添加在第二日志信息中,以得到目标日志信息。基于此,便于后续结合账户信息,追踪分析该账户的访问操作。In step S230, after acquiring the first log information and the second log information, the
例如,电子设备10可以基于预设时长内获取到的目标日志,可以通过统计同一账户在预设时长内,对应用系统中相应的业务模块的访问次数、访问停留时长、访问频率等。预设时长可以根据实际情况进行设置,例如,可以为一天、一周等时长。业务模块可以根据实际情况进行确定。例如,可以为提供新闻的业务模块,提供视频资源的业务模块等。For example, the
作为一种可选的实施方式,方法还可以包括:根据预设清洗分类策略,对所述目标日志信息进行清洗分类,得到清洗分类后的日志信息。As an optional implementation manner, the method may further include: cleaning and classifying the target log information according to a preset cleaning and classification strategy to obtain cleaned and classified log information.
在本实施例中,用户可以根据实际的业务需求,设置预设清洗分类策略,以对日志进行筛选以及分类。其中,对日志信息进行分类的方式可以是:根据账户信息对日志信息进行分类;根据日志中表征所访问的模块种类对日志信息进行分类。例如,同一个账户对应的日志归为一个大类,不同的账户对应的日志归为不同的大类。假设在应用系统中,可以包括多个业务模块,电子设备10可以针对访问不同的业务模块,对得到的日志进行分类,访问同一个业务模块的日志可以分为一个大类;访问不同的业务模块可以归为不同的大类。In this embodiment, the user can set a preset cleaning and classification policy according to actual business requirements, so as to filter and classify the logs. The method of classifying the log information may be: classifying the log information according to the account information; classifying the log information according to the type of the accessed module in the log. For example, logs corresponding to the same account are classified into one category, and logs corresponding to different accounts are classified into different categories. Assuming that the application system can include multiple business modules, the
在本实施例中,根据预设清洗分类策略,对所述目标日志信息进行清洗分类,得到清洗分类后的日志信息,包括:从所述目标日志信息中保留指定数据类型的数据,删除非所述指定数据类型的数据,得到清洗分类后的日志信息。In this embodiment, cleaning and classifying the target log information according to a preset cleaning and classification strategy, to obtain cleaned and classified log information, includes: retaining data of a specified data type from the target log information, deleting non-target log information Describe the data of the specified data type, and obtain the log information after cleaning and classification.
可理解地,指定数据类型可以根据实际情况进行设置。例如,指定数据类型可以为表征访问应用系统中的指定业务模块的数据类型。指定业务模块可以根据实际情况进行确定,例如为提供视频资源的业务模块,提供音频资源的业务模块、提供新闻资源的业务模块等。基于此,有利于用户根据实际需求灵活地保留相应类型的日志数据,从而有利于减少日志的处理量。Understandably, the specified data type can be set according to the actual situation. For example, the specified data type may be a data type representing a specified service module in the access application system. The designated service module can be determined according to the actual situation, for example, a service module that provides video resources, a service module that provides audio resources, and a service module that provides news resources. Based on this, it is beneficial for the user to flexibly retain corresponding types of log data according to actual needs, thereby helping to reduce the processing volume of the log.
需要说明的是,在其他实施方式中,电子设备10可以将得到的第一日志信息、第二日志信息、目标日志信息发送至其他服务器,由其他服务器对第一日志信息、第二日志信息、目标日志信息进行清洗处理。It should be noted that, in other embodiments, the
作为一种可选的实施方式,方法还可以包括:通过键值数据库存储所述第一日志信息、所述第二日志信息,在所述键值数据库中,键为所述用户终端的身份标识,值为账户信息。As an optional implementation manner, the method may further include: storing the first log information and the second log information through a key-value database, where the key is the identity identifier of the user terminal in the key-value database , the value is account information.
键值数据库可以理解为Key-Value数据库。在Key-Value数据库中,Key为用户终端的身份标识,Value为账户信息。Key还可以包括应用系统的编号或标识,用于区分不同的应用系统。Key-Value数据库存储日志信息的过程可以如下:A key-value database can be understood as a Key-Value database. In the Key-Value database, Key is the identity identifier of the user terminal, and Value is account information. The Key may also include the serial number or identification of the application system, which is used to distinguish different application systems. The process of storing log information in the Key-Value database can be as follows:
电子设备10提取登录日志(第一日志信息)当中的账号信息,并按照key:value的形式(key为应用系统编号_用户终端IP;value为账号信息,比如,比如应用系统编号_用户终端IP为“100001_1.1.1.1”,账号信息为“admin”,此时,key为“100001_1.1.1.1”,对应的value为“admin”),存储于redis数据库当中。如果key在redis数据库中不存在,则插入该key;如果key在redis数据库当中已存在,则更新value的值,比如,对value的值加一,以记录同一用户终端的访问次数。The
电子设备10提取访问日志(第二日志信息)中的用户终端的应用系统编号_用户终端IP(第二身份标识),形成上述的key。根据该key,关联查询redis数据库,获取到对应key键的value值,也就是账号信息。此时,可以将账号信息填充到第二日志信息中,以作为目标日志信息。The
在本实施例中,得到的目标日志信息可以存储在相应的集群系统中,以便于用户后续根据需求,从集群系统中下载相应的日志信息。集群系统可以根据实际情况进行选择。例如,集群系统可以为Elasticsearch存储集群,可以实现分布式存储。In this embodiment, the obtained target log information may be stored in the corresponding cluster system, so that the user can subsequently download the corresponding log information from the cluster system according to requirements. The cluster system can be selected according to the actual situation. For example, the cluster system can be an Elasticsearch storage cluster, which can implement distributed storage.
作为一种可选的实施方式,方法还可以包括:通过显示模块输出展示所述清洗分类后的日志信息。As an optional implementation manner, the method may further include: outputting and displaying the log information after cleaning and classification through a display module.
在本实施例中,若电子设备10为终端设备,终端设备可以包括显示屏类的显示模块。终端设备可以通过显示屏输出展示清洗分类后的日志信息。展示日志信息的方式可以根据实际情况进行设置。例如,可以对日志信息进行分类,通过统计图(例如折线统计图、柱状统计图、扇形统计图等)的方式、文本文档的方式展示日志信息。可理解地,通过将日志信息可视化显示,便于用户直观地关注各类日志信息,有利于日志数据的分析审计,提升用户体验感。In this embodiment, if the
若电子设备10为服务器。该服务器可以将清洗分类后的日志信息发生至具有显示模块的终端设备,由终端设备输出展示该日志信息。If the
请参照图4,本申请实施例还提供一种日志管理装置100,可以应用于上述的电子设备10中,用于执行或实现方法中的各步骤。日志管理装置100包括至少一个可以软件或固件(Firmware)的形式存储于存储模块12中或固化在电子设备10操作系统(OperatingSystem,OS)中的软件功能模块。处理模块11用于执行存储模块12中存储的可执行模块,例如日志管理装置100所包括的软件功能模块及计算机程序等。Referring to FIG. 4 , an embodiment of the present application further provides a
日志管理装置100可以包括第一获取单元110、第二获取单元120及日志处理单元130。The
第一获取单元110,用于通过预先配置的登录日志采集适配规则,获取登录应用系统的登录操作的得到的第一日志信息,所述第一日志信息包括账户信息、与所述账户信息对应的第一用户终端的第一身份标识。The first obtaining
第二获取单元120,通过预先配置的访问日志采集适配规则,获取访问所述应用系统的访问操作的得到的第二日志信息,所述第二日志信息包括第二用户终端的第二身份标识。The second obtaining
日志处理单元130,用于当所述第二日志信息中存在与所述第一身份标识相同的第二身份标识时,将所述第一身份标识对应的账户信息与所述第二身份标识对应的日志关联,得到目标日志信息。The
可选地,第一获取单元110还可以用于:Optionally, the first obtaining
通过所述登录日志采集适配规则,确定表征登录所述应用系统的第一URL信息;Determine the first URL information representing the login to the application system by collecting the adaptation rules through the login log;
从所述第一URL信息中获取包括所述账户信息的日志信息;Obtain log information including the account information from the first URL information;
在所述日志信息中添加第一标识,得到所述第一日志信息,所述第一标识用于表征所述第一日志信息为登录日志。A first identifier is added to the log information to obtain the first log information, where the first identifier is used to represent that the first log information is a login log.
可选地,第二日志单元还可以用于:Optionally, the second log unit can also be used to:
通过所述访问日志采集适配规则,确定表征访问所述应用系统的第二URL信息;Through the access log collection and adaptation rules, determine the second URL information representing the access to the application system;
从所述第二URL信息中获取日志信息;Obtain log information from the second URL information;
在所述日志信息中添加第二标识,得到所述第二日志信息,所述第二标识用于表征所述第二日志信息为访问日志。A second identifier is added to the log information to obtain the second log information, where the second identifier is used to represent that the second log information is an access log.
可选地,日志管理装置100还可以包括清洗单元,用于根据预设清洗分类策略,对所述目标日志信息进行清洗分类,得到清洗分类后的日志信息。Optionally, the
清洗单元还可以用于:从所述目标日志信息中保留指定数据类型的数据,删除非所述指定数据类型的数据,得到清洗分类后的日志信息。The cleaning unit may also be configured to: retain data of a specified data type from the target log information, delete data not of the specified data type, and obtain cleaned and classified log information.
可选地,日志管理装置100还可以包括展示单元,用于通过显示模块输出展示所述清洗分类后的日志信息。Optionally, the
可选地,日志管理装置100还可以包括存储单元,用于通过键值数据库存储所述第一日志信息、所述第二日志信息,在所述键值数据库中,键为所述用户终端的身份标识,值为账户信息。Optionally, the
在本实施例中,处理模块11可以是一种集成电路芯片,具有信号的处理能力。上述处理模块11可以是通用处理器。例如,该处理器可以是中央处理器(Central ProcessingUnit,CPU)、图形处理器(Graphics Processing Unit,GPU)、网络处理器(NetworkProcessor,NP)等;还可以是数字信号处理器(Digital Signal Processing,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现场可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件,可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。In this embodiment, the
通信模块13用于通过网络建立电子设备10与应用系统的通信连接,并通过网络收发数据。The
存储模块12可以是,但不限于,随机存取存储器,只读存储器,可编程只读存储器,可擦除可编程只读存储器,电可擦除可编程只读存储器等。在本实施例中,存储模块12可以用于存储日志信息。当然,存储模块12还可以用于存储程序,处理模块11在接收到执行指令后,执行该程序。The
需要说明的是,所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的电子设备10、日志管理方法的具体工作过程,可以参考前述方法中的各步骤对应过程,在此不再过多赘述。It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the
本申请实施例还提供一种计算机可读存储介质。可读存储介质中存储有计算机程序,当计算机程序在计算机上运行时,使得计算机执行如上述实施例中所述的日志管理方法。Embodiments of the present application further provide a computer-readable storage medium. A computer program is stored in the readable storage medium, and when the computer program runs on the computer, the computer executes the log management method described in the above embodiments.
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到本申请可以通过硬件实现,也可以借助软件加必要的通用硬件平台的方式来实现,基于这样的理解,本申请的技术方案可以以软件产品的形式体现出来,该软件产品可以存储在一个非易失性存储介质(可以是CD-ROM,U盘,移动硬盘等)中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本申请各个实施场景所述的方法。From the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by hardware or by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions of the present application It can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.), and includes several instructions to make a computer device (which can be It is a personal computer, a server, or a network device, etc.) to execute the methods described in various implementation scenarios of this application.
综上所述,本申请提供一种日志管理方法、装置、电子设备及可读存储介质。方法可以包括:通过预先配置的登录日志采集适配规则,获取登录应用系统的登录操作的得到的第一日志信息,第一日志信息包括账户信息、与账户信息对应的第一用户终端的第一身份标识;通过预先配置的访问日志采集适配规则,获取访问应用系统的访问操作的得到的第二日志信息,第二日志信息包括第二用户终端的第二身份标识;当第二日志信息中存在与第一身份标识相同的第二身份标识时,将第一身份标识对应的账户信息与第二身份标识对应的日志关联,得到目标日志信息。在本方案中,通过预先配置的登录日志采集适配规则,可以获取到登录应用系统中产生的第一日志信息,第一日志信息中通常可以包括账户信息。然后结合访问日志采集适配规则获取的第二日志信息,可以在账户密码模式登录的应用系统中获取到账户信息,改善因采用Session/Cookie机制保存账户信息时,无法从日志获取到账户信息而导致获取的日志存在信息缺失的问题。In summary, the present application provides a log management method, apparatus, electronic device, and readable storage medium. The method may include: obtaining first log information obtained from a login operation of logging in to an application system through a preconfigured login log collection and adaptation rule, where the first log information includes account information and first log information of the first user terminal corresponding to the account information. identity identifier; through the preconfigured access log collection and adaptation rules, obtain the second log information obtained by accessing the access operation of the application system, and the second log information includes the second identity identifier of the second user terminal; when the second log information is in the second log information When there is a second identity that is the same as the first, the account information corresponding to the first identity is associated with the log corresponding to the second identity to obtain target log information. In this solution, the first log information generated in the login application system can be acquired through the preconfigured login log collection and adaptation rules, and the first log information can usually include account information. Then, in combination with the second log information obtained from the access log collection and adaptation rules, account information can be obtained in the application system logged in in the account password mode. There is a problem of missing information in the obtained logs.
在本申请所提供的实施例中,应该理解到,所揭露的装置、系统和方法,也可以通过其它的方式实现。以上所描述的装置、系统和方法实施例仅仅是示意性的,例如,附图中的流程图和框图显示了根据本申请的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上,流程图或框图中的每个方框可以代表一个模块、程序段或代码的一部分,所述模块、程序段或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也要注意的是,框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合,可以用执行规定的功能或动作的专用的基于硬件的系统来实现,或者可以用专用硬件与计算机指令的组合来实现。另外,在本申请各个实施例中的各功能模块可以集成在一起形成一个独立的部分,也可以是各个模块单独存在,也可以两个或两个以上模块集成形成一个独立的部分。In the embodiments provided in this application, it should be understood that the disclosed apparatus, system and method may also be implemented in other manners. The apparatus, system and method embodiments described above are merely schematic, for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture of possible implementations of systems, methods and computer program products according to various embodiments of the present application Architecture, functionality and operation. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more functions for implementing the specified logical function(s) executable instructions. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions. In addition, each functional module in each embodiment of the present application may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.
以上所述仅为本申请的优选实施例而已,并不用于限制本申请,对于本领域的技术人员来说,本申请可以有各种更改和变化。凡在本申请的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本申请的保护范围之内。The above descriptions are only preferred embodiments of the present application, and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the protection scope of this application.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010780439.6A CN112069425A (en) | 2020-08-05 | 2020-08-05 | Log management method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010780439.6A CN112069425A (en) | 2020-08-05 | 2020-08-05 | Log management method and device, electronic equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112069425A true CN112069425A (en) | 2020-12-11 |
Family
ID=73657156
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010780439.6A Pending CN112069425A (en) | 2020-08-05 | 2020-08-05 | Log management method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112069425A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112711570A (en) * | 2021-01-19 | 2021-04-27 | 北京数字认证股份有限公司 | Log information processing method and device, electronic equipment and storage medium |
| CN113138971A (en) * | 2021-04-30 | 2021-07-20 | 深圳市度申科技有限公司 | Visual log analysis method, device and system |
| CN113194088A (en) * | 2021-04-28 | 2021-07-30 | 广东电网有限责任公司广州供电局 | Access interception method, device, log server and computer readable storage medium |
| CN113869360A (en) * | 2021-08-18 | 2021-12-31 | 国网思极网安科技(北京)有限公司 | Method and device for equipment safety management, electronic equipment and storage medium |
| CN113887889A (en) * | 2021-09-15 | 2022-01-04 | 北京市农林科学院信息技术研究中心 | A method and device for generating assessment data |
| CN116136901A (en) * | 2023-04-19 | 2023-05-19 | 杭州美创科技股份有限公司 | Application program anti-counterfeiting method and device, computer equipment and storage medium |
| CN116800628A (en) * | 2022-08-26 | 2023-09-22 | 中移(苏州)软件技术有限公司 | Log processing method, device, electronic equipment and computer-readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104484389A (en) * | 2014-12-11 | 2015-04-01 | 焦点科技股份有限公司 | Method and system for discovering maximum likelihood geographic position of internet users |
| CN111310139A (en) * | 2020-01-21 | 2020-06-19 | 腾讯科技(深圳)有限公司 | Behavior data identification method and device and storage medium |
-
2020
- 2020-08-05 CN CN202010780439.6A patent/CN112069425A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104484389A (en) * | 2014-12-11 | 2015-04-01 | 焦点科技股份有限公司 | Method and system for discovering maximum likelihood geographic position of internet users |
| CN111310139A (en) * | 2020-01-21 | 2020-06-19 | 腾讯科技(深圳)有限公司 | Behavior data identification method and device and storage medium |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112711570A (en) * | 2021-01-19 | 2021-04-27 | 北京数字认证股份有限公司 | Log information processing method and device, electronic equipment and storage medium |
| CN113194088A (en) * | 2021-04-28 | 2021-07-30 | 广东电网有限责任公司广州供电局 | Access interception method, device, log server and computer readable storage medium |
| CN113138971A (en) * | 2021-04-30 | 2021-07-20 | 深圳市度申科技有限公司 | Visual log analysis method, device and system |
| CN113869360A (en) * | 2021-08-18 | 2021-12-31 | 国网思极网安科技(北京)有限公司 | Method and device for equipment safety management, electronic equipment and storage medium |
| CN113887889A (en) * | 2021-09-15 | 2022-01-04 | 北京市农林科学院信息技术研究中心 | A method and device for generating assessment data |
| CN116800628A (en) * | 2022-08-26 | 2023-09-22 | 中移(苏州)软件技术有限公司 | Log processing method, device, electronic equipment and computer-readable storage medium |
| CN116136901A (en) * | 2023-04-19 | 2023-05-19 | 杭州美创科技股份有限公司 | Application program anti-counterfeiting method and device, computer equipment and storage medium |
| CN116136901B (en) * | 2023-04-19 | 2023-07-14 | 杭州美创科技股份有限公司 | Application program anti-counterfeiting method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112069425A (en) | Log management method and device, electronic equipment and readable storage medium | |
| CN105516821B (en) | The method and device of barrage screening | |
| CN110245069A (en) | The methods of exhibiting and device of the test method and device of page versions, the page | |
| CN107704497A (en) | Web data crawling method, device, web data crawl platform and storage medium | |
| CN110225104A (en) | Data capture method, device and terminal device | |
| CN104219230B (en) | Identify method and the device of malicious websites | |
| CN107786992B (en) | Method and device for detecting quality of mobile communication network | |
| CN111143650B (en) | Method, device, medium and electronic device for obtaining page data | |
| CN114861054B (en) | Information collection method, device, electronic device and storage medium | |
| CN112135199B (en) | Video playing method based on multiple types of video sources and related equipment | |
| CN109933508B (en) | Method and apparatus for transmitting information | |
| CN112100534A (en) | Information processing method, device, medium and electronic equipment in page sharing | |
| CN116204438A (en) | Test case generation method, automatic test method and related device | |
| CN115423030A (en) | Method and device for device identification | |
| CN111767481A (en) | Access processing method, apparatus, device and storage medium | |
| CN108804501B (en) | A method and device for detecting valid information | |
| CN113010587B (en) | Data source configuration method, device, terminal, server and medium | |
| CN111046393B (en) | Vulnerability information uploading method and device, terminal equipment and storage medium | |
| CN117093627A (en) | Information mining methods, devices, electronic equipment and storage media | |
| CN113114611A (en) | Method and device for managing blacklist | |
| CN115730170A (en) | Message data processing method and device and server | |
| CN114579398A (en) | Log storage method, device, equipment and storage medium | |
| CN110020166B (en) | Data analysis method and related equipment | |
| CN110674386A (en) | Resource recommendation method, device and storage medium | |
| CN112783754B (en) | A method and device for testing a page |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201211 |
