CN108512943A - Embedded device data store system based on message queue and store method - Google Patents
Embedded device data store system based on message queue and store method Download PDFInfo
- Publication number
- CN108512943A CN108512943A CN201810425085.6A CN201810425085A CN108512943A CN 108512943 A CN108512943 A CN 108512943A CN 201810425085 A CN201810425085 A CN 201810425085A CN 108512943 A CN108512943 A CN 108512943A
- Authority
- CN
- China
- Prior art keywords
- data
- message
- cloud platform
- transmission unit
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 230000005540 biological transmission Effects 0.000 claims abstract description 132
- 230000002085 persistent effect Effects 0.000 claims abstract description 15
- 238000013480 data collection Methods 0.000 claims abstract description 13
- 238000013500 data storage Methods 0.000 claims abstract description 9
- 238000012545 processing Methods 0.000 claims description 38
- 238000004891 communication Methods 0.000 claims description 18
- 238000005538 encapsulation Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 6
- 230000002688 persistence Effects 0.000 claims description 6
- 108010001267 Protein Subunits Proteins 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 5
- 230000035800 maturation Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/32—Flow control; Congestion control by discarding or delaying data units, e.g. packets or frames
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
- H04L47/62—Queue scheduling characterised by scheduling criteria
- H04L47/625—Queue scheduling characterised by scheduling criteria for service slots or service orders
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/55—Push-based network services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/565—Conversion or adaptation of application format or content
- H04L67/5651—Reducing the amount or size of exchanged application data
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Present invention is disclosed a kind of embedded device data store system and store method based on message queue, system includes data acquisition unit, data transmission unit, data center unit and cloud platform, and method includes data collection steps, data sending step, data center's step and cloud storage step.The present invention under the premise of ensure that the data of embedded device routine acquire and upload decoupling function is unaffected, realizes the persistent storage to gathered data by adding the data center unit with information storage function.Even if there is a situation where embedded devices to cause network connection to disconnect because of uncontrollable reason with cloud platform, having completed the data of acquisition will not lose easily, considerably enhance the safety of data storage.
Description
Technical field
The present invention relates to a kind of data store system and store methods, and in particular to a kind of based on the embedded of message queue
Device data preserves system and store method, belongs to industry internet field.
Background technology
In Internet of Things or industry internet platform, embedded device after gathered data can by MQTT, HTTP or
Other transport protocols based on TCP upload the data to cloud platform, and cloud platform stores data in database again.
In such system architecture system, the database of cloud platform and and the transport protocol based on TCP in contrast
All it is relatively stable, reliable, therefore after the completion of data upload process, there is no lose for the data in cloud platform or database
The risk of mistake.
But for embedded device, it collects data and forward the data to cloud platform this during, if
Because certain reason causes network connection to be failed, the connection between embedded device and cloud platform is interrupted suddenly, cause data without
When method is completed to upload, just easily causes the collected information of embedded device and lose.Due to embedded devices many at present
Lack the function of data buffer storage processing, therefore the case where information loss occurs often.For manufacturing enterprise, certain key numbers
According to（Such as equipment alarm data）Loss be unacceptable.
In conclusion how to design a kind of embedded device data store system and data corresponding thereto preserve
Method ensures that it does not lose institute after network linking disconnection to realize the upgrading of embedded device data buffer storage processing function
The data information of acquisition just becomes those skilled in the art's urgent problem to be solved.
Invention content
In view of the prior art there are drawbacks described above, the purpose of the present invention is to propose to a kind of based on message queue embedded sets
Standby data store system and store method.
The purpose of the present invention will be achieved by the following technical programs：
A kind of embedded device data store system based on message queue, including：
Data acquisition unit generates message data, and upload to message data for acquiring, parsing data；
Data transmission unit connection and checks connection status for carrying out communication with cloud platform, and is connect from data center unit
It receives message data and is forwarded to cloud platform；
Data center unit, for receiving the event request for coming from data acquisition unit and data transmission unit, and according to thing
Part request carries out event handling and message push；
Cloud platform carries out storage record for receiving message data, and to message data；
The data transmission unit includes,
Connection checks subelement, for carrying out communication connection with cloud platform and checking connection status,
Data transmission sub-unit, for receiving message data from data center unit and being forwarded to cloud platform；
The data center unit includes,
Event handling subelement, for managing message data and receiving the subscription information of message data,
Message pushes subelement, is used for data transmission unit PUSH message data, and stores the message data that do not consume.
Preferably, the data acquisition unit includes：
Collecting device initialization module, for carrying out initialization process to collecting device；
Data center's link block, for establishing the connection between data acquisition unit and data center unit；
Data acquisition module, for the collecting device data from collecting device；
Data resolution module, for collected device data to be resolved to the required format of cloud platform；
Data package module, for device data to be encapsulated into message data；
Data transmission blocks, for message data to be sent to data center unit；
Circle collection module, after being established for the connection between data acquisition unit and data center unit, sequentially circulating
Row data acquisition module, data resolution module, data package module and data transmission blocks.
Preferably, the connection checks that subelement includes：
Cloud platform link block, for establishing communication connection with cloud platform；
Connection status initial survey module, for checking the connection status between data transmission unit and cloud platform,
When data transmission unit and cloud platform are in successful connection operating mode, mark is connected as normal condition,
When data transmission unit and cloud platform are in connection failure operating mode, mark is connected as off-state；
Connection retries module, for when data transmission unit and cloud platform are in connection failure operating mode, sequentially circulating to rack
Platform link block and connection status initial survey module are stopped until after data transmission unit is in successful connection operating mode with cloud platform
Only recycle.
Preferably, the data transmission sub-unit includes：
Data subscription module, for subscribing to message data to data center unit；
Data reception module, for receiving the message data pushed from data center unit；
Connection status examines module again, for checking again for the connection status between data transmission unit and cloud platform and making corresponding anti-
Feedback,
When data transmission unit and cloud platform are in successful connection operating mode, message authentication event is sent to data center unit,
When data transmission unit and cloud platform are in connection failure operating mode, message failure event is sent to data center unit；
Receiving module is repeated, for the data reception module that reruns.
Preferably, the data transmission sub-unit further includes：
Data extract sending module, under being in successful connection operating mode in data transmission unit and cloud platform, from message data
Middle extraction equipment data, and device data is sent to cloud platform.
Preferably, the event handling subelement includes：
Data initialization module, for initial message queue and related data structures and module；
Digital independent is inserted into module, for reading the message data not sent from local persistent storage, and by message data
It is inserted into message queue；
Request receiving module, for receiving the request for coming from data acquisition unit and data transmission unit；
Request processing module, for handling received request event,
When what is received is the subscription request of data transmission unit, which is added consumer queue, subsequently
The data transmission unit being pushed to after message data in consumer queue is received, sort run request is then pressed again and receives mould
Block and request processing module,
When what is received is message data request, message data is added to message queue, task pair is handled by message queue
Message data is handled, and then presses sort run request receiving module and request processing module again,
When what is received is the message authentication event that data transmission unit is beamed back, message data is held from message queue and locally
It is deleted in longization storage, then presses sort run request receiving module and request processing module again,
When what is received is the message failure event that data transmission unit is beamed back, message data is stored to local persistence and is deposited
Chu Zhong then presses sort run request receiving module and request processing module again,
When what is received is non-above-mentioned event type, message data and without any processing is abandoned.
Preferably, the message push subelement includes：
Data read module, for reading message data from message queue；
It subscribes to and checks module, for checking whether there is the message subscribing request from data transmission unit,
When there is message subscribing request, message data is pushed into data transmission unit, then sequentially operation data is read again
Module and subscription check module,
When there is no message subscribing request, by message data storage to local persistent storage, deletes and disappear from message queue
Data are ceased, then sequentially operation data read module and subscription check module again.
A kind of embedded device data save method based on message queue, includes the following steps：
S1, data collection steps, acquisition, parsing data, generate message data, and upload to message data；
S2, data sending step carry out communication connection with cloud platform and check connection status, and received from data center unit
Message data is simultaneously forwarded to cloud platform；
S3, data center's step receive the event request for coming from data acquisition unit and data transmission unit, and according to event
Request carries out event handling and message push；
S4, cloud storage step receive message data, and carry out storage record to message data；
The data sending step includes,
S21, connection check sub-step, carry out communication connection with cloud platform and check connection status,
S22, data send sub-step, receive message data from data center unit and are forwarded to cloud platform；
Data center's step includes,
S31, event handling sub-step manage message data and receive the subscription information of message data,
S32, message push sub-step, to data transmission unit PUSH message data, and store the message data that do not consume.
Preferably, the data collection steps include：
S11, collecting device initialization step carry out initialization process to collecting device；
S12, data center's Connection Step, establish the connection between data acquisition unit and data center unit；
S13, data collection steps, the collecting device data from collecting device；
Collected device data is resolved to the required format of cloud platform by S14, data analyzing step；
S15, data encapsulation step, message data is encapsulated by device data；
Message data is sent to data center unit by S16, data sending step；
S17, circle collection step, after connection between data acquisition unit and data center unit is established, sequentially recycle into
Row data collection steps, data analyzing step, data encapsulation step and data sending step.
Preferably, the connection checks that sub-step includes：
S211, cloud platform Connection Step establish communication connection with cloud platform；
S212, connection status initial survey step check the connection status between data transmission unit and cloud platform,
When data transmission unit and cloud platform are in successful connection operating mode, mark is connected as normal condition,
When data transmission unit and cloud platform are in connection failure operating mode, mark is connected as off-state；
S213, connection retry step, when data transmission unit and cloud platform are in connection failure operating mode, sequentially recycle into racking
Platform Connection Step and connection status initial survey step are stopped until after data transmission unit is in successful connection operating mode with cloud platform
Only recycle.
Preferably, the data transmission sub-step includes：
S221, data subscription step subscribe to message data to data center's step；
S222, data reception step receive the message data pushed from data center unit；
S223, connection status examine step again, check again for the connection status between data transmission unit and cloud platform and make corresponding
Feedback,
When data transmission unit and cloud platform are in successful connection operating mode, message authentication event is sent to data center unit,
When data transmission unit and cloud platform are in connection failure operating mode, message failure event is sent to data center unit；
S225, receiving step is repeated, repeats data reception step.
Preferably, the data transmission sub-step further includes：
S224, data extract forwarding step, in the case where data transmission unit and cloud platform are in successful connection operating mode, from message data
Middle extraction equipment data, and device data is sent to cloud platform.
Preferably, the event handling sub-step includes：
S311, data initialization step, initial message queue and related data structures and module；
S312, digital independent inserting step, read the message data not sent from local persistent storage, and by message data
It is inserted into message queue；
S313, request receiving step, receive the request for coming from data acquisition unit and data transmission unit；
S314, request processing step handle received request event,
When what is received is the subscription request of data transmission unit, which is added consumer queue, subsequently
The data transmission unit being pushed to after message data in consumer queue is received, then sequentially makes requests on reception step again
Rapid and request processing step,
When what is received is message data request, message data is added to message queue, task pair is handled by message queue
Message data is handled, and then sequentially makes requests on receiving step and request processing step again,
When what is received is the message authentication event that data transmission unit is beamed back, message data is held from message queue and locally
It is deleted in longization storage, then sequentially makes requests on receiving step and request processing step again,
When what is received is the message failure event that data transmission unit is beamed back, message data is stored to local persistence and is deposited
Chu Zhong then sequentially makes requests on receiving step and request processing step again,
When what is received is non-above-mentioned event type, message data and without any processing is abandoned.
Preferably, the message push sub-step includes：
S321, data reading step, read message data from message queue；
S322, checking step is subscribed to, checks whether there is the message subscribing request from data transmission unit,
When there is message subscribing request, message data is pushed into data sending step, then sequentially operation data is read again
Step and subscription checking step,
When there is no message subscribing request, by message data storage to local persistent storage, deletes and disappear from message queue
Cease data, then again sequentially operation data read step and subscribe to checking step.
Compared in the prior art, protrusion effect of the invention is as follows：
The present invention is by adding the data center unit with information storage function, in the number that ensure that embedded device routine
According to acquisition and upload decoupling function it is unaffected under the premise of, realize the persistent storage to gathered data.Even if occurring embedding
Enter formula equipment and, because of the case where uncontrollable reason causes network connection to disconnect, completes the data of acquisition with cloud platform
It will not lose easily, considerably enhance the safety of data storage.
Meanwhile the present invention by the way of message queue to the data of embedded device acquisition and resolving carried out it is excellent
Change is handled, and data acquisition is separated with data sending function, that is, improves the stability during data decoupler, and facilitate
The storage of data information records.
In addition, the message queue used in the present invention can select the third-party application of the maturation such as Redis, technology opposite
Maturation, acquisition modes are various, and system compatibility is high, provide the foundation for its subsequent expansion use.
In conclusion using effect of the present invention is excellent, there is very high use and promotional value.
Just attached drawing in conjunction with the embodiments below, the embodiment of the present invention is described in further detail, so that of the invention
Technical solution is more readily understood, grasps.
Description of the drawings
Fig. 1 is the topological relation figure of the present invention；
Fig. 2 is the operational flow diagram of data acquisition unit in the present invention；
Fig. 3 is the operational flow diagram that connection checks subelement in the present invention；
Fig. 4 is the operational flow diagram of data transmission sub-unit in the present invention；
Fig. 5 is the operational flow diagram of event handling subelement in the present invention；
Fig. 6 is the operational flow diagram of message push subelement in the present invention.
Specific implementation mode
As shown, present invention is disclosed a kind of embedded device data store system based on message queue, including：
Data acquisition unit, i.e., the traditional message producer generate message data, and right for acquiring, parsing data
Message data is uploaded.
Data transmission unit, i.e., traditional message consumer, for carrying out communication connection with cloud platform and checking
Connection status, and receive message data from data center unit and be forwarded to cloud platform.
Data center unit, i.e., traditional message center come from data acquisition unit and data for receiving
The event request of transmission unit, and carry out event handling and message push according to event request.
Cloud platform carries out storage record for receiving message data, and to message data.
The data transmission unit includes,
Connection checks subelement, for carrying out communication connection with cloud platform and checking connection status,
Data transmission sub-unit, for receiving message data from data center unit and being forwarded to cloud platform.
The data center unit includes,
Event handling subelement, for managing message data and receiving the subscription information of message data,
Message pushes subelement, is used for data transmission unit PUSH message data, and stores the message data that do not consume.
The data acquisition unit includes：
Collecting device initialization module, for carrying out initialization process to collecting device.
Data center's link block, for establishing the connection between data acquisition unit and data center unit.
Data acquisition module, for the collecting device data from collecting device.
Data resolution module, for collected device data to be resolved to the required format of cloud platform.
Data package module, for device data to be encapsulated into message data.
Data transmission blocks, for message data to be sent to data center unit.
Circle collection module is sequentially followed after being established for the connection between data acquisition unit and data center unit
Inscription of loop data acquisition module, data resolution module, data package module and data transmission blocks.
The connection checks that subelement includes：
Cloud platform link block, for establishing communication connection with cloud platform.
Connection status initial survey module, for checking the connection status between data transmission unit and cloud platform,
When data transmission unit and cloud platform are in successful connection operating mode, mark is connected as normal condition,
When data transmission unit and cloud platform are in connection failure operating mode, mark is connected as off-state.
Connection retries module, is used for when data transmission unit and cloud platform are in connection failure operating mode, sequentially circulating
Platform link block of racking and connection status initial survey module, until data transmission unit is in successful connection operating mode with cloud platform
Afterwards, stop cycle.
The data transmission sub-unit includes：
Data subscription module, for subscribing to message data to data center unit.
Data reception module, for receiving the message data pushed from data center unit.
Connection status examines module again, for checking again for the connection status between data transmission unit and cloud platform and making phase
It should feed back,
When data transmission unit and cloud platform are in successful connection operating mode, message authentication event is sent to data center unit,
When data transmission unit and cloud platform are in connection failure operating mode, message failure event is sent to data center unit.
Receiving module is repeated, for the data reception module that reruns.
The data transmission sub-unit further includes：
Data extract sending module, under being in successful connection operating mode in data transmission unit and cloud platform, from message data
Middle extraction equipment data, and device data is sent to cloud platform.
The event handling subelement includes：
Data initialization module, for initial message queue and related data structures and module.
Digital independent is inserted into module, for reading the message data not sent from local persistent storage, and by message
Data are inserted into message queue.
Request receiving module, for receiving the request for coming from data acquisition unit and data transmission unit.
Request processing module, for handling received request event,
When what is received is the subscription request of data transmission unit, which is added consumer queue, subsequently
The data transmission unit being pushed to after message data in consumer queue is received, sort run request is then pressed again and receives mould
Block and request processing module,
When what is received is message data request, message data is added to message queue, task pair is handled by message queue
Message data is handled, and then presses sort run request receiving module and request processing module again,
When what is received is the message authentication event that data transmission unit is beamed back, message data is held from message queue and locally
It is deleted in longization storage, then presses sort run request receiving module and request processing module again,
When what is received is the message failure event that data transmission unit is beamed back, message data is stored to local persistence and is deposited
Chu Zhong then presses sort run request receiving module and request processing module again,
When what is received is non-above-mentioned event type, message data and without any processing is abandoned.
The message pushes subelement：
Data read module, for reading message data from message queue.
It subscribes to and checks module, for checking whether there is the message subscribing request from data transmission unit,
When there is message subscribing request, message data is pushed into data transmission unit, then sequentially operation data is read again
Module and subscription check module,
When there is no message subscribing request, by message data storage to local persistent storage, deletes and disappear from message queue
Data are ceased, then sequentially operation data read module and subscription check module again.
It should be noted that in the technical scheme, the third-party application of the maturation such as Redis may be used in message queue,
Cloud platform can be the privately owned deployment Internet of things system of open the Internet port.
Present invention further teaches a kind of embedded device data save method based on message queue, includes the following steps：
S1, data collection steps, acquisition, parsing data, generate message data, and upload to message data.
S2, data sending step carry out communication connection with cloud platform and check connection status, and from data center unit
It receives message data and is forwarded to cloud platform.
S3, data center's step receive the event request for coming from data acquisition unit and data transmission unit, and foundation
Event request carries out event handling and message push.
S4, cloud storage step receive message data, and carry out storage record to message data.
The data sending step includes,
S21, connection check sub-step, carry out communication connection with cloud platform and check connection status,
S22, data send sub-step, receive message data from data center unit and are forwarded to cloud platform.
Data center's step includes,
S31, event handling sub-step manage message data and receive the subscription information of message data,
S32, message push sub-step, to data transmission unit PUSH message data, and store the message data that do not consume.
The data collection steps include：
S11, collecting device initialization step carry out initialization process to collecting device.
S12, data center's Connection Step, establish the connection between data acquisition unit and data center unit.
S13, data collection steps, the collecting device data from collecting device.
Collected device data is resolved to the required format of cloud platform by S14, data analyzing step.
S15, data encapsulation step, message data is encapsulated by device data.
Message data is sent to data center unit by S16, data sending step.
S17, circle collection step are sequentially followed after the connection between data acquisition unit and data center unit is established
Ring carries out data collection steps, data analyzing step, data encapsulation step and data sending step.
The connection checks that sub-step includes：
S211, cloud platform Connection Step establish communication connection with cloud platform.
S212, connection status initial survey step check the connection status between data transmission unit and cloud platform,
When data transmission unit and cloud platform are in successful connection operating mode, mark is connected as normal condition,
When data transmission unit and cloud platform are in connection failure operating mode, mark is connected as off-state.
S213, connection retry step, when data transmission unit and cloud platform are in connection failure operating mode, sequentially recycle into
It racks platform Connection Step and connection status initial survey step, until data transmission unit is in successful connection operating mode with cloud platform
Afterwards, stop cycle.
The data send sub-step：
S221, data subscription step subscribe to message data to data center's step.
S222, data reception step receive the message data pushed from data center unit.
S223, connection status examine step again, check again for the connection status between data transmission unit and cloud platform and make
Respective feedback,
When data transmission unit and cloud platform are in successful connection operating mode, message authentication event is sent to data center unit,
When data transmission unit and cloud platform are in connection failure operating mode, message failure event is sent to data center unit.
S225, receiving step is repeated, repeats data reception step.
The data send sub-step：
S224, data extract forwarding step, in the case where data transmission unit and cloud platform are in successful connection operating mode, from message data
Middle extraction equipment data, and device data is sent to cloud platform.
The event handling sub-step includes：
S311, data initialization step, initial message queue and related data structures and module.
S312, digital independent inserting step, read the message data not sent from local persistent storage, and by message
Data are inserted into message queue.
S313, request receiving step, receive the request for coming from data acquisition unit and data transmission unit.
S314, request processing step handle received request event,
When what is received is the subscription request of data transmission unit, which is added consumer queue, subsequently
The data transmission unit being pushed to after message data in consumer queue is received, then sequentially makes requests on reception step again
Rapid and request processing step,
When what is received is message data request, message data is added to message queue, task pair is handled by message queue
Message data is handled, and then sequentially makes requests on receiving step and request processing step again,
When what is received is the message authentication event that data transmission unit is beamed back, message data is held from message queue and locally
It is deleted in longization storage, then sequentially makes requests on receiving step and request processing step again,
When what is received is the message failure event that data transmission unit is beamed back, message data is stored to local persistence and is deposited
Chu Zhong then sequentially makes requests on receiving step and request processing step again,
When what is received is non-above-mentioned event type, message data and without any processing is abandoned.
The message pushes sub-step：
S321, data reading step, read message data from message queue.
S322, checking step is subscribed to, checks whether there is the message subscribing request from data transmission unit,
When there is message subscribing request, message data is pushed into data sending step, then sequentially operation data is read again
Step and subscription checking step,
When there is no message subscribing request, by message data storage to local persistent storage, deletes and disappear from message queue
Cease data, then again sequentially operation data read step and subscribe to checking step.
The present invention ensure that embedded device routine by adding the data center unit with information storage function
Data acquisition and upload decoupling function it is unaffected under the premise of, realize the persistent storage to gathered data.Even if hair
Raw embedded device, because of the case where uncontrollable reason causes network connection to disconnect, completes the number of acquisition with cloud platform
According to that will not lose easily, the safety of data storage is considerably enhanced.
Meanwhile the present invention by the way of message queue to the data of embedded device acquisition and resolving carried out it is excellent
Change is handled, and data acquisition is separated with data sending function, that is, improves the stability during data decoupler, and facilitate
The storage of data information records.
In addition, the message queue used in the present invention can select the third-party application of the maturation such as Redis, technology opposite
Maturation, acquisition modes are various, and system compatibility is high, provide the foundation for its subsequent expansion use.
It is obvious to a person skilled in the art that invention is not limited to the details of the above exemplary embodiments, Er Qie
In the case of without departing substantially from spirit and essential characteristics of the invention, the present invention can be realized in other specific forms.Therefore, no matter
From the point of view of which point, the present embodiments are to be considered as illustrative and not restrictive, and the scope of the present invention is by appended power
Profit requires rather than above description limits, it is intended that all by what is fallen within the meaning and scope of the equivalent requirements of the claims
Variation is included within the present invention, and should not be considered as the note of any attached drawing table in claim and be limited the claims involved.
In addition, it should be understood that although this specification is described in terms of embodiments, but not each embodiment is only wrapped
Containing an independent technical solution, this description of the specification is merely for the sake of clarity, and those skilled in the art should
It considers the specification as a whole, the technical solutions in the various embodiments may also be suitably combined, forms those skilled in the art
The other embodiment being appreciated that.
Claims (14)
1. a kind of embedded device data store system based on message queue, which is characterized in that including：
Data acquisition unit generates message data, and upload to message data for acquiring, parsing data；
Data transmission unit connection and checks connection status for carrying out communication with cloud platform, and is connect from data center unit
It receives message data and is forwarded to cloud platform；
Data center unit, for receiving the event request for coming from data acquisition unit and data transmission unit, and according to thing
Part request carries out event handling and message push；
Cloud platform carries out storage record for receiving message data, and to message data；
The data transmission unit includes,
Connection checks subelement, for carrying out communication connection with cloud platform and checking connection status,
Data transmission sub-unit, for receiving message data from data center unit and being forwarded to cloud platform；
The data center unit includes,
Event handling subelement, for managing message data and receiving the subscription information of message data,
Message pushes subelement, is used for data transmission unit PUSH message data, and stores the message data that do not consume.
2. the embedded device data store system according to claim 1 based on message queue, which is characterized in that described
Data acquisition unit includes：
Collecting device initialization module, for carrying out initialization process to collecting device；
Data center's link block, for establishing the connection between data acquisition unit and data center unit；
Data acquisition module, for the collecting device data from collecting device；
Data resolution module, for collected device data to be resolved to the required format of cloud platform；
Data package module, for device data to be encapsulated into message data；
Data transmission blocks, for message data to be sent to data center unit；
Circle collection module, after being established for the connection between data acquisition unit and data center unit, sequentially circulating
Row data acquisition module, data resolution module, data package module and data transmission blocks.
3. the embedded device data store system according to claim 1 based on message queue, which is characterized in that described
Connection checks that subelement includes：
Cloud platform link block, for establishing communication connection with cloud platform；
Connection status initial survey module, for checking the connection status between data transmission unit and cloud platform,
When data transmission unit and cloud platform are in successful connection operating mode, mark is connected as normal condition,
When data transmission unit and cloud platform are in connection failure operating mode, mark is connected as off-state；
Connection retries module, for when data transmission unit and cloud platform are in connection failure operating mode, sequentially circulating to rack
Platform link block and connection status initial survey module are stopped until after data transmission unit is in successful connection operating mode with cloud platform
Only recycle.
4. the embedded device data store system according to claim 1 based on message queue, which is characterized in that described
Data transmission sub-unit includes：
Data subscription module, for subscribing to message data to data center unit；
Data reception module, for receiving the message data pushed from data center unit；
Connection status examines module again, for checking again for the connection status between data transmission unit and cloud platform and making corresponding anti-
Feedback,
When data transmission unit and cloud platform are in successful connection operating mode, message authentication event is sent to data center unit,
When data transmission unit and cloud platform are in connection failure operating mode, message failure event is sent to data center unit；
Receiving module is repeated, for the data reception module that reruns.
5. the embedded device data store system according to claim 4 based on message queue, which is characterized in that described
Data transmission sub-unit further includes：
Data extract sending module, under being in successful connection operating mode in data transmission unit and cloud platform, from message data
Middle extraction equipment data, and device data is sent to cloud platform.
6. the embedded device data store system according to claim 1 based on message queue, which is characterized in that described
Event handling subelement includes：
Data initialization module, for initial message queue and related data structures and module；
Digital independent is inserted into module, for reading the message data not sent from local persistent storage, and by message data
It is inserted into message queue；
Request receiving module, for receiving the request for coming from data acquisition unit and data transmission unit；
Request processing module, for handling received request event,
When what is received is the subscription request of data transmission unit, which is added consumer queue, subsequently
The data transmission unit being pushed to after message data in consumer queue is received, sort run request is then pressed again and receives mould
Block and request processing module,
When what is received is message data request, message data is added to message queue, task pair is handled by message queue
Message data is handled, and then presses sort run request receiving module and request processing module again,
When what is received is the message authentication event that data transmission unit is beamed back, message data is held from message queue and locally
It is deleted in longization storage, then presses sort run request receiving module and request processing module again,
When what is received is the message failure event that data transmission unit is beamed back, message data is stored to local persistence and is deposited
Chu Zhong then presses sort run request receiving module and request processing module again,
When what is received is non-above-mentioned event type, message data and without any processing is abandoned.
7. the embedded device data store system according to claim 1 based on message queue, which is characterized in that described
Message pushes subelement：
Data read module, for reading message data from message queue；
It subscribes to and checks module, for checking whether there is the message subscribing request from data transmission unit,
When there is message subscribing request, message data is pushed into data transmission unit, then sequentially operation data is read again
Module and subscription check module,
When there is no message subscribing request, by message data storage to local persistent storage, deletes and disappear from message queue
Data are ceased, then sequentially operation data read module and subscription check module again.
8. a kind of embedded device data save method based on message queue, which is characterized in that include the following steps：
S1, data collection steps, acquisition, parsing data, generate message data, and upload to message data；
S2, data sending step carry out communication connection with cloud platform and check connection status, and received from data center unit
Message data is simultaneously forwarded to cloud platform；
S3, data center's step receive the event request for coming from data acquisition unit and data transmission unit, and according to event
Request carries out event handling and message push；
S4, cloud storage step receive message data, and carry out storage record to message data；
The data sending step includes,
S21, connection check sub-step, carry out communication connection with cloud platform and check connection status,
S22, data send sub-step, receive message data from data center unit and are forwarded to cloud platform；
Data center's step includes,
S31, event handling sub-step manage message data and receive the subscription information of message data,
S32, message push sub-step, to data transmission unit PUSH message data, and store the message data that do not consume.
9. the embedded device data save method according to claim 8 based on message queue, which is characterized in that described
Data collection steps include：
S11, collecting device initialization step carry out initialization process to collecting device；
S12, data center's Connection Step, establish the connection between data acquisition unit and data center unit；
S13, data collection steps, the collecting device data from collecting device；
Collected device data is resolved to the required format of cloud platform by S14, data analyzing step；
S15, data encapsulation step, message data is encapsulated by device data；
Message data is sent to data center unit by S16, data sending step；
S17, circle collection step, after connection between data acquisition unit and data center unit is established, sequentially recycle into
Row data collection steps, data analyzing step, data encapsulation step and data sending step.
10. the embedded device data save method according to claim 1 based on message queue, which is characterized in that institute
It states connection and checks that sub-step includes：
S211, cloud platform Connection Step establish communication connection with cloud platform；
S212, connection status initial survey step check the connection status between data transmission unit and cloud platform,
When data transmission unit and cloud platform are in successful connection operating mode, mark is connected as normal condition,
When data transmission unit and cloud platform are in connection failure operating mode, mark is connected as off-state；
S213, connection retry step, when data transmission unit and cloud platform are in connection failure operating mode, sequentially recycle into racking
Platform Connection Step and connection status initial survey step are stopped until after data transmission unit is in successful connection operating mode with cloud platform
Only recycle.
11. the embedded device data save method according to claim 8 based on message queue, which is characterized in that institute
Stating data transmission sub-step includes：
S221, data subscription step subscribe to message data to data center's step；
S222, data reception step receive the message data pushed from data center unit；
S223, connection status examine step again, check again for the connection status between data transmission unit and cloud platform and make corresponding
Feedback,
When data transmission unit and cloud platform are in successful connection operating mode, message authentication event is sent to data center unit,
When data transmission unit and cloud platform are in connection failure operating mode, message failure event is sent to data center unit；
S225, receiving step is repeated, repeats data reception step.
12. the embedded device data save method according to claim 11 based on message queue, which is characterized in that institute
Stating data transmission sub-step further includes：
S224, data extract forwarding step, in the case where data transmission unit and cloud platform are in successful connection operating mode, from message data
Middle extraction equipment data, and device data is sent to cloud platform.
13. the embedded device data save method according to claim 1 based on message queue, which is characterized in that institute
Stating event handling sub-step includes：
S311, data initialization step, initial message queue and related data structures and module；
S312, digital independent inserting step, read the message data not sent from local persistent storage, and by message data
It is inserted into message queue；
S313, request receiving step, receive the request for coming from data acquisition unit and data transmission unit；
S314, request processing step handle received request event,
When what is received is the subscription request of data transmission unit, which is added consumer queue, subsequently
The data transmission unit being pushed to after message data in consumer queue is received, sort run request is then pressed again and receives step
Rapid and request processing step,
When what is received is message data request, message data is added to message queue, task pair is handled by message queue
Message data is handled, and then presses sort run request receiving step and request processing step again,
When what is received is the message authentication event that data transmission unit is beamed back, message data is held from message queue and locally
It is deleted in longization storage, then presses sort run request receiving step and request processing step again,
When what is received is the message failure event that data transmission unit is beamed back, message data is stored to local persistence and is deposited
Chu Zhong then presses sort run request receiving step and request processing step again,
When what is received is non-above-mentioned event type, message data and without any processing is abandoned.
14. the embedded device data save method according to claim 1 based on message queue, which is characterized in that institute
Stating message push sub-step includes：
S321, data reading step, read message data from message queue；
S322, checking step is subscribed to, checks whether there is the message subscribing request from data transmission unit,
When there is message subscribing request, message data is pushed into data sending step, then sequentially operation data is read again
Step and subscription checking step,
When there is no message subscribing request, by message data storage to local persistent storage, deletes and disappear from message queue
Cease data, then again sequentially operation data read step and subscribe to checking step.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810425085.6A CN108512943A (en) | 2018-05-07 | 2018-05-07 | Embedded device data store system based on message queue and store method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810425085.6A CN108512943A (en) | 2018-05-07 | 2018-05-07 | Embedded device data store system based on message queue and store method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108512943A true CN108512943A (en) | 2018-09-07 |
Family
ID=63400133
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810425085.6A Pending CN108512943A (en) | 2018-05-07 | 2018-05-07 | Embedded device data store system based on message queue and store method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108512943A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109491617A (en) * | 2018-11-16 | 2019-03-19 | 广州创想云科技有限公司 | Data storage retransmission method, electronic equipment, storage medium |
| CN112559198A (en) * | 2019-09-10 | 2021-03-26 | 北京东土科技股份有限公司 | Method for storing bus data in memory, storage device and storage medium |
| WO2021223110A1 (en) * | 2020-05-06 | 2021-11-11 | 深圳市大疆创新科技有限公司 | Data processing apparatus and method, medium, computer device, and mobile device |
| CN115914210A (en) * | 2022-11-17 | 2023-04-04 | 天翼云科技有限公司 | File transmission method, device and system based on virtual network |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102402209A (en) * | 2010-09-14 | 2012-04-04 | 广州致远电子有限公司 | Embedded data acquisition and equipment control system |
| CN103927218A (en) * | 2014-04-30 | 2014-07-16 | 广州唯品会网络技术有限公司 | Event dispatching method and system |
| CN105740083A (en) * | 2016-01-28 | 2016-07-06 | 努比亚技术有限公司 | Information processing method, device and system |
| CN105808363A (en) * | 2016-03-11 | 2016-07-27 | 浪潮通用软件有限公司 | Method for realizing asynchronous execution based on Active MQ message queue |
| CN106257874A (en) * | 2016-08-01 | 2016-12-28 | 上海海事大学 | Big data quantity remote collection method and system in a kind of unstable network |
| CN107391269A (en) * | 2016-03-28 | 2017-11-24 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus being used for by persistence queue processing message |
| CN208210014U (en) * | 2018-05-07 | 2018-12-07 | 苏州德姆斯信息技术有限公司 | Embedded device data store system based on message queue |
-
2018
- 2018-05-07 CN CN201810425085.6A patent/CN108512943A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102402209A (en) * | 2010-09-14 | 2012-04-04 | 广州致远电子有限公司 | Embedded data acquisition and equipment control system |
| CN103927218A (en) * | 2014-04-30 | 2014-07-16 | 广州唯品会网络技术有限公司 | Event dispatching method and system |
| CN105740083A (en) * | 2016-01-28 | 2016-07-06 | 努比亚技术有限公司 | Information processing method, device and system |
| CN105808363A (en) * | 2016-03-11 | 2016-07-27 | 浪潮通用软件有限公司 | Method for realizing asynchronous execution based on Active MQ message queue |
| CN107391269A (en) * | 2016-03-28 | 2017-11-24 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus being used for by persistence queue processing message |
| CN106257874A (en) * | 2016-08-01 | 2016-12-28 | 上海海事大学 | Big data quantity remote collection method and system in a kind of unstable network |
| CN208210014U (en) * | 2018-05-07 | 2018-12-07 | 苏州德姆斯信息技术有限公司 | Embedded device data store system based on message queue |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109491617A (en) * | 2018-11-16 | 2019-03-19 | 广州创想云科技有限公司 | Data storage retransmission method, electronic equipment, storage medium |
| CN109491617B (en) * | 2018-11-16 | 2022-10-04 | 广州创想云科技有限公司 | Data storage and forwarding method, electronic equipment and storage medium |
| CN112559198A (en) * | 2019-09-10 | 2021-03-26 | 北京东土科技股份有限公司 | Method for storing bus data in memory, storage device and storage medium |
| WO2021223110A1 (en) * | 2020-05-06 | 2021-11-11 | 深圳市大疆创新科技有限公司 | Data processing apparatus and method, medium, computer device, and mobile device |
| CN115914210A (en) * | 2022-11-17 | 2023-04-04 | 天翼云科技有限公司 | File transmission method, device and system based on virtual network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6928467B2 (en) | Apparatus and methods for providing data synchronization by facilitating data synchronization system design | |
| CN108512943A (en) | Embedded device data store system based on message queue and store method | |
| CN114866528B (en) | A data communication method based on MQTT and Websocket | |
| CN105630977A (en) | Application recommending method, device and system | |
| CN103139157A (en) | Network communication method based on socket, device and system | |
| CN103517375B (en) | A kind of to multiple mobile equipment identification, the method for communication and device | |
| CN106897159A (en) | A kind of system and method for gathering Data of Automatic Weather | |
| CN103179193A (en) | Data transmission system and data transmission method used for distributed application system | |
| CN103167428A (en) | Picture adaptive method, mobile phone newspaper delivering device and system | |
| CN105791399B (en) | More trunk interconnection net big data method for pushing and system | |
| CN112417050A (en) | Data synchronization method and device, system, storage medium and electronic device | |
| CN109451078A (en) | Transaction methods and device under a kind of distributed structure/architecture | |
| CN105447146A (en) | Massive data collecting and exchanging system and method | |
| CN113794622B (en) | Message processing method and device, electronic equipment and storage medium | |
| CN208210014U (en) | Embedded device data store system based on message queue | |
| CN110362548B (en) | Log extraction method and related equipment | |
| EP1764691A1 (en) | Mobile data management using association table | |
| CN111338821B (en) | Method, system and electronic equipment for realizing data load balance | |
| CN111614645A (en) | Big data forwarding method and device for system cooperation | |
| CN108763970A (en) | Embedded device data store system and store method | |
| CN113407491B (en) | Data processing method and device | |
| CN103107991B (en) | A kind of many NVT event-handling method, NVC and safety monitoring platform | |
| CN102172051A (en) | Multimedia message processing method and device | |
| CN107506379A (en) | Method based on database row mode construction streaming movie real-time ecological models | |
| CN107317865A (en) | A kind of general off-line data cut-in method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180907 |