CN106953900A - A kind of industrial environment outdoor scene enhanced interactive terminal and system - Google Patents

A kind of industrial environment outdoor scene enhanced interactive terminal and system Download PDF

Info

Publication number
CN106953900A
CN106953900A CN201710138192.6A CN201710138192A CN106953900A CN 106953900 A CN106953900 A CN 106953900A CN 201710138192 A CN201710138192 A CN 201710138192A CN 106953900 A CN106953900 A CN 106953900A
Authority
CN
China
Prior art keywords
equipment
real
module
industrial
time data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710138192.6A
Other languages
Chinese (zh)
Inventor
何积丰
毛侠
史建琦
黄滟鸿
李昂
方徽星
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Fenglei Information Technology Co Ltd
East China Normal University
Original Assignee
Shanghai Fenglei Information Technology Co Ltd
East China Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Fenglei Information Technology Co Ltd, East China Normal University filed Critical Shanghai Fenglei Information Technology Co Ltd
Priority to CN201710138192.6A priority Critical patent/CN106953900A/en
Publication of CN106953900A publication Critical patent/CN106953900A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/02Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
    • H04L67/025Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00Program-control systems
    • G05B19/02Program-control systems electric
    • G05B19/04Program control other than numerical control, i.e. in sequence controllers or logic controllers
    • G05B19/05Programmable logic controllers, e.g. simulating logic interconnections of signals according to ladder diagrams or function charts
    • G05B19/054Input/output
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16Sound input; Sound output
    • G06F3/167Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W4/00Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/02Services making use of location information
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/10Plc systems
    • G05B2219/15Plc structure of the system
    • G05B2219/15011Configuration of operating system
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/10Plc systems
    • G05B2219/15Plc structure of the system
    • G05B2219/15042Synoptic display of process, mimic diagram

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Automation & Control Theory (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The invention discloses a kind of industrial environment outdoor scene enhanced interactive terminal and system, outdoor scene enhanced interactive terminal therein includes AR equipment, Intelligent positioning module, real-time data acquisition module and wireless communication module;AR equipment therein includes AR backstages configuration module and AR outdoor scenes from adaptation module, AR backstages configuration module is used for the parameters index and additional real time data for obtaining industrial equipment, it is used for the instruction for receiving and explaining AR equipment simultaneously, to realize the generation and/or renewal of configuration environment;AR outdoor scenes therein are used to catch the actual observation object in the AR equipment visual field from adaptation module, the position of appearing and mode of the real time data for the industrial equipment that adjust automatically is gathered, the observation operation interface of virtualization is shown, calls and interfaceization shows the configuration environment for being generated and/or being updated by AR backstages configuration module.The mode called, show and observed there is provided brand-new configuration.

Description

一种工业环境实景增强式交互终端及系统An industrial environment enhanced interactive terminal and system

技术领域technical field

本发明涉及工业控制领域,具体地,涉及工业环境实景增强式交互系统。这种交互系统涉及了AR(增强现实)技术、PLC(可编程逻辑控制器)数据监控采集技术、蓝牙无线组网和短距定位技术、以及HMI(人机接口)组态技术。The invention relates to the field of industrial control, in particular to an industrial environment real-scene enhanced interactive system. This interactive system involves AR (Augmented Reality) technology, PLC (Programmable Logic Controller) data monitoring and acquisition technology, Bluetooth wireless networking and short-distance positioning technology, and HMI (Human-Machine Interface) configuration technology.

背景技术Background technique

美国智能制造领导力联盟于2011年提出了智能制造的概念。德国于2013年在汉诺威工博会上提出了工业4.0,并将其纳入由德国政府发布的《德国2020高技术战略》的十大未来项目中,希望在新一轮的工业革命中占领先机。中国在2015年由国家工信部牵头制定了《中国制造2025》等战略级计划。由此可见,工业控制系统受到越来越多国家的重视。The American Smart Manufacturing Leadership Alliance put forward the concept of smart manufacturing in 2011. Germany proposed Industry 4.0 at the Hannover Messe in 2013 and included it in the top ten future projects of the "Germany 2020 High-Tech Strategy" issued by the German government, hoping to take the lead in the new round of industrial revolution . In 2015, China led by the Ministry of Industry and Information Technology to formulate strategic plans such as "Made in China 2025". It can be seen that the industrial control system has been valued by more and more countries.

工业控制系统是强国之基础,大到核电厂、航空航天、轨道交通,小到民营生产等等,据统计,超过80%的涉及国计民生的领域都需要工业控制系统来实现自动化作业,其先进性和现代化程度紧密关系着国家的战略布局。PLC作为工业现场新型、通用的控制器,有着组装灵活、安全性高、抗干扰能力强等显著优点,已经被广泛而大量地运用在了各大工业现场。控制器的不断升级和革新为工业控制的生产、维护以及管理等诸多方面带来了质的突破,然而,配套辅助设施的突破性及创新性不够明显。大量的工业现场都存在以下这些现象:传统的只使用物理按键的机械式HMI操作方式沿用至今,控制方式固定死板;为设备配置独立的物理显示器,成本高昂,数据相对孤立;工业现场采用全有线数据传输,布线极其繁琐复杂等等。工业4.0时代如何改善传统工业制造模式并使其向智能制造转型很大程度上和现有实现技术是息息相关的。The industrial control system is the foundation of a strong country, ranging from nuclear power plants, aerospace, rail transit, to small private production, etc. According to statistics, more than 80% of the fields involving the national economy and people's livelihood need industrial control systems to realize automated operations. Its advanced nature The country's strategic layout is closely related to the degree of modernization. As a new and general-purpose controller in industrial sites, PLC has significant advantages such as flexible assembly, high safety, and strong anti-interference ability, and has been widely and massively used in various industrial sites. The continuous upgrading and innovation of controllers has brought qualitative breakthroughs in many aspects of industrial control production, maintenance and management. However, the breakthrough and innovation of supporting auxiliary facilities are not obvious enough. The following phenomena exist in a large number of industrial sites: the traditional mechanical HMI operation method that only uses physical buttons is still in use today, and the control method is fixed and rigid; it is expensive to configure an independent physical display for the device, and the data is relatively isolated; the industrial site adopts fully wired Data transmission, wiring is extremely cumbersome and complicated, and so on. How to improve the traditional industrial manufacturing mode and transform it to intelligent manufacturing in the era of Industry 4.0 is closely related to the existing realization technology to a large extent.

虽然市场中的PLC品牌的种类繁多,但是主流的控制器厂商却屈指可数,被广泛使用且具有影响力的工业现场通信协议数量并不多,这使得针对市场主流PLC进行协议对接和设备数据采集成为可能。另外,物联网以及无线通信技术的迅速发展为工业级数据传输方式带来了新的扩展途径,国家3G/4G传输速率大幅提升且覆盖范围日益完善,蓝牙自4.0+开始逐步实现了低功耗、自组网以及安全加密等优越的新特性,更加贴合工业使用环境。除此之外,视觉上,AR相关的技术热潮正冲击着传统物理屏幕的静态的固定显示方式;肢体上,景深摄像头的使用逐渐开辟了一个人机自然交互的新市场。与此同时,大数据的不断发展和人们对安全性的空前重视使得在工业环境中使用在线专家诊断系统的技术成为热门,上述每一点所涉及的新技术都可能在工业界带来开创性的技术融合与革新。Although there are many types of PLC brands in the market, there are only a handful of mainstream controller manufacturers, and there are not many widely used and influential industrial field communication protocols, which makes protocol docking and device data for mainstream PLCs in the market collection is possible. In addition, the rapid development of the Internet of Things and wireless communication technology has brought new expansion methods for industrial-grade data transmission methods. The national 3G/4G transmission rate has been greatly improved and the coverage is increasingly perfect. Bluetooth has gradually realized low power consumption since 4.0+ , self-organizing network and security encryption and other superior new features, more suitable for industrial use environment. In addition, visually, the upsurge of AR-related technologies is impacting the static and fixed display methods of traditional physical screens; physically, the use of depth-of-field cameras has gradually opened up a new market for natural human-computer interaction. At the same time, the continuous development of big data and the unprecedented emphasis on safety make the technology of using online expert diagnosis system in the industrial environment become popular, and the new technologies involved in each of the above points may bring groundbreaking in the industry Technology integration and innovation.

本发明针对传统工业现场目前实际存在的上述问题,以及观测设备固定、数量众多且不能被复用、成本较高、占用较大的空间,组态调用、显示及观测方式不够灵活、人机交互方式不够灵活、设备诊断效果需要进一步改善等问题,提出了一种工业环境实景增强式交互终端和系统。The present invention aims at the above-mentioned problems currently existing in traditional industrial sites, as well as fixed observation equipment, a large number and cannot be reused, high cost, large space occupation, inflexible configuration calling, display and observation methods, and human-computer interaction. The method is not flexible enough, and the effect of equipment diagnosis needs to be further improved. A real-scene enhanced interactive terminal and system for industrial environment is proposed.

发明内容Contents of the invention

为了至少部分解决传统工业现场目前实际存在的上述技术问题,本发明提出了工业环境实景增强式交互终端和系统。In order to at least partly solve the above-mentioned technical problems currently existing in traditional industrial sites, the present invention proposes an interactive terminal and system for augmented reality in industrial environments.

本发明提出了一种工业环境实景增强式交互终端,包括:AR设备;智能定位模块,用于实时定位追踪AR设备的位置信息,同时给出工业设备的物理服务范围;实时数据采集模块,用于采集工业设备的实时数据和对工业设备进行实时数据的反馈;以及,无线通信模块,用于在AR设备进入工业设备的物理服务范围内时,对实时数据采集模块所采集的工业设备的实时数据、与AR设备产生的交互数据进行双向传输,实时连续地更新;其中的AR设备包括AR后台组态模块和AR实景自适配模块,AR后台组态模块用于获取工业设备的各项参数指标及附加实时数据,同时用于接收和解释AR设备的指令,以实现组态环境的生成和/或更新;其中的AR实景自适配模块用于捕捉AR设备视野中的实际观测对象,自动调整所采集的工业设备的实时数据的呈现位置及方式,显示虚拟化的观测操作界面,调用并界面化显示由AR后台组态模块所生成和/或更新的组态环境。The present invention proposes an industrial environment enhanced interactive terminal, including: AR equipment; an intelligent positioning module for real-time positioning and tracking of the location information of the AR equipment, and at the same time providing the physical service range of the industrial equipment; a real-time data acquisition module for It is used to collect real-time data of industrial equipment and provide real-time data feedback to industrial equipment; and the wireless communication module is used for real-time monitoring of industrial equipment collected by the real-time data acquisition module when AR equipment enters the physical service range of industrial equipment. The data and the interactive data generated by the AR equipment are transmitted bidirectionally and continuously updated in real time; the AR equipment includes the AR background configuration module and the AR real scene self-adaptation module, and the AR background configuration module is used to obtain various parameters of industrial equipment Indicators and additional real-time data are used to receive and interpret the instructions of the AR device, so as to realize the generation and/or update of the configuration environment; the AR real scene self-adaptation module is used to capture the actual observation object in the field of view of the AR device, and automatically Adjust the presentation position and mode of the collected real-time data of industrial equipment, display the virtualized observation operation interface, call and interface display the configuration environment generated and/or updated by the AR background configuration module.

根据本发明的这种实景增强式交互终端已经提供了一种全新的组态调用、显示及观测的方式,通过随身佩戴工业环境实景增强式交互终端(例如,其可以被实现为无线接入的AR设备的形式,还可以是可以移动的其他显示和交互装置)的移动式复用,可在一定程度上减少工业环境传统显示设备的数量,降低采购成本,更加节约能耗和设备占用空间。This real-scene enhanced interactive terminal according to the present invention has provided a brand-new mode of configuration call, display and observation, by wearing an industrial environment enhanced interactive terminal (for example, it can be realized as a wireless access The form of AR equipment can also be mobile reuse of other display and interactive devices), which can reduce the number of traditional display equipment in industrial environments to a certain extent, reduce procurement costs, and save energy consumption and equipment space.

而且,根据本发明的这种实景增强式交互终端由于采用了物理与体感相结合的人机交互方式,所以不仅能够以高精度的操作来使用对物理设备的控制,从而保证其具有传统工业中的稳定性、安全性和可靠性;而且能够以低精度和提示性的操作来使用手势、语音等更为自然的方式进行人机交互,从而使数据交互更加直观、便捷、人性化。Moreover, since the real-scene enhanced interactive terminal according to the present invention adopts the human-computer interaction mode combining physics and somatosensory, it can not only use the control of physical equipment with high-precision operation, thereby ensuring that it has the advantages of traditional industrial stability, security and reliability; and it can use low-precision and suggestive operations to use gestures, voice and other more natural ways for human-computer interaction, so that data interaction is more intuitive, convenient and humanized.

根据本发明的实景增强式交互终端,其中的实时数据采集模块包括:数据获取装置,用于采集工业设备的实时数据;以及,数据反馈装置,用于对所采集的工业设备进行实时数据的反馈。According to the real-scene enhanced interactive terminal of the present invention, the real-time data collection module includes: a data acquisition device for collecting real-time data of industrial equipment; and a data feedback device for feeding back real-time data to the collected industrial equipment .

根据本发明的实景增强式交互终端,其中的实时数据采集模块采用有线接入方式或无线接入方式,且在网络接入时绑定所述工业设备的IP地址。According to the real-scene enhanced interactive terminal of the present invention, the real-time data acquisition module adopts a wired access mode or a wireless access mode, and binds the IP address of the industrial equipment when accessing the network.

根据本发明的实景增强式交互终端,其中的实时数据采集模块分别采用以下有线接入方式或无线接入方式中的至少一种:串口、网线、WiFi、2G/3G/4G。According to the real-scene enhanced interactive terminal of the present invention, the real-time data acquisition module adopts at least one of the following wired access methods or wireless access methods: serial port, network cable, WiFi, 2G/3G/4G.

根据本发明的实景增强式交互终端,其中的实时数据采集模块基于网络的方案采用C/S架构的客户端或B/S架构的web服务的数据访问方式。According to the real-scene enhanced interactive terminal of the present invention, the real-time data acquisition module adopts the data access mode of the client of C/S structure or the web service of B/S structure based on the network solution.

根据本发明的实景增强式交互终端,其中的智能定位模块是采用带组网物联的无线定位技术的定位模块。According to the real-scene enhanced interactive terminal of the present invention, the intelligent positioning module is a positioning module adopting the wireless positioning technology with networking of things.

根据本发明的实景增强式交互终端,其中的智能定位模块采用以下带组网物联的无线定位技术:蓝牙无线定位技术,用于给出工业设备的服务范围;蓝牙组网技术,用于多设备互联且汇总数据从而提供模拟工业现场的虚拟节点设备。According to the real-scene enhanced interactive terminal of the present invention, the intelligent positioning module adopts the following wireless positioning technology with networking and IoT: Bluetooth wireless positioning technology, used to provide the service range of industrial equipment; Bluetooth networking technology, used for multiple Devices are interconnected and data is aggregated to provide virtual node devices that simulate industrial sites.

根据本发明的实景增强式交互终端,其中的无线通信模块采用蓝牙作为基础通信方案,并且扩展到2G/3G/4G、WiFi、以及LoRa通信方案中的至少一种方案,用于在所述AR设备进入智能定位模块的控制节点设备的服务范围内时,将AR设备的指令按照相关协议发送到控制节点设备,从而启用控制节点设备的设备控制属性,其中所述AR设备采用蓝牙作为基础通信方案,或者扩展到2G/3G/4G、WiFi、以及LoRa通信方案中的至少一种方案。According to the real-scene enhanced interactive terminal of the present invention, the wireless communication module adopts bluetooth as the basic communication scheme, and extends to at least one of 2G/3G/4G, WiFi, and LoRa communication schemes, which are used in the AR When the device enters the service range of the control node device of the intelligent positioning module, the command of the AR device is sent to the control node device according to the relevant protocol, thereby enabling the device control attribute of the control node device, wherein the AR device uses Bluetooth as the basic communication scheme , or extended to at least one of 2G/3G/4G, WiFi, and LoRa communication solutions.

根据本发明的实景增强式交互终端,AR实景自适配模块自动调整和选择组态界面或通用界面的配置,用于在AR设备上显示所采集的工业设备的实时数据。According to the real-scene enhanced interactive terminal of the present invention, the AR real-scene self-adapting module automatically adjusts and selects the configuration of the configuration interface or the general interface for displaying the collected real-time data of industrial equipment on the AR equipment.

根据本发明的第二个方面,提出了一种工业环境实景增强式交互系统,包括以下设备:需要进行实时数据采集和对其进行反馈的工业设备;以及,如上文所述的实景增强式交互终端,其中的工业设备可以采用有线连接和/或无线连接适配的方式来接入实时数据采集模块,而且,实时数据采集模块不与工业设备(即节点设备)串口直连,而是通过网络方式与其进行连接时,实时数据采集模块本身可作为虚拟节点。According to the second aspect of the present invention, an industrial environment augmented reality interactive system is proposed, including the following equipment: industrial equipment that needs to collect real-time data and provide feedback to it; and, the augmented reality interactive system as described above The terminal, where the industrial equipment can be connected to the real-time data acquisition module by means of wired connection and/or wireless connection adaptation, and the real-time data acquisition module is not directly connected to the serial port of the industrial equipment (that is, node equipment), but through the network When it is connected with it in a different way, the real-time data acquisition module itself can be used as a virtual node.

如本领域的技术人员能够理解的,根据本发明的实景增强式交互终端和系统能够获得如下有益效果:As those skilled in the art can understand, the reality-enhanced interactive terminal and system according to the present invention can obtain the following beneficial effects:

本发明的实景增强式交互终端和系统提供了一种全新的组态调用、显示及观测的方式,通过随身佩戴AR设备的移动式复用,可在一定程度上减少工业环境传统显示设备的数量,降低采购成本,更加节约能耗和设备占用空间,同时缓解现场复杂布线问题。The real-scene enhanced interactive terminal and system of the present invention provide a brand-new way of configuration calling, displaying and observing, and can reduce the number of traditional display devices in the industrial environment to a certain extent through the mobile multiplexing of wearing AR devices , reduce procurement costs, save more energy consumption and equipment space, and at the same time alleviate complex wiring problems on site.

本发明的实景增强式交互终端和系统由于采用了物理与体感相结合的人机交互方式,所以不仅能够以高精度的操作来使用对物理设备的控制,从而保证其具有传统工业中的稳定性、安全性和可靠性;而且能够以低精度和提示性的操作来使用手势、语音等更为自然的方式进行人机交互,从而使数据交互更加直观、便捷、人性化。Since the real-scene enhanced interactive terminal and system of the present invention adopts the human-computer interaction mode combining physics and somatosensory, it can not only use the control of physical equipment with high-precision operation, so as to ensure its stability in traditional industries , security and reliability; and can use low-precision and prompt operations to use gestures, voice and other more natural ways for human-computer interaction, so that data interaction is more intuitive, convenient and humanized.

本发明的实景增强式交互终端和系统结合在线诊断技术,扩展配合在线专家诊断系统,不仅可以提供丰富的例如参数指标预警提示等辅助服务,还可达到减少培训成本、降低失误操作概率、提升设备管理效率等目的。The real-scene enhanced interactive terminal and system of the present invention are combined with online diagnosis technology and extended to cooperate with the online expert diagnosis system, which can not only provide abundant auxiliary services such as parameter index warning prompts, but also reduce training costs, reduce the probability of misoperation, and improve equipment management efficiency, etc.

附图说明Description of drawings

通过阅读下文具体实施方式的详细描述,各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图仅用于示出具体实施方式的目的,而并不认为是对本发明的限制。而且在附图中,用相同的参考符号表示相同的部件。在附图中:Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the specific embodiments. The drawings are only for the purpose of illustrating specific embodiments and are not to be considered as limiting the invention. Also in the drawings, the same components are denoted by the same reference symbols. In the attached picture:

图1示出了根据本发明的实施方式的工业环境实景增强式交互系统的总体示意图;FIG. 1 shows an overall schematic diagram of an industrial environment augmented interactive system according to an embodiment of the present invention;

图2示出了根据本发明的实施方式的工业环境实景增强式交互系统中的工业环境实景增强式交互终端的结构框图;Fig. 2 shows a structural block diagram of an industrial environment enhanced interactive terminal in an industrial environment enhanced interactive system according to an embodiment of the present invention;

图3示出了根据本发明的实施方式的工业环境实景增强式交互终端中的实时数据采集模块的示例性的结构框图;Fig. 3 shows an exemplary structural block diagram of a real-time data acquisition module in an industrial environment enhanced interactive terminal according to an embodiment of the present invention;

图4示出了与根据本发明的实施方式的工业环境实景增强式交互终端进行交互的示例性的工业设备的结构框图;Fig. 4 shows a structural block diagram of an exemplary industrial device interacting with an industrial environment augmented interactive terminal according to an embodiment of the present invention;

图5示出了根据本发明的实施方式的实景增强式交互系统所实现的方法流程图;FIG. 5 shows a flow chart of a method implemented by a reality-enhanced interactive system according to an embodiment of the present invention;

图6示出了根据本发明的实施方式的实景增强式交互终端与工业设备进行交互的方法流程图;以及FIG. 6 shows a flow chart of a method for interacting with an industrial device between a reality-enhanced interactive terminal and an industrial device according to an embodiment of the present invention; and

图7示出了根据本发明具体实施方式的工业环境实景增强式交互系统的系统框架图。Fig. 7 shows a system frame diagram of an industrial environment augmented interactive system according to a specific embodiment of the present invention.

具体实施方式detailed description

下面将参照附图更详细地描述本发明的示例性实施方式。虽然附图中显示了本发明的示例性实施方式,然而应当理解,能够以各种形式实现本发明而不应被这里阐述的实施方式所限制。相反,提供这些实施方式是为了能够更透彻地理解本发明,并且能够将本发明的范围完整的传达给本领域的技术人员。Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and is not limited to the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

图1示出了根据本发明的实施方式的工业环境实景增强式交互系统的总体示意图。如图1所示,整个系统示意性地包括实景增强式交互终端和工业现场设备,所述实景增强式交互终端包括实时数据采集模块、智能定位模块、无线通信模块、AR设备。Fig. 1 shows an overall schematic diagram of an industrial environment augmented reality interactive system according to an embodiment of the present invention. As shown in FIG. 1 , the whole system schematically includes a reality-enhanced interactive terminal and industrial field devices, and the real-enhanced interactive terminal includes a real-time data acquisition module, an intelligent positioning module, a wireless communication module, and an AR device.

其中,实时数据采集模块是整个所述系统的基础模块,其通过适配市面主流PLC(例如西门子、三菱、欧姆龙等),支持多种工业现场总线通讯协议(例如PROFINET、EtherCAT、Modbus等),用于采集工业设备(即,工业现场节点设备)的实时数据,同时需要对工业现场设备进行数据反馈,二者之间的基础连接方式为有线连接方式(例如,在图1中示出了工业用网线和采用UART的串口方式,但是并不仅限于这两种)。实时数据采集模块可以采用有线接入方式或无线接入方式,可以分别采用以下有线接入方式或无线接入方式中的至少一种:串口、网线、WiFi、2G/3G/4G。实时数据采集模块基于网络采集的方案可以采用C/S架构的客户端或B/S架构的web服务来访问数据,并且数据监测及管理的API可开放给第三方进行多方扩展。Among them, the real-time data acquisition module is the basic module of the entire system, which supports various industrial field bus communication protocols (such as PROFINET, EtherCAT, Modbus, etc.) by adapting to mainstream PLCs (such as Siemens, Mitsubishi, Omron, etc.) It is used to collect real-time data of industrial equipment (that is, industrial field node equipment), and at the same time needs to perform data feedback to industrial field equipment. The basic connection method between the two is a wired connection (for example, the industrial Use a network cable and a serial port using UART, but it is not limited to these two). The real-time data acquisition module can adopt wired access or wireless access, and can respectively adopt at least one of the following wired or wireless access methods: serial port, network cable, WiFi, 2G/3G/4G. The real-time data acquisition module based on the network acquisition scheme can use the client of C/S architecture or the web service of B/S architecture to access the data, and the API of data monitoring and management can be opened to third parties for multi-party expansion.

其中,无线通信模块(例如,在图1中示出了包含蓝牙、WiFi的无线通信模块,但是并不仅限于这两种),用于在AR设备和工业现场设备之间进行通信。由此,无线通信模块可以采用蓝牙作为基础通信方案,并且扩展到2G/3G/4G、WiFi、以及LoRa通信方案中的至少一种方案,用于在带有通过上述通信方案实现的无线功能的AR设备进入智能定位模块的控制节点设备的服务范围内时,将AR设备的指令按照相关协议发送到控制节点设备,从而启用控制节点设备的设备控制属性。具体地,一旦所述AR设备进入所述系统服务范围内,无线通信模块将所述实时数据采集模块获取的数据与所述系统的AR设备产生的交互数据进行双向传输,实时连续地更新。Wherein, the wireless communication module (for example, shown in FIG. 1 includes wireless communication modules including Bluetooth and WiFi, but not limited to these two types) is used for communication between the AR device and the industrial field device. Thus, the wireless communication module can adopt bluetooth as the basic communication scheme, and expand to at least one scheme in 2G/3G/4G, WiFi, and LoRa communication schemes, for wireless functions realized through the above-mentioned communication schemes When the AR device enters the service range of the control node device of the intelligent positioning module, the command of the AR device is sent to the control node device according to the relevant protocol, thereby enabling the device control attribute of the control node device. Specifically, once the AR device enters the service range of the system, the wireless communication module bidirectionally transmits the data acquired by the real-time data collection module and the interaction data generated by the AR device of the system, and continuously updates in real time.

其中,智能定位模块,采用主流的无线定位技术及方案(例如,蓝牙定位、LoRa定位、室内GPS定位等),一方面给出系统中每个节点设备的物理服务的范围,另一方面实时检测并追踪所述系统的使用者进入服务范围后的位置信息。智能定位模块可以是采用带组网物联的无线定位技术的定位模块,可以采用以下带组网物联的无线定位技术:蓝牙无线定位技术,用于给出工业设备的服务范围;若存在服务范围部分重叠的多台设备则可相互组成网络,实现多设备互联,模拟虚拟化的I/O数据节点。Among them, the intelligent positioning module adopts the mainstream wireless positioning technology and solutions (for example, Bluetooth positioning, LoRa positioning, indoor GPS positioning, etc.), on the one hand, it provides the physical service range of each node device in the system, and on the other hand, it detects in real time And track the location information of the users of the system after entering the service area. The intelligent positioning module can be a positioning module that adopts the wireless positioning technology with networking and IoT, and can adopt the following wireless positioning technology with networking and IoT: Bluetooth wireless positioning technology, which is used to give the service range of industrial equipment; if there is a service Multiple devices with partially overlapping ranges can form a network with each other to realize multi-device interconnection and simulate virtualized I/O data nodes.

其中,AR设备包含AR实景自适配模块和AR后台组态模块。当所述系统的AR设备获取所述无线通信模块传来的所述实时数据采集模块采集到的数据,AR实景自适配模块和AR后台组态模块协同运行,所述AR实景自适配模块根据捕获到的所述AR设备视野中的实际观测对象,自动调整数据呈现位置及方式并且显示虚拟化的观测操及作界面,即,可以自动调整和选择组态界面或通用界面的配置,为所接收数据提供更加合适的方式显示于所述系统AR设备中;所述AR后台组态模块通过获取设备各项参数指标、待接收和解释的AR设备操作指令等实时数据,及时完成组态环境的生成和更新功能,AR后台组态模块还可以包括在线专家诊断系统,用于进行指标提取分析,并且用于进行综合评判以给出优选决策提示,以供使用者针对多种方案做出决策。Among them, the AR device includes an AR real scene self-adaptation module and an AR background configuration module. When the AR device of the system obtains the data collected by the real-time data acquisition module transmitted from the wireless communication module, the AR real-scene self-adaptation module and the AR background configuration module work together, and the AR real-scene self-adaptation module According to the captured actual observation object in the field of view of the AR device, automatically adjust the position and mode of data presentation and display the virtualized observation operation and operation interface, that is, the configuration of the configuration interface or the general interface can be automatically adjusted and selected, for The received data is displayed in the AR device in the system in a more suitable way; the AR background configuration module completes the configuration environment in time by obtaining real-time data such as various parameter indicators of the device, AR device operation instructions to be received and explained, etc. The generation and update functions of the AR background configuration module can also include an online expert diagnosis system for index extraction and analysis, and for comprehensive evaluation to give optimal decision prompts for users to make decisions on various plans .

如图2,其以功能模块的形式示出了根据本发明的实施方式的工业环境实景增强式交互系统中的工业环境实景增强式交互终端的结构框图,该工业环境实景增强式交互终端包括实时数据采集模块、智能定位模块、无线通信模块、AR设备,上述各模块的功能与图1中相应模块的功能相对应。As shown in Fig. 2, it shows the structural block diagram of the industrial environment augmented reality interactive terminal in the industrial environment augmented reality interactive system according to the embodiment of the present invention in the form of functional modules, the industrial environment augmented reality interactive terminal includes real-time Data acquisition module, intelligent positioning module, wireless communication module, AR device, the functions of the above modules correspond to the functions of the corresponding modules in Figure 1.

其中,智能定位模块,用于实时定位追踪AR设备的位置信息,同时给出工业设备的物理服务范围;实时数据采集模块,用于采集工业设备的实时数据和对工业设备进行实时数据的反馈;以及,无线通信模块,用于在AR设备进入工业设备的物理服务范围内时,对实时数据采集模块所采集的工业设备的实时数据、以及对AR设备的指令进行转发,实时连续地更新;其中的AR设备包括AR后台组态模块和AR实景自适配模块,AR后台组态模块用于获取工业设备的各项参数指标及附加实时数据,同时用于接收和解释AR设备的指令,以实现组态环境的生成和/或更新;其中的AR实景自适配模块用于捕捉AR设备视野中的实际观测对象,自动调整所采集的工业设备的实时数据的呈现位置及方式,显示虚拟化的观测操作界面,调用并界面化显示由AR后台组态模块所生成和/或更新的组态环境。Among them, the intelligent positioning module is used to locate and track the location information of AR equipment in real time, and at the same time provide the physical service range of industrial equipment; the real-time data acquisition module is used to collect real-time data of industrial equipment and provide real-time data feedback to industrial equipment; And, the wireless communication module is used to forward and continuously update the real-time data of the industrial equipment collected by the real-time data acquisition module and the instructions of the AR equipment when the AR equipment enters the physical service range of the industrial equipment; The AR equipment includes the AR background configuration module and the AR real scene self-adaptation module. The AR background configuration module is used to obtain various parameters and additional real-time data of industrial equipment, and is used to receive and interpret the instructions of the AR equipment to realize Generation and/or update of the configuration environment; the AR real scene self-adaptation module is used to capture the actual observation objects in the field of view of the AR equipment, automatically adjust the presentation position and mode of the collected real-time data of the industrial equipment, and display the virtualized Observe the operation interface, invoke and display the configuration environment generated and/or updated by the AR background configuration module.

如图2所示,在其中示意性地用双向箭头和单向箭头分别表示了各个模块之间的双向和单向数据通信关系。但是本领域的技术人员应该了解的是,图2中所示出的各模块之间的数据通信关系仅仅是示意性的,可以设想各模块之间的其他数据通信关系。As shown in FIG. 2 , two-way arrows and one-way arrows are used to schematically represent the two-way and one-way data communication relationships between modules, respectively. However, those skilled in the art should understand that the data communication relationship between the modules shown in FIG. 2 is only schematic, and other data communication relationships between the modules can be conceived.

如本领域的技术人员所能了解的,首先,根据本发明的实景增强式交互终端、以及该终端与工业现场设备相结合的系统,提供了一种全新的组态调用、显示及观测的方式,通过随身佩戴AR设备的移动式复用,可在一定程度上减少工业环境传统显示设备的数量,降低采购成本,更加节约能耗和设备占用空间。其次,由于采用了物理与体感相结合的人机交互方式,所以不仅能够以高精度的操作来使用对物理设备的控制,从而保证其具有传统工业中的稳定性、安全性和可靠性;而且能够以低精度和提示性的操作来使用手势、语音等更为自然的方式进行人机交互,从而使数据交互更加直观、便捷、人性化。而且,根据本发明的这种实景增强式交互终端可以缓解现场复杂布线问题。As those skilled in the art can understand, first of all, according to the present invention, the augmented reality interactive terminal and the system combining the terminal with industrial field equipment provide a brand new way of configuration call, display and observation , through the mobile multiplexing of wearable AR devices, the number of traditional display devices in the industrial environment can be reduced to a certain extent, the purchase cost can be reduced, and energy consumption and equipment space can be saved. Secondly, due to the combination of physical and somatosensory human-computer interaction, it is not only possible to use the control of physical equipment with high-precision operation, so as to ensure its stability, safety and reliability in traditional industries; and Human-computer interaction can be carried out in a more natural way such as gestures and voices with low-precision and suggestive operations, so that data interaction is more intuitive, convenient, and humanized. Moreover, the real-scene enhanced interactive terminal according to the present invention can alleviate the complex wiring problem on site.

再次,结合了在线诊断技术的这种实景增强式交互终端,因为扩展配合了在线专家诊断系统,所以不仅可以提供丰富的例如参数指标预警提示等辅助服务,还可达到减少培训成本、降低失误操作概率、提升设备管理效率等目的。Thirdly, this real-scene enhanced interactive terminal combined with online diagnosis technology can not only provide rich auxiliary services such as parameter indicator warning prompts, but also reduce training costs and reduce erroneous operations because of the expansion and cooperation with the online expert diagnosis system. Probability, improve equipment management efficiency and other purposes.

图3示出了根据本发明的实施方式的工业环境实景增强式交互终端中的实时数据采集模块的示例性的结构框图。如图3所示,实时数据采集模块包括数据获取装置和数据反馈装置,其中的数据获取装置用于采集工业设备的实时数据;而其中的数据反馈装置对所采集的工业设备进行实时数据的反馈。Fig. 3 shows an exemplary structural block diagram of a real-time data collection module in an industrial environment enhanced interactive terminal according to an embodiment of the present invention. As shown in Figure 3, the real-time data acquisition module includes a data acquisition device and a data feedback device, wherein the data acquisition device is used to collect real-time data of industrial equipment; and wherein the data feedback device performs real-time data feedback on the collected industrial equipment .

图4示出了与根据本发明的实施方式的工业环境实景增强式交互终端进行交互的工业设备的结构框图。如图4中所示,可选地,工业设备通过串口、网线、WiFi、2G/3G/4G中的一种或多种方式与实景增强式交互终端中的实时数据采集模块连接适配,从而实现工业设备的实时数据采集及反馈功能。Fig. 4 shows a structural block diagram of an industrial device interacting with an industrial environment augmented interactive terminal according to an embodiment of the present invention. As shown in Figure 4, optionally, the industrial equipment is connected and adapted to the real-time data acquisition module in the real-scene enhanced interactive terminal through one or more methods of serial port, network cable, WiFi, 2G/3G/4G, so that Realize the real-time data acquisition and feedback function of industrial equipment.

图5示出了根据本发明的实施方式的实景增强式交互方法的步骤的框图,其中包括了以实线框的形式表示的实景增强式交互终端可以操作的基本步骤S501至S505。Fig. 5 shows a block diagram of the steps of a reality-enhanced interaction method according to an embodiment of the present invention, which includes basic steps S501 to S505 that can be operated by a reality-enhanced interactive terminal in the form of a solid line box.

在步骤S501中,智能定位模块实时定位追踪带无线功能的AR设备的位置信息,同时给出工业设备的物理服务范围。In step S501, the intelligent positioning module locates and tracks the location information of the AR device with wireless function in real time, and at the same time provides the physical service range of the industrial device.

在步骤S502中,实时数据采集模块采集工业设备的实时数据。In step S502, the real-time data collection module collects real-time data of industrial equipment.

在步骤S503中,当AR设备进入工业设备的物理服务范围内时,无线通信模块对实时数据采集模块所采集的工业设备的实时数据、AR设备产生的指令进行数据交互,实时连续地更新。In step S503, when the AR device enters the physical service range of the industrial device, the wireless communication module performs data interaction with the real-time data of the industrial device collected by the real-time data collection module and the instructions generated by the AR device, and continuously updates in real time.

在步骤S504中,AR后台组态模块获取工业设备的各项参数指标及附加实时数据,同时接收和解释AR设备的指令,以实现组态环境的生成和/或更新。In step S504, the AR background configuration module acquires various parameter indicators and additional real-time data of industrial equipment, and at the same time receives and interprets instructions from the AR equipment, so as to realize the generation and/or update of the configuration environment.

在步骤S505中,AR实景自适配模块捕捉AR设备视野中的实际观测对象,自动调整所采集的工业设备的实时数据的呈现位置及方式,显示虚拟化的观测操作界面,调用并界面化显示由AR后台组态模块所生成和/或更新的组态环境。In step S505, the AR real-scene self-adapting module captures the actual observation object in the field of view of the AR equipment, automatically adjusts the presentation position and mode of the collected real-time data of the industrial equipment, displays the virtualized observation operation interface, calls and interfaces the display The configuration environment generated and/or updated by the AR background configuration module.

根据步骤S504和步骤S505的操作,以及图2中示出的AR实景自适配模块与AR后台组态模块之间的连接关系,可以看出AR实景自适配模块和AR后台组态模块是协同运行的,但是本领域的技术人员也可以考虑实现相同功能的其他实现方式。According to the operations of step S504 and step S505, and the connection relationship between the AR real-scene self-adaptation module and the AR background configuration module shown in FIG. 2, it can be seen that the AR real-scene self-adaptation module and the AR background configuration module are However, those skilled in the art can also consider other implementation manners to achieve the same function.

可选择地,当根据本发明的实景增强式交互终端的无线通信模块采用蓝牙作为基础通信方案,并且扩展到2G/3G/4G、WiFi、以及LoRa通信方案中的至少一种方案时,实景增强式交互终端还执行步骤S506。在图5中以虚线框的形式表示的步骤S506中,在带有通过上述通信方案实现的无线功能的AR设备(相应地,AR设备采用蓝牙作为基础通信方案,或者扩展到2G/3G/4G、WiFi、以及LoRa通信方案中的至少一种方案)进入智能定位模块的控制节点设备的服务范围内时,将AR设备的指令按照相关协议发送到控制节点设备,从而启用控制节点设备的设备控制属性。Optionally, when the wireless communication module of the real-scene enhanced interactive terminal according to the present invention adopts Bluetooth as the basic communication scheme and extends to at least one of 2G/3G/4G, WiFi, and LoRa communication schemes, the real-scene enhanced The interactive terminal also executes step S506. In step S506 shown in the form of a dotted box in FIG. 5 , in the AR device with the wireless function realized through the above-mentioned communication scheme (correspondingly, the AR device adopts Bluetooth as the basic communication scheme, or extends to 2G/3G/4G , WiFi, and at least one of the LoRa communication schemes) when entering the service range of the control node device of the intelligent positioning module, the command of the AR device is sent to the control node device according to the relevant protocol, thereby enabling the device control of the control node device Attributes.

可选择地,根据本发明的实景增强式交互终端执行步骤S507。在图5中以虚线框的形式表示的步骤S507中,AR实景自适配模块自动调整和选择组态界面或通用界面的配置,用于在AR设备上显示所采集的工业设备的实时数据。Optionally, the augmented reality interactive terminal according to the present invention executes step S507. In step S507 shown in the form of a dotted box in FIG. 5 , the AR real-scene self-adapting module automatically adjusts and selects the configuration of the configuration interface or the general interface for displaying the collected real-time data of industrial equipment on the AR device.

可选择地,当AR后台组态模块包括在线专家诊断系统时,根据本发明的实景增强式交互终端执行步骤S508。在图5中以虚线框的形式表示的步骤S508中,在线专家诊断系统进行指标提取分析,并且进行综合评判以给出优选决策提示,以供使用者针对多种方案做出决策。Optionally, when the AR background configuration module includes an online expert diagnosis system, the augmented reality interactive terminal according to the present invention executes step S508. In step S508 shown in the form of a dotted box in FIG. 5 , the online expert diagnosis system performs index extraction and analysis, and conducts comprehensive evaluation to give optimal decision prompts for users to make decisions on various schemes.

图6示出了根据本发明的实施方式的实景增强式交互终端与工业设备进行交互的方法流程图。Fig. 6 shows a flowchart of a method for interacting between a reality-enhanced interactive terminal and an industrial device according to an embodiment of the present invention.

如图6所示,其中包括了以实线框的形式表示的工业设备可以操作的基本步骤S601。在步骤S601中,实景增强式交互终端中的实时数据采集模块通过适配连入工业设备,该适配连入的方式采用有线直连(例如串口、网线)或WiFi、2G/3G/4G中的一种或多种方式。As shown in FIG. 6 , it includes basic steps S601 that industrial equipment can operate in the form of a solid line box. In step S601, the real-time data acquisition module in the augmented reality interactive terminal is connected to the industrial equipment through adaptation. one or more ways.

可选择地,与根据本发明的实景增强式交互终端进行交互的工业设备执行步骤S602。在图6中以虚线框的形式表示的步骤S602中,服务范围部分重叠的多个工业设备组成网络,实现设备互联,用于模拟虚拟化的I/O数据节点。Optionally, the industrial equipment interacting with the reality-enhanced interactive terminal according to the present invention executes step S602. In step S602 shown in the form of a dotted box in FIG. 6 , a plurality of industrial devices with partially overlapping service areas form a network to realize device interconnection for simulating virtualized I/O data nodes.

可选择地,在使用其的实景增强式交互系统还包括控制节点设备时,与根据本发明的实景增强式交互终端进行交互的工业设备执行步骤S603。在图6中以虚线框的形式表示的步骤S603中,当实景增强式交互终端的无线通信模块采用蓝牙作为基础通信方案,并且扩展到2G/3G/4G、WiFi、以及LoRa通信方案中的至少一种方案时,且在带有通过上述通信方案实现的无线功能的AR设备(相应地,AR设备采用蓝牙作为基础通信方案,或者扩展到2G/3G/4G、WiFi、以及LoRa通信方案中的至少一种方案)进入智能定位模块的控制节点设备的服务范围内时,将AR设备的指令按照相关协议发送到控制节点设备,从而启用控制节点设备的设备控制属性。Optionally, when the reality-enhanced interactive system using it further includes a control node device, the industrial equipment interacting with the reality-enhanced interactive terminal according to the present invention executes step S603. In step S603 shown in the form of a dashed box in Figure 6, when the wireless communication module of the enhanced interactive terminal uses Bluetooth as the basic communication solution, and extends to at least 2G/3G/4G, WiFi, and LoRa communication solutions One solution, and AR devices with wireless functions realized through the above communication solutions (correspondingly, AR devices use Bluetooth as the basic communication solution, or extend to 2G/3G/4G, WiFi, and LoRa communication solutions At least one solution) when entering the service range of the control node device of the intelligent positioning module, the command of the AR device is sent to the control node device according to the relevant protocol, thereby enabling the device control attribute of the control node device.

尽管在图5、图6中分别以图示的示例顺序示出了可以由实景增强式交互终端和工业设备执行的可选择的示例性的步骤。但是,如本领域的技术人员所能够理解的,这些步骤能够以其他的顺序执行,可以包括可以设想的其他步骤,而且可以包括子步骤。Although FIG. 5 and FIG. 6 respectively show optional exemplary steps that can be performed by the reality-enhanced interactive terminal and the industrial equipment in the exemplary order shown in the illustration. However, as will be appreciated by those skilled in the art, the steps can be performed in other orders, can include other steps that are conceivable, and can include sub-steps.

已经根据上述实施方式实现了根据本发明的工业环境实景增强式交互终端、系统及方法。仅仅出于说明的目的,在此列出了部分实现的细节,以帮助本领域的技术人员理解。The industrial environment augmented interactive terminal, system and method according to the present invention have been realized according to the above-mentioned embodiments. For the purpose of illustration only, some implementation details are listed here to help those skilled in the art understand.

如图7所示,根据本发明的一个实施方式的数据采集模块使用了AnyLinkTM产品,AnyLinkTM盒子集成了大量工业级通讯协议(例如PROFINET、EtherCAT、Modbus等)并适配了市场上的主流PLC(例如西门子、三菱、欧姆龙等),并且提供了任务驱动、设备驱动以及云端开发的API,在其基础上进行二次开发方便、快捷;其中的智能定位模块选用蓝牙定位模块,蓝牙芯片采用东芝公司的TC35676芯片或TC35678芯片,它们支持蓝牙4.1、具有低耗能和自组网等卓越特点,能够给出每个工业设备的物理服务范围,并且若存在服务范围部分重叠的多台工业设备则可相互组成网络,实现多设备互联,模拟虚拟化的I/O数据节点;其中的无线通信模块使用上述蓝牙芯片,并且在嵌入式设备板(如cortex-M3/M4开发板)的操作系统中进行数据的透传和任务调度,当AR设备进入智能定位模块的控制节点设备服务范围内时,通过无线通信模块将所采集的工业设备的数据转发到实景增强式交互终端;实景增强式交互终端可以采用Microsoft公司的Hololens产品,其重量适中、时延低且带有景深摄像头,可以为自然人机交互提供硬件基础,在此基础上,AR实景自适配模块根据AR设备视野中的观测对象自动调整数据呈现位置及方式,并且显示虚拟化的观测操作界面,而AR后台组态模块则通过获取工业设备的各项参数指标、待接收和解释的AR设备的指令等实时数据,及时完成组态环境的生成和更新功能。As shown in Figure 7, the data acquisition module according to an embodiment of the present invention uses AnyLinkTM products, and the AnyLinkTM box integrates a large number of industrial-grade communication protocols (such as PROFINET, EtherCAT, Modbus, etc.) and adapts to mainstream PLCs on the market ( Such as Siemens, Mitsubishi, Omron, etc.), and provides task-driven, device-driven and cloud-developed APIs, based on which the secondary development is convenient and fast; the intelligent positioning module uses the Bluetooth positioning module, and the Bluetooth chip adopts Toshiba The TC35676 chip or TC35678 chip, which supports Bluetooth 4.1, has excellent features such as low energy consumption and ad hoc networking, and can give the physical service range of each industrial device, and if there are multiple industrial devices with overlapping service ranges, it can be Form a network with each other, realize multi-device interconnection, and simulate virtualized I/O data nodes; the wireless communication module uses the above-mentioned Bluetooth chip, and is implemented in the operating system of the embedded device board (such as cortex-M3/M4 development board) Data transparent transmission and task scheduling, when the AR device enters the service range of the control node device of the intelligent positioning module, the data collected by the industrial equipment is forwarded to the real-scene enhanced interactive terminal through the wireless communication module; the real-scene enhanced interactive terminal can The Hololens product of Microsoft is adopted, which has moderate weight, low latency and a depth-of-field camera, which can provide a hardware foundation for natural human-computer interaction. On this basis, the AR real-scene self-adaptation module automatically adjusts according to the observed objects in the field of view of the AR device. The location and method of data presentation, and display the virtualized observation operation interface, while the AR background configuration module obtains real-time data such as various parameter indicators of industrial equipment, instructions of AR equipment to be received and interpreted, and completes the configuration environment in time generation and update functions.

如本领域技术人员所能理解的,根据本发明的上述实施方式的实景增强式交互系统至少包括了与根据本发明的上述实施方式的实景增强式交互终端相同或相应的技术特征,因此也解决了上文所述的根据本发明的上述实施方式的实景增强式交互终端所解决的技术问题,并且能够获得其所获得的有益效果。As those skilled in the art can understand, the reality-enhanced interactive system according to the above-mentioned embodiment of the present invention at least includes the same or corresponding technical features as the reality-enhanced interactive terminal according to the above-mentioned embodiment of the present invention, so it also solves the The above-mentioned technical problems solved by the reality-enhanced interactive terminal according to the above-mentioned embodiments of the present invention are solved, and the beneficial effects obtained by it can be obtained.

以上所述,仅为本发明的示例性的具体实施方式,但本发明的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本发明揭露的技术范围内,可轻易想到的变化或替换,都应涵盖在本发明的保护范围之内。因此,本发明的保护范围应以权利要求的保护范围为准。The above is only an exemplary specific implementation of the present invention, but the scope of protection of the present invention is not limited thereto, any changes that can be easily imagined by those skilled in the art within the technical scope disclosed in the present invention Or replacement, all should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (10)

1. a kind of industrial environment outdoor scene enhanced interactive terminal, it is characterised in that including:
AR equipment;
Intelligent positioning module, for the positional information of AR equipment described in real-time location tracking, while providing the physics of industrial equipment Service range;
Real-time data acquisition module, for gathering the real time data of the industrial equipment and being counted in real time to the industrial equipment According to feedback;And
Wireless communication module, for when the AR equipment enters in the range of the physical services of the industrial equipment, to the reality When the industrial equipment that is gathered of data acquisition module the real time data, carry out with the interaction data that the AR equipment is produced it is double To transmission, update to real-time continuous;
Wherein, the AR equipment includes AR backstages configuration module and AR outdoor scenes from adaptation module, and AR backstages configuration module is used In the parameters index and additional real time data that obtain the industrial equipment, while for receiving and explaining the AR equipment Instruction, to realize the generation and/or renewal of configuration environment;The AR outdoor scenes are used to catch the AR equipment visual field from adaptation module In actual observation object, the position of appearing and mode of the real time data for the industrial equipment that adjust automatically is gathered, display The observation operation interface of virtualization, is called and interfaceization shows the group for being generated and/or being updated by AR backstages configuration module State environment.
2. outdoor scene enhanced interactive terminal according to claim 1, it is characterised in that the real-time data acquisition module bag Include:
Data acquisition facility, the real time data for gathering the industrial equipment;And
Data feedback device, the feedback for carrying out real time data to the industrial equipment gathered.
3. outdoor scene enhanced interactive terminal according to claim 1 or 2, it is characterised in that the real-time data acquisition mould Block uses Wired access mode or wireless access way, and binds in network insertion the IP address of the industrial equipment.
4. outdoor scene enhanced interactive terminal according to claim 3, it is characterised in that the real-time data acquisition module point At least one that Cai Yong be in following Wired access mode or wireless access way:Serial ports, netting twine, WiFi, 2G/3G/4G.
5. outdoor scene enhanced interactive terminal according to claim 1 or 2, it is characterised in that the real-time data acquisition mould The network scheme of block uses the data access mode of the client of C/S frameworks or the web services of B/S frameworks.
6. outdoor scene enhanced interactive terminal according to claim 1 or 2, it is characterised in that the Intelligent positioning module is Using the locating module of the wireless location technology with networking Internet of Things.
7. outdoor scene enhanced interactive terminal according to claim 6, it is characterised in that the Intelligent positioning module use with The wireless location technology of lower band networking Internet of Things:Bluetooth wireless location technology, the service range for providing the industrial equipment;It is blue Tooth networking technology, is interconnected for many equipment and combined data is so as to provide the dummy node equipment of simulation industry spot.
8. outdoor scene enhanced interactive terminal according to claim 1 or 2, it is characterised in that the wireless communication module is adopted With communication plan based on bluetooth, and at least one side expanded in 2G/3G/4G, WiFi and LoRa communication plan Case, during in the service range for the control node equipment for entering the Intelligent positioning module in the AR equipment, by the AR The instruction of equipment is sent to the control node equipment according to related protocol, so as to enable the equipment control of the control node equipment Attribute processed, wherein the AR equipment use bluetooth based on communication plan, or expand to 2G/3G/4G, WiFi and At least one scheme in LoRa communication plans.
9. outdoor scene enhanced interactive terminal according to claim 1 or 2, it is characterised in that the AR outdoor scenes are adapted to mould certainly The configuration of block adjust automatically and selection configuration interface or General Purpose Interface, for showing the work gathered in the AR equipment The real time data of industry equipment.
10. a kind of industrial environment outdoor scene enhanced interactive system, it is characterised in that including following equipment:
Need the industrial equipment for carrying out real-time data acquisition and being fed back to it;And
Outdoor scene enhanced interactive terminal according to any one of claim 1-9 claim.
CN201710138192.6A 2017-03-09 2017-03-09 A kind of industrial environment outdoor scene enhanced interactive terminal and system Pending CN106953900A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710138192.6A CN106953900A (en) 2017-03-09 2017-03-09 A kind of industrial environment outdoor scene enhanced interactive terminal and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710138192.6A CN106953900A (en) 2017-03-09 2017-03-09 A kind of industrial environment outdoor scene enhanced interactive terminal and system

Publications (1)

Publication Number Publication Date
CN106953900A true CN106953900A (en) 2017-07-14

Family

ID=59466736

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710138192.6A Pending CN106953900A (en) 2017-03-09 2017-03-09 A kind of industrial environment outdoor scene enhanced interactive terminal and system

Country Status (1)

Country Link
CN (1) CN106953900A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107508737A (en) * 2017-08-10 2017-12-22 杰克缝纫机股份有限公司 EtherCAT-based wireless communication device and communication method thereof
CN107515674A (en) * 2017-08-08 2017-12-26 山东科技大学 A method for implementing multi-interaction in mining operations based on virtual reality and augmented reality
CN108415386A (en) * 2018-03-12 2018-08-17 范业鹏 Augmented reality system and its working method for intelligent workshop
CN108809971A (en) * 2018-06-01 2018-11-13 广州亿图自动控制系统有限公司 A kind of opened industrial Internet of Things
CN109688450A (en) * 2017-10-19 2019-04-26 阿里巴巴集团控股有限公司 Multi-screen interaction method, device and electronic equipment
CN110347259A (en) * 2019-07-09 2019-10-18 北京猫眼视觉科技有限公司 A kind of method and system being presented IOT device data in AR application apparatus
CN113256836A (en) * 2021-04-27 2021-08-13 交通运输部天津水运工程科学研究所 Immersive intelligent inspection method for port facilities

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103823935A (en) * 2014-02-28 2014-05-28 武汉大学 Three-dimensional remote monitoring system for wind power plant
CN104486586A (en) * 2014-12-18 2015-04-01 深圳先进技术研究院 Disaster lifesaving simulated training method and system based on video map
CN104794583A (en) * 2015-04-30 2015-07-22 武汉光谷北斗控股集团有限公司 Perceptual mine internet-of-things model

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103823935A (en) * 2014-02-28 2014-05-28 武汉大学 Three-dimensional remote monitoring system for wind power plant
CN104486586A (en) * 2014-12-18 2015-04-01 深圳先进技术研究院 Disaster lifesaving simulated training method and system based on video map
CN104794583A (en) * 2015-04-30 2015-07-22 武汉光谷北斗控股集团有限公司 Perceptual mine internet-of-things model

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107515674A (en) * 2017-08-08 2017-12-26 山东科技大学 A method for implementing multi-interaction in mining operations based on virtual reality and augmented reality
CN107508737A (en) * 2017-08-10 2017-12-22 杰克缝纫机股份有限公司 EtherCAT-based wireless communication device and communication method thereof
CN109688450A (en) * 2017-10-19 2019-04-26 阿里巴巴集团控股有限公司 Multi-screen interaction method, device and electronic equipment
CN108415386A (en) * 2018-03-12 2018-08-17 范业鹏 Augmented reality system and its working method for intelligent workshop
CN108809971A (en) * 2018-06-01 2018-11-13 广州亿图自动控制系统有限公司 A kind of opened industrial Internet of Things
CN110347259A (en) * 2019-07-09 2019-10-18 北京猫眼视觉科技有限公司 A kind of method and system being presented IOT device data in AR application apparatus
CN113256836A (en) * 2021-04-27 2021-08-13 交通运输部天津水运工程科学研究所 Immersive intelligent inspection method for port facilities

Similar Documents

Publication Publication Date Title
CN106953900A (en) A kind of industrial environment outdoor scene enhanced interactive terminal and system
CN103731645B (en) Law enforcement and case handling system based on wireless video transmission technology
CN102184014B (en) Intelligent appliance interaction control method and device based on mobile equipment orientation
CN110753218A (en) Digital twinning system and method and computer equipment
CN104090827B (en) Two-dimensional platform and three-dimensional platform integrated pipeline data synchronization and linkage method
CN105922262A (en) Robot and remote control equipment and remote control method thereof
CN106899599A (en) A kind of industrial environment outdoor scene enhanced exchange method
CN106648297A (en) Intelligent device control method and device based on VR device
TWI644235B (en) Wearable instant interaction system
CN204810307U (en) Unmanned aerial vehicle wearing formula maintenance guarantee support system
CN116489149A (en) Construction method of cloud-edge-device collaboration mixed reality collaboration environment for industrial applications
CN117226847B (en) Control method and system of teleoperation equipment
CN114374710A (en) Distribution network monitoring method and system for monitoring 5G ultra-high-definition videos and Internet of things
CN206960934U (en) A kind of Real-time Expert Instruction System for electric field maintenance
CN110288207A (en) It is a kind of that the method and apparatus of scene information on duty is provided
CN108908342A (en) Intelligent Mobile Robot and scheduling system business operations linkage system and method
CN116450092A (en) Multi-display terminal simulation data synchronization and interaction method and system
CN205563456U (en) Real estate sells display system based on virtual reality technology
CN205647813U (en) Family's intelligence cinema based on wireless network deployment of wiFi technique
CN104348920B (en) DCS MANET monitoring systems based on OPC standards
CN103024351A (en) Multi-view linkage and switching method for power grid operation monitoring picture
CN105635650B (en) Three-dimensional video monitor method and its monitoring system
CN203057366U (en) Wireless image transmission system based on Zigbee network
CN105100229A (en) A system and method based on intelligent hardware networking and mobile device game interaction
CN203178775U (en) Mobile remote wireless monitoring system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20170714

RJ01 Rejection of invention patent application after publication