TWI765558B - Intelligent locomotive road inspection and detection system - Google Patents

Abstract

An intelligent locomotive road inspection and detection system, comprising: the locomotive has a power supply device and an on-board automatic diagnosis system electrically connected to the power supply device, so that the power supply device can supply power to the on-board automatic diagnosis system, and the on-board automatic diagnosis system detects the mileage of the locomotive The camera is installed on the locomotive to capture a road image; the mobile device is installed on the locomotive and has a screen; the adapter can be electrically combined with the power supply device, the on-board automatic diagnosis system, the lens and the mobile device, The power supply device can provide the mobile device for charging, the road image captured by the lens is displayed on the screen, and the on-board automatic diagnosis system transmits the mileage record of the locomotive to the mobile device; and the artificial intelligence recognition algorithm can decompose the road image into continuous photos , and retrieve a photo to identify whether there is a sabotage information.

Description

Intelligent locomotive road inspection and detection system

The present invention relates to an intelligent locomotive road inspection and detection system, in particular to an adapter that integrates the power supply device of the locomotive with the vehicle-mounted automatic diagnosis system, lens and mobile device, and cooperates with the artificial intelligence identification algorithm to lead the most convenient road detection. method.

Press, traditional road inspection and inspection operations, in addition to pavement elevation measurement and pavement image capture, usually cooperate with other equipment to collect data, and try to keep all the information on the road in one inspection process. Digital cameras shoot street scenes to collect road images, but the images captured by digital cameras can only show part of the image, and cannot effectively present the road conditions, and then develop the street scene recording technology, which can assist the digital data in the rest of the system. For the road conditions that cannot be presented, engineers will obtain more information for visual judgment. However, in order to make the shooting field of view wider, the video recorder is installed on the roof of the car, but this method causes the video recorder to be exposed to the sun. It is easily damaged by rain. In addition, although there is an automatic road analyzer (Automatic Road Analyzer, ARAN) that specializes in analyzing roads abroad, this equipment can be tested at high speeds, and its complete system can simultaneously collect and measure road conditions, surface roughness, cracks, etc. Potholes, rut depth, road curve radius, longitudinal section, transverse section, number of structural layers, surface friction coefficient, global satellite positioning, geographic coordinates, right-of-way photography, marking reflectivity and other data.

However, in recent years, the domestic introduction of a test vehicle that only includes road conditions, surface roughness, cracks, potholes, rut depth, global satellite positioning, and right-of-way photography. Not only is the price expensive, but the inspection vehicle is imported from abroad, and maintenance is not easy. In addition, the applicability and popularization of this inspection vehicle in Taiwan still needs to be tested. It is impossible to get out of the car to take pictures in traffic conditions, and the traditional way of patrolling is possible In order to achieve the inspection coverage rate, manufacturers cannot carefully find and record the defects of the road surface. They can only make temporary repairs for potholes, and cannot monitor the hidden road damage, which is the biggest problem.

However, the inspection vehicle of the traditional road inspection system is not only large in size and many equipments, but also very expensive, and the digital camera is only a single type. How to develop an innovative structure with more ideal practicability is what the present invention intends to solve. the subject.

The main purpose of the present invention is to provide an intelligent locomotive road inspection and detection system, which integrates the power supply device of the locomotive, the vehicle-mounted automatic diagnosis system, the lens and the mobile device with an adapter, and cooperates with the artificial intelligence recognition algorithm to be used. To solve the problems of the conventional road inspection system of the prior art, the inspection vehicle not only has a large vehicle model, but also has many equipments and is expensive, and further has the effect of reducing the volume, simplifying the equipment and reducing the cost.

Another object of the present invention is to provide an intelligent locomotive road inspection and detection system, the lens of which is an IP lens, and the artificial intelligence identification algorithm is used to identify whether the photo is destructive information at the back end, or the lens is an industrial lens, and the artificial intelligence The identification algorithm at the front end identifies whether the photo is destructive information, which is used to solve the problem that the detection vehicle of the conventional road detection system in the prior art has only a single type of digital camera, and thus has the function of scalability.

In order to achieve the above object, the technical means adopted by the present invention: a locomotive, the locomotive has a power supply device and an on-board automatic diagnosis system electrically connected to the power supply device, so that the power supply device can supply power to the on-board automatic diagnosis system, and The on-board automatic diagnosis system detects the mileage record of the locomotive; a lens is installed on the locomotive to capture a road image; a mobile device is installed on the locomotive and has a A screen; an adapter, which can electrically combine the power supply device with the on-board automatic diagnosis system, the lens and the mobile device, so that the power supply device can provide the mobile device for charging, the lens captured by the lens Road images are displayed on the screen, and the on-board automatic diagnosis system The mileage record of the locomotive is systematically transmitted to the mobile device; and an artificial intelligence identification algorithm, the artificial intelligence identification algorithm can decompose the road image into continuous photos, and retrieve the photos to identify whether there is a damage information.

According to the aforementioned features, the power supply device is one of a mobile power supply or a battery.

According to the aforementioned features, the camera is an IP camera. After the IP camera is disconnected, the photo temporarily stored in the mobile device is uploaded through at least 4G, and the mobile device is connected to a cloud. The artificial intelligence identification algorithm identifies whether the photo is the sabotage information, and transmits the sabotage information to a platform.

According to the features disclosed above, the lens is an industrial lens, and the mobile device uses the mileage record of the locomotive to calculate that a trigger signal is sent and received every d meters of movement, and the mobile device uses the artificial intelligence recognition algorithm to identify the photo in real time. Whether it is the sabotage information, and the photo has the sabotage information uploaded through at least 4G, and the mobile device is connected to a cloud, and the sabotage information is transmitted to a platform.

According to the aforementioned feature, the mobile device has a built-in GPS, uploads the GPS track every second, and converts the GPS location to the urban address.

According to the aforementioned features, the artificial intelligence recognition algorithm includes an image training model, the image training model selects the damage information on the photo with a region of interest, and the image training model learns the samples of the damage information for training After that, the damage information can be identified.

According to the aforementioned features, the damage information includes a damage type, and the damage type is composed of a pothole, a patch, a crocodile-shaped crack, a vertical and horizontal crack, a hand hole or a concave-convex.

According to the aforementioned features, the damage information includes a degree of damage. When the type of damage is the pothole, the criterion for determining the severity of the degree of damage is the diameter; when the type of damage is the patch, the severity of the degree of damage is The judgment criterion is whether it is square or polygonal; when the damage type is the crocodile-shaped crack, the severity judgment criterion of the damage degree is the crack width; when the damage type is the vertical and horizontal crack If the damage type is the hand hole, the severity judgment criterion of the damage degree is the recording position.

According to the characteristics of the previous disclosure, the diameter of the hole is less than 10 cm, it is light, 10~20 cm is medium, and it is more than 20 cm. Heavy grade; the crack width of the crocodile-shaped crack is less than 6mm, it is light, 6mm~20mm is medium, and more than 20mm is heavy; the crack width of the vertical and horizontal cracks is less than 10mm and the crack length is not limited to light, and the crack width is within More than 10mm and the crack length less than 7.5m are intermediate, and the crack width is more than 10mm and the crack length is more than 7.5m is heavy.

According to the aforementioned features, the artificial intelligence recognition algorithm includes an opencv database, and the opencv database cooperates with the image training model.

With the help of the above-mentioned technical structure, the intelligent locomotive road inspection and detection system disclosed in the present invention integrates the power supply device of the locomotive with the on-board automatic diagnosis system, the lens and the mobile device by the adapter, and cooperates with the manual The intelligent identification algorithm, and the lens is an IP lens, cooperate with the artificial intelligence identification algorithm to identify whether the photo is the destructive information at the back end, or the lens is an industrial lens, cooperate with the artificial intelligence identification algorithm to identify the information at the front end. Whether the photo identification is the destructive information, it also solves the problems of traditional road inspection systems such as large vehicle models, many equipment, high cost, and a single type of digital camera, thereby reducing the size, simplifying equipment, reducing costs and scalability. The efficacy is enhanced.

10: Locomotive

11: Power supply device

12: On-board automatic diagnosis system

20: Shots

30: Mobile Devices

31: Screen

40: Adapter

AI: Artificial Intelligence Recognition Algorithm

AI ₁ : Image training model

AI ₂ : opencv database

I: Destruction of information

I ₁ : Destruction type

I ₁₁ : Pothole

I ₁₂ : Complementary Blossoms

I ₁₃ : Alligator Crack

I ₁₄ : Longitudinal and lateral cracks

I ₁₅ : Hand hole

I ₁₆ : bump

I ₂ : Degree of damage

I ₂₁ : Diameter

I ₂₂ : Whether it is square and polygonal

I ₂₃ : Crack width

I ₂₄ : crack width and crack length

I ₂₅ : record position

P: road image

Pn: photo

R: region of interest

S ₁ ~S ₅ : Steps

T ₁ ~T ₅ : Steps

W: line

FIG. 1 is a perspective view of the intelligent locomotive road inspection and detection system of the present invention.

FIG. 2 is a top view of the intelligent locomotive road inspection and detection system of the present invention.

FIG. 3 is a schematic diagram of the adapter of the present invention integrating a power supply device of a locomotive with an on-board automatic diagnosis system, a lens and a mobile device.

FIG. 4 is a flow chart showing that the lens of the present invention is an IP lens or an industrial lens.

FIG. 5 is a schematic diagram of a sample of training destruction information according to the present invention.

FIG. 6A is an image recognition diagram of capturing a road image according to the present invention.

FIG. 6B is another image recognition diagram of capturing a road image according to the present invention.

7A is an image recognition diagram of the present invention automatically identifying the failure type as a crocodile-shaped crack.

FIG. 7B is an image recognition diagram of the present invention automatically identifying the damage type as patching.

FIG. 8 is a schematic diagram of an image of the region of interest boxed to select the damage type and the damage degree according to the present invention.

In order to fully understand the present invention, the present invention is described in detail by the following specific embodiments and the accompanying drawings. Those skilled in the art can understand the objects, features and effects of the present invention from the contents disclosed in this specification. It should be noted that the present invention can be implemented or applied through other different specific embodiments, and various details in this specification can also be modified based on different viewpoints without departing from the spirit of the present invention. In addition, the drawings attached to the present invention are merely schematic illustrations, and are not drawn according to actual dimensions. The following embodiments will further describe the related technical contents of the present invention in detail, but the disclosed contents are not intended to limit the technical scope of the present invention. The description is as follows: First, as shown in FIG. 1 to FIG. 8 , the preferred implementation of the intelligent locomotive road inspection and detection system of the present invention includes: a locomotive 10 , the locomotive 10 has a power supply device 11 and a power supply device that is electrically connected to the power supply. On-Board Diagnostics (OBD) 12 of the device 11, so that the power supply device 11 can supply power to the OBD system 12, and the OBD system 12 detects the mileage record of the locomotive 10 or records the Various physical data of the locomotive 10 during driving, such as the power of the power supply device 11, engine speed, vehicle speed, etc. In this embodiment, the power supply device 11 is either a mobile power supply or a battery, but not limited to this.

A lens 20 is installed on the locomotive 10 for capturing a road image (P). In this embodiment, the lens 20 is either an IP lens or an industrial lens, but not limited thereto.

a mobile device 30, the mobile device 30 is installed on the locomotive 10 and has a screen 31, In this embodiment, the mobile device 30 is a tablet computer, a mobile phone, etc., and the best specification is an Apple Ipad, and the mobile device 30 has a built-in GPS, and the GPS track is uploaded every second, and the mobile device 30 can Knowing the inspection vehicle number, it can also convert the urban address with GPS positioning, but it is not limited to this.

An adapter 40, the adapter 40 can electrically combine the power supply device 11 with the vehicle-mounted automatic diagnosis system 12, the lens 20 and the mobile device 30, so that the power supply device 11 can provide the mobile device 30 for charging, The road image (P) captured by the lens 20 is displayed on the screen 31, and the on-board automatic diagnosis system 12 transmits the mileage record of the locomotive 10 to the mobile device 30. In this embodiment, the adapter 40 is a charging and transmitting hub adapter, the charging and transmitting hub adapter has an HDMI hole, a PD charging hole, a card reader slot, a USB hole, a USB connector, etc., so as to facilitate the integration of the power supply device 11, the vehicle-mounted automatic diagnosis system 12 and the The line (W) of the lens 20 is connected to the mobile device 30 through the USB connector, but not limited thereto.

An artificial intelligence identification algorithm (AI), the artificial intelligence identification algorithm (AI) can decompose the road image (P) into a continuous photo (Pn), and extract the photo (Pn) to identify whether there is a damage information ( i).

As shown in FIG. 4, two modes are adopted according to the difference of the lens 20. In the first mode, the lens 20 is an IP lens (network lens), and the IP lens is temporarily stored in the action after the disconnection The photo (Pn) of the device 30 is uploaded through at least 4G, and the mobile device 30 is connected to a cloud, and a backend of the cloud uses the artificial intelligence identification algorithm (AI) to identify whether the photo (Pn) is the damage. information (I), and the destruction information (I) is sent to a platform, and cooperates with steps S ₁ -S ₅ , so the artificial intelligence recognition algorithm (AI) is set in the background, and the photo (Pn ) to identify whether it is the destruction information (I), or in the second mode, the lens 20 is an industrial lens, and the mobile device 30 calculates the mileage record of the locomotive 11 to send and receive a trigger signal every d meters of movement , cooperate with the mobile device 30 to use the artificial intelligence recognition algorithm (AI) to instantly identify whether the photo (Pn) is the damage information (I), and the photo (Pn) has the damage information (I) uploaded through at least 4G, And connect the mobile device 30 to a cloud, and transmit the damage information (I) to a platform, and cooperate with steps T ₁ -T ₅ , so the artificial intelligence recognition algorithm (AI) is set in the mobile device 30 , The front end identifies whether the photo (Pn) is the damage information (I), so that the identification information of the damage, GPS points, road mileage and damage photos can be instantly transmitted to the cloud to the platform, It is for the system personnel of the office to make inquiries or dispatch workers.

As shown in FIG. 5 , the artificial intelligence recognition algorithm (AI) includes an image training model (AI ₁ ), and the image training model (AI ₁ ) selects a region of interest (R) on the photo (Pn) After the destruction information (I) is trained by the image training model (AI ₁ ) to learn the samples of the destruction information (I), the destruction information (I) can be identified, and the destruction information (I) refers to the United States ASTM D6433 formulated by the American Society for Testing and Materials (ASTM), and the artificial intelligence recognition algorithm (AI) includes an opencv database (AI ₂ ), the opencv database (AI ₂ ) and The image training model (AI ₁ ) cooperates with each other. In this embodiment, the image training model (AI ₁ ) may include either a YOLO training model or a Mask-RCNN training model, but is not limited thereto.

Continuing from the above, in this embodiment, the damage information (I) includes a damage type (I ₁ ), and the damage type (I ₁ ) is Potholes (I ₁₁ ), Patches (I ₁₂ ) ), Alligator Cracking (I ₁₃ ), Vertical and Horizontal Cracking (Cracking) (I ₁₄ ), Cover (I ₁₅ ) or Concavity (I ₁₆ ), but not limited to this.

Continuing from the above, in this embodiment, the damage information (I) includes a damage degree (I ₂ ). When the damage type (I ₁ ) is the pothole (I ₁₁ ), the damage degree (I ₂ ) is equal to Severity judgment criterion is diameter (I ₂₁ ); when the damage type (I ₁ ) is the patch (I ₁₂ ), the severity judgment criterion of the damage degree (I ₂ ) is whether it is square or polygon (I ₂₂ ) ; When the damage type (I ₁ ) is the crocodile crack (I ₁₃ ), the severity judgment criterion of the damage degree (I ₂ ) is the crack width (I ₂₃ ); when the damage type (I ₁ ) is the Longitudinal and transverse cracks (I ₁₄ ), the severity judgment criteria of the damage degree (I ₂ ) are the crack width and crack length (I ₂₄ ); when the damage type (I ₁ ) is the hand hole (I ₁₅ ), then The severity judgment criterion of the damage degree (I ₂ ) is the recording position (I ₂₅ ), but it is not limited to this.

Continuing above, in this embodiment, the diameter of the pit (I ₁₁ ) is light grade below 10 cm, intermediate grade is 10-20 cm, and heavy grade is above 20 cm; the patch (I ₁₂ ) is square and light The crocodile-shaped cracks (I ₁₃ ) are classified as light grades with a width of less than 6 mm, intermediate grades from 6 mm to 20 mm, and heavy grades if they are more than 20 mm; the vertical and horizontal cracks (I 13 ) ₁₄ ) The crack width is less than 10mm and the crack length is not limited to light, the crack width is more than 10mm and the crack length is less than 7.5m, it is intermediate, the crack width is more than 10mm and the crack length is more than 7.5m, it is heavy, but not limited to this.

As shown in FIGS. 6A and 6B , the photo (Pn) of the road image (P) displays the damage information (I). Further explanation, as shown in FIG. 7A , the damage information (I) automatically identifies the damage The type (I ₁ ) is the crocodile crack (I ₁₃ ) and the damage degree (I ₂ ) is 74%. As shown in FIG. 7B , the damage information (I) automatically identifies the damage type (I ₁ ) as the The repair (I ₁₂ ) and the degree of damage (I ₂ ) are 62%. The effect achieved above is mainly that, as shown in Figure 8, the damage is selected by the region of interest (R) in the photo (Pn) The type (I ₁ ) and the damage degree (I ₂ ), and for training the samples of the damage information (I), the higher the training times, the higher the accuracy.

Based on such a structure, the intelligent locomotive road inspection and detection system disclosed in the present invention integrates the power supply device 11 of the locomotive 10 with the on-board automatic diagnosis system 12 , the lens 20 and the mobile device 30 by the adapter 40 . , and cooperate with the artificial intelligence identification algorithm (AI), and the lens 20 is an IP lens, cooperate with the artificial intelligence identification algorithm (AI) to identify whether the photo (P) is the damaged information (I) at the back end, Or the lens 20 is an industrial lens, cooperate with the artificial intelligence recognition algorithm (AI) to identify whether the photo (P) is the damage information (I) at the front end, and solve the problem of the traditional road detection system inspection vehicles in large models, large vehicles, etc. The problem is that there are many equipment, high cost, and a single type of digital camera, which has the effect of reducing volume, simplifying equipment, reducing cost and scalability. Deep learning (convolutional neural network) technology is used for identification and machine vision. Technology for destroying circle selection, using instrument control technology for distance sensing Technical features such as external triggering, GPS positioning and conversion of urban addresses.

To sum up, the technical means disclosed in the present invention do meet the requirements for invention patents such as "novelty", "progressiveness" and "industrial utilization". Moral sense.

However, the drawings and descriptions disclosed above are only preferred embodiments of the present invention, and modifications or equivalent changes made by those skilled in the art according to the spirit of the present case should still be included in the scope of the patent application of the present case.

10: Locomotive

11: Power supply device

12: On-board automatic diagnosis system

20: Shots

30: Mobile Devices

31: Screen

40: Adapter

P: road image

W: line

Claims (9)

An intelligent locomotive road inspection and detection system, comprising: a locomotive, the locomotive has a power supply device and an on-board automatic diagnosis system electrically connected to the power supply device, so that the power supply device can supply power to the on-board automatic diagnosis system, and the on-board automatic diagnosis system The automatic diagnosis system detects the mileage record of the locomotive; a lens, which is installed on the locomotive, is used to capture a road image, the lens is an IP lens, and the IP lens is temporarily stored in the locomotive after the disconnection The photo of the mobile device is uploaded through at least 4G, and the mobile device is connected to a cloud. A backend of the cloud uses the artificial intelligence identification algorithm to identify whether the photo is the damage information, and transmit the damage information to a cloud. platform; a mobile device, which is installed on the locomotive and has a screen; an adapter, which can electrically combine the power supply device with the on-board automatic diagnosis system, the lens and the mobile device , so that the power supply device can provide the mobile device for charging, the road image captured by the lens is displayed on the screen, and the on-board automatic diagnosis system transmits the mileage record of the locomotive to the mobile device; and an artificial intelligence recognition an algorithm, the artificial intelligence identification algorithm can decompose the road image into continuous photos, and retrieve the photos to identify whether there is a damage information. An intelligent locomotive road inspection and detection system, comprising: a locomotive, the locomotive has a power supply device and an on-board automatic diagnosis system electrically connected to the power supply device, so that the power supply device can supply power to the on-board automatic diagnosis system, and the on-board automatic diagnosis system The automatic diagnosis system detects the mileage record of the locomotive; a lens, which is installed on the locomotive, is used to capture a road image, the lens is an industrial lens, and the mobile device calculates each mileage record by the locomotive Move d meters to send and receive a trigger signal, and cooperate with the mobile device to use the artificial intelligence recognition algorithm to instantly identify whether the photo is the sabotage information, and the photo has the sabotage information uploaded through at least 4G, and connected with the mobile device to a cloud terminal, and transmit the damage information to a platform; a mobile device, which is installed on the locomotive and has a screen; an adapter, which can be electrically combined with the power supply device and the vehicle An automatic diagnosis system, the lens and the mobile device, so that the power supply device can provide the mobile device for charging, the road image captured by the lens is displayed on the screen, and the on-board automatic diagnosis system transmits the mileage record of the locomotive to the mobile device; and an artificial intelligence identification algorithm, the artificial intelligence identification algorithm can decompose the road image into continuous photos, and retrieve the photos to identify whether there is a damage information. The intelligent locomotive road inspection and detection system according to claim 1 or 2, wherein the power supply device is one of a mobile power supply or a battery. The intelligent locomotive road inspection and detection system according to claim 1 or 2, wherein the mobile device has a built-in GPS, uploads the GPS track every one second, and the GPS positioning converts the urban address. The intelligent locomotive road inspection and detection system as claimed in claim 1 or 2, wherein the artificial intelligence recognition algorithm comprises an image training model, and the image training model selects the damage information on the photo with a region of interest, After the image training model learns the samples of the damage information for training, the damage information can be identified. The intelligent locomotive road inspection and detection system according to claim 5, wherein the damage information includes a damage type, and the damage type is one of potholes, patches, crocodile cracks, vertical and horizontal cracks, hand holes or bumps constituted. The intelligent locomotive road inspection and detection system according to claim 6, wherein the damage information includes a damage degree, and when the damage type is the pothole, the severity judgment criterion of the damage degree is diameter; For the patch, the judging criterion for the severity of the damage degree is whether it is a square or a polygon; when the damage type is the crocodile-shaped crack, the severity judgment criterion for the damage degree is the crack width; when the damage type is the vertical and horizontal cracks, the severity of the damage The judging criteria are the crack width and the crack length; when the damage type is the hand hole, the judging criteria for the severity of the damage degree is the recording location. The intelligent locomotive road inspection and detection system according to claim 7, wherein the diameter of the pothole is less than 10 centimeters as light, 10-20 centimeters is medium, and over 20 centimeters is heavy; Grade, square but uneven is intermediate, irregular shape is heavy; the crack width of the crocodile-shaped crack is less than 6mm is light, 6mm~20mm is intermediate, and more than 20mm is heavy; the crack width of the vertical and horizontal cracks is less than 10mm The crack length is not limited to light, the crack width is more than 10mm and the crack length is less than 7.5m, and the crack width is more than 10mm and the crack length is more than 7.5m. Heavy. The intelligent vehicle road inspection and detection system according to claim 5, wherein the artificial intelligence identification algorithm includes an opencv database, and the opencv database cooperates with the image training model.

Images