KR102752939B1 - Method and system for providing route of unmanned air vehicle - Google Patents

Abstract

A method and system for constructing an unmanned aerial vehicle route are presented. The method for constructing an unmanned aerial vehicle route may include: a step of identifying a subject from ground scanning data and shaping a space in which autonomous flight is possible into layers; a step of collecting ground image data for a flight path from the shaped layers; and a step of analyzing a change in image resolution according to a distance from the subject through the collected ground image data to extract an altitude value on the flight path.

Description

{METHOD AND SYSTEM FOR PROVIDING ROUTE OF UNMANNED AIR VEHICLE}

The following examples relate to a method and system for constructing an unmanned aerial vehicle route, and more specifically, to a method and system for constructing an unmanned aerial vehicle route that provides an autonomous flight route beyond visual line of sight.

Indiscriminate flights of unmanned aircraft such as drones are taking place in free flight zones below the minimum flight altitude (the minimum altitude for avoiding collisions with obstacles on the ground) of manned aircraft. As a result, the need for safety and security regulations regarding the flights of unmanned aircraft has recently become an issue, including collisions between passenger aircraft and drones, accidents due to intrusions into military security zones, and collisions between manned fire helicopters and unmanned camera devices. Accordingly, the International Civil Aviation Organization (ICAO) is reviewing regulatory measures to protect no-fly zones and maintain a safe distance (horizontal separation and vertical separation) between aircraft in airspace below the minimum flight altitude.

Current safety regulations for unmanned aerial vehicles require that a qualified pilot operate the unmanned aerial vehicle within the range where the pilot can visually identify the unmanned aerial vehicle, i.e. within the pilot’s visual line of sight. However, if the use of unmanned aerial vehicles expands to areas such as densely populated residential areas, disaster prevention, and crime prevention, situation control is required for areas outside the visual line of sight (e.g., night, fog, smoke, urban shadows (blind spots), etc.) and areas that cannot be recognized (e.g., military security and airport areas, etc.).

In particular, in the case of unmanned aerial vehicles, there are technical limitations in utilizing the cognitive ability of the pilot's five senses during flight. Therefore, unlike manned aircraft, the risk of accidents due to situational awareness problems is relatively high. The current safety regulations reflect these problems, but as the industrial demand and complexity of unmanned aerial vehicle utilization increase, the safety of autonomous flight for non-visual flight of the pilot must first be systematically secured and verified.

Korean Patent Publication No. 10-2013-0002492 describes technology regarding a flight control system for such unmanned aerial vehicles.

The embodiments describe a method and system for constructing an unmanned aerial vehicle route, and more specifically, provide a method and system for constructing an unmanned aerial vehicle route that provides an autonomous flight route beyond visual line of sight.

The embodiments provide a method and system for constructing an unmanned aerial vehicle route, which constructs a safe autonomous flight route for an unmanned aerial vehicle by extracting elevation and obstacle height information using scanning data, analyzing changes in image resolution of ground image data, and utilizing the extracted ground object height information to verify calibration and correct measurements of a radio altitude sensor of the unmanned aerial vehicle.

A method for constructing a flight path for an unmanned aerial vehicle according to one embodiment comprises: a step of identifying a subject from ground scanning data and shaping a space in which autonomous flight is possible into a layer; a step of collecting ground image data for a flight path from the shaped layer; and a step of analyzing a change in image resolution according to a distance from the subject through the collected ground image data to extract an altitude value on the flight path.

The method may further include a step of correcting the measurement value of the radio altitude sensor through route verification from the extracted altitude value.

The step of shaping the space in which autonomous flight is possible into a layer may include the step of obtaining a point cloud of the subject scanned by a ground scanning device mounted on a ground photographing aircraft; the step of analyzing the collected point cloud to identify the subject; the step of extracting a height value of a specific point of the identified subject using terrain altitude data; and the step of shaping an area and altitude in which autonomous flight of an unmanned aerial vehicle is possible into the layer by connecting the extracted height values of the specific points of the subject.

The step of obtaining the above point cluster can obtain the point cluster of the subject on which the LiDAR pulse is projected through a LiDAR device mounted on the above-mentioned ground photographing aircraft.

The step of shaping the space in which autonomous flight is possible into layers may include a step of creating a plurality of two-dimensional layers in the space.

The step of collecting the above-mentioned ground image data may include a step of acquiring the above-mentioned ground image data through a photographing device with a calibration value set at a specific altitude mounted on the above-mentioned ground photographing aircraft.

The step of collecting the above-mentioned ground image data may include a step of checking spatial geographic information to search for a safe path for flight; and a step of generating a flight path reflecting the safe path and collecting the above-mentioned ground image data for the flight path.

The step of collecting the above-mentioned index image data may further include a step of setting a flight altitude limit value and verifying the measurement value of the radio altitude sensor through a subject capable of verifying the flight altitude limit height.

The step of collecting the above-mentioned index image data may include a step of checking the calibration information of the photographing device and checking the flight information recorded in the flight data recorder (FDR) mounted on the unmanned aerial vehicle.

The step of extracting the altitude value on the above flight path can be performed by matching at least one of coordinate, altitude, attitude, and time information from a flight data recorder (FDR) mounted on the unmanned aerial vehicle with the photographed ground image data, and by referring to the calibration information of the photographing device, calculating the altitude value on the flight path through image distortion correction and analysis of the change in the image resolution.

The step of correcting the measurement value of the radio altitude sensor may include the step of extracting an altitude value from a subject existing in the route and substituting it into the route coordinates of the unmanned aerial vehicle at a predetermined interval, and recognizing the resolution height of an image corresponding to the coordinates at which the unmanned aerial vehicle contacts the subject when the unmanned aerial vehicle reaches the route coordinates; and the step of correcting the measurement value of the radio altitude sensor of the unmanned aerial vehicle according to the resolution height.

The step of correcting the measurement value of the above radio altitude sensor can support an offline image processing method to minimize risks to the communication and aircraft infrastructure environment during autonomous flight.

The step of correcting the measurement value of the above radio altitude sensor can be performed by repeatedly collecting the above ground image data through autonomous flight of an unmanned aerial vehicle, and reflecting the collected above ground image data in route control and ground control and route map data through resolution change analysis, thereby creating or verifying a new route.

In another embodiment, a system for constructing a flight path for an unmanned aerial vehicle includes: a layer shaping unit for identifying a subject from ground scanning data and shaping a space in which autonomous flight is possible into layers; a data collecting unit for collecting ground image data for a flight path from the shaping layer; and an altitude calculating unit for analyzing a change in image resolution according to a distance from the subject through the collected ground image data and extracting an altitude value of a flight path coordinate.

The method may further include a verification unit that corrects the measurement value of the radio altitude sensor through route verification from the extracted altitude value.

The above layer shaping unit may include a collection unit that obtains a point cloud of the subject scanned by a ground scanning device mounted on a ground photographing aircraft; an identification unit that analyzes the collected point cloud to identify the subject; an extraction unit that extracts a height value of a specific point of the identified subject using terrain elevation data; and a layer unit that connects the height values of the extracted specific points of the subject to shape an area and altitude in space where autonomous flight of the unmanned aerial vehicle is possible as the layer.

The above data collection unit can check spatial geographic information to search for a safe path for flight, generate a flight path reflecting the safe path, collect the surface image data for the flight path, and obtain the surface image data through a photographing device with a calibration value set at a specific altitude mounted on the surface photographing aircraft.

The above data collection unit can set a flight altitude limit value and check the measurement value of the radio altitude sensor through a subject capable of verifying the flight altitude limit height.

The above data collection unit verifies the calibration information of the photographing device and verifies the flight information recorded in the flight data recorder (FDR) mounted on the unmanned aerial vehicle, and the altitude calculation unit aligns at least one of the coordinate, altitude, attitude, and time information from the flight data recorder (FDR) mounted on the unmanned aerial vehicle with the photographed ground image data, and calculates the altitude value on the flight path by correcting the distortion of the image and analyzing the change in the image resolution with reference to the calibration information of the photographing device.

The above verification unit extracts an altitude value from a subject existing in the route and substitutes it into the route coordinates of the unmanned aerial vehicle at regular intervals, and when the unmanned aerial vehicle reaches the route coordinates, recognizes the resolution height of the image corresponding to the coordinates at which it makes contact with the subject, and - can correct the measurement value of the radio altitude sensor of the unmanned aerial vehicle according to the resolution height.

The above verification unit can support offline image processing to minimize risks to communication and aircraft infrastructure environments during autonomous flight.

The above verification unit repeatedly collects the above-mentioned ground image data through autonomous flight of an unmanned aerial vehicle, and reflects the collected above-mentioned ground image data in route control and ground control and route map data through resolution change analysis, and can create or verify a new route.

According to another embodiment, a method for constructing a flight path for an unmanned aerial vehicle may include: a step of identifying a subject from ground scanning data and forming a space in which an autonomous flight of an unmanned aerial vehicle is possible into a layer; a step of determining a waypoint for creating a flight path for the unmanned aerial vehicle on the formed layer; a step of collecting ground image data for the waypoint from the formed layer; a step of analyzing a change in image resolution according to a distance from the subject through the collected ground image data and extracting altitude values for each waypoint; and a step of generating flight path information for the unmanned aerial vehicle including at least one or more of flight path information from the formed layer, the waypoints, the altitude values, and the flight paths that are connecting lines between the waypoints.

Here, the waypoint may represent a location of a ground object existing on the surface of the earth at a point where the unmanned aerial vehicle performs autonomous flight on the layer, or may represent a location where a predetermined mission is performed.

The step of generating flight path information of the above unmanned aerial vehicle may include a step of determining an arrival layer to which the unmanned aerial vehicle is scheduled to move when movement from a departure layer, which is a layer to which the unmanned aerial vehicle is initially assigned, to another layer; and a step of generating layer movement information for moving from the departure layer to the arrival layer.

The above layer movement information may include at least one of a layer changeable section including a waypoint section for layer change among the route for autonomous flight of the unmanned aerial vehicle, a layer movement time, a change section entry time, and a change section entry angle.

According to embodiments, by providing an autonomous flight route beyond the visual line of sight, it is possible to overcome the limitations of operation within the pilot's visual range in areas where it is difficult to maintain a constant altitude value due to ground objects, etc.

According to embodiments, a method and system for constructing an unmanned aerial vehicle route for constructing a safe autonomous flight route of an unmanned aerial vehicle can be provided by extracting elevation and obstacle height information using scanning data, analyzing changes in image resolution of ground image data, and utilizing the extracted ground object height information to verify calibration and correct measurements of a radio altitude sensor of an unmanned aerial vehicle.

Figure 1 is a drawing for explaining the flight altitude restrictions for operating an unmanned aerial vehicle.
FIG. 2 is a drawing for explaining a sensor unit of an unmanned aerial vehicle according to one embodiment.
FIG. 3 is a flowchart illustrating a map creation method for unmanned aerial vehicle flight according to one embodiment.
FIG. 4 is a block diagram illustrating a map creation system for unmanned aerial vehicle flight according to one embodiment.
FIGS. 5 and 6 are flowcharts showing a method for establishing an unmanned aerial vehicle route according to one embodiment.
FIG. 7 is a block diagram showing an unmanned aerial vehicle route construction system according to one embodiment.
FIGS. 8 to 10 are diagrams for explaining autonomous flight space shaping from surface scanning and image capturing data according to one embodiment.
FIG. 11 is a diagram for explaining alignment of geospatial data according to one embodiment.
FIG. 12 is a diagram for explaining a method of creating a map by aligning geospatial data according to one embodiment.
FIG. 13 is a diagram illustrating the collection of point clusters through laser scanning according to one embodiment.
FIG. 14 is a drawing for explaining a layer having a specific height in a three-dimensional space according to one embodiment.
FIG. 15 is a drawing for explaining the change in image resolution according to the distance from the subject according to one embodiment.
FIGS. 16 to 19 are diagrams for explaining a flight control and ground control process through image recognition and processing of an unmanned aerial vehicle according to one embodiment.
Figure 20 is a diagram showing a simulation of a constructed route according to one embodiment.
Figure 21 is a drawing showing a gas recognition and route control shape according to one embodiment.
FIG. 22 is a drawing for explaining a process of extracting the height of a specific point on an object scanned by LiDAR according to one embodiment.
Figure 23 is a diagram for explaining the DSM and DTM used in one embodiment.
FIG. 24 is a drawing for explaining a method of setting a waypoint of a ground object according to one embodiment.
FIG. 25 is a diagram for explaining a process of adding a waypoint within a waypoint valid range of a ground object according to one embodiment.
Figure 26 is a flowchart showing the operation of an unmanned aerial vehicle according to one embodiment.
Figure 27 is a flowchart showing the operation of an unmanned aerial vehicle according to another embodiment.
Figure 28 is a flowchart showing an operation method of a route construction system and a control system for autonomous flight of an unmanned aerial vehicle according to one embodiment.
FIG. 29 is a drawing explaining how, when a ground object is present while an unmanned aerial vehicle is flying along a predetermined route according to one embodiment, the flight altitude is maintained within a preset layer range by using a resolution height for the ground object.
Figure 30 is a block diagram of an unmanned aerial vehicle according to another embodiment.
Figure 31 is a flowchart showing an operating method of an unmanned aerial vehicle operating system according to another embodiment.
Figure 32 is a flowchart showing an operating method of an unmanned aerial vehicle operating system according to another embodiment.
Figure 33 is a flowchart showing an unmanned aerial vehicle operation method of an operating system according to another embodiment.
Figure 34 is a flowchart showing the operation of an unmanned aerial vehicle according to another embodiment.
Fig. 35 is a flowchart showing an unmanned aerial vehicle control method of a control system according to another embodiment.
Figure 36 is a flowchart showing an unmanned aerial vehicle operation method of an operating system according to another embodiment.
Figure 37 is a block diagram of an unmanned aerial vehicle according to another embodiment.
FIG. 38 is a block diagram illustrating an unmanned aerial vehicle, an operating system, and a control system according to another embodiment.
FIG. 39 is a flowchart illustrating a method for generating flight path information of an unmanned aerial vehicle for autonomous flight between layers of an unmanned aerial vehicle according to one embodiment.
FIG. 40 is a flowchart illustrating a method for generating flight path information of an unmanned aerial vehicle for autonomous flight between layers of an unmanned aerial vehicle according to another embodiment.
FIG. 41 is a block diagram showing an unmanned aerial vehicle route construction system that constructs a route for inter-layer movement of an unmanned aerial vehicle according to another embodiment.
FIG. 42 is a flowchart illustrating a method of operating an unmanned aerial vehicle for moving between layers according to one embodiment.
Fig. 43 is a block diagram of a route construction system for constructing a route for an unmanned aerial vehicle to move between layers according to another embodiment.
FIG. 44 is a drawing for explaining a procedure for an unmanned aerial vehicle to perform autonomous flight between layers according to one embodiment.
FIG. 45 is a drawing illustrating a layer changeable section set for an unmanned aerial vehicle to move between layers according to one embodiment.
FIG. 46 is a vertical cross-sectional view illustrating a procedure for an unmanned aerial vehicle to move between layers according to one embodiment.
FIG. 47 is a drawing illustrating a procedure for an unmanned aerial vehicle to move between layers according to another embodiment.
FIG. 48 is a drawing illustrating a procedure for an unmanned aerial vehicle to move between layers according to another embodiment.
FIG. 49 is a method flow diagram of an unmanned aerial vehicle and a control system for layer movement of an unmanned aerial vehicle according to another embodiment.
Figure 50 is a flowchart illustrating a method for controlling an unmanned aerial vehicle according to one embodiment.
Figure 51 is a block diagram showing an unmanned aerial vehicle control system according to one embodiment.
Figure 52 is a block diagram showing an unmanned aerial vehicle control system according to another embodiment.

Hereinafter, embodiments will be described with reference to the attached drawings. However, the described embodiments may be modified in various other forms, and the scope of the present invention is not limited by the embodiments described below. In addition, various embodiments are provided to more completely explain the present invention to a person having average knowledge in the relevant technical field. The shapes and sizes of elements in the drawings may be exaggerated for a clearer description.

Figure 1 is a drawing for explaining the flight altitude restrictions for operating an unmanned aerial vehicle.

According to the Drone Highway concept of NASA's Unmanned Aerial System Traffic Management (UASTM) plan, the Drone Highway is a concept regarding drone utilization (e.g., delivery service) and control (e.g., building a system to ensure safety such as collision avoidance) with the participation of approximately 120 related organizations and companies in the U.S., including Amazon, Google, NASA, and the Federal Aviation Administration (FAA).

The no-fly zone (110) is formed in the range of 400 ft to 500 ft in rural, suburban, and urban areas, but the entire altitude range near the airport is included in the no-fly zone due to the takeoff and landing of manned aircraft. According to one embodiment, unmanned aerial vehicles are also prohibited from flying in the no-fly zone, so they cannot fly higher than 400 ft in rural, suburban, and urban areas. In addition, the zone can be divided into a high-speed flight zone (120) and a low-speed flight zone (130) based on the mission performed by the unmanned aerial vehicle. For example, a company providing logistics services such as Amazon will use the range of the high-speed flight zone for quick delivery service, and the range of the low-speed flight zone for agriculture, facility inspection, and photography.

In the examples below, the vertical separation for each range can be conceptualized as a “layer,” and separations such as a high-speed flight zone (120) and a low-speed flight zone (130) are simply distinguished by the characteristics of the mission execution of the unmanned aerial vehicle (a mission to deliver at high speed or a mission to inspect a facility slowly at low speed). In reality, more layers can be created to reflect more diverse mission execution characteristics of unmanned aerial vehicles.

Autonomous flight of unmanned aerial vehicles requires altitude (Z) information in addition to location coordinates (X, Y). Existing autonomous flight technology of unmanned aerial vehicles inputs the altitude (Z) value before flight and maintains the altitude (Z) value measured by ultrasonic sensors using the principle of echo of radio waves. For example, existing autonomous flight systems are used for the purpose of spraying pesticides on farmland with a constant ground height, targeting farmers who are not familiar with operating unmanned aerial vehicles. However, in order to overcome the limitations of safety regulations (operation within the pilot's visual range) due to changes in industrial demand, supplementary measures are needed for areas where it is difficult to maintain a constant altitude (Z) value due to ground objects, etc. Here, ground objects basically include the ground, and may include structures and obstacles formed or connected from the ground. Since radio altitude sensors operate on the principle of echo of the subject, they maintain the altitude (Z) value relative to the subject. That is, if an altitude (Z) value of 150 meters is input to an unmanned aerial vehicle, an altitude of 150 meters from the sea level is continuously maintained, but if there is a large ground object with a height of 50 meters in the middle of the route, the flight altitude of the unmanned aerial vehicle is maintained at 200 meters within the range of the ground object. Therefore, if there is a limit to the flight altitude, a flight that depends on a radio sensor that measures the altitude by the principle of wave echo may ultimately violate the flight altitude restrictions for safety regulations. In particular, since the flight altitude restriction as illustrated in Fig. 1 is a safety measure to maintain a safe distance (vertical separation) from the lowest flight altitude of a manned aircraft, violating it may lead to a risk of an aircraft collision.

Accordingly, in order to ensure safe autonomous flight of an unmanned aerial vehicle, the absolute altitude (Z) value from the ground (i.e., flight altitude limit) must be maintained, and there must be compensation for maintaining the absolute altitude (Z) value for ground objects in the middle of the route, and an avoidance route must be provided for ground objects that are difficult to vertically separate from the absolute altitude (Z) value.

FIG. 2 is a drawing for explaining a sensor unit of an unmanned aerial vehicle according to one embodiment.

Referring to FIG. 2, the sensor unit (200) of the unmanned aerial vehicle according to one embodiment may include an attitude control unit (210), a fail-safe unit (220), and a position measurement unit (230). In addition, it may include a wireless communication sensor, a sensor for image capture, and a laser scan sensor.

The attitude control unit (210) detects the rotation angle of the aircraft to control the attitude, and for example, a gyro sensor, a geomagnetic sensor, an accelerator, etc. can be used.

The fail-safe unit (220) is for flight errors (Falt Safe), and for example, a barometric altimeter (radio altitude sensor), ultrasonic waves, radar, voltage, current measuring meters, etc. can be used.

Meanwhile, since the radio altitude sensor operates on the principle of echo of the subject, it maintains the altitude (Z) value relative to the subject. Therefore, if there is a limit to the flight altitude, a flight dependent on a radio sensor that measures the altitude by the principle of echo of the wave may end up violating the flight altitude limit for safety regulations. Accordingly, in order to ensure safe autonomous flight of an unmanned aerial vehicle, the absolute altitude (Z) value from the ground (i.e., the flight altitude limit) must be maintained, and there must be compensation for maintaining the absolute altitude (Z) value for ground objects in the middle of the route, and an avoidance route must be provided for ground objects that are difficult to vertically separate from the absolute altitude (Z) value.

The position measuring unit (230) is a sensor that detects the position of an unmanned aerial vehicle, and for example, a GPS (Global Positioning System) sensor can be used.

On the other hand, when measuring altitude using a GPS sensor, the error range must be reduced through surrounding infrastructure, assuming the limitation of calculating altitude (Z) values. However, GPS altitude measurement is primarily affected by the (geometric) arrangement of GPS satellites and is secondarily affected by ground obstacles and terrain, so altitude (Z) value calculation may be impossible or errors may occur even at the same point.

In one embodiment, to extract the altitude (Z) value on the route through an actual flight, first, a point cloud of the subject extracted by LiDAR scanning necessary for constructing a 2D layer in a three-dimensional space can be analyzed.

The first layer obtained by connecting the height values of specific points of the subject through analysis of point clusters extracted from radio waves or light echoes cannot exclude errors caused by radio interference or distortions (e.g., radio shadows) caused by the material of the subject and the angle of incidence. Therefore, a safer autonomous flight route can be established through verification and correction of the extracted values.

FIG. 3 is a flowchart illustrating a map creation method for unmanned aerial vehicle flight according to one embodiment.

Referring to FIG. 3, a method for creating a map for an unmanned aerial vehicle flight according to one embodiment may include a step (310) of identifying a subject from ground scanning data and forming a space in which autonomous flight is possible into a layer, and a step (320) of constructing an autonomous navigation map for the flight of an unmanned aerial vehicle in the space by aligning at least one of flight altitude restriction data, a precision numerical map, and route information for avoiding a military security area or a no-fly zone to the layer formed in the space. Here, the layer may mean a two-dimensional space formed by applying an altitude value (height value) to a three-dimensional space.

Here, the step of shaping a space in which autonomous flight is possible into a layer may include a step of acquiring a point cloud of a subject scanned by a ground scanning device mounted on a ground photographing aircraft, a step of analyzing the collected point cloud to identify the subject, a step of extracting a height value of a specific point of the identified subject using terrain elevation data, and a step of shaping an area and altitude in which autonomous flight of an unmanned aerial vehicle is possible into a layer by connecting the height values of the specific points of the extracted subject.

In addition, the step (330) of synchronizing an autonomous navigation map for the flight of an unmanned aerial vehicle built on a layer with the unmanned aerial vehicle within a preset safety standard through information from a GPS or position coordinate correction device and shaping it into a spatial map applicable to the unmanned aerial vehicle can be further included.

According to some embodiments, by providing a non-visual autonomous flight map, it is possible to overcome the limitations of operating within a pilot's visual range in areas where it is difficult to maintain a constant altitude value due to ground objects, etc.

In addition, a method and system for producing a map for unmanned aerial vehicle flight can be provided by shaping an autonomous flight space into layers from surface scanning and image capturing data and aligning data to the shaped layers to reflect altitude values.

According to another aspect, a method for creating a map for an unmanned aerial vehicle flight may include a step of setting a layer having a certain altitude value from the ground surface over which the unmanned aerial vehicle can fly according to a mission for the unmanned aerial vehicle, a step of setting a route for the unmanned aerial vehicle on the set layer, and a step of constructing an autonomous navigation map including the set layer and the route. Here, the route may be composed of at least two waypoints including the positions of ground objects existing on the ground surface of the route. And the method may further include a step of constructing an autonomous navigation map for each mission according to identification information of the unmanned aerial vehicle. A waypoint is a point where the assigned mission can be performed by the unmanned aerial vehicle. Accordingly, an autonomous flight map of a non-visible area can be provided to overcome the limitations of operation within the pilot's visible range in an area where it is difficult to maintain a constant altitude value due to ground objects, etc. In addition, a method and system for creating a map for an unmanned aerial vehicle flight in which an autonomous flight space is visualized as a layer from ground scanning and image capturing data and data is aligned to the visualized layer can be provided.

Below, each step of the map creation method for unmanned aerial vehicle flight according to one embodiment is described in more detail.

Fig. 4 is a block diagram showing a map production system for unmanned aerial vehicle flight according to one embodiment. As shown in Fig. 4, the map production system for unmanned aerial vehicle flight according to one embodiment may include a layer shaping unit (410), an autonomous navigation map unit (420), and a space map unit (430). Each component of the map production system for unmanned aerial vehicle flight may be a processor included in a server.

These components can be implemented to execute steps (310 to 330) included in the method of FIG. 3 through an operating system including a memory and at least one program code.

In step (310), the layer shaping unit (410) can identify a subject from ground scanning data and shape a space in which autonomous flight is possible into a layer. Here, the layer can be expressed as a plane including a height concept.

The layer shaping unit (410) can create a number of two-dimensional layers in space, and the layers can form a vertical separation.

Here, the layer shaping unit (410) may include a collection unit, an identification unit, an extraction unit, and a layer unit.

The collection unit of the layer shaping unit (410) can obtain a point cloud of a subject scanned by a ground scanning device mounted on a ground photographing aircraft. At this time, the obtained point cloud of the subject can be used to extract the height of a specific point of a building. The height at this time can be the top height of the building or the height of a specific point of the building, such as the middle height of the building. According to one embodiment, a method of extracting the height of a specific point of a building from a point cloud of a scanned subject will be described with reference to FIG. 22.

FIG. 22 is a drawing for explaining a process of extracting the height of a specific point in a subject scanned by LiDAR according to one embodiment. FIG. 22 (a) is an image of an actual subject, and (b) shows a point cluster of the subject scanned by a scanning device such as LiDAR equipment. At this time, a color spectrum (2200) that can refer to the height at each point of the subject can be shown. According to one embodiment, the collection unit of the layer shaping unit (410) can extract the height of a specific point of the scanned subject by referring to the color spectrum.

In one embodiment, when extracting the height of a specific point from a scanned subject using a point cloud, the height spectrum value of the color spectrum (2200) can be used. However, since LiDAR scans the subject in a pulse manner of laser light, problems in recognizing light dispersion, boundaries, and breaklines depending on the material of the subject may occur, and the extraction result for the subject height value may vary depending on the algorithm of the software tool used to analyze the color spectrum (2200). Therefore, in one embodiment, for a layer initially set by a point cloud, which is LiDAR data, an error in the layer can be corrected through calibration of an optical imaging device by verifying the height of the layer by an initial flight - a flight within the pilot's line of sight.

For example, the collection unit of the layer shaping unit (410) can obtain a point cluster of an object on which a LiDAR pulse is projected through a LiDAR device mounted on a ground photographing aircraft.

The identification unit of the layer shaping unit (410) can identify a subject by analyzing a point cluster collected from the collection unit. At this time, the identification unit of the layer shaping unit (410) can recognize the boundary or outline of a ground object through the point cluster, and can identify the ground object identified through this as a bridge, building, wire, etc.

The extraction unit of the layer shaping unit (410) can extract the height value of a specific point of the subject identified in the identification unit by utilizing a digital surface model (DSM) or a digital terrain model (DTM) among the terrain elevation data. The DSM data and the DTM data can be acquired from a government agency (for example, the National Geographic Information Institute in the case of Korea) or an aerial surveying company that is constructing a database of geographic information of each country. FIG. 23 is a drawing for explaining the DSM and DTM used in one embodiment, and as illustrated in FIG. 23, the DSM can be a height value of a ground object and the DTM can be a height value (elevation) of the terrain.

The layer section of the layer shaping section (410) can shape the area and altitude in space where an unmanned aerial vehicle can fly autonomously as a layer by connecting the height values of specific points of the subject extracted from the extraction section.

A method of identifying such subjects and shaping an autonomous flight space will be described below with examples referring to FIGS. 8 to 10.

In step (320), the autonomous navigation guidance unit (420) can build an autonomous navigation map for the flight of an unmanned aerial vehicle in space by aligning at least one of flight altitude restriction data, a precision numerical map, and route information for avoiding military security areas or no-fly zones to a layer visualized in space.

In step (330), the space management unit (430) can synchronize the autonomous navigation map for the flight of the unmanned aerial vehicle built in the layer with the unmanned aerial vehicle within the preset safety standards through information from a GPS or position coordinate correction device, and shape it into a space map applicable to the unmanned aerial vehicle.

The space map (430) can align GPS coordinates with an autonomous navigation map for the flight of an unmanned aerial vehicle built in a layer, and process the altitude value of an image of a ground object from the autonomous navigation map for the flight of an unmanned aerial vehicle to correct the sensor measured altitude value.

That is, the space map (430) aligns GPS coordinates with an autonomous navigation map for the flight of an unmanned aerial vehicle built on a layer, analyzes changes in the resolution of ground objects by the set (calibrated with respect to the ground) incidence angle of an imaging device (e.g., various optical-based imaging devices that can be mounted) mounted on the unmanned aerial vehicle based on the GPS coordinates aligned from the autonomous navigation map, and corrects the altitude measurement value of an altitude measurement device that uses the principle of echo, such as ultrasonic waves, by aligning the height value of the resolution extracted through the analysis of the change in resolution with the GPS coordinates.

The method of aligning such geospatial data and constructing a map will be explained in more detail with examples below, referring to Figures 11 and 12.

In another embodiment, a method for producing a map for an unmanned aerial vehicle flight on a three-dimensional precision map may be provided, the method including the steps of forming a plurality of layers in a three-dimensional precision map to form vertical separations, and the steps of shaping symbols representing routes formed at separation intervals of the vertical separations and collected waypoints formed in the layers.

Here, the step of forming a plurality of layers to form a vertical separation may include the step of acquiring a point cloud of a subject scanned by a ground scanning device mounted on a ground photographing aircraft, the step of analyzing the collected point cloud to identify the subject, the step of extracting a height value of a specific point of the identified subject using terrain elevation data, and the step of connecting the height values of the specific points of the extracted subject to form an area and altitude in space where autonomous flight of the unmanned aerial vehicle is possible as a layer.

A method for producing a map for an unmanned aerial vehicle flight on a three-dimensional precision map according to another embodiment can be described more specifically using a map producing system for an unmanned aerial vehicle flight on a three-dimensional precision map according to another embodiment. Here, the map producing system for an unmanned aerial vehicle flight on a three-dimensional precision map can include a layer shaping unit and a route and symbol shaping unit. Each component of the map producing system for an unmanned aerial vehicle flight on a three-dimensional precision map can be a processor included in a server.

The layer shaping unit (410) can form multiple layers by vertically separating them on a three-dimensional precision map. At this time, the three-dimensional precision map can be produced directly by using an existing three-dimensional precision map or by collecting data.

The layer shaping unit (410) may include a collection unit, an identification unit, an extraction unit, and a layer unit, as described in FIG. 4.

The collection unit of the layer shaping unit (410) can obtain a point cloud of a subject scanned by a ground scanning device mounted on a ground photographing aircraft. For example, the collection unit of the layer shaping unit can obtain a point cloud of a subject projected with a LiDAR pulse by a LiDAR device mounted on a ground photographing aircraft.

The identification unit of the layer shaping unit (410) can identify a subject by analyzing a point cluster collected from the collection unit.

The extraction unit of the layer shaping unit (410) can extract the height value of a specific point of an object identified in the identification unit by utilizing a digital surface model (DSM) or a digital terrain model (DTM) among the terrain elevation data.

The layer section of the layer shaping section (410) can shape the area and altitude where an unmanned aerial vehicle can autonomously fly in space as a layer by connecting the height values of specific points of the subject extracted from the extraction section.

Here, a layer may contain at least one of the following information: formation altitude, performable missions, and aircraft specifications.

And the symbol of the route formed in the layer includes the position coordinates and the altitude value of the image based on the layer for the corresponding coordinates, and the altitude value of the image may be a value that must be corrected for the measurement value by the altitude measuring sensor to maintain the formation altitude of the layer in which the unmanned aerial vehicle is flying autonomously.

FIGS. 5 and 6 are flowcharts showing a method for establishing an unmanned aerial vehicle route according to one embodiment.

Referring to FIGS. 5 and 6, a method for constructing an unmanned aerial vehicle route according to an embodiment may include a step (510) of identifying a subject from ground scanning data and shaping a space in which autonomous flight is possible into a layer, a step (520) of collecting ground image data for a flight path from the shaped layer, and a step (530) of analyzing a change in image resolution according to the distance between a camera that scans the ground surface and the subject through the collected ground image data to extract an altitude value on the flight path. Here, the distance between the camera and the subject can be calculated through internal and external parameter values of the camera confirmed through camera calibration. In addition, in an embodiment to which the present invention is applied, it is assumed that the position and direction of the camera at the time of capturing an image of a ground object are known, and therefore the distance between the camera and the subject can be calculated by considering the parameters of the camera described above.

In addition, the camera may include not only a general optical camera having a structure of a light collection unit, a light collection control unit, and an imaging unit capable of analyzing changes in the resolution of a subject, but also other replaceable devices capable of recognizing and recording changes in the resolution of a subject based on parameter values of calibration.

Here, the step (510) of shaping a space in which autonomous flight is possible into layers may include a step (511) of acquiring a point cloud of a subject scanned by a ground scanning device mounted on a ground photographing aircraft, a step (512) of identifying a subject by analyzing the collected point cloud, a step (513) of extracting a height value of a specific point of the identified subject by utilizing terrain elevation data, and a step (514) of shaping an area and altitude in which autonomous flight of an unmanned aerial vehicle is possible into layers by connecting the height values of the specific points of the extracted subject.

In addition, the method for establishing an unmanned aerial vehicle route according to one embodiment may further include a step (540) of correcting a measurement value of a radio altitude sensor through route verification from an extracted altitude value.

According to some embodiments, an autonomous flight route beyond the visual line of sight can be provided to overcome the limitations of operation within the pilot's visual range in areas where it is difficult to maintain a constant altitude value due to ground objects, etc.

In addition, by extracting elevation and obstacle height information using scanning data, analyzing changes in image resolution of ground image data, and utilizing the extracted ground object height information to verify calibration and correct the measurement values of the radio altitude sensor of the unmanned aerial vehicle, a method and system for constructing a safe autonomous flight route for the unmanned aerial vehicle can be provided.

Calibration verification according to one embodiment includes performing verification of whether the distance between the camera lens mounted on the drone and the subject and the focus on the subject are accurate, and correction of the measurement value of the radio altitude sensor of the drone may also include correcting the error range of the radio altitude sensor using the image resolution change value.

And in one embodiment, the correction of the radio altitude sensor measurement value of the unmanned aerial vehicle may be performed for the purpose of verifying camera calibration for the purpose of initial flight settings corresponding to the flight purpose before the flight of the unmanned aerial vehicle, and may also be performed for continuously correcting the radio altitude measurement value of the unmanned aerial vehicle while operating the unmanned aerial vehicle after the initial setting is completed.

In addition, calibration verification for initial setting before the first flight of an unmanned aerial vehicle may be verification of camera calibration based on the measurement value of the radio altitude sensor of the unmanned aerial vehicle on a flat surface, and this will be described in more detail as follows. In one embodiment, when an operator of an unmanned aerial vehicle operation system or an autonomous flight map construction operation company sets the layer height to around 80 m from the elevation, the radio altitude value of the unmanned aerial vehicle before the flight is set to 80 m, and by checking whether the camera mounted on the unmanned aerial vehicle hovering at the corresponding altitude is focused on 80 m, the distance (focal length) from the center of the camera optical lens to the image sensor can be checked. Therefore, it is possible to check whether the subject at an incident angle (angle) from a height of 80 m is focused.

The reason for performing this calibration verification before each flight of the UAV is that the values may be distorted due to extreme vibrations that occur when the UAV takes off, and as a result, the flight altitude recognized by the UAV may be different from the altitude of the set layer.

In addition, according to one embodiment, after the initial setting for the flight of the unmanned aerial vehicle is completed, it may be performed to continuously calibrate the measurement values of the radio altitude sensor of the unmanned aerial vehicle while the unmanned aerial vehicle is being operated, which will be described in more detail as follows. The value measured from the radio altitude sensor actually mounted on the unmanned aerial vehicle may have a range that exceeds the height of a predetermined layer depending on the flight purpose of the unmanned aerial vehicle, and in this case, there is a risk that the unmanned aerial vehicle may exceed the highest flight limit altitude at which it can fly or may collide with other aircraft. Therefore, in one embodiment, in order to prevent such a problem, the unmanned aerial vehicle may be able to fly while maintaining the height of the layer constant through calibration verification using the image resolution change value of the optical equipment mounted on the unmanned aerial vehicle.

Therefore, in one embodiment, even if an unmanned aerial vehicle suddenly recognizes the presence of a ground object by a radio altitude sensor during flight, the flight altitude height that deviates from the layer by the height of the ground object is adjusted to a predetermined layer height using the resolution height obtained by resolution conversion analysis, so that the unmanned aerial vehicle can fly while maintaining the layer height constant. In particular, in order to do so, the calibration of the camera must first be accurately verified so as to obtain an image capable of accurately analyzing the resolution along the path that the unmanned aerial vehicle is moving, so as to obtain a correct captured image.

Below, each step of the method for constructing an unmanned aerial vehicle route according to one embodiment is described in more detail.

Fig. 7 is a block diagram showing an unmanned aerial vehicle route construction system according to one embodiment. As shown in Fig. 7, an unmanned aerial vehicle route construction system (700) according to one embodiment may include a layer shaping unit (710), a data collection unit (720), an altitude calculation unit (730), and a verification unit (740). Each component of the unmanned aerial vehicle route construction system may be a processor included in a server.

These components can be implemented to execute steps (510 to 540) included in the methods of FIGS. 5 and 6 through an operating system including a memory and at least one program code.

In step (510), the layer shaping unit (710) can identify a subject from ground scanning data and shape a space in which autonomous flight is possible into a layer. Here, the layer can be a plane including a height concept.

By analyzing the point cloud of objects scanned by various ground scanning devices (e.g., Synthetic Aperture Radar (SAR), LiDAR, short-wave infrared sensors, etc.) from ground photographing aircraft, objects such as buildings and bridges can be identified.

The layer shaping unit (710) calculates the height of an object identified from scan data based on the ground altitude of the corresponding coordinates, and can shape a two-dimensional layer in a three-dimensional space by connecting the heights of specific points.

This layer shaping unit (710) can create a plurality of two-dimensional layers in space, and the layers can form a vertical separation.

Here, the layer shaping unit (710) may include a collection unit, an identification unit, an extraction unit, and a layer unit.

The collection unit of the layer shaping unit (710) can obtain a point cloud of an object scanned by a ground scanning device mounted on a ground photographing aircraft. At this time, the height can be extracted according to the height of the building, and the middle height of the building can also be extracted.

For example, the collection unit of the layer shaping unit (710) can obtain a point cluster of an object on which a LiDAR pulse is projected through a LiDAR device mounted on a ground photographing aircraft.

The identification unit of the layer shaping unit (710) can identify a subject by analyzing a point cluster collected from the collection unit.

The extraction unit of the layer shaping unit (710) can extract the height value of a specific point of the subject identified in the identification unit by utilizing terrain elevation data.

The layer section of the layer shaping section (710) can shape the area and altitude where an unmanned aerial vehicle can autonomously fly in space as a layer by connecting the height values of specific points of the subject extracted from the extraction section.

In step (520), the data collection unit (720) can collect ground image data for the flight path from the shaped layer.

At this time, the data collection unit (720) can initially collect ground image data from a layer at a flight altitude limit height.

The data collection unit (720) can acquire ground image data through a photographing device with a calibration value set at a specific altitude mounted on a ground photographing aircraft.

And the data collection unit (720) can check the spatial geographic information to collect ground image data, search for a safe route for flight, generate a specific flight route, and collect ground image data for the corresponding flight route. In particular, the collection of the initial ground image data required for route analysis for route construction can secure maximum safety by only allowing flights within the visual range of a pilot with pilot qualification.

The data collection unit (720) can set the height value of the flight altitude limit and check the measurement value of a radio altitude sensor (e.g., a radio altimeter, etc.) through an object for which the flight altitude limit height can be verified. Here, the object for which the flight altitude limit height can be verified can be a ground structure, etc., that is higher than or equal to the flight altitude limit.

Additionally, the data collection unit (720) can check information such as specifications of the photographing device, such as resolution and image acquisition method, and calibration parameters according to the angle of incidence, and can check the flight information of the aircraft recorded in the flight data recorder (FDR) mounted on the unmanned aerial vehicle.

In step (530), the altitude calculation unit (730) can extract the altitude value along the flight path by analyzing the change in image resolution according to the distance between the camera and the subject through the collected ground image data.

The altitude calculation unit (730) calculates the altitude (Z) value on the route by matching coordinate, altitude, attitude, and time information from the flight data recorder (FDR) of the aircraft with the captured ground image data, and by referencing the calibration information and parameters of the shooting device to extract the height value of the collected image, and by performing image distortion correction and analysis of image resolution change.

More specifically, the altitude estimation unit (730) can extract the altitude value along the flight path by analyzing the change in the resolution of the image according to the distance between the camera and the subject. Here, the change in the resolution of the image can be determined by checking the altitude through the difference in the number of pixels between the previous frame and the current frame or the difference in the number of pixels of the subject photographed from various angles.

Accordingly, as illustrated in Fig. 15b, the resolution height (HR) can be calculated as the difference between the height (HF) by the radio altitude sensor and the height (HO) by point cluster analysis, and correction of the resolution height (HR) can be performed by verifying the sensor and scanning data through triangulation analysis.

In this way, the existing image analysis distance measurement method, which measures distance by analyzing changes in the image of a subject, can be applied to altitude measurement, and the altitude (Z) value of the image can be extracted by analyzing changes in image resolution.

In step (540), the verification unit (740) can correct the measurement value of the radio altitude sensor through route verification from the extracted altitude value.

When the verification unit (740) extracts the altitude (Z) value from a subject (obstacle) existing in the route and substitutes the result into the route coordinates of the unmanned aerial vehicle at regular intervals, when the unmanned aerial vehicle reaches the corresponding route coordinates, the resolution height (HR) of the image corresponding to the coordinates at which it comes into contact with the subject (obstacle) is recognized, thereby enabling the measurement value of the radio altitude sensor being used to be corrected.

This verification unit (740) can support offline image processing to minimize risks to communication and aircraft infrastructure environments during autonomous flight.

The verification unit (740) repeatedly collects ground image data through autonomous flight of an unmanned aerial vehicle, and reflects the collected ground image data in route control and ground control and route map data through resolution change analysis, thereby creating or verifying a new route.

When an unmanned aerial vehicle reaches a specific coordinate of the route, it can align the route map data stored in advance on the aircraft with the GPS coordinates, and correct the sensor-measured altitude (Z) value using the altitude (Z) value of the image from the route map data. The corrected altitude (Z) value can maintain the flight altitude limit and the vertical separation of the route by layers through the gear control of the unmanned aerial vehicle.

To verify the route and maintain the latest data, the unmanned aerial vehicle repeatedly collects ground image data through autonomous flight missions, and the collected ground image data can be reflected in route control and ground control and route map data through image change or resolution change analysis. As autonomous flight missions are repeated, the reliability of the route increases, and the creation and verification of new routes through simulations are possible.

FIGS. 8 to 10 are diagrams for explaining autonomous flight space shaping from surface scanning and image capturing data according to one embodiment.

Referring to FIGS. 8 and 9, for autonomous flight spatial shaping from ground scanning and image capturing data, a LiDAR device and a calibrated photographing device mounted on a ground capturing aircraft (810) can obtain a point cloud of reflection points of a subject (building, etc.) (821, 822, 823) on which a LiDAR pulse is projected, points (831, 832, 833) formed at the height of a specific point of the subjects (821, 822, 823), and image data. Such data can be utilized to provide various spatial and geographic information services, such as subject identification and three-dimensional modeling of the subject.

By analyzing point clusters collected by a lidar pulse and points (911, 912, 913) formed at the height of a specific point among the point clusters, subjects (821, 822, 823) are identified, and by utilizing existing terrain elevation data, the height value h of a specific point (912) of the identified subject (822) can be extracted, and a plane (910) connecting specific points (911, 913) of subjects (821, 823) having the same height as the height value h can be generated. For example, the height value (h) of 120 m of a specific point of the identified subject (building, etc.) can be extracted. Here, the specific point can be arbitrarily selected, and in FIG. 8, it is a point selected assuming that there is a space on the roof of the subject (822) where an unmanned aerial vehicle can take off and land.

Afterwards, as shown in Fig. 10, by connecting the height values (1011, 1012, 1013) of the extracted subject (1010), the area and altitude where autonomous flight of an unmanned aerial vehicle is possible in space can be visualized as a concept called a layer (1020).

Meanwhile, in FIGS. 8 to 10, it is assumed that the maximum flight limit altitude of an unmanned aerial vehicle is 120 m, and the height of the subject (822) corresponds to 120 m, and a layer (1020) capable of autonomous flight is generated based on that height. In one embodiment, when generating a layer capable of autonomous flight of an unmanned aerial vehicle, it is assumed that the maximum flight limit altitude of the unmanned aerial vehicle will be set by the safety regulation policy described above in order to prevent collision with a manned aerial vehicle, and a plurality of vertically separated layers can be set (shaped) in a space below the maximum flight limit altitude and used to determine the route of the unmanned aerial vehicle.

FIG. 11 is a diagram for explaining alignment of geospatial data according to one embodiment.

Referring to FIG. 11, data such as altitude restriction policy (1110), precision digital map (1130), and route information (1120) for avoiding military security areas and no-fly zones can be aligned and applied to layers visualized in space to build an autonomous navigation map (1100) for unmanned aerial vehicles. Accordingly, by guiding safe routes for unmanned aerial vehicles, a service utilizing multiple unmanned aerial vehicles simultaneously becomes possible even in safety-sensitive areas. Here, the autonomous navigation map for unmanned aerial vehicles can be expressed as an unmanned aerial vehicle safety autonomous navigation map.

FIG. 12 is a diagram for explaining a method of creating a map by aligning geospatial data according to one embodiment.

Referring to FIG. 12a, an autonomous navigation map (1210) for flight of an unmanned aerial vehicle built on a layer according to one embodiment can be synchronized with an unmanned aerial vehicle (1230) through information from GPS (1220) and various position coordinate correction devices and meet target safety standards.

That is, as illustrated in Fig. 12b, the tutorial content of the 3D virtual flight simulator (1240) can be visualized as a space map that can be physically applied to an unmanned aerial vehicle in reality.

By applying non-contact altitude measurement technology, vertical altitude values can be built into spatial data so that the waypoints visualized in a 3D virtual flight simulator (1240) can be recognized by an actual aircraft, and by applying this, the safety of unmanned aerial vehicle operation can be secured.

In order to establish a safe autonomous flight path in space for an unmanned aerial vehicle according to one embodiment, scanning (echo) data of radio waves/light, etc. to which the operating principle of a radio wave sensor is applied and ground image data acquired by a photographing device with a calibration value of a specific altitude set can be used.

The altitude (Z) value obtained by the distance and altitude measurement method using scanning (echo) data from the subject can extract information on the elevation and height of the subject (obstacle).

And the altitude (Z) value by the image change analysis method of the ground image data collected by the photographing device with the calibration value set at a specific altitude can be used to verify the calibration and correct the extracted value from the radio altitude sensor of the unmanned aerial vehicle by utilizing the height information of the extracted subject (obstacle).

FIG. 13 is a diagram illustrating the collection of point clusters through laser scanning according to one embodiment.

As illustrated in Figure 13, a ground-based imaging aircraft can identify a subject by collecting a point cloud of the subject using GPS (position), inertial navigation system (INS, en-route position), laser scanning, etc.

From the scanning results that appear as countless point clouds, it is possible to create orthophotos, stereoscopic maps, contour lines, and DEMs.

FIG. 14 is a drawing for explaining a layer having a specific height in a three-dimensional space according to one embodiment.

As illustrated in Fig. 14a, the unmanned aerial vehicle route construction system calculates the height of an object identified from scan data based on the ground altitude of the corresponding coordinates, and by connecting the heights of specific points, it is possible to visualize a two-dimensional layer in a three-dimensional space.

When countless 2D layers (1410) having specific altitude (Z) values are generated in a three-dimensional space, a terrain like that of Fig. 14a can be formed, and furthermore, it can be shaped into a plurality of grid terrains (1420) as illustrated in Fig. 14b. In other words, the layer illustrated in Fig. 14a can be expanded to be shaped into a grid terrain in which a plurality of layers of Fig. 14b are connected, and thus it can be used to construct a route for an unmanned aerial vehicle that can fly long distances. Therefore, if the grids are expanded one by one from Fig. 14b, it eventually shows a connected shape of layers shaped at a specified height along the elevation.

Referring to FIG. 14c, the unmanned aerial vehicle route construction system can collect ground image data for the flight path from the shaped layer.

At this time, surface image data can be collected from layers with flight altitude limit heights.

The unmanned aerial vehicle route construction system can identify spatial geographic information to collect ground image data, search for a safe route for flight, generate a specific flight route, and collect ground image data for the route. In particular, the collection of the initial ground image data required for route analysis for route construction can only allow flights within the visual range of a pilot with piloting qualifications, thereby ensuring maximum safety.

The unmanned aerial vehicle route construction system can set the height value of the flight altitude limit and check the measurement value of the radio altitude sensor (e.g., radio altimeter, etc.) through an object for which the flight altitude limit height can be verified. Here, the object for which the flight altitude limit height can be verified can be a ground structure that is higher than or equal to the flight altitude limit.

Additionally, the unmanned aerial vehicle route construction system can check information such as the resolution of the photographing device, the image acquisition method, and the calibration parameters according to the angle of incidence, and can check the flight information of the aircraft recorded in the Flight Data Recorder (FDR) mounted on the unmanned aerial vehicle.

The altitude (Z) value on the route can be calculated by matching the coordinate, altitude, attitude, and time information from the flight data recorder (FDR) of the aircraft with the captured ground image data, and by referencing the calibration information and parameters of the shooting device to extract the height value of the collected image, and by performing image distortion correction and analysis of image resolution change.

Referring to FIGS. 14c and 14d, the unmanned aerial vehicle route construction system sets an autonomous flight route (1430), which is a flight path, according to the flight purpose of the unmanned aerial vehicle on a layer (1410), and then generates points (1440) for ground resolution conversion analysis to maintain a constant flight altitude of the unmanned aerial vehicle at each constant point on the autonomous flight route. Through this, an unmanned aerial vehicle flying along the autonomous flight route (1430) can periodically check whether its flight altitude is accurately maintained.

As illustrated in FIG. 14c, a route (1430) for autonomous flight set on a layer (1410) may be indicated. The route (1430) for autonomous flight may be indicated as a route composed of one way point (1452) designated as the same destination as the departure point for repeated mission performance of an unmanned aerial vehicle, and four way points (1450, 1454, 1456, 1458) for avoiding dangerous areas and changing direction of movement, for a total of five way points (1450, 1452, 1454, 1456, 1458). At this time, the five waypoints (1450, 1452, 1454, 1456, 1458) may correspond to one of the multiple waypoints (1440) for measuring the height of the resolution illustrated in FIG. 14d.

These waypoints are the concept of specific coordinate points to achieve a certain purpose set on the route, and basically, they can be points designated in advance to set a path from the starting point to the destination or to avoid obstacles. In the same concept, they can also be said to be coordinate points that contain the image resolution values of ground objects that must be recognized by an unmanned aerial vehicle to maintain a preset layer on the route.

That is, in Fig. 14d, the point (1440) for analyzing the resolution conversion of ground objects to maintain a constant flight altitude of the unmanned aerial vehicle at each point on the autonomous flight path is a point where the image change of each ground object existing on the moving path (1430) must be measured in order to maintain the safety regulation altitude of the unmanned aerial vehicle, which is indicated as a waypoint. In simple terms, the point where the image change analysis is performed centered on the ground object below the path that interferes with the height maintenance of the layer (1410) while the unmanned aerial vehicle moves along the path (1430) formed in the layer (1410) is indicated as a waypoint. Here, the waypoint can be a point where the unmanned aerial vehicle hovers for a while to perform a specific mission or acquire data.

A method for setting a route and waypoint according to one embodiment will be specifically described with reference to FIG. 24. FIG. 24 describes a method for maintaining a flight altitude corresponding to a layer assigned to a route through image analysis when an unmanned aerial vehicle passes over a ground object existing in the route of the unmanned aerial vehicle according to one embodiment.

Referring to FIG. 24, an unmanned aerial vehicle according to one embodiment may set a waypoint (2403) that requires correction of an altitude measurement value of an ultrasonic sensor through image analysis in order to maintain a constant flight altitude corresponding to a layer. The route (2401) of the unmanned aerial vehicle is formed to pass from a ground object A (2410) to a ground object B (2420) and a ground object C (2430), and in the case of ground objects A (2410) and B (2420), the height of the waypoint of the entry point (2450, 2454) and the height of the waypoint of the exit point (2452, 2456) for the ground objects may be formed to be the same. In this case, the corresponding section may be set as a waypoint valid section (2405). Here, the waypoint valid section (2405) may mean a section in which the resolution height of the ground object image at the entry point and the exit point of the waypoint are the same.

In contrast, if the heights are different within the waypoint valid section (2405), an additional waypoint may be set in the middle of the waypoint valid section (2405). Referring to FIG. 25, for example, if it is assumed that there is a rooftop (2550) on top of the ground object A, if the rooftop (2550) is located on the flight path, a waypoint corresponding to it may be added to the waypoint valid section (2570).

Preferably, there is one waypoint for each ground object, but in the case of ground objects A (2410) and B (2420) of Fig. 24, it can be assumed that there are two waypoints assuming that they are very large buildings.

FIG. 15 is a drawing for explaining the change in image resolution according to the distance between a camera mounted on an unmanned aerial vehicle and a subject according to one embodiment.

Referring to FIGS. 15a and 15b, the unmanned aerial vehicle route construction system can extract an altitude value along a flight path by analyzing a change in image resolution according to the distance between the unmanned aerial vehicle (1500) and the subject.

In one embodiment, the unmanned aerial vehicle can basically always determine its current position, speed, etc. through an inertial navigation system (INS), but the aircraft altitude can be corrected through changes in the resolution of the image and the measurement value of the radio altitude sensor for precise altitude determination. Specifically, the altitude can be determined through the difference in the number of pixels for a specific point of the subject in the previous frame and the current frame captured by an optical device (1510) such as a camera mounted on the unmanned aerial vehicle, and the change in the resolution of the image can be analyzed by utilizing the image plane (1520) on which an image is formed inside the camera (1510).

First, referring to FIG. 15a, assuming that an unmanned aerial vehicle (1500) flies in the vector directions of the X-axis and the Y-axis according to a waypoint, pixel values captured by the optical equipment (1510) of the unmanned aerial vehicle (1500) at two points (1550, 1552) where no subject exists on the ground surface can be analyzed on an image plane (1520). Here, the points (1550, 1552) can be points on the ground surface corresponding to waypoints included in the unmanned aerial vehicle route.

The unmanned aerial vehicle (1500) can verify the accuracy of the measurement value of the radio altitude sensor measured by the unmanned aerial vehicle (1500) above the point (1550) by using the resolution height obtained through the resolution analysis of the image captured above the waypoint before the point (1550). At this time, the flight altitude can be measured (1554) using the radio altitude sensor mounted on the unmanned aerial vehicle (1500).

And, the point (1552) can be a point on the ground surface for calibrating the shooting incidence angle in order to check the change in resolution of the ground object existing in the flight direction in which the unmanned aerial vehicle (1500) is flying. The resolution height for the point (1552) can be calculated using the incident angle direction (1556) of the camera (1510) considering the movement direction for the point (1552). For the ground surface of reference number 1552, the unmanned aerial vehicle (1500) is captured when it is located above the point (1550), and accordingly, the unmanned aerial vehicle (1500) can check the resolution size (1530) of the image for the point (1552). This process can be performed for each waypoint while the unmanned aerial vehicle (1500) performs autonomous flight according to the set waypoint.

That is, for each waypoint, the unmanned aerial vehicle (1500) measures the flight altitude from a specific point corresponding to the current waypoint via a radio altitude sensor, and at the same time, measures the resolution height from a specific point corresponding to the next waypoint to be moved to, compares it with the radio altitude sensor measurement value on the next waypoint, and if the radio altitude sensor measurement value and the resolution height are different, changes the radio altitude sensor measurement value, thereby maintaining the flight altitude of the unmanned aerial vehicle (1500) constant at the next waypoint.

Referring to FIG. 15b, a point (1562) is a point on the ground surface for verifying the accuracy of a measurement value of a radio altitude sensor, and accuracy verification can be performed continuously. In addition, in FIG. 15b, since a ground object (1560) exists on the flight path of an unmanned aerial vehicle (1500), the resolution of an image captured at a point (1552) in FIG. 15a and the resolution of an image captured at a point (1564) in FIG. 15b change, and by analyzing the change, the unmanned aerial vehicle (1500) can be controlled to maintain its set flight altitude by considering the height of the ground object (1560). That is, the resolution size (1530) of the image of FIG. 15a and the resolution size (1566) of the image of FIG. 15b show different resolution changes depending on the presence or absence of the ground object (1560).

In other words, in Fig. 15b, due to the presence of the ground object (1560), the distance between the camera and a specific point (1564) of the ground object is closer than the distance between the camera (1510) and the point (1552) of Fig. 15a, so that the resolution for the point (1564) and the resolution for the point (1552) are different. In this way, according to one embodiment, the altitude of the unmanned aerial vehicle and the height of the ground object (1560) can be estimated by utilizing this difference in resolution, and thereby the flight altitude of the unmanned aerial vehicle (1500) can be measured.

Accordingly, the resolution height (HR) and the correction of the resolution height (HR) can be expressed as follows.

[Mathematical formula 1]

Height by radio altitude sensor (HF) - Height by point cluster analysis (HO) = Resolution height (HR)

Calibration of sensor and scanning data through triangulation analysis = Correction of resolution height (HR)

Meanwhile, a distance measurement method using image change analysis prepares an OMR (Optical Mark Recognition) target for image processing, places the targets at regular intervals (for example, 0.5 m), and then uses the results of analysis of the correlation between the distance and resolution between the camera and the subject to measure the distance.

In this way, the existing image analysis distance measurement method, which measures distance by analyzing changes in the image of a subject, can be applied to altitude measurement, and the altitude (Z) value of the image can be extracted by analyzing changes in image resolution.

The verification unit (740) of the unmanned aerial vehicle route construction system can correct the measurement value of the radio altitude sensor through route verification using the extracted altitude value.

When the verification unit (740) extracts the altitude (Z) value from a subject (obstacle) existing in the route and substitutes the result into the route coordinates of the unmanned aerial vehicle at regular intervals, when the unmanned aerial vehicle reaches the corresponding route coordinates, the resolution height (HR) of the image corresponding to the coordinates at which it comes into contact with the subject (obstacle) is recognized, thereby enabling the measurement value of the radio altitude sensor being used to be corrected.

As illustrated in Fig. 14d, in this embodiment, the extracted altitude (Z) value can be uniformly placed in the corresponding route coordinates.

FIGS. 16 to 19 are diagrams for explaining a flight control and ground control process through image recognition and processing of an unmanned aerial vehicle according to one embodiment.

Referring to Fig. 16, the method by which an unmanned aerial vehicle identifies and processes the altitude (Z) value of an image placed at each waypoint in the route coordinates is inherently susceptible to vulnerabilities such as delays in processing time and battery consumption depending on the characteristics of the image processing device and data link. Accordingly, in order to ensure the safety of autonomous flight and minimize risks to the communication and aircraft infrastructure environments, an offline image processing method may be supported.

An unmanned aerial vehicle (1660) can align GPS coordinates with route map data (1601) stored in advance in the aircraft (1602), process the altitude (Z) value of the image from the route map data (1601) (1603), and correct the sensor measured altitude (Z) value (1604).

If a GPS signal is not received, the unmanned aerial vehicle (1660) can perform flight using route map data (1601) stored in advance by inertial navigation without GPS coordinate processing (1602) and perform image processing (1603) at each set waypoint. In addition, if a GPS signal is not received, the unmanned aerial vehicle (1660) can further include a means for communicating with a mobile communication base station so that it can determine its location through communication with the surrounding mobile communication base stations.

If the unmanned aerial vehicle is not on its first flight or if there are resolution values of ground objects at each waypoint, the pre-stored route map data (1601) according to one embodiment stores the resolution values of ground objects acquired in advance at each waypoint. Accordingly, the unmanned aerial vehicle (1660) can compare the resolution values of ground objects stored at each waypoint with the altitude values measured and maintained by the radio altitude sensor while performing autonomous flight, and correct or utilize the radio altitude sensor measurement values to comply with the flight height of the predefined layer. In addition, for each new flight, the unmanned aerial vehicle (1660) can store the resolution values of ground objects acquired at each waypoint.

Additionally, in another embodiment, the unmanned aerial vehicle (1660) can correct the radio altitude sensor measurement value using the average value of the resolution value of the ground object acquired in advance before autonomous flight and the resolution value of the ground object acquired during autonomous flight to maintain the flight altitude of a predetermined layer.

If an unmanned aerial vehicle (1660) is in the initial flight state and has not yet acquired the resolution value of ground objects existing at each waypoint, the unmanned aerial vehicle (1660) can align route map data (1601) and GPS coordinates (1602), and perform a flight while maintaining a pre-input flight altitude through a radio altitude sensor, and acquire and store the resolution value of ground objects at each waypoint.

The unmanned aerial vehicle (1660) can perform gear control and flight altitude control (1605) to maintain a flight altitude corresponding to the height of a layer by maintaining a flight altitude restriction and a vertical separation of a route by layers using a corrected altitude (Z) value. In addition, the unmanned aerial vehicle (1660) can perform a flight report (1608) by transmitting flight information, etc. generated while performing steps 1601 to 1605 to a ground control system (1650) via a wireless transceiver (1607). The control system (1650) may also mean a ground control system.

In addition, when the unmanned aerial vehicle (1660) receives (1606) route control information (1609) from the ground control system (1650) through the wireless transceiver (1607), the unmanned aerial vehicle (1660) may perform gear control and flight altitude control (1605) to perform a flight according to the received route control information (control data). In addition, according to one embodiment, the data processed in 1601 to 1605 may be recorded in the flight data recorder (FDR) of the unmanned aerial vehicle (1660) every time it reaches a corresponding coordinate point on the route.

When an unmanned aerial vehicle (1660) performs a flight report (1608), it can transmit flight information such as speed, altitude, and direction of movement in the form of a message to a control center (1610) of a ground control system (1650), and the ground control system (1650) can transmit received flight information data and route control data (1609) for controlling the unmanned aerial vehicle (1660) according to a situation to the unmanned aerial vehicle (1660). The flight information data and route control data (1609) can be transmitted through a wireless communication network, and can also be transmitted through a mobile communication network such as LTE (Long Term Evolution).

In addition, the control center (1610) of the ground control system (1650) can obtain (1611) a route image captured by an unmanned aerial vehicle (1660). Accordingly, the control center (1610) can obtain images and videos that can analyze the height information of ground objects obtained while the unmanned aerial vehicle (1660) flies along the set route through the obtained route image (1611), and when the obtained image is analyzed, the analyzed information can be used to update the route information corresponding to the obtained route image.

Accordingly, when route update data (1612) is generated through the acquired route image, the control center (1610) can apply (1613) the updated route data to the unmanned aerial vehicle (1660) online or offline. In addition, in one embodiment, the application (1613) can be performed during the procedures of reference numbers 1901 to 1903 illustrated in FIG. 19, and can be performed online or offline.

And route image acquisition (1611) can be acquired through wireless or wired methods. The method of acquiring route images through wireless methods can be performed in real time through a wireless communication network, a mobile communication network, or a satellite communication network, and the method of acquiring route images through wired methods can be acquired directly from the storage unit storing the route images inside the unmanned aerial vehicle (1660) after the unmanned aerial vehicle (1660) has finished its flight and landed.

In addition, the ground control system (1650) can acquire route images both wirelessly and wiredly, and in this case, the average value of the route images acquired wirelessly and the route images acquired wiredly can be used as the final route image information.

Additionally, the control center (1610) of the ground control system (1650) can determine a layer changeable section for inter-layer flight of the unmanned aerial vehicle (1660) and transmit route data reflecting the determined layer changeable section to the operating system or the unmanned aerial vehicle (1660). At this time, when a layer change request message is received from the unmanned aerial vehicle (1660), the control center (1610) can control other aircrafts existing in the layer changeable section determined for the unmanned aerial vehicle (1660) to fly for layer change, and perform control of the unmanned aerial vehicle (1660) so that it does not collide with other aircrafts until it is located on the layer (arrival layer) to which it must move.

For example, a situation may arise where an unmanned aerial vehicle (1660) needs to move to layer C according to a pre-programmed command or remote control while flying over a waypoint set on layer A. For convenience of explanation, layer A is referred to as a departure layer, and layer C is referred to as an arrival layer. In addition, if layer B exists between layer A and layer C, layer B is referred to as a transit layer.

In one embodiment, when an unmanned aerial vehicle (1660) needs to move from layer A to layer C, the control center (1610) can continuously control the unmanned aerial vehicle (1660) and other unmanned aerial vehicles located in the layer change possible section during the layer change possible time of the unmanned aerial vehicle (1660) to prevent collisions with other unmanned aerial vehicles during the layer movement of the unmanned aerial vehicle (1660).

When an unmanned aerial vehicle (1660) needs to move from layer A to layer C, it transmits a layer change request message to the control center (1610), and after the control center (1610) completes identifying the unmanned aerial vehicle (1660) that has transmitted the layer change request message, it checks a layer changeable section that the unmanned aerial vehicle (1660) can use for moving between layers, and transmits a confirmation message accordingly to the unmanned aerial vehicle (1660). At this time, the control center (1660) may consider the congestion of the layer changeable section, and transmit layer movement information including at least one of layer changeable section information, layer changeable time, layer change entry point information, and layer change entry angle information to the unmanned aerial vehicle (1660). An unmanned aerial vehicle (1660) that has received layer movement information can move from layer A to layer B according to the information included in the layer movement information, and then move to layer C again according to the layer movement information received from the control center (1660) while positioned on layer B. Of course, the unmanned aerial vehicle (1660) can also perform a flight for layer movement from layer A to layer C using only the layer movement information received in layer A.

According to another embodiment, the unmanned aerial vehicle (1660) may perform a flight for moving between layers without control from the control center (1610). According to another embodiment, when a flight for moving between layers is required, the unmanned aerial vehicle (1660) may perform a flight between layers based on pre-stored layer changeable section information, and in this case, the unmanned aerial vehicle (1660) may perform an autonomous flight while preventing collisions using sensors provided therein before entering the layer changeable section and after entering and before reaching the destination layer.

Referring to FIG. 17, an example of a method for recognizing an object (obstacle) and correcting sensor measurements when an unmanned aerial vehicle reaches a specific coordinate can be shown.

An unmanned aerial vehicle that has reached a specific coordinate of a route can align (1702) GPS coordinates with route map data (1701) previously stored in the aircraft, process (1703) the altitude (Z) value of an image from the route map data (1701), and correct (1704) the sensor measured altitude (Z) value. The corrected altitude (Z) value is used to control the gear shift and altitude (1705) of the unmanned aerial vehicle, and through this, the unmanned aerial vehicle can maintain a flight altitude restriction and vertical separation of the route by layers.

And the unmanned aerial vehicle can store flight information collected during the flight in a device such as a flight data recorder (FDR), and report (1706) the stored flight information to a control system or a flight path construction system through an undisclosed communication means.

Referring to FIGS. 18 to 20, examples of a route control and route generation-verification system and a route control and route generation-verification processor can be illustrated. 1801 to 1813 illustrated in FIG. 18 are identical to 1601 to 1613 described in FIG. 16, and therefore, a description thereof will be omitted.

Referring to FIG. 18, unlike FIG. 16, a simulation verification system (1820) may be added to perform simulation verification on route data (1812) updated by a control center (1810) in a ground control system (1850). Accordingly, the ground control system (1850) may perform simulation verification (1820) in advance before applying route update data (1812) to an unmanned aerial vehicle (1860) and apply the verified route update data (1822) as a new route for the unmanned aerial vehicle (1860), thereby improving stability.

In order to verify the route and maintain the latest data, the unmanned aerial vehicle (1860) repeatedly collects ground image data through an autonomous flight mission, performs gear control and flight altitude control (1805) through analysis of image change or resolution change of the collected ground image data (1803), and transmits the data processed from 1801 to 1805 to the ground control system (1850), thereby enabling the control center (1810) to perform route control and ground control.

In addition, the ground control system (1850) can verify the generated route update data (1812) through a simulation verification system (1820) and apply (1813) the verified route update data (1822) to the route map data (1801) online or offline.

FIG. 19 is a block diagram of a control device for controlling a route for an unmanned aerial vehicle and generating a route for the unmanned aerial vehicle according to another embodiment.

First, when an unmanned aerial vehicle operating company applies for autonomous flight of an unmanned aerial vehicle, the autonomous flight application aircraft/mission reporting unit (1901) can transmit the autonomous flight aircraft and the purpose (mission) of the autonomous flight applied for by the operating company to the control unit (1902). For example, the operating company can be a logistics company such as Amazon, DHL, FEDEX, or UPS, a private security company, an oil refinery company for managing large-scale oil pipelines, a railway operating company for monitoring abnormalities in massive railways, or an institution that promotes public safety such as a prison, the military, police, or fire department.

The control unit (1902) stores identification information about specifications and basic tasks of an unmanned aerial vehicle in advance in the storage unit (1912), registers an unmanned aerial vehicle acquired from the autonomous flight application aircraft/mission report unit (1901) in the storage unit (1912), and can also store identification information about specifications and basic tasks of the unmanned aerial vehicle. In addition, the control unit (1902) analyzes information reported by an operating company of the unmanned aerial vehicle through the autonomous flight application aircraft/mission report unit (1901) to identify the unmanned aerial vehicle, and can confirm whether the identified unmanned aerial vehicle is a suitable aircraft for the reported mission. If it is confirmed to be a suitable aircraft, a layer and a route corresponding to the mission of the unmanned aerial vehicle can be assigned through the aircraft identification and route assignment unit (1904).

It would be desirable for operational efficiency to classify the unmanned aerial vehicles owned by the above operating companies into certain standards according to their weight and output, and accordingly, the control unit (1902) can easily construct an autonomous flight map (route) for autonomous flight.

For example, a specification that can classify unmanned aerial vehicles that have applied for autonomous flight according to certain specifications, such as the weight of the unmanned aerial vehicle, the purpose of the mission, the flight time, and the loadable weight, can be stored in the storage unit (1912), and when a new route creation is requested from the new route application unit (1910), the control unit (1902) can control the simulation verification unit (1906) to perform simulation verification if there is a route for the unmanned aerial vehicle corresponding to the specifications and mission of the unmanned aerial vehicle to fly along the requested new route among the route maps for autonomous flight stored in the storage unit (1912). Accordingly, similar routes can be generated and provided for unmanned aerial vehicles that fall within certain specifications. Of course, the control unit (1902) can set different routes for the flight time, flight distance, altitude, etc. so that the unmanned aerial vehicles do not collide during flight, and can continuously monitor whether the unmanned aerial vehicles maintain the route.

When the assignment of layers and routes for an unmanned aerial vehicle is completed in the aircraft identification and route assignment unit (1904), the completion of assignment can be transmitted to the flight information acquisition unit (1905). The operating company of the unmanned aerial vehicle can control the unmanned aerial vehicle to fly autonomously along the assigned layer and route, and control the flight information to be recorded and transmitted to the flight information acquisition unit (1905).

The flight information acquisition unit (1905) acquires flight information reported by the unmanned aerial vehicle and transmits it to the control unit (1902), and the control unit (1902) can examine the acquired flight information to determine whether it deviates from a pre-assigned layer and route. If the flight information of the unmanned aerial vehicle acquired through the flight information acquisition unit (1905) deviates from a pre-set layer and route, this can be transmitted to the unmanned aerial vehicle operating company or the unmanned aerial vehicle to control the flight along the pre-set layer and route.

Meanwhile, if additional simulation verification is required, the control unit (1902) can transmit the flight information acquired from the unmanned aerial vehicle by the flight information acquisition unit (1905) to the simulation verification unit (1906).

The simulation verification unit (1906) can simulate and verify the safety of layers and routes assigned to unmanned aerial vehicles by considering information input from the flight information and three-dimensional geographic information security, risk, and obstacle input unit (1907), weather information input unit (1908), and safety regulation information input unit (1909), and transmit the results to the control unit (1902).

Meanwhile, when a new route is applied for from an unmanned aerial vehicle operating company in addition to an existing route, the new route application unit (1910) transmits the applied new route to the simulation verification unit (1906), and the simulation verification unit (1906) can determine the validity of the applied new route through information obtained from the three-dimensional geographic information security, risk, and obstacle input unit (1907), the weather information input unit (1908), and the safety regulation information input unit (1909).

And if the applied new route is valid, the new route construction unit (1911) can be requested to build the applied new route. If the new route is built at the request of the simulation verification unit (1906), the new route construction unit (1911) can transmit the new route and the identification information of the unmanned aerial vehicle that applied for the new route to the control unit (1902). Accordingly, even if an autonomous flight request for the unmanned aerial vehicle that applied for the new route is received, the control unit (1902) can immediately permit the flight of the unmanned aerial vehicle because the new route has already been verified in advance.

As illustrated in Figure 19, as autonomous flight missions are repeated, the reliability of the route increases, and the creation (1911) and verification (1906) of new routes through simulations are possible.

Additionally, we provide an example of how to improve positioning accuracy in an autonomous flight system for an unmanned aerial vehicle.

The infrastructure for improving the positioning accuracy of unmanned aerial vehicles can be composed of a satellite communication module that can receive GPS satellites and non-GPS satellite signals, a GPS receiver equipped with the module, a communication module and system that broadcasts various satellite signals to ground stations and the GCS (Ground Control System) of manual flight aircraft that are not identified as autonomous flight, and a system that processes and displays the positioning correction Reference (data) accumulated through machine learning through the TDOA (Time Difference of Arrival) method utilizing the communication infrastructure used for civilian services such as LTE (moving from 4G to 5G) and the TDOA process and operation that reflects the altitude difference of the base station.

A method for improving the positioning accuracy of an autonomous flight of an unmanned aerial vehicle according to one embodiment can apply all of GPS and GNSS information reception and message broadcasting, TDOA method application using LTE ground base stations (or next-generation mobile communication infrastructure), and machine learning positioning correction Reference (data) utilization.

Figure 20 is a diagram showing a simulation of a constructed route according to one embodiment.

Referring to FIG. 20, the simulation and route verification shape of the constructed route is shown, which may include vertically separated layers (2010, 2011) and a route (2020), and a waypoint (2030) shaped as a symbol. The simulation shape may be shaped as a route corresponding to the separation interval and a symbol corresponding to the collected waypoint (Way Point) by forming a vertical separation of multiple 2D layers on a stereoscopic precision map.

Here, the layer includes information such as formation altitude and performable missions, aircraft specifications, etc., and the symbol of the route formed in the layer (connected to Way Point) may include position coordinates and the altitude (Z) value of the image based on the layer for the corresponding coordinates. In this case, the altitude (Z) value of the image may mean a value that must be corrected for the measurement value by the altitude measuring sensor in order for the unmanned aerial vehicle to maintain the formation altitude of the layer during autonomous flight.

FIG. 21 is a drawing showing an aircraft recognition and route control configuration according to one embodiment, showing information on a manually flown unmanned aerial vehicle and information on an autonomously flown unmanned aerial vehicle being displayed on a control screen.

Referring to FIG. 21, the aircraft recognition and route control configuration (2100) may include layer identification and vertical separation information, identification of an autonomous flying drone (2120), a flight path of an autonomous flying drone (2110), identification of a manually flying drone and radius information, etc.

In the control information (2160) of a manual flight unmanned aerial vehicle, if the pilot's position is displayed (2130), a flight radius stipulated by law is displayed centered on that position, and the identifier (ID) of the manual flight unmanned aerial vehicle, information on whether it is a manual flight unmanned aerial vehicle (in the case of a manual flight unmanned aerial vehicle, "Manual" is displayed), GPS, INS, altitude information, and flight data can be displayed in real time.

The control information (2150) of an autonomous unmanned aerial vehicle can display in real time the identifier (ID) of the registered aircraft, the business code for which route assignment has been requested, information on whether the aircraft is autonomous, GPS, INS, sensor altitude information, flight data, etc.

Additionally, each waypoint and route information of an autonomous flying unmanned aerial vehicle (2120) can be displayed as coordinates and image resolution values for each waypoint.

The aircraft recognition route control shape can display layer information (2170, 2180) of the corresponding screen. The layers can be configured in various ways through vertical separation according to the specifications of the aircraft and the mission to be performed below the regulated altitude. For example, reference number 2170 on the screen can display information of the currently displayed layer, and reference number 2180 can display the vertical separation interval from other layers.

The aircraft recognition and route control shape (2100) displays information for maintaining the vertical separation interval of multiple 2D layers on a 2D precision map, and can be visualized as symbols corresponding to the route built based on each layer and the collected waypoints.

The aircraft recognition and route control shape (2100) can minimize the delay in image processing time, thereby reducing the risk of control and flight control delay, and display layer identification, aircraft assigned an autonomous flight route, flight information, autonomous flight route and waypoint, and the altitude (Z) value of the image assigned to the waypoint based on each layer.

Meanwhile, to ensure safety, the identification and flight radius of manually controlled aircraft by pilots can be limited, and autonomous flight route information can be shared.

When an autonomous drone reaches a waypoint, it recognizes the altitude (Z) value of the image analyzed and assigned based on the layer it is flying in by loading the map installed on the drone, and corrects the measurement value by the sensor to maintain the formation altitude of the corresponding layer.

Verification of this process can be achieved by receiving messages from route control, which include flight recorder data containing sensor altitude, GPS, and INS information, as the aircraft broadcasts them via the message transmission module, and analyzing the GPS and sensor altitude values and the layer-formed altitude information to determine whether the UAV is maintaining vertical separation and flight altitude restrictions while flying autonomously.

An example of the provision of maps for route control support can be presented below.

[Display route information]

- Security area display

-Danger zone marking

-No-fly zone sign

- Display of height and area of ground objects extracted through surface scan

- Display formation height information for each layer

- Display construction route for each layer

-Display waypoints on the construction route for each layer

- Displays the altitude (Z) value of the image assigned to the waypoint on the construction route for each layer.

[Autonomous flying drone demonstration]

-Display of identification code of autonomous flying drone

-Display of mission code of autonomous unmanned aerial vehicle

-Display of route assigned to autonomous unmanned aerial vehicle

- Display of horizontal separation interval based on the route assigned to the autonomous unmanned aerial vehicle

-Display GPS location coordinates of autonomous flying drones

-Display sensor altitude values of autonomous flying drones

-Display of flight error (Fail Safe) status of autonomous unmanned aerial vehicle

[Display of a manually controlled unmanned aerial vehicle by a pilot]

-Display of identification code of manual flight unmanned aerial vehicle

- Display of pilot identification code and current location of a manually piloted drone

-Display of permitted flight range based on the pilot of a manually operated unmanned aerial vehicle

-Display GPS location coordinates of a manually flown drone

-Display sensor altitude values of a manual flight unmanned aerial vehicle

-Display of flight error (Fail Safe) status of a manual flight unmanned aerial vehicle

Figure 26 is a flowchart showing the operation of an unmanned aerial vehicle according to one embodiment.

Referring to FIG. 26, in step (2601), the unmanned aerial vehicle can align route map data with GPS, and in step (2602), the unmanned aerial vehicle can fly while passing through waypoints set on the route according to the aligned data while maintaining a pre-input flight altitude through a radio altitude sensor. In addition, in step (2603), the unmanned aerial vehicle can continuously measure its own position during the flight and check whether it has reached a pre-set waypoint. If the waypoint is reached as a result of the check in step (2603), the unmanned aerial vehicle can check whether there is a pre-stored resolution height for a ground object existing at the waypoint at step (2604). At this time, the route map data may have a pre-stored resolution height for each waypoint.

As a result of the inspection in step (2604), if there is a pre-stored resolution height, it means that there is a ground object, so the unmanned aerial vehicle can compare the radio altitude sensor measurement value measured at the current waypoint with the pre-stored resolution height in step (2605). Then, if there is a difference between the two resolution height information compared in step (2606), it is determined in step (2607) that there is an error in the radio altitude sensor measurement value, and in order to maintain a flight altitude set on the layer, the radio altitude sensor setting value is corrected to the resolution height so as to fly while maintaining a constant altitude according to the corrected radio altitude sensor setting value. Here, the constant altitude can be a flight altitude defined in the layer assigned to the unmanned aerial vehicle. In addition, the unmanned aerial vehicle can perform gear shift control and altitude control by controlling the motor control unit in order to maintain a constant flight altitude corresponding to the radio altitude sensor setting value of the radio altitude sensor measurement value measured by the radio altitude sensor.

On the other hand, if there is no previously stored resolution height in step (2604) or there is no error in the inspection result in step (2606), the ground resolution height acquired from the waypoint can be stored (2608). At this time, similar to Fig. 25, if addition or change of ground objects occurs, a new waypoint can be added using the acquired resolution height.

In step (2609), the unmanned aerial vehicle that has stored the resolution height checks whether the flight to the final waypoint has been completed, and if the flight has not been completed, it can move to the next waypoint (2610).

Meanwhile, in step (2605), it was described that the radio altitude sensor value is corrected by comparing the measured value of the radio altitude sensor with the resolution height stored in advance, but the present invention is not limited thereto, and the resolution height acquired at each waypoint in the unmanned aerial vehicle and the resolution height stored in advance for each waypoint can be compared, and the average value calculated through the comparison value can be used to determine whether the radio altitude sensor measured value is erroneous. Of course, the radio altitude sensor value can be corrected according to the determination result so that the unmanned aerial vehicle can maintain a predetermined altitude.

Additionally, the unmanned aerial vehicle may transmit flight information, including flight speed, location, altitude, etc., to a control system or an operating system according to predetermined conditions. Here, the predetermined conditions may include when a certain period has arrived, when a waypoint has been reached, or when an emergency situation occurs.

Figure 27 is a flowchart showing the operation of an unmanned aerial vehicle according to another embodiment.

Referring to Fig. 27, unlike Fig. 26, it illustrates flying while maintaining the layer height by using the resolution height acquired in real time whenever a waypoint is reached, rather than the pre-stored resolution height.

In step (2701), the unmanned aerial vehicle can align route map data and GPS, and in step (2702), the unmanned aerial vehicle can fly while passing through waypoints set on the route according to the aligned data while maintaining a pre-input flight altitude through a radio altitude sensor. In step (2703), the unmanned aerial vehicle can continuously measure its own position during the flight and check whether it has reached a preset waypoint. If the waypoint is reached as a result of the check in step (2703), the unmanned aerial vehicle can analyze the resolution height for the waypoint in real time in step (2704). In step (2705), the unmanned aerial vehicle can compare the analyzed resolution height at the current waypoint with the height of a preset layer. If the analyzed resolution height and the layer height are the same as a result of the check in step (2705), the unmanned aerial vehicle can perform autonomous flight by maintaining the layer height using the analyzed resolution height in step (2706).

On the other hand, if the pre-stored resolution height and layer height are different as a result of the inspection in step (2705), the aircraft can fly at a constant altitude using the radio altitude sensor measurement value to maintain the flight altitude set on the layer in step (2707). The constant altitude here can be the flight altitude defined in the layer assigned to the unmanned aerial vehicle.

In addition, in steps (2706) and (2707), the unmanned aerial vehicle may perform gear shift control and altitude control by controlling the flight control unit to maintain a constant flight altitude. In step (2708), the unmanned aerial vehicle may check whether the flight to the final waypoint is completed, and if the flight is not completed, may move to the next waypoint (2709).

Figure 28 is a flowchart showing an operation method of a route construction system and a control system for autonomous flight of an unmanned aerial vehicle according to one embodiment.

Referring to FIG. 28, a route construction system according to one embodiment can be included and performed in a control system.

In step (2801), the control system receives an autonomous flight report of an unmanned aerial vehicle from an unmanned aerial vehicle operating company and an unmanned aerial vehicle user, and in step (2802), can confirm the identification information and mission of the unmanned aerial vehicle. In step (2803), the control system checks whether simulation verification is required for route assignment of the unmanned aerial vehicle for which autonomous flight has been reported, and if simulation verification is required, can perform route verification by performing a simulation using information required for autonomous flight of the unmanned aerial vehicle in step (2804).

At this time, if simulation verification is not required in step (2803) or verification is completed in step (2804), the control system can assign layers and routes that match the specifications and mission of the unmanned aerial vehicle in step (2805), and transmit the assigned layers and routes to the unmanned aerial vehicle operating company or the user of the unmanned aerial vehicle in step (2806).

When an unmanned aerial vehicle flying through a layer and route assigned from the above control system performs an unmanned flight, the unmanned aerial vehicle transmits flight information to the control system. Therefore, in step (2807), the control system receives the flight information of the unmanned aerial vehicle, and by using the received flight information, continuously monitors whether the unmanned aerial vehicle has deviated from the assigned route layer or whether there is a possibility of collision in step (2808), thereby controlling the flight of the unmanned aerial vehicle.

When the unmanned aerial vehicle completes the flight, the control system performs a validity check on the route of the unmanned aerial vehicle that completed the flight at step (2809), and if the result is valid at step (2810), the valid route information is transmitted to the unmanned aerial vehicle operating company or the unmanned aerial vehicle user at step (2812), and if the result is invalid, the unmanned aerial vehicle route is modified at step (2811), and then the modified route information is transmitted to the unmanned aerial vehicle operating company at step (2812).

FIG. 29 is a drawing explaining how to maintain a flight altitude within a preset layer range by using a resolution height for a ground object (2960) when a ground object (2960) exists while an unmanned aerial vehicle is flying along a preset route according to one embodiment. At this time, the layer height is assumed to be 150 m, and accordingly, the radio altitude sensor setting value of the unmanned aerial vehicle is set to 150 m, so that the unmanned aerial vehicle can fly such that the flight altitude is maintained at 150 m from the ground surface by using the value measured by the radio altitude sensor.

First, according to one embodiment, an unmanned aerial vehicle can fly at a constant flight altitude within a range (2950) of a layer and route assigned to it from the ground surface. Waypoints (2910, 2912, 2914, 2916, 2918, 2920) exist in a route along which the unmanned aerial vehicle flies, and the unmanned aerial vehicle can measure the flight altitude from a point (2970, 2972, 2974, 2976, 2978, 2980) using a radio altitude sensor at each waypoint (2910, 2912, 2914, 2916, 2918, 2920).

And the unmanned aerial vehicle can calculate the resolution height through the camera incidence distance to the ground surface or ground object located at the next waypoint at each waypoint. For example, when the unmanned aerial vehicle is located at a waypoint (2910), the resolution height between the next waypoint (2912) and the point (2972) can be calculated by measuring the camera incidence distance to the point (2972) while measuring the flight altitude (2991) from the point (2970) through the radio altitude sensor. The unmanned aerial vehicle can perform this procedure at each waypoint.

When the unmanned aerial vehicle is located at the waypoint (2914), the unmanned aerial vehicle can measure the flight altitude through the radio altitude sensor for the point (2974), and calculate the resolution height of 130 m between the waypoint (2916) and the point (2976) through the camera incidence distance to the point (2976) above the ground object (2960) existing at the point below the next waypoint (2916). Therefore, the unmanned aerial vehicle can prevent the flight altitude from exceeding the layer by controlling the flight altitude from the ground object (2960) at the waypoint (2916) to be 130 m. This operation can also be performed at the waypoint (2918) where the ground object exists. As shown in FIG. 29, since the height of the ground object (2960) is 20 m, the flight altitude at the waypoints (2916, 2918) existing on the ground object (2960) must be 130 m above the ground object (2960), but the unmanned aerial vehicle may not exceed a flight altitude of 150 m.

Meanwhile, if the flight altitude of an unmanned aerial vehicle is maintained on a layer by using only the setting value of the radio altitude sensor without considering the resolution height for the ground or ground objects, the point where the radio altitude sensor measurement value from the ground (2960) becomes 150 m is judged as a waypoint (2922, 2924), and thus the flight altitude of the preset layer may be deviated, and a collision with another aerial vehicle flying on a different vertically separated layer may occur.

However, according to one embodiment, the unmanned aerial vehicle can fly without exceeding the flight altitude on the layer by correcting the radio altitude sensor setting value to the resolution height (130 m) from the ground object (2960) even if there is a ground object (2960).

Below, an unmanned aerial vehicle control system according to one embodiment is described.

Figure 50 is a flowchart illustrating a method for controlling an unmanned aerial vehicle according to one embodiment.

Referring to FIG. 50, a method for controlling an unmanned aerial vehicle according to one embodiment may include a step (5010) of aligning route map data and location coordinates stored in a body of an unmanned aerial vehicle, a step (5020) of processing an altitude value of an image from the route map data, a step (5030) of correcting a measurement value of a radio altitude sensor using the altitude value of the image, and a step (5040) of controlling a flight altitude through gear control according to the corrected measurement value of the radio altitude sensor.

Here, the step (5010) of matching route map data and location coordinates is a step of matching the GPS coordinates of the unmanned aerial vehicle to the route map data for the flight of the unmanned aerial vehicle constructed in the layer, and the layer can identify a subject from ground scanning data and visualize a space in which autonomous flight is possible.

The route map data can be used to construct an autonomous navigation map for the flight of an unmanned aerial vehicle in space by matching at least one of flight altitude restriction data, a precision numerical map, and route information for avoiding military security areas or no-fly zones to a spatially shaped layer.

According to one embodiment, by processing the altitude value of an image by aligning the route map data and location coordinates stored in the unmanned aerial vehicle and correcting the measurement value of a radio altitude sensor, a method and system for controlling an unmanned aerial vehicle capable of safe autonomous flight of the unmanned aerial vehicle even in a non-visual area can be provided.

Below, each step of the unmanned aerial vehicle control method according to one embodiment is described in more detail.

Figure 51 is a block diagram showing an unmanned aerial vehicle control system according to one embodiment.

As illustrated in FIG. 51, an unmanned aerial vehicle control system (5100) according to one embodiment may include a position coordinate processing unit (5110), an image processing unit (5120), a measurement value correction unit (5130), and a flight control unit (5140). These components may be implemented to execute steps (5010 to 5040) included in the method of FIG. 50.

In step (5010), the position coordinate processing unit (5110) can align the position coordinates with the route map data stored in the body of the unmanned aerial vehicle.

More specifically, the position coordinate processing unit (5110) can align the GPS coordinates of the unmanned aerial vehicle with the route map data for the flight of the unmanned aerial vehicle constructed in the layer. Here, the layer can identify the subject from the ground scanning data and visualize a space in which autonomous flight is possible. In addition, the route map data can align at least one or more of flight altitude restriction data, precision numerical map, and route information for avoiding military security areas or no-fly zones to the layer visualized in the space, thereby building an autonomous navigation map for the flight of the unmanned aerial vehicle in the space.

The position coordinate processing unit (5110) can identify a subject from ground scanning data and shape a space in which autonomous flight is possible into layers.

Here, the location coordinate processing unit (5110) may include a collection unit, an identification unit, an extraction unit, and a layer unit.

The collection unit can obtain a point cloud of a subject scanned by a ground scanning device mounted on a ground photographing aircraft. For example, the collection unit can obtain a point cloud of a subject on which a LiDAR pulse is projected by a LiDAR device mounted on a ground photographing aircraft. In addition, the identification unit can identify a subject by analyzing the point cloud collected by the collection unit. The extraction unit can extract a height value of a specific point of the subject identified by the identification unit by utilizing terrain elevation data.

The layer section can form a layer of area and altitude where an unmanned aerial vehicle can fly autonomously by connecting the height values of specific points of the subject extracted from the extraction section.

In addition, the position coordinate processing unit (5110) can check spatial geographic information to search for a safe path for flight, generate a flight path reflecting the safe path, and collect ground image data for the flight path.

And the position coordinate processing unit (5110) can set a flight altitude limit value and check the measured value of the radio altitude sensor through a subject capable of verifying the flight altitude limit height. In addition, the position coordinate processing unit (5110) can check the calibration information of the photographing device and check the flight information recorded in the flight data recorder (FDR) mounted on the unmanned aerial vehicle.

The image processing unit (5120) can process the altitude value of the image from the route map data.

The image processing unit (5120) can extract the altitude value of the image on the route by analyzing the change in the resolution of the image according to the distance from the subject. At this time, the measured value of the corrected radio altitude sensor can be used to control the speed of the unmanned aerial vehicle to maintain the flight altitude limit and the vertical separation of the route by the layer.

Furthermore, the image processing unit (5120) can align at least one of coordinate, altitude, attitude, and time information from a flight data recorder (FDR) mounted on an unmanned aerial vehicle with captured ground image data, and calculate an altitude value along a flight path by analyzing image distortion correction and image resolution change by referring to calibration information of a photographing device.

The measurement value correction unit (5130) can correct the measurement value of the radio altitude sensor using the altitude value of the image. This measurement value correction unit (5130) extracts the altitude value from the subject existing in the route and substitutes it into the route coordinate of the unmanned aerial vehicle at a certain interval, and when the unmanned aerial vehicle reaches the route coordinate, it recognizes the resolution height of the image corresponding to the coordinate where it contacts the subject, and can correct the measurement value of the radio altitude sensor of the unmanned aerial vehicle according to the resolution height.

In addition, the measurement value correction unit (5130) can support an offline image processing method to minimize risks to the communication and aircraft infrastructure environment during autonomous flight.

The measurement value correction unit (5130) repeatedly collects ground image data through autonomous flight of an unmanned aerial vehicle, reflects the collected ground image data in route control and ground control and route map data through resolution change analysis, and can create or verify a new route through simulation. For this purpose, a simulation verification system can be formed.

The flight control unit (5140) can control the flight altitude through gear control according to the measurement value of the calibrated radio altitude sensor.

Meanwhile, an unmanned aerial vehicle control system (5100) according to one embodiment may further include a route control unit, and the route control unit may receive FDR (Flight Data Recorder) data including radio altitude sensor, GPS, and INS information transmitted by the unmanned aerial vehicle through a transmitter, analyze the measured values of the GPS and radio altitude sensor, and the layer formation altitude information, thereby determining whether the unmanned aerial vehicle is maintaining vertical separation and flight altitude restrictions while flying autonomously.

Accordingly, the present embodiments provide a control technology for an unmanned aerial vehicle capable of autonomous flight beyond the visual line of sight, thereby overcoming the limitations of operation within a pilot's visible range in areas where it is difficult to maintain a constant altitude value due to ground objects, etc. In addition, by processing the altitude value of an image of a ground object by aligning the route map data and location coordinates stored in the unmanned aerial vehicle and correcting the measurement value of a radio altitude sensor, a method and system for controlling an unmanned aerial vehicle capable of safe autonomous flight of the unmanned aerial vehicle even beyond the visual line of sight can be provided.

Below, we will specifically describe the unmanned aerial vehicle control system according to different aspects.

Figure 52 is a block diagram showing an unmanned aerial vehicle control system according to another embodiment.

Referring to FIG. 52, an unmanned aerial vehicle control system (5200) according to another embodiment may include a flight driving unit (5210), a sensor unit (5220), a memory unit (5230), and a control unit (5240). According to an embodiment, the unmanned aerial vehicle control system (5200) may further include a wireless communication unit (5250).

The flight drive unit (5210) can generate lift and flight force for the flight of an unmanned aerial vehicle.

The sensor unit (5220) can measure the flight altitude of an unmanned aerial vehicle.

The memory unit (5230) can store route map data generated by the control center according to the mission of the unmanned aerial vehicle and program commands for the flight of the unmanned aerial vehicle.

The control unit (5240) can control the flight drive unit (5210) to fly a route on a layer defined in the stored route map data, and maintain the flight altitude defined in the layer by using the comparison result of the radio altitude sensor measurement value that measures the flight altitude from the sensor unit (5220) and the resolution height corresponding to the waypoint on the route.

Here, the layers are vertically separated and shaped in three-dimensional space to have a certain altitude value from the ground surface at which the unmanned aerial vehicle can fly depending on the mission, and the route is constructed on the layer and can include at least two waypoints.

The control unit (5210) compares the resolution height with the measured radio altitude sensor measurement value when the resolution height for the waypoint is stored in advance, and if there is a difference between the resolution height and the radio altitude sensor measurement value as a result of the comparison between the resolution height and the radio altitude sensor measurement value, the control unit (5210) can correct the radio altitude sensor setting value with the resolution height and control the flight drive unit (5210) to maintain the flight altitude using the corrected radio altitude sensor setting value.

This control unit (5240) can store the resolution value of the ground object located at the waypoint in the memory unit (5230) while maintaining the radio altitude sensor measurement value if the resolution height stored in advance at the waypoint is not stored.

When a failure occurs during a mission, the control unit (5240) can control the flight drive unit (5210) to move to a preset safe zone.

And when an emergency situation occurs during the mission, the control unit (5240) can switch to manual operation mode and control the flight drive unit (5210) to perform flight by operating the operating system of the unmanned aerial vehicle.

When a control unit (5240) needs to move to another layer while performing autonomous flight on an initially assigned layer, it can control the flight drive unit (5210) to move to a layer changeable section according to layer movement information and fly to the layer to be changed from the layer changeable section.

Here, layer movement information may be stored in advance in the memory unit (5230) or may be received from the control system via the wireless communication unit (5250).

And the wireless communication unit (5250) can communicate with the operating system of the unmanned aerial vehicle. Accordingly, if a flight failure occurs during the flight, the control unit (5240) can report the failure to the operating system of the unmanned aerial vehicle through the wireless communication unit (5250), or if an emergency situation occurs during the performance of a mission, transmit the information captured about the emergency situation to the operating system of the unmanned aerial vehicle through the wireless communication unit (5250).

In addition, the wireless communication unit (5250) can transmit a layer change request message of the unmanned aerial vehicle to the control system through communication with the control system, and receive layer movement information from the control system. Below, the unmanned aerial vehicle control system will be described in more detail using one embodiment.

Figure 30 is a block diagram of an unmanned aerial vehicle according to another embodiment, and each component can be connected electronically or mechanically.

Referring to FIG. 30, an unmanned aerial vehicle according to another embodiment may include a control unit (3000), a GPS receiver unit (3002), a pressure sensor (3004), an image sensor unit (3006), a radio altitude sensor unit (3008), an ultrasonic sensor unit (3010), a memory unit (3012), an acceleration sensor (3014), a payload drive unit (3016), a communication unit (3018), a flight drive unit (3020), a geomagnetic sensor (3022), and a gyroscope sensor (3024).

The GPS receiver (3002) can receive signals from GPS satellites and measure the current location, through which the control unit (3000) can determine the location of the unmanned aerial vehicle (3050). The control unit (3000) can include at least one central processing unit, which is a general purpose processor, and/or dedicated processors, such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field-programmable gate array), or a DSP (Digital Signal Processor).

The air pressure sensor (3004) measures the ambient atmospheric pressure of the unmanned aerial vehicle (3050) and transmits the value to the control unit (3000) to measure the flight altitude of the unmanned aerial vehicle (3050).

The image sensor unit (3006) can capture images of subjects using optical equipment such as a camera, and convert optical image signals received from the captured subjects into electrical image signals and transmit them to the control unit (3000).

The radio altitude sensor unit (3008) can measure the distance based on the radio wave arrival time according to the signal reflected from the ground surface by transmitting microwaves to the ground surface. Then, the measured value is transmitted to the control unit (3000), and an ultrasonic sensor unit or a synthetic aperture radar (SAR), etc. can be used. Accordingly, the control unit (3000) of the unmanned aerial vehicle (3050) can perform altitude measurement through the radio altitude sensor unit (3008) and observe ground objects and the ground surface at the same time.

The ultrasonic sensor unit (3010) is composed of a transmitter that transmits ultrasonic waves and a receiver that receives ultrasonic waves. The ultrasonic sensor unit measures the time until the transmitted ultrasonic waves are received and transmits this to the control unit (3000), thereby enabling the control unit (3000) to determine whether there is an object around the unmanned aerial vehicle (3050). Accordingly, the control unit (3000) controls the flight actuation unit (3020) to control the position and speed to avoid collisions when an obstacle exists around the unmanned aerial vehicle (3050) based on the measurement value measured by the ultrasonic sensor (3010).

The memory unit (3012) can store information (program commands) required for the operation of the unmanned aerial vehicle (3050), flight information related to route maps and autonomous flight, and various flight information identified during the flight. In addition, the memory unit (3012) can store resolution height information measured for each waypoint and radio altitude sensor measurement values.

The accelerometer (3014) is a sensor that measures the acceleration of the unmanned aerial vehicle (3050), and can measure the acceleration of the unmanned aerial vehicle (3050) in the x-axis, y-axis, and z-axis directions and transmit it to the control unit (3000).

The communication unit (3018) communicates with the ground control center and the unmanned aerial vehicle (3050) operating company via wireless communication, and periodically transmits and receives flight information and control information to and from the control center and the operating company. In addition, the communication unit (3018) may connect to a mobile communication network via a nearby mobile communication base station to communicate with the control center or the operating company. The control unit (3000) communicates with the operating system or the control system via the communication unit (3018), and when a remote control command is received from the operating system, the control unit (3000) may transmit a control signal to the flight drive unit (3020) to control the flight of the unmanned aerial vehicle according to the received remote control command, or may generate a control signal to the payload drive unit (3016) to drive the payload drive unit (3016) to collect or deliver an object.

Additionally, the control unit (3000) may transmit images collected through the image sensor unit (3006) to the operating system or control system through the communication unit (3018).

The geomagnetic sensor (3022) is a sensor that measures the Earth's magnetic field and transmits the measured value to the control unit (3000) so that it can be used to measure the direction of an unmanned aerial vehicle (3050).

And the gyro sensor (3024) measures the angular velocity of the unmanned aerial vehicle (3050) and transmits it to the control unit (3000), and the control unit (3000) can measure the inclination of the unmanned aerial vehicle (3050).

The control unit (3000) controls the overall functions of the unmanned aerial vehicle (3050) according to an embodiment of the present invention, and can perform the methods of FIGS. 26 and 27. The control unit (3000) performs overall control so that the unmanned aerial vehicle (3050) flies according to a route stored in the memory unit (3012), and compares the altitude value measured by the radio altitude sensor (3008) with the resolution height acquired from the image sensor unit (3006) for each preset waypoint, so that the unmanned aerial vehicle (3050) can maintain a set flight altitude even when there is a ground object on the waypoint.

In addition, the control unit (3000) controls the payload drive unit (3016) to drop or collect cargo loaded on the payload of the unmanned aerial vehicle (3050) at a specific point or deliver it to a specific point, depending on the cargo delivery method of the unmanned aerial vehicle (3050).

At this time, if the payload drive unit (3016) of the unmanned aerial vehicle (3050) includes a hoist, when dropping or collecting cargo, the control unit (3000) can control the payload drive unit (3016) to lower the cargo to the delivery point or collect the cargo from the collection point using the hoist. More specifically, the unmanned aerial vehicle (3050) can deliver the cargo to the delivery point by lowering the rope to which the cargo is fixed by the distance from the flight altitude to the delivery point while maintaining the flight altitude corresponding to the set layer. In addition, when collecting cargo, after lowering the rope by the distance from the flight altitude to the collection point, if it is confirmed that the cargo is fixed to the hook of the rope, the control unit (3000) can control the payload drive unit (3016) to have the hoist roll up the rope.

In addition, the control unit (3000) can control the lift and flight speed of the unmanned aerial vehicle (3050) by controlling the flight drive unit (3020). Considering the flight altitude and resolution height measured by the radio altitude sensor unit (3008), the flight drive unit (3020) can be controlled so that the current flight altitude does not go beyond a set layer.

And, the control unit (3000) can control the flight drive unit (3020) to move to a layer changeable section, and after moving to the layer changeable section, control the flight drive unit (3020) to perform a flight for a layer change procedure according to information included in the layer movement information.

The flight drive unit (3020) generates lift and flight force of the unmanned aerial vehicle (3050), and may include a plurality of propellers or a motor or engine for controlling each propeller. The flight drive unit (3020) can control the three movement directions of the unmanned aerial vehicle (3050), namely roll, yaw, and pitch, under the control of the control unit (3000), thereby maintaining the movement direction, attitude, and flight altitude of the unmanned aerial vehicle (3050).

Figure 31 is a flowchart showing an operating method of an unmanned aerial vehicle operating system according to another embodiment.

Referring to FIG. 31, an operating method of an unmanned aerial vehicle operating system according to another embodiment may be performed by an operating system of an unmanned aerial vehicle (simply referred to as an operating system). At this time, the operating method of an unmanned aerial vehicle operating system according to another embodiment may be an operating method for operating an unmanned aerial vehicle according to another embodiment described in FIG. 30.

In step (3100), the operation system of the unmanned aerial vehicle may generate autonomous flight information of the unmanned aerial vehicle to perform autonomous flight, and in step (3105), may transmit an autonomous flight registration request message including the autonomous flight information to the control system. Here, the autonomous flight information of the unmanned aerial vehicle may include airframe information and flight mission information of the unmanned aerial vehicle.

Table 1 shows examples of aircraft information included in the autonomous flight information of unmanned aerial vehicles.

Table 2 shows examples of flight mission information included in the autonomous flight information of an unmanned aerial vehicle.

In the above Table 2, the takeoff and landing method field is a field that defines the takeoff and landing method of the unmanned aerial vehicle. If the field is "manual," the operator or manager of the unmanned aerial vehicle controls the takeoff and landing of the unmanned aerial vehicle, and if it is "automatic," the unmanned aerial vehicle performs takeoff and landing according to pre-programmed commands. The failure system field is a field that defines alternatives and procedures in case an emergency occurs in the unmanned aerial vehicle and it becomes uncontrollable. For example, information on a landing zone, which is a safe zone where the unmanned aerial vehicle can land in an emergency situation such as when communication with the control system is cut off, information on returning to the takeoff point, flight route information to the safe zone, and procedures such as parachute deployment for a safe landing in case of loss of flight power may be included. In addition, the Failure System field includes information on points to prevent collisions with civilians or residential facilities in the event of an emergency for the unmanned aerial vehicle, so that in the event of an emergency, the unmanned aerial vehicle can fly away from densely populated civilian areas according to the information defined in the Failure System field, thereby preventing damage to civilians to the greatest extent possible.

In step (3110), the operating system can check whether a recommended transponder loading request message for the unmanned aerial vehicle that requested autonomous flight registration to the control system in step (3105) has been received from the control system. Here, the recommended transponder message received from the control system may mean that the transponder already loaded on the unmanned aerial vehicle does not satisfy certain conditions for control or the transponder is not loaded. In order for the control system to identify the unmanned aerial vehicle and monitor the flight of the unmanned aerial vehicle, it is desirable for the unmanned aerial vehicle to be loaded with a transponder capable of communicating with the control system.

In step (3110), if a message requesting installation of a recommended transponder is received by the operating system, the operating system can install the transponder recommended by the control system on the unmanned aerial vehicle in step (3115), and receive a test result from the control system in step (3120).

On the other hand, if the recommended transponder loading request message is not received or the test result is received, it means that the control system has completed the authentication (test) of the transponder loaded on the UAV, so in step (3125), the operating system can check whether an autonomous flight approval message has been received.

If an autonomous flight approval message is not received at step (3125), the operating system may regenerate autonomous flight information for autonomous flight of the unmanned aerial vehicle and retransmit an autonomous flight registration request message at step (3100).

Additionally, in step (3130), the operating system may be assigned an authorized route and layer from the control system when an autonomous flight approval message is received, and in step (3135), the operating system may download the authorized route and layer to the unmanned aerial vehicle to perform autonomous flight.

At step (3140), the operating system can operate the unmanned aerial vehicle along an authorized layer and route according to the start and end times of the flight of the unmanned aerial vehicle.

An example of an autonomous flight approval message transmitted from a control system to an operating system according to another embodiment may be presented in Table 3 below.

Referring to Table 3, the authentication code and identifier are identification information for identifying the unmanned aerial vehicle, and the layer information is layer information assigned according to the mission of the unmanned aerial vehicle. The mission code is information indicating which mission the unmanned aerial vehicle corresponds to among the missions of delivery, crime surveillance, reconnaissance, forest fire surveillance, surveying, rescue activities, weather measurement, and air pollution measurement. In addition, the unmanned aerial vehicle identification information is information for identifying the unmanned aerial vehicle from the control system after the authentication procedure between the unmanned aerial vehicle and the control system, and the safety regulation information can indicate information about safety regulations corresponding to the mission code, if any.

FIG. 32 is a flowchart illustrating a method of an unmanned aerial vehicle operating system according to another embodiment.

Referring to FIG. 32, an operating method of an unmanned aerial vehicle operating system according to another embodiment may be performed by an operating system of an unmanned aerial vehicle. At this time, the operating method of an unmanned aerial vehicle operating system according to another embodiment may be included in an operating method for operating an unmanned aerial vehicle according to another embodiment described in step (3140) of FIG. 31.

At step (3200), the operating system can select an unmanned aerial vehicle capable of performing a mission from among a plurality of unmanned aerial vehicles.

And, in step (3202), the operating system can receive route information and layer information corresponding to the mission from the control system. Here, the route information and layer information can be included in the autonomous flight approval message and received from the control center. At this time, step (3202) can include steps (3100) to (3125) described in FIG. 31.

In step (3204), the operating system may download the route information and layer information received by the selected unmanned aerial vehicle, and when the mission start time of the selected unmanned aerial vehicle arrives in step (3206), the operating system may instruct the start of the mission. Here, when the operating system transmits a mission start message to the unmanned aerial vehicle, the unmanned aerial vehicle may start a flight to perform the set mission according to the flight start time information included in Table 3 above.

In step (3208), the operating system may periodically receive flight information from the unmanned aerial vehicle, or, when an event occurs in the unmanned aerial vehicle, receive flight information about the event. Here, the event may include the results of self-diagnosis continuously performed by the unmanned aerial vehicle during flight, when a failure occurs, or when an incident or accident occurs during mission execution.

In step (3210), if it is confirmed from the received flight information that a failure of the unmanned aerial vehicle has occurred, the operating system may command the unmanned aerial vehicle to move to a pre-determined recovery point in step (3222). Here, the recovery point may be determined in advance between the control system and the operating system, and may be a point determined not to be assigned to other unmanned aerial vehicles in normal times so that it can be used only in emergency situations, or may be a point defined in advance as a safe zone. In addition, the layer and route used to move to the recovery point when a failure of the unmanned aerial vehicle occurs may be an emergency layer and route set by the control system so that it can be used only in emergency situations.

Meanwhile, the operating system may also activate means to prevent impact from ground collision, such as a parachute, in the event of a catastrophic failure that prevents the drone from reaching a recovery point.

In step (3224), the operating system can select an unmanned aerial vehicle that can replace the mission of the unmanned aerial vehicle that has failed among the unmanned aerial vehicles in the standby state. At this time, it is assumed that the unmanned aerial vehicle that can be replaced has already undergone an authentication procedure from the control system.

In step (3226), the operating system downloads route information and layer information of the disabled unmanned aerial vehicle to the selected unmanned aerial vehicle, instructs the unmanned aerial vehicle to move to the point of failure in step (3228), and can instruct the unmanned aerial vehicle to continuously receive flight information from the substituted unmanned aerial vehicle and perform a mission.

In step (3230), the operating system can recover the malfunctioning unmanned aerial vehicle from the recovery point, identify the cause of the malfunction (3232), and transmit the cause of the malfunction to the control system (3234). At this time, the operating system can recover the malfunctioning unmanned aerial vehicle through a separate recovery unmanned aerial vehicle in step (3230). At this time, the recovery unmanned aerial vehicle is an aircraft that has completed an authentication procedure from the control system in advance, and stores emergency layer information and flight information for recovery in advance, so that when a recovery situation occurs, the flight for recovering the malfunctioning unmanned aerial vehicle can be performed immediately. On the other hand, if no malfunction occurs in the unmanned aerial vehicle performing the mission in step (3210), the operating system can receive unmanned aerial vehicle acquisition information in step (3212). Here, the information acquired may be images acquired through video equipment mounted on an unmanned aerial vehicle performing a mission, and may include, for example, images of crime scenes or incident scenes, images required for use in places requiring continuous surveillance, such as railroads, factory plants, pipelines, military fences, and prisons. It may also be images captured by a thermal imaging camera for maintenance of railroads, factories, buildings, etc. Additionally, if the mission of the unmanned aerial vehicle is weather measurement, air pollution measurement, etc., data measured during the flight may be the acquired information.

In step (3214), the operating system can check whether there is an abnormality in the route information and layer information through the flight information stored by the unmanned aerial vehicle during the flight when the unmanned aerial vehicle returns after performing the mission (3216), and in step (3218), if a change in the layer and route is required, the operating system can request a change in the layer and route of the unmanned aerial vehicle to the control system (3220).

On the other hand, in step (3214), the operating system can receive the drone flight information if the drone has not returned (3208). Also, in step (3218), the operating system can receive the route information and layer information corresponding to the mission if no change is required (3202).

Fig. 33 is a flow chart showing a method for operating an unmanned aerial vehicle of an operating system according to another embodiment, and is a flow chart showing a method in the case of a company that operates a product delivery service using an unmanned aerial vehicle. At this time, the method for operating an unmanned aerial vehicle of an operating system according to another embodiment may be included in the operating method for operating an unmanned aerial vehicle according to another embodiment described in step (3140) of Fig. 31.

Steps (3300) to (3320) of Fig. 33 overlap with those described in steps (3200) to (3220) of Fig. 32, so further description will be omitted.

In step (3322), the operating system may transmit a safe zone movement message to the malfunctioning unmanned aerial vehicle to command it to move to a predetermined safe zone. Here, the safe zone may be a point determined to be used only in emergency situations and not normally assigned to other unmanned aerial vehicles. In addition, the layer and route used by the unmanned aerial vehicle to move to the safe zone in the event of a malfunction may also be an emergency layer and route set by the control system so that they may be used only in emergencies.

In step (3324), the operating system can select an unmanned aerial vehicle that can replace the mission of a malfunctioning unmanned aerial vehicle among the unmanned aerial vehicles in a standby state. At this time, it is assumed that the replacement unmanned aerial vehicle has already undergone an authentication procedure from the control system.

At step (3326), the operating system may download route information and layer information of the disabled unmanned aerial vehicle to the selected unmanned aerial vehicle, and at step (3328), instruct the unmanned aerial vehicle to move to a safe zone.

In step (3330), the operating system can notify information about the occurrence of a failure and the delay in delivery of goods due to the failure to a control system controlling the unmanned aerial vehicle or a computer or portable terminal of the goods recipient.

In step (3332), the operating system can recover the malfunctioning drone, identify the cause of the malfunction through the drone's FDR, etc. (3334), and transmit the cause of the malfunction to the control system (3336).

Below, a method for performing a flight in an unmanned aerial vehicle according to one embodiment is described. This method for operating an unmanned aerial vehicle can be operated in conjunction with the unmanned aerial vehicle operating system described above.

A method for performing a flight in an unmanned aerial vehicle according to one embodiment may include a step of performing authentication for autonomous flight with a control center, a step of downloading route map data generated by the control center according to a mission of the unmanned aerial vehicle, a step of flying a route on a layer defined in the downloaded route map data when a time to perform the mission has arrived, and a step of maintaining a flight altitude defined in the layer by using a resolution height corresponding to a waypoint on the route. Here, the layer is vertically separated and shaped in a three-dimensional space to have a certain altitude value from the ground surface at which the unmanned aerial vehicle can fly according to the mission, and the route is constructed on the layer and may include at least two waypoints.

In addition, if a flight failure occurs during the flight, a step of reporting the failure to the operating system of the unmanned aerial vehicle may be further included.

Additionally, if a failure occurs during the mission, the step of moving to a preset safe zone may be included.

In the event of an emergency during the mission, the step of switching to manual operation mode and performing the flight by operating the operating system of the unmanned aerial vehicle may be further included.

In the event of an emergency during the mission, the method may further include transmitting information captured regarding the emergency to the operating system of the unmanned aerial vehicle.

Below we provide a more specific example of how to perform flight on an unmanned aerial vehicle.

Figure 34 is a flowchart showing the operation of an unmanned aerial vehicle according to another embodiment.

Referring to FIG. 34, the operation of the unmanned aerial vehicle according to another embodiment may be performed by the unmanned aerial vehicle. At this time, the operation of the unmanned aerial vehicle according to another embodiment may be the operation method of the unmanned aerial vehicle according to another embodiment described in FIG. 30.

At step (3400), the unmanned aerial vehicle may receive power supply from the operating system, and then perform a transponder authentication procedure with the control system at step (3402). The authentication procedure performed here may include not only a transponder authentication procedure, but also any procedure not described herein for authenticating the control system to control the unmanned aerial vehicle.

At step (3404), the unmanned aerial vehicle can download route information and layer information from the operating system, and at step (3406), check whether the flight start time has arrived.

At this time, the unmanned aerial vehicle waits until the flight start time if the flight start time has not arrived (3408), and starts the flight if the flight start time has arrived (3410). At this time, before starting the flight, a procedure for starting the flight (such as an operation check of the elevator, ailerons, rudder, etc.) can be performed, and the flight can be set to start if there is no abnormality in the procedure for starting the flight.

In step (3412), the unmanned aerial vehicle performs a flight along the waypoints defined in the downloaded layer and route, stores the flight information in step (3414) and reports it to the operating system or control system, and performs a predefined mission in step (3416). At this time, the unmanned aerial vehicle can perform the mission while maintaining a constant flight altitude defined in the layer.

At step (3418), the unmanned aerial vehicle can perform self-function diagnosis while performing a predetermined mission to check whether a failure has occurred (3420).

In step (3420), if a failure occurs in the unmanned aerial vehicle, the failure may be reported to the operating system or control system in step (3422), and the unmanned aerial vehicle may move to a predetermined recovery point in step (3424). At this time, the location to which the unmanned aerial vehicle moves may be a safe zone rather than a recovery point.

On the other hand, if no failure occurs in step (3420), the unmanned aerial vehicle checks whether the flight completion time has arrived in step (3426), and if the flight completion time has not arrived, the unmanned aerial vehicle continues the flight according to the waypoint in step (3412). On the other hand, if the flight completion time has arrived in step (3426), the unmanned aerial vehicle can return to the starting point in step (3428).

Fig. 35 is a flowchart showing an unmanned aerial vehicle control method of a control system according to another embodiment.

Referring to FIG. 35, a method for controlling an unmanned aerial vehicle of a control system according to another embodiment may be performed by the control system. At this time, the method for controlling an unmanned aerial vehicle of a control system according to another embodiment may be a method for controlling an unmanned aerial vehicle according to another embodiment described in FIG. 30.

In step (3500), the control system may receive an autonomous flight registration request from the operating system of the unmanned aerial vehicle, and perform an authentication procedure with the unmanned aerial vehicle for which registration was requested in step (3502). The authentication procedure performed here may include a procedure for authenticating whether a transponder mounted on the unmanned aerial vehicle is a recommended transponder.

If the control system fails to complete authentication for the unmanned aerial vehicle at step (3504), a recommended transponder loading request message requesting loading of a recommended transponder may be transmitted to the unmanned aerial vehicle or the unmanned aerial vehicle's operating system at step (3506).

On the other hand, if the authentication for the unmanned aerial vehicle is completed, in step (3508), the control system can obtain information and mission information of the unmanned aerial vehicle from the operating system and check whether there is pre-built route and layer information corresponding to the obtained information (step (3510)).

In step (3510), the control system performs a simulation using the acquired mission information if a pre-built route and layer exist in the database (3512), assigns the route and layer verified by the simulation result to the unmanned aerial vehicle (3514), and transmits the assigned route and layer information to the operating system (3516). Then, the control system performs control for the unmanned aerial vehicle reported to have started flight in step (3518).

Meanwhile, if the pre-built route and layer in step (3510) do not exist in the database, in step (3520), the control system selects an unmanned aerial vehicle that meets the conditions of the unmanned aerial vehicle that has applied for autonomous flight from among the unmanned aerial vehicles that are held in advance, and can build a new route and layer through the selected unmanned aerial vehicle.

In step (3522), the control system can transmit the constructed new route and layer information to the operating system of the unmanned aerial vehicle, and store the constructed new route and layer information in a database in step (3524).

Figure 36 is a flowchart showing an unmanned aerial vehicle operation method of an operating system according to another embodiment.

Referring to FIG. 36, an unmanned aerial vehicle operating method of an operating system according to another embodiment may be performed by the operating system. At this time, the unmanned aerial vehicle operating method of an operating system according to another embodiment may be a method for operating an unmanned aerial vehicle according to another embodiment described in step (3140) of FIG. 31.

At step (3600), the operating system may receive flight-related information and images acquired by the unmanned aerial vehicle from the unmanned aerial vehicle, and at step (3602), check whether an event has occurred. Here, the event may include the occurrence of a crime, the occurrence of an incident such as a fire, the occurrence of cracks in structures such as buildings, etc.

If an event occurs, the operating system can check whether the operation mode of the unmanned aerial vehicle has changed to manual control at step (3604). The procedure for checking whether the operating system has changed to manual control at step (3604) can be checked by checking whether a manual control command has been input from the operator.

In step (3604), if the operation mode for the unmanned aerial vehicle is not changed to manual control, in step (3612), the operating system can process an event that has occurred by pre-programmed commands. For example, if the occurrence event is a crime, if program commands for operations such as taking a high-magnification picture of an object or a moving object at the corresponding location or transmitting a picture taken using equipment such as a night vision device to the operating system or tracking a moving object and flying are stored in the unmanned aerial vehicle, the unmanned aerial vehicle can process the occurrence event according to the pre-stored commands.

On the other hand, if the control is changed to manual control in step (3604), the operating system transmits a message to the unmanned aerial vehicle for controlling the unmanned aerial vehicle by control commands input from the operator in step (3606), and transmits processing commands for the point where the event occurred to the unmanned aerial vehicle in step (3608) to process the event that occurred in step (3602). For example, the operating system can process the event according to commands such as a camera angle adjustment command, a magnification adjustment command, a frame rate command, a voice transmission command, and a tracking command input by the operator.

And, in step (3610), the operating system can transmit event-related information received from the unmanned aerial vehicle to related organizations such as a police station, fire station, security company, military unit, or facility maintenance company.

Figure 37 is a block diagram of an unmanned aerial vehicle according to another embodiment.

Referring to FIG. 37, an unmanned aerial vehicle according to another embodiment may include a control unit (3700), a GPS receiver unit (3702), a pressure sensor (3704), an image sensor unit (3706), a radio altitude sensor unit (3708), an ultrasonic sensor unit (3710), a memory unit (3712), an acceleration sensor (3714), a payload drive unit (3716), a communication unit (3718), a flight drive unit (3720), a geomagnetic sensor (3722), a gyroscope sensor (3724), a power supply unit (3730), a fuel storage unit (3732), and a transponder (3734).

Components of an unmanned aerial vehicle according to another embodiment may perform some of the same functions as components of an unmanned aerial vehicle according to another embodiment described in FIG. 30. For example, according to another embodiment, the GPS receiver (3702), the air pressure sensor (3704), the image sensor unit (3706), the radio altitude sensor unit (3708), the ultrasonic sensor unit (3710), the memory unit (3712), the acceleration sensor (3714), the payload drive unit (3716), the communication unit (3718), the flight drive unit (3720), the geomagnetic sensor (3722), and the gyroscope sensor (3724) of the unmanned aerial vehicle are the GPS receiver (3002), the air pressure sensor (3004), the image sensor unit (3006), the radio altitude sensor unit (3008), the ultrasonic sensor unit (3010), the memory unit (3012), the acceleration sensor (3014), the payload drive unit (3016), the communication unit (3018), the flight drive unit (3020), the geomagnetic sensor (3022), and the gyroscope of the unmanned aerial vehicle according to another embodiment described in FIG. Since it can perform the same function as the sensor (3024), a redundant description will be omitted. Here, each component of the unmanned aerial vehicle according to another embodiment can be connected electronically or mechanically.

The power supply unit (3730) supplies power required for the operation of the unmanned aerial vehicle (3750) and may include an internal combustion engine such as an engine or a battery, and the fuel storage unit (3732) may store fuel such as petroleum when the power supply source of the unmanned aerial vehicle (3750) is an internal combustion engine such as an engine.

The transponder (3734) can perform authentication for the control system to identify the unmanned aerial vehicle (3750) and periodically transmit flight information, etc. for controlling the unmanned aerial vehicle (3750) to the control system.

The control unit (3700) may include at least one central processing unit, which is a general purpose processor, and/or dedicated processors, such as an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a digital signal processor (DSP), and may control the overall functions of the unmanned aerial vehicle (3750) according to the present invention and perform the methods described in FIG. 34.

The control unit (3700) performs overall control so that the unmanned aerial vehicle (3750) flies according to the route stored in the memory unit (3712), and compares the altitude value measured by the radio altitude sensor (3708) with the resolution height acquired from the image sensor unit (3706) for each preset waypoint, so that the unmanned aerial vehicle (3750) can maintain the set flight altitude even when there is a ground object on the waypoint.

In addition, if a failure occurs in the unmanned aerial vehicle (3750), the control unit (3700) can control the flight drive unit (3720) to move to a safe zone or recovery point stored in the memory unit (3712) and transmit failure-related information to the operating system through the communication unit (3718).

And, if the power supply of the power supply unit (3730) is lower than the power required for the operation of the unmanned aerial vehicle, the fuel of the fuel storage unit (3732) is below the minimum storage amount, or the operation of the flight drive unit (3720) is interrupted, the control unit (3700) can determine that a failure has occurred in the unmanned aerial vehicle (3750), and can transmit the fact of the failure to the operating system or the control system through the communication unit (3718).

In addition, when an event occurs, the control unit (3700) controls the image sensor unit (3706) according to a pre-determined event response procedure, thereby selecting an image acquisition direction and an image acquisition mode (infrared, X-ray, etc.), and controls the image acquired by the image sensor unit (3706) to be stored in the memory unit (3712) and transmitted to the operating system through the communication unit (3718).

In addition, the control unit (3700) controls the functional blocks for performing the corresponding command according to the command received from the operating system or the control system through the communication unit (3718). For example, when an image acquisition command is received from the operating system or the control system through the communication unit (3718), the control unit (3700) controls the angle of view, direction, resolution, magnification, etc. of the image acquisition module of the image sensor unit (3706) for image acquisition according to the image acquisition command, or controls the flight driving unit (3720) to control the flight direction for the operating system or the control system to acquire the desired image.

In addition, when a command to retrieve or deliver goods is received from an operating system or a control system through a communication unit (3718), the control unit (3700) controls the flight drive unit (3720) to move to the goods retrieve point or delivery point according to the received command, and controls the payload drive unit (3716) at the goods retrieve point or delivery point to perform an operation to retrieve or deliver the goods.

FIG. 38 is a block diagram illustrating an unmanned aerial vehicle, an operating system, and a control system according to another embodiment.

Referring to FIG. 38, another embodiment may include a control system (3800), an operating system (3850), and an unmanned aerial vehicle (3870). Each of the components of the control system (3800), the operating system (3850), and the unmanned aerial vehicle (3870) may be electrically connected through a bus (3812) interface to transmit and receive data and control signals.

The control system (3800) may include a simulation database (3802), a processor (3804), a memory (3806), a communication unit (3808), and a network interface (3810).

The communication unit (3808) of the control system (3800) performs wireless communication with the unmanned aerial vehicle (3870), and the network interface unit (3810) is connected to the network interface (3856) of the operating system (3850) to transmit and receive information. The memory (3806) can store program commands for the processor (3804) of the control system (3800) to operate according to embodiments of the present invention.

The simulation database (3802) of the control system (3800) stores simulation result information on layer information and route information for missions and specifications of unmanned aerial vehicles performing autonomous flight established by the control system. When a request for autonomous flight registration is received from the operating system (3850), the processor (3804) checks whether the simulation database (3802) contains layer information and route information corresponding to the requested autonomous flight, performs a simulation using specification information of the unmanned aerial vehicle performing the requested autonomous flight, and transmits the result to the operating system (3850).

The operating system (3850) may include an unmanned aerial vehicle interface unit (3851), a processor (3852), a memory (3854), a network interface (3856), and a communication unit (3858).

The communication unit (3858) of the operating system (3850) can transmit and receive various types of information with the communication unit (3874) of the unmanned aerial vehicle (3870) via wireless communication. The memory (3854) stores program commands that cause the processor (3852) of the operating system (3850) to operate according to embodiments of the present invention, and can also store specification information and mission information for a plurality of unmanned aerial vehicles (3870) operated by the operating system (3850).

The unmanned aerial vehicle interface unit (3851) is connected to a plurality of unmanned aerial vehicles located in the hangar of the operating system (3850) and can transmit various control information such as power supply, route information and layer information download, and flight mission assignment.

An unmanned aerial vehicle (3870) may include a processor (3872), a memory (3873), a communication unit (3874), a flight drive unit (3875), and an image acquisition unit (3876).

The processor (3872) of the unmanned aerial vehicle (3870) controls various operations related to the flight of the unmanned aerial vehicle to be performed, and the memory (3873) can store program commands performed by the processor (3872), route information, layer information, various flight information stored during the flight, and images acquired by the image acquisition unit (3876). The processor (3872) can generate a control signal for controlling each component for autonomous flight of the unmanned aerial vehicle (3870) according to the program commands related to the operation of the unmanned aerial vehicle (3870) stored in the memory (3873), route information, layer information, etc.

The communication unit (3874) can transmit and receive flight information, various data, and control information through wireless communication with the communication unit (3808) of the control system (3800) and the communication unit (3858) of the operating system (3850). The flight drive unit (3875) generates lift or flight force of the unmanned aerial vehicle (3870) under the control of the processor (3872), and the image acquisition unit (3876) can capture objects during flight under the control of the processor (3872). In particular, the communication unit (3874) transmits the image captured by the image acquisition unit (3876) to the communication unit (3858) of the operating system (3850) or the communication unit (3808) of the control system (3800), and transmits the control signal of the image acquisition unit (3876) received from the communication unit (3858) of the operating system (3850) or the communication unit (3808) of the control system (3800) to the processor (3872), thereby enabling the image acquisition unit (3876) to control various flight control operations for capturing images and operations for image acquisition.

In the above-described embodiments, it has been described that the unmanned aerial vehicle flies along a route on a pre-built layer, but below, it will be described that the unmanned aerial vehicle performs autonomous flight by changing layers.

According to another embodiment, a method for constructing a flight path for an unmanned aerial vehicle may include a step of identifying a subject from ground scanning data and forming a space in which an unmanned aerial vehicle can autonomously fly into a layer, a step of determining a waypoint for creating a flight path for the unmanned aerial vehicle on the formed layer, a step of collecting ground image data for the waypoint from the formed layer, a step of analyzing a change in image resolution according to a distance from the subject through the collected ground image data to extract altitude values for each waypoint, and a step of generating flight path information for the unmanned aerial vehicle including flight path information of at least one or more of the formed layer, waypoints, altitude values, and flight paths that are connecting lines between waypoints.

Here, a waypoint can represent a location of a ground object on the surface of the Earth at a point where an unmanned aerial vehicle performs autonomous flight on a layer, or a location where a predetermined task is performed.

The step of generating flight path information of an unmanned aerial vehicle may include a step of determining an arrival layer to which the unmanned aerial vehicle is scheduled to move when movement from a departure layer, which is the layer to which the unmanned aerial vehicle is initially assigned, to another layer, and a step of generating layer movement information for moving from the departure layer to the arrival layer.

The layer movement information may include at least one of a layer changeable section including a waypoint section for layer change during an autonomous flight of an unmanned aerial vehicle, a layer movement time, a change section entry time, and a change section entry angle.

Below, a method for constructing an unmanned aerial vehicle route according to another embodiment is described in more detail with an example.

FIG. 39 is a flowchart illustrating a method for generating flight path information of an unmanned aerial vehicle for autonomous flight between layers of an unmanned aerial vehicle according to one embodiment.

Referring to FIG. 39, a method for generating flight path information for an unmanned aerial vehicle for autonomous flight between layers of an unmanned aerial vehicle according to one embodiment may be performed by an unmanned aerial vehicle route construction system. Here, the unmanned aerial vehicle route construction system may be an unmanned aerial vehicle route construction system according to embodiments of the present invention, and may be the unmanned aerial vehicle route construction system described above or below.

In step (3900), the unmanned aerial vehicle route construction system identifies a subject from ground scanning data and shapes a space in which autonomous flight is possible into layers, collects ground image data for a flight path from the layers shaped in step (3910), and analyzes a change in image resolution according to the distance between a camera scanning the ground surface and the subject through the ground image data collected in step (3920) to extract an altitude value along the flight path. In addition, the unmanned aerial vehicle route construction system can correct the measurement value of a radio altitude sensor through route verification from the altitude value extracted in step (3930), and this may be optionally performed.

Thereafter, in step (3940), the unmanned aerial vehicle route construction system can generate layer information and a flight path for the unmanned aerial vehicle to fly autonomously. Here, the layer information can include information for identifying a layer assigned for flight among a plurality of layers in which the unmanned aerial vehicle can fly, information on layer altitude, area, etc., and the flight path information can include the positions of waypoints existing on the flight path of the unmanned aerial vehicle in each layer, altitude values, and a flight path that is a connecting line of each waypoint. Here, the flight path and the route can be used with the same meaning to indicate a path for the unmanned aerial vehicle to fly in a three-dimensional space.

In step (3950), the unmanned aerial vehicle route construction system can determine whether inter-layer movement is possible based on the airframe information or mission information of the unmanned aerial vehicle. For example, if the analysis results of weight, battery, fuel, and propellant information included in the airframe information of the unmanned aerial vehicle indicate that the unmanned aerial vehicle has the performance to enable inter-layer movement, it can be determined that inter-layer movement is possible. In addition, in step (3950), the unmanned aerial vehicle route construction system can determine whether inter-layer movement of the unmanned aerial vehicle is necessary based on the mission content included in the mission information of the unmanned aerial vehicle.

An example of flight mission information can be presented as in Table 4.

Referring to Table 4, in step (3950), the unmanned aerial vehicle route construction system determines whether the unmanned aerial vehicle can move between layers during flight based on flight mission information, and if movement between layers is possible, layer movement information for movement between layers of the unmanned aerial vehicle can be set in step (3960). The layer movement information may include at least one of layer changeable section position information including a waypoint section for changing layers during the route for autonomous flight of the unmanned aerial vehicle, layer movement time, change section entry time, and change section entry angle. In addition, the unmanned aerial vehicle route construction system reflects the layer movement information set in step (3960) in the autonomous flight route of the unmanned aerial vehicle in step (3970), and then generates unmanned aerial vehicle flight path information to which the layer movement information is reflected in step (3980), and transmits it to the unmanned aerial vehicle and the unmanned aerial vehicle's operating system in step (3990). At this time, the unmanned aerial vehicle route construction system can transmit a route map constructed only by the unmanned aerial vehicle if the unmanned aerial vehicle operation system does not exist. In addition, the method by which the unmanned aerial vehicle route construction system transmits the route map can be transmitted through a wired or wireless communication network or through a storage medium.

On the other hand, if inter-layer movement is not possible at step (3950), the unmanned aerial vehicle route construction system can generate unmanned aerial vehicle flight path information that does not include layer movement information at step (3980).

FIG. 40 is a flowchart illustrating a method for generating flight path information of an unmanned aerial vehicle for autonomous flight between layers of an unmanned aerial vehicle according to another embodiment.

Referring to FIG. 40, a method for generating flight path information for an unmanned aerial vehicle for autonomous flight between layers of an unmanned aerial vehicle according to another embodiment is shown, which can be performed by an unmanned aerial vehicle route construction system. In step (4000), the unmanned aerial vehicle route construction system can visualize layers according to the mission of the unmanned aerial vehicle. Here, the method for visualizing the layers can be performed according to the embodiments described above. Information about the mission of the unmanned aerial vehicle can be obtained from an operating system that has requested generation of flight path information for the unmanned aerial vehicle, and can also be obtained directly from the unmanned aerial vehicle.

When the layer is shaped, the unmanned aerial vehicle route construction system can determine waypoints for the flight of the unmanned aerial vehicle on the shaped layer at step (4005). The procedure for setting each waypoint has also been described above, so a detailed description will be omitted.

In step (4010), the unmanned aerial vehicle route construction system can generate a flight path within a layer connecting the determined waypoints, and in step (4015), can check whether the unmanned aerial vehicle needs to change layers during flight. Here, whether the layer change is needed can be determined based on the mission information or the aircraft information of the unmanned aerial vehicle, and can be confirmed based on whether a request is received from an operating system or the unmanned aerial vehicle.

Meanwhile, if a layer change is required, the unmanned aerial vehicle route construction system can determine the arrival layer to which the unmanned aerial vehicle will move in step (4020). Then, in step (4025), layer movement information for moving to the arrival layer can be generated, and unmanned aerial vehicle flight path information including the flight path generated in step (4030) and the layer movement information can be generated. At this time, the unmanned aerial vehicle flight path information generated in step (4030) can also include flight path information on the arrival layer.

On the other hand, if no layer change is required, the unmanned aerial vehicle route construction system can generate an unmanned aerial vehicle flight path including a flight path within the layer at step (4030).

In step (4035), the unmanned aerial vehicle route construction system can transmit the generated unmanned aerial vehicle flight path information to at least one of the operating system, the control system, and the unmanned aerial vehicle.

Figure 41 is a block diagram showing an unmanned aerial vehicle route construction system that constructs a route for inter-layer movement of an unmanned aerial vehicle according to another embodiment.

As illustrated in FIG. 41, an unmanned aerial vehicle route construction system (4100) according to another embodiment may include a layer shaping unit (4110), a data collection unit (4120), an altitude calculation unit (4130), a verification unit (4140), a layer change determination unit (4150), and a route generation unit (4160). Each component of the unmanned aerial vehicle route construction system may be a processor included in a server. These components may be implemented to execute steps (3900 to 3990) included in the method of FIG. 39 or steps (4000 to 4035) included in the method of FIG. 40 through an operating system including a memory and at least one program code.

In Fig. 41, the layer shaping unit (4110) calculates the height of the subject identified from scan data based on the ground altitude of the corresponding coordinates, and by connecting the heights of specific points, it is possible to shape a plurality of two-dimensional layers in a three-dimensional space, and the layers can form a vertical separation.

The data collection unit (4120) can initially collect ground image data from a layer at a flight altitude limit height. The data collection unit (4120) can obtain ground image data through a photographing device with a calibration value set at a specific altitude mounted on a ground photographing aircraft.

And the data collection unit (4120) can check the spatial geographic information to collect ground image data, search for a safe route for flight, generate a specific flight route, and collect ground image data for the route. In particular, the collection of the initial ground image data required for route analysis for route construction can secure maximum safety by only allowing flights within the visual range of a pilot with pilot qualification.

The data collection unit (4120) can set the height value of the flight altitude limit and check the measurement value of a radio altitude sensor (e.g., a radio altimeter, etc.) through an object for which the flight altitude limit height can be verified. Here, the object for which the flight altitude limit height can be verified can be a ground structure that is higher than or equal to the flight altitude limit.

Additionally, the data collection unit (4120) can check information such as specifications of the photographing device, such as resolution and image acquisition method, and calibration parameters according to the angle of incidence, and can check the flight information of the aircraft recorded in the flight data recorder (FDR) mounted on the unmanned aerial vehicle.

The altitude calculation unit (4130) can extract the altitude value along the flight path by analyzing the change in image resolution according to the distance between the camera and the subject through the collected ground image data.

The verification unit (4140) can correct the measurement value of the radio altitude sensor through route verification from the extracted altitude value. In addition, the verification unit (4140) can also perform verification on route information to be assigned to an unmanned aerial vehicle that requires a layer change in advance through simulation.

The layer change judgment unit (4150) determines whether a layer change of the unmanned aerial vehicle is necessary during autonomous flight of the unmanned aerial vehicle, and if a layer change is necessary, layer movement information for changing the layer can be defined and reflected in the flight path information for the unmanned aerial vehicle.

Here, the layer movement information may include at least one of the layer changeable section location information, the changeable section entry time, the layer movement time (layer movement start time and layer movement end time), and the changeable section entry angle. Accordingly, the layer change determination unit (4150) may determine whether the unmanned aerial vehicle can fly between layers, and then generate layer movement information for each unmanned aerial vehicle to prevent collision with other unmanned aerial vehicles performing autonomous flight, and include it in the route map. In addition, the layer change determination unit (4150) may not generate layer movement information when establishing a route, but may generate it when the unmanned aerial vehicle requests it.

The route generation unit (4160) can define waypoints for the flight of an unmanned aerial vehicle on a layer generated by the layer shaping unit (4110) for autonomous flight of the unmanned aerial vehicle, and can connect the defined waypoints to generate a route. In addition, if the layer change determination unit (4150) determines that a layer change is necessary during the flight of the unmanned aerial vehicle, an unmanned aerial vehicle route including inter-layer flight path information including flight path information within the layer and layer movement information for layer change can be generated. In addition, the route generation unit (4160) can set in advance a layer changeable section, which is a section for the unmanned aerial vehicle to move between layers. At this time, the route generation unit (4160) can set a section with a relatively lowest number of flights of unmanned aerial vehicles for each layer, and connect the sections to set a layer changeable section. In addition, when the route generation unit (4160) generates an unmanned aerial vehicle route for an unmanned aerial vehicle that requires a layer change, it can use the inter-layer flight path information simulated by the verification unit (4140). A description of the layer movement information and the procedure for the unmanned aerial vehicle to perform a flight to move between layers will be described in detail with reference to FIGS. 44 to 48.

FIG. 42 is a flowchart illustrating a method of operating an unmanned aerial vehicle for moving between layers according to one embodiment.

Referring to FIG. 42, in step (4202), the unmanned aerial vehicle can perform autonomous flight according to a set route. At this time, the autonomous flight of the unmanned aerial vehicle can be performed according to the embodiments described above. In step (4204), the unmanned aerial vehicle can determine whether a layer change is necessary while performing autonomous flight. The unmanned aerial vehicle can determine whether a layer change is necessary according to route information stored in advance, a predefined program, or a control command from a control system or an operating system.

For example, if there is a layer transfer route for inter-layer transfer to layer B while autonomously flying on layer A according to a route pre-stored in the unmanned aerial vehicle at step (4204), the unmanned aerial vehicle can determine that a layer change is necessary. For example, the unmanned aerial vehicle may fly a set route on layer A once according to a pre-defined program and then move to layer B through the layer transfer route, or may repeatedly fly the route on layer A and then move to layer B through the layer transfer route. These layer transfer procedures of the unmanned aerial vehicle may be defined on a route constructed in advance by a route construction system.

As another example, in step (4204), if a layer movement command is received from the control system or the operating system, the unmanned aerial vehicle may perform movement between layers if it determines that a layer change is necessary. In this case, if a layer movement command is received from the control system or the operating system, the unmanned aerial vehicle may move to a section where a layer change is possible and then move to another layer through the layer movement route.

As another example, at step (4204), the UAV may determine whether a layer change is necessary according to a predefined program. In this case, the predefined program may be a pre-stored action command such as "after a predetermined number of repeated flights of the route on layer A, move to layer B."

If no layer change is required, the unmanned aerial vehicle can determine whether autonomous flight has ended at step (4206), and if not, can perform autonomous flight along the route at step (4202).

On the other hand, if a layer change is required, the unmanned aerial vehicle moves to a layer changeable section in step (4208), and it can be confirmed in step (4210) whether it has entered the layer changeable section. At this time, whether the unmanned aerial vehicle has entered the layer changeable section can be confirmed by comparing the current location information of the unmanned aerial vehicle identified through GPS information, an altitude sensor, etc. with the predetermined layer changeable section location information. If it has not entered the layer changeable section, the unmanned aerial vehicle continues to move to the layer changeable section in step (4208). In step (4208), the unmanned aerial vehicle can move to the layer changeable section by using at least one of the layer changeable section information (layer changeable section location information, height information, size information), layer changeable time, entry point information for moving to another layer, entry angle information, entry speed information, arrival layer identification information, route information within the arrival layer, changeable section information, and information on other unmanned aerial vehicles within the changeable section included in the layer movement information.

On the other hand, if it enters the layer changeable section in step (4210), the unmanned aerial vehicle may perform an operation for changing the layer in step (4212), and may move to the layer (arrival layer) to be changed in step (4214). The operation of the unmanned aerial vehicle may include a series of operations in which the control unit or the processor controls the flight drive unit to adjust the lift and flight power for altitude increase or decrease. Here, the unmanned aerial vehicle may avoid collision with other aerial vehicles during layer movement by using at least one or more pieces of information from the layer changeable section information, the layer changeable time, the entry point information for moving to another layer, the entry angle information, the entry speed information, the arrival layer identification information, the route information within the arrival layer, the changeable section information, and the information of other unmanned aerial vehicles within the changeable section included in the layer movement information. In step (4214), the unmanned aerial vehicle may pass through at least one layer in order to move from the layer in which it is currently located to the arrival layer.

In step (4216), if the unmanned aerial vehicle reaches the layer to be changed (arrival layer), it can fly along the route assigned to the arrival layer in step (4220). On the other hand, if the unmanned aerial vehicle has not reached the arrival layer, it can continue to move until it reaches the arrival layer in step (4218). In step (4220), the unmanned aerial vehicle can receive route information assigned to the arrival layer via a wireless communication unit. In addition, the route information assigned to the arrival layer in step (4220) can be stored in advance in the memory of the unmanned aerial vehicle.

Fig. 43 is a block diagram of a route construction system for constructing a route for an unmanned aerial vehicle to move between layers according to another embodiment.

Referring to FIG. 43, the layer shaping unit (4302), the route determination unit (4304), the verification unit (4306), the flight path information generation unit (4308), the interface unit (4310), the memory (4312), and the control unit (4314) included in the route construction system (4300) are electrically connected through a bus interface (4316) and can transmit and receive data and control signals to each other.

The layer shaping unit (4302) can shape layers for autonomous flight of each unmanned aerial vehicle as described above. More specifically, the layer shaping unit (4302) identifies a subject from ground scanning data acquired through aerial photographs, etc., shapes a space in which autonomous flight is possible into layers, collects ground image data for a flight path (route) from the shaped layer, analyzes a change in image resolution according to a distance from the subject through the collected ground image data, extracts an altitude value of a flight path coordinate, and then sets a flight altitude value at a waypoint of each route. Here, the route may include a line connecting waypoints indicated in the same layer, and may also include a line connecting waypoints located in different layers.

The route determination unit (4304) determines the route along which the unmanned aerial vehicle will fly according to the mission of the unmanned aerial vehicle or the purpose required by the operation system, and can transmit the determined route information to the control unit (4314). At this time, the route determination unit (4304) determines a route for autonomous flight of the unmanned aerial vehicle among a plurality of waypoints (intra-layer waypoints) existing on the layer shaped by the layer shaping unit (4302). If the unmanned aerial vehicle needs to move between layers, the route determination unit (4304) can also generate layer movement information including a route connecting waypoints between layers (Inter-Layer waypoints) along which the unmanned aerial vehicle will move. The procedure by which the route determination unit (4304) determines a route for movement between layers of the unmanned aerial vehicle will be described with reference to FIGS. 44 to 48 below.

The verification unit (4306) uses the measured altitude values obtained through route verification for the altitude values included in the layer generated by the layer shaping unit (4302) to correct the measured values of the altitude sensor and transmit them to the flight path information generation unit (4308), so that they can be used in the future when creating an autonomous flight map or establishing a route.

The interface unit (4310) can transmit and receive data information and control information by communicating with an undesignated control system, operating system, or unmanned aerial vehicle via a wireless network/wired network.

The memory (4312) can store layers generated by the layer shaping unit (4302) for autonomous flight of the unmanned aerial vehicle and route information determined by the route determining unit (4304), and can store unmanned aerial vehicle flight route information generated by the flight route information generating unit (4308). In addition, the memory (4312) can store unmanned aerial vehicle identifiers for identifying unmanned aerial vehicles, layer identifiers, and flight route information assigned to each unmanned aerial vehicle. In addition, the flight route information generating unit (4308) can set in advance a layer changeable section, which is a section for the unmanned aerial vehicle to move between layers, like the route generating unit (4160) of FIG. 41. The control unit (4314) transmits and receives control signals and data signals with the layer shaping unit (4302), the route determination unit (4304), the verification unit (4306), the flight path information generation unit (4308), the interface unit (4310), and the memory (4312) through the bus interface (4316), and when a request for flight path information required for autonomous flight of an unmanned aerial vehicle is received, if the requested flight path information does not exist in the memory (4312), the control unit (4302), the route determination unit (4304), the verification unit (4306), and the flight path information generation unit (4308) can be controlled to generate flight path information.

On the other hand, if the requested flight path information exists in the memory (4312), the flight path information pre-stored in the memory (4312) can be read out and transmitted to the operating system, control system, or unmanned aerial vehicle through the interface unit (4310).

FIG. 44 is a drawing for explaining a method for an unmanned aerial vehicle to perform autonomous flight between layers according to one embodiment.

Referring to FIG. 44, it is assumed that there are N layers, including Layer 1 (4400), Layer 2 (4402), Layer 3 (4404), ..., Layer N (4406). Here, reference numbers 4410a, 4410b, 4410c, and 4410n may represent waypoints existing on each layer and flight paths connecting each waypoint.

In a method for an unmanned aerial vehicle to perform inter-layer autonomous flight according to one embodiment, the unmanned aerial vehicle can fly between layers (4450) from layer 1 (4400) to layer 2 (4402), layer 3 (4404), and then to layer N (4406), and the unmanned aerial vehicle can move between layer 1 (4400) and layer 2 (4402) via a layer movement path (4460), and the unmanned aerial vehicle can move between layer 2 (4405) and layer 3 (4404) via a layer movement path (4470).

That is, the drone can move between layers according to layer movement information, or pass through multiple layers to reach the destination layer.

FIG. 45 is a drawing illustrating a layer changeable section set for an unmanned aerial vehicle to move between layers according to one embodiment.

Referring to FIG. 45, it can be seen that an unmanned aerial vehicle moves from layer 1 (4510) to layer 3 (4530) via layer 2 (4520) while flying along a route. For convenience of explanation, layer 1 (4510) from which an unmanned aerial vehicle departs to move to another layer while performing autonomous flight in FIG. 45 is referred to as a departure layer, layer 3 (4530) to which the unmanned aerial vehicle intends to move is referred to as a destination layer, and layer 2 (4520) through which the unmanned aerial vehicle passes to arrive at layer 3 (4530) is referred to as a transit layer. Layer 2 (4520), which is a transit layer, is a layer that exists between the departure layer and the arrival layer, and may not exist if there is no layer through which the unmanned aerial vehicle passes.

In Fig. 45, it can be confirmed that there is a layer changeable section (4550) for an unmanned aerial vehicle to move from layer 1 (4510) to layer 3 (4530). The layer changeable section (4550) is a section that is set in advance to prevent collisions with other unmanned aerial vehicles when the unmanned aerial vehicle moves between layers, and can be defined as a section that all aircraft moving between layers must comply with. In other words, it is a section used only for the flight of an unmanned aerial vehicle that wishes to use a different layer. The layer changeable section (4550) can be determined by the route generation unit (4160) of Fig. 40 or the flight path information generation unit (4308) of Fig. 43. Reference numbers 4560 and 4570 show that an unmanned aerial vehicle moves between layers through ascending or descending flight within the layer changeable section (4550). In addition, in order to prevent collisions when changing layers of an unmanned aerial vehicle, the layer changeable section (4550) may be divided into an ascending section where the unmanned aerial vehicle is only allowed to ascend, and a descending section where the unmanned aerial vehicle is only allowed to descend.

In Fig. 45, the unmanned aerial vehicle may enter a layer changeable section (4550) by using at least one of the layer changeable time included in the layer movement information generated by the route construction system, information on an entry point for moving to another layer, information on an entry angle, information on an entry speed, information on an arrival layer identification, route information within the arrival layer, information on a changeable section, and information on another unmanned aerial vehicle within the changeable section, and may perform a flight for moving to another layer while preventing collision with another unmanned aerial vehicle.

Table 5 below shows information included in layer movement information according to one embodiment.

FIG. 46 is a vertical cross-sectional view illustrating a procedure for an unmanned aerial vehicle to move between layers according to one embodiment.

Reference number 4640 represents a layer changeable section for an unmanned aerial vehicle (4650) to move from layer 1 (4600) to layer 3 (4620) via layer 2 (4610). Referring to FIG. 46, as shown in reference number 4630, an unmanned aerial vehicle (4650) flies a route on layer 1 (4600) and then moves from layer 1 (4600) to layer 3 (4620) according to layer changeable section information, layer changeable time, entry point information for moving to another layer, entry angle information, and entry speed information included in layer movement information. Reference number 4660 represents a route for moving from layer 1 (4600) to layer 2 (4610), and reference number 4670 represents a route for moving from layer 2 (4610) to layer 3 (4620). Reference number 4680 shows an unmanned aerial vehicle (UAV) that has arrived at the arrival layer, layer 3 (4620), and is performing autonomous flight along a route on layer 3 (4620).

FIG. 47 is a drawing illustrating a procedure for an unmanned aerial vehicle to move between layers according to another embodiment.

FIG. 47 is a drawing showing that an unmanned aerial vehicle can perform a flight according to a route on layer 2 (4710), which is a transit layer, and move to another layer while moving between layers. That is, reference numeral 4730 in FIG. 47 shows that an unmanned aerial vehicle can depart from layer 1 (4700), perform a flight according to a route on layer 2 (4710), which is a transit layer, and move to layer 3 (4720). The unmanned aerial vehicle that has flown the route on layer 3 (4720) returns to layer 1 (4700), which is a departure layer, after passing through layer 2 (4710) again. FIG. 47 shows that the layer change section has a separate section for descending flight (4740) and a section for ascending flight (4750). That is, section reference number 4740 is a section set for the purpose of layer movement through descending flight, and section reference number 4750 is a section set for the purpose of layer movement through ascending flight.

FIG. 48 is a drawing illustrating a procedure for an unmanned aerial vehicle to move between layers according to another embodiment.

Fig. 48 shows that, unlike Fig. 47, there are no separate ascending and descending flight sections for layer movement. Fig. 48 shows that an unmanned aerial vehicle flies a route (4830) on layer 1 (4800), then performs a descending flight (4840) to move from the last waypoint to layer 2 (4810), then flies a route (4850) on layer 2 (4810), then performs a descending flight (4860) to move from the last waypoint to layer 3 (4820), and then flies a route (4870) on layer 3 (4820). A flight like Fig. 48 can also be used for an unmanned aerial vehicle flying for purposes such as patrol and surveillance.

As illustrated in FIGS. 47 and 48, the procedure for an unmanned aerial vehicle to perform autonomous flight while moving between layers can be defined in advance at the time of establishing a route, and can also be carried out according to control commands from a control system, operating system, etc. during the flight of the unmanned aerial vehicle.

FIG. 49 is a method flow diagram of an unmanned aerial vehicle and a control system for layer movement of an unmanned aerial vehicle according to another embodiment.

In step (4902), the unmanned aerial vehicle (4900) performs a flight report for autonomous flight to the control system (4950), and can perform autonomous flight in step (4904).

In step (4906), the control system (4950) performs control of the unmanned aerial vehicle (4900), and the unmanned aerial vehicle (4900) performs set operations such as reporting acquired information, surveillance work, delivery work, and rescue work according to the control of the control system (4950), and can periodically report its own status information.

At step (4908), the unmanned aerial vehicle (4900) determines whether a layer change is necessary, and if a layer change is necessary, it can transmit a layer change request message to the control system at step (4910).

At step (4912), the control system (4950) can determine whether to approve a layer change for the unmanned aerial vehicle (4900). At this time, the control system (4950) can determine whether to accept the layer change request by considering the possibility of collision with other unmanned aerial vehicles existing within the layer changeable section and the airframe information and mission information of the unmanned aerial vehicle (4900).

If the layer change is not accepted at step (4912), the control system transmits a layer change not possible message at step (4913). On the other hand, if the layer change is accepted at step (4912), the control system can generate layer movement information at step (4914) and transmit the layer movement information to the unmanned aerial vehicle (4900) at step (4916).

The unmanned aerial vehicle (4900) that has received the above layer movement information can move to a layer changeable section according to the above layer movement information in step (4918) and then perform movement flight to the arrival layer.

In step (4920), the unmanned aerial vehicle (4900) that has reached the arrival layer can determine whether route information on the arrival layer is required to perform autonomous flight on the arrival layer. If the arrival layer route information is required in step (4920), the unmanned aerial vehicle (4900) can transmit a route request message to the control system (4950) in step (4922).

The control system (4950) that receives the above route request message transmits route information on the arrival layer that the unmanned aerial vehicle (4900) has reached to the unmanned aerial vehicle (4900) at step (4924), and the unmanned aerial vehicle (4900) can perform autonomous flight on the arrival layer according to the route information received from the control system (4950) at step (4926).

On the other hand, if arrival layer route information is not required in the above step (4920), the unmanned aerial vehicle (4900) can perform autonomous flight according to route information stored in advance in step (4926).

Table 6 below shows the information included in the layer change request message transmitted by the unmanned aerial vehicle (4900) to the control system (4950) at step (4910).

In the above-described embodiments, it has been described that the unmanned aerial vehicle performs layer movement in a changeable section. However, in cases where the route on the departure layer and the route on the arrival layer do not overlap, it is possible for the unmanned aerial vehicle to perform a flight procedure for layer movement without moving to the changeable section.

In this way, by providing an autonomous flight route beyond the visual line of sight according to the embodiments, it is possible to overcome the limitations of operation within the pilot's visual range in areas where it is difficult to maintain a constant altitude value due to ground objects, etc.

In addition, according to embodiments, by using scanning data to extract elevation and obstacle height information, and analyzing changes in image resolution of ground image data to utilize the extracted ground object height information, calibration verification and correction of measurement values of the radio altitude sensor of the unmanned aerial vehicle can be performed, thereby establishing a safe autonomous flight route for the unmanned aerial vehicle.

Furthermore, while the construction of an autonomous flight route is described as being done on a layer previously constructed using ground scanning data, if a layer is set in advance without ground scanning data and a safe flight altitude is determined using only the resolution height information obtained through a test flight of an actual drone for the autonomous flight route constructed on the set layer, it is also possible to construct an autonomous flight route using this.

In addition, by using the height value of the ground object resolution to calibrate the ultrasonic altitude sensor value for the ground object to verify the set layer by utilizing the scanned point cloud and the extracted DTM and DSM for the existing ground object, the safety of the route can be verified, so that new layers and routes can be set through simulation without additional scanning data for the new route. In addition, by setting the maximum flight limit altitude of the unmanned aircraft, collision with the manned aircraft can be prevented.

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone, but those skilled in the art will appreciate that the processing device may include multiple processing elements and/or multiple types of processing elements. For example, the processing device may include multiple processors, or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

The software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may, independently or collectively, command the processing device. The software and/or data may be embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave, for interpretation by the processing device or for providing instructions or data to the processing device. The software may be distributed over network-connected computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the embodiment or may be those known to and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, flash memories, etc. Examples of the program commands include not only machine language codes generated by a compiler but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

Although the embodiments have been described above by way of limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, appropriate results can be achieved even if the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or are replaced or substituted by other components or equivalents.

Therefore, other implementations, other embodiments, and equivalents to the claims are also included in the scope of the claims described below.

Claims (20)

A step of identifying a subject from ground scanning data and shaping a space where autonomous flight is possible into layers;
A step of collecting surface image data for a flight path from the above-mentioned shaped layer; and
A step of extracting the altitude value along the flight path by analyzing the change in image resolution according to the distance from the subject through the collected above-mentioned index image data.
A method for establishing an unmanned aerial vehicle route including: In the first paragraph,
A step for correcting the measurement value of the radio altitude sensor through route verification from the extracted altitude value.
A method for establishing an unmanned aerial vehicle route including: In the first paragraph,
The step of shaping the space where autonomous flight is possible into layers is:
A step of acquiring a point cloud of the subject scanned by a surface scanning device mounted on a surface photographing aircraft;
A step of identifying the subject by analyzing the collected point cluster;
A step of extracting a height value of a specific point of the identified subject by utilizing terrain elevation data; and
A step of connecting the height values of specific points of the extracted subject to visualize the area and altitude where autonomous flight of an unmanned aerial vehicle is possible in space as the layer.
A method for establishing an unmanned aerial vehicle route including: In the third paragraph,
The steps for obtaining the above point clusters are:
Obtaining the point cluster of the subject to which the LiDAR pulse is projected through the LiDAR device mounted on the above-mentioned surface photographing aircraft.
A method for establishing an unmanned aerial vehicle route, characterized by: In the first paragraph,
The step of shaping the space where autonomous flight is possible into layers is:
A step of creating multiple two-dimensional layers in the above space.
A method for establishing an unmanned aerial vehicle route including: In the first paragraph,
The steps of collecting the above indicator image data are:
A step of acquiring the ground image data through a photographing device with a calibration value set at a specific altitude mounted on a ground photographing aircraft.
A method for establishing an unmanned aerial vehicle route including: In the first paragraph,
The steps of collecting the above indicator image data are:
A step of checking spatial geographic information to find a safe path for flight; and
A step of generating a flight path reflecting the above safety path and collecting the above-mentioned surface image data for the above-mentioned flight path.
A method for establishing an unmanned aerial vehicle route including: In the second paragraph,
The step of correcting the measurement value of the above radio altitude sensor is:
A step of extracting an altitude value from an object existing in a route and substituting it into the route coordinates of an unmanned aerial vehicle at regular intervals, and recognizing the resolution height of an image corresponding to the coordinates at which the unmanned aerial vehicle contacts the object when the unmanned aerial vehicle reaches the route coordinates; and
- A step for correcting the measurement value of the radio altitude sensor of the unmanned aerial vehicle according to the above resolution height.
A method for establishing an unmanned aerial vehicle route including: In the second paragraph,
The step of correcting the measurement value of the above radio altitude sensor is:
Repeatedly collect the above-mentioned ground image data through autonomous flight of an unmanned aerial vehicle, and reflect the collected above-mentioned ground image data in route control and ground control and route map data through analysis of resolution changes, and create or verify a new route.
A method for establishing an unmanned aerial vehicle route, characterized by: A layer shaping unit that identifies subjects from ground scanning data and shapes spaces where autonomous flight is possible into layers;
A data collection unit for collecting surface image data for a flight path from the above-described shaped layer; and
An altitude calculation unit that extracts the altitude value of the flight path coordinates by analyzing the change in image resolution according to the distance from the subject through the collected above-mentioned index image data.
An unmanned aerial vehicle route construction system including: In Article 10,
A verification unit that corrects the measurement value of the radio altitude sensor through route verification from the extracted altitude value.
An unmanned aerial vehicle route construction system including: In Article 10,
The above layer shaping part is,
A collection unit that acquires a point cloud of the subject scanned by a surface scanning device mounted on a surface photographing aircraft;
An identification unit that identifies the subject by analyzing the collected point cluster;
An extraction unit that extracts the height value of a specific point of the subject identified by utilizing terrain elevation data; and
A layer section that connects the height values of specific points of the extracted subject to form the area and altitude where autonomous flight of an unmanned aerial vehicle is possible in space as a layer.
An unmanned aerial vehicle route construction system including: In Article 10,
The above data collection unit,
Checking spatial geographic information to find a safe path for flight, generating a flight path reflecting the safe path, collecting the surface image data for the flight path, and obtaining the surface image data through a photographing device with a calibration value set at a specific altitude mounted on a surface photographing aircraft.
An unmanned aerial vehicle route construction system featuring: In Article 10,
The above data collection unit,
Check the calibration information of the shooting device and check the flight information recorded in the Flight Data Record (FDR) installed on the unmanned aerial vehicle.
The above altitude mountain range is,
Matching at least one of coordinate, altitude, attitude, and time information from the flight data recorder (FDR) mounted on the unmanned aerial vehicle with the captured ground image data, and calculating the altitude value on the flight path by correcting the distortion of the image and analyzing the change in the image resolution by referring to the calibration information of the photographing device.
An unmanned aerial vehicle route construction system featuring: In Article 11,
The above verification department,
Extracting altitude values from a subject existing in a route and substituting them into the route coordinates of an unmanned aerial vehicle at regular intervals, and when the unmanned aerial vehicle reaches the route coordinates, recognizing the resolution height of the image corresponding to the coordinates at which it contacts the subject, and - correcting the measurement value of the radio altitude sensor of the unmanned aerial vehicle according to the resolution height.
An unmanned aerial vehicle route construction system featuring: In Article 11,
The above verification department,
Repeatedly collect the above-mentioned ground image data through autonomous flight of an unmanned aerial vehicle, and reflect the collected above-mentioned ground image data in route control and ground control and route map data through analysis of resolution changes, and create or verify a new route.
An unmanned aerial vehicle route construction system featuring: A step of identifying a subject from ground scanning data and shaping a space where an unmanned aerial vehicle can fly autonomously into a layer;
A step of determining waypoints for generating a route of the unmanned aerial vehicle on the shaped layer;
A step of collecting surface image data for the waypoint from the above-mentioned shaped layer;
A step of extracting altitude values at each waypoint by analyzing the change in image resolution according to the distance from the subject through the collected index image data; and
A step of generating flight path information of an unmanned aerial vehicle including at least one flight path information among the above-described layer, the above-described waypoints, the above-described altitude values, and the flight path which is a connecting line between the above-described waypoints.
A method for establishing an unmanned aerial vehicle route including: In Article 17,
The above waypoint is,
Indicates the location of ground objects on the ground surface at the point where the above-mentioned unmanned aerial vehicle performs autonomous flight on the above-mentioned layer or indicates the location where the above-mentioned unmanned aerial vehicle performs a predetermined mission.
A method for establishing an unmanned aerial vehicle route, characterized by: In Article 17,
The step of generating flight path information of the above unmanned aerial vehicle is:
When the drone needs to move from the departure layer, which is the layer to which the drone was initially assigned, to another layer, a step of determining the arrival layer to which the drone is scheduled to move; and
A step of generating layer movement information for moving from the departure layer to the arrival layer;
A method for establishing an unmanned aerial vehicle route including: In Article 19,
The above layer movement information is,
A layer changeable section including a waypoint section for changing layers among the route for autonomous flight of the above unmanned aerial vehicle, including at least one of the layer movement time, change section entry time, and change section entry angle
A method for establishing an unmanned aerial vehicle route, characterized by:

Images