JP4491293B2 - Model forming apparatus and model forming method - Google Patents

Description

  According to the present invention, based on given conditions such as image resolution, position accuracy, range of a model non-formation part, etc., an imaging condition such as an imaging position, an imaging direction, etc. is presented to a photographer. The present invention relates to a model forming apparatus and a model forming method used for forming an original model.

  As described in Patent Document 1 relating to the proposal of the present applicant, due to recent advances in digital technology, detailed 3D data and high-resolution images of a measurement object can be obtained using a laser scanner or digital photogrammetry. Data can be easily acquired. For example, according to devices such as close-up photogrammetry using a digital camera and ground-type surveying instruments, detailed measurement can be performed with respect to an object side surface from the ground.

JP 2002-352224 A FIG.

  However, it is obtained by a surveying instrument that measures the position of the measurement point or the distance and direction to the measurement point, such as an automatic tracking surveyor or a non-prism surveying instrument, or a laser scanner (hereinafter simply referred to as “surveying instrument”). The 3D data basically consists of three-dimensional coordinate data including distance data. Therefore, it is difficult to associate the 3D data acquired by the surveying instrument with the on-site situation, and it is difficult to determine which position of the measurement object when performing the operation of associating the 3D data with the image information of the measurement object. Thus, it has been found that there is a problem that the three-dimensional model formation work of the object becomes complicated. In addition, 3D data acquired by a surveying instrument installed on the ground cannot acquire 3D data from the sky of a building roof or a road, for example, and it is difficult to form a three-dimensional model of an object. was there.

  On the other hand, digital cameras and laser scanner devices for installation on flying objects such as helicopters and aircraft have been developed in recent years. However, when a 3D model of an object is created by combining such a photographed image from the sky and a photographed image from the ground, the image distortion based on the difference in the shooting direction and the difference in shooting scale and lens aberration is large. Thus, there is a problem that the assembled load of photographed images is large and it is difficult to form a three-dimensional model of the object.

  SUMMARY OF THE INVENTION The present invention solves the above-described problem, and uses a three-dimensional model data of an object already obtained as a starting point, and a model forming apparatus and a model forming device that can easily and efficiently form a three-dimensional model of the object It aims to provide a method.

  For example, as shown in FIG. 1, the model forming apparatus of the present invention that achieves the above object includes a photographing unit 110 that photographs a target object 10, an image data storage unit 130 that stores three-dimensional model data of the target object, and a target A display unit 140 that displays a three-dimensional model based on the three-dimensional model data of the object 10, and a recognition unit 150 that recognizes a model-unformed part of the object 10 based on the three-dimensional model data stored in the image data storage unit 130. And an imaging instruction information unit 160 for obtaining imaging instruction information related to imaging of the model-unformed part, and based on the imaging instruction information obtained by the imaging instruction information unit 160, Shooting is performed.

  In the apparatus configured as described above, it is necessary when the recognition unit 150 recognizes the missing shooting direction with respect to the object, and the shooting instruction information unit 160 performs supplementary shooting of the missing shooting direction. The photographing instruction information to be obtained is obtained and displayed on the display unit 140 so that the photographer can recognize it. Then, it is suitable for the photographer to replenish image data according to an appropriate photographing mode from the photographing direction missing with respect to the object.

  For example, as shown in FIG. 19, the model forming apparatus of the present invention that achieves the above object includes an imaging unit 110 that images the object 10 and at least three-dimensional model data of the object, or an object imaged by the imaging unit 110. The image data storage unit 130 that stores the object image data, the model forming unit 155 that obtains the model data of the object 10 from the target image data stored in the image data storage unit 130, and the three-dimensional model data of the object 10 A display unit 140 for displaying a three-dimensional model based on the three-dimensional model data stored in the image data storage unit 130, a recognition unit 150 for recognizing a model-unformed portion, and imaging of the model-unformed portion. And a display unit 140 that displays both the three-dimensional model and the shooting instruction information.

  In the apparatus configured as described above, it is necessary when the recognition unit 150 recognizes the missing shooting direction with respect to the object, and the shooting instruction information unit 160 performs supplementary shooting of the missing shooting direction. The photographing instruction information to be obtained is obtained and displayed on the display unit 140 so that the photographer can recognize it. Then, it is suitable for the photographer to replenish image data according to an appropriate photographing mode from the photographing direction missing with respect to the object. The model forming unit 155 obtains model data of the target object 10 from the target image data stored in the image data storage unit 130. Since the 3D model is displayed on the display unit 140 based on the 3D model data of the object 10, the object recognized by the photographer in addition to the model unformed part recognized by the recognition unit 150. Recognize the shooting direction to be supplemented. Therefore, the photographer can replenish image data in an appropriate photographing mode from the photographing direction to be missing or supplemented with respect to the object.

  In the model forming apparatus of the present invention, for example, as shown in FIG. 19, the model is further divided into a plurality of areas for the model formed by the model forming unit 155. And a texture pasting unit 192 for pasting the texture of the object image data to each of the divided areas, and the display unit 140 displays the textured model formed by the texture pasting unit 192. Good.

  In the model forming apparatus of the present invention, for example, as shown in FIG. 1 and FIG. 19, preferably, in the display unit 140, the shooting instruction information output from the shooting instruction information unit 160 is a plan view of a range including the object 10. Or it is good to be comprised so that it may be displayed graphically in piles during the display of a three-dimensional model.

  In the model forming apparatus of the present invention, for example, as shown in FIGS. 1 and 19, preferably, a measurement setting unit 170 in which necessary measurement conditions (measurement accuracy, baseline length, overlap) are further set, and measurement settings are set. An imaging condition calculation unit 180 that obtains imaging conditions that satisfy the measurement conditions set by the unit 170 may be provided.

  The model forming method of the present invention that achieves the above object displays a three-dimensional model of the object 10 based on the three-dimensional model data of the object 10 stored in the image data storage unit 130, for example, as shown in FIG. (S110), based on the three-dimensional model data, recognizes an unformed part of the object 10 (S120), obtains photographing instruction information related to photographing the unformed part (S130), and performs each step. It is executed by a computer. Then, the photographer shoots the object 10 by the photographing unit 110 in accordance with the photographing instruction information (S140).

  For example, as shown in FIG. 20, the model formation method of the present invention that achieves the above-described object is the three-dimensional model data of the object 10 stored in the image data storage unit 130 or the object image captured by the image capturing unit 110. From the data, model data of the target object 10 is obtained (S210), a three-dimensional model is displayed based on the three-dimensional model data of the target object 10 (S220), and based on the three-dimensional model data of the target object 10, Recognizing the unmodeled part (S230), obtaining photographing instruction information related to photographing the unmodeled part (S240), and displaying the three-dimensional model and photographing instruction information on the display unit 140 (S250). Each step is executed by a computer. Then, based on the 3D model displayed on the display unit 140 and the imaging instruction information, the imaging unit 110 captures the object 10 by the photographer (S260).

  According to the model forming apparatus of the present invention, it is necessary when the recognition unit 150 recognizes a missing shooting direction with respect to the object, and the shooting instruction information unit 160 performs supplementary shooting of the missing shooting direction. By enabling the photographer to recognize the shooting instruction information, it is possible to supplement image data in an appropriate shooting mode from the missing shooting direction with respect to the object. Therefore, for example, by performing bundle adjustment of an aerial photograph and a photographed image from the ground, a 3D model relating to the object can be easily constructed.

  The present invention will be described below with reference to the drawings.

  FIG. 1 is an overall functional block for explaining an embodiment of the present invention. FIG. 3 is a system configuration diagram illustrating an embodiment of the present invention. The object 10 is a tangible object that is a measurement object or a production object, and includes, for example, various works such as buildings included in the areas of city planning, architecture, maintenance, and cultural properties, and people and landscapes. In the figure, the model forming apparatus of the present invention includes a photographing unit housing 100, a photographing unit 110, photographing position measuring units 120 and 125 for obtaining photographing position information of the photographing unit 110, and a plurality of images of an object 10 having position known points. It has an image data storage unit 130 for storing data, a display unit 140 such as a liquid crystal display or CRT, and an arithmetic processing unit 200, and is called, for example, a photographing guiding system (Virtual Guiding System).

  The photographing unit housing 100 integrally holds the photographing unit 110 and the photographing position measuring units 120 and 125, and is small and lightweight so that the photographer can easily operate. For the photographing unit 110, for example, a digital camera for consumer use can be used as a digital camera, and a camera capable of outputting video is preferable. The number of pixels is selected according to the required resolution. The photographing position measuring unit 120 uses GPS so that the photographing position (X, Y, Z) of the photographing unit 110 can be measured as latitude, longitude, and altitude on the earth. The photographing position measuring unit 120 is preferably capable of switching between kinematic and real-time kinematics, for example. The photographing position measuring unit 125 uses a three-axis angle sensor that measures the photographing posture (Yaw, Pitch, Law) of the photographing unit 110. The photographing position measuring unit 125 has, for example, a resolution of 0.05 °, a stationary accuracy of 1 ° RMS, and a dynamic accuracy of 3 ° RMS.

  The image data storage unit 130 uses a medium for storing image information such as a flexible disk, MD, and DVD. The key input unit 132 is used by an operator to input keys to the arithmetic processing unit 200. For example, a keyboard, a mouse, a touch panel, or the like is used. The display unit 140 is viewed from the mode I having a display area divided into three areas of a monitor image display unit 142, a model display unit 144, and a measurement condition display unit 146, a measurement condition display unit 146, a shooting position input screen 148, and a shooting position. The object model display screen 149 has a mode II having a display area divided into three sections. Mode I is suitable for recognizing an unmodeled portion of the object 10. Mode II is suitable for displaying shooting instruction information related to shooting of a non-modeled portion of the object 10 and supplementary shooting of ground photographs based on aerial photographs.

  As the arithmetic processing unit 200, for example, a general-purpose notebook computer can be used. The image information captured by the image capturing unit 110 and the position information (X, X) of the image capturing unit 110 measured by the image capturing position measuring units 120 and 125 are used. Y, Z) and posture information (Yaw, Pitch, Law) are input. A general-purpose notebook personal computer is provided with an electromagnetic storage device such as a flexible disk storage device or a CDROM, and is used as the image data storage unit 130. A general-purpose notebook computer stores a recognition unit 150, a model formation unit 155, a shooting instruction information unit 160, a measurement setting unit 170, a shooting condition calculation unit 180, and a reference point display position calculation unit 190 as software.

  The recognition unit 150 recognizes a model-unformed part of the object 10 based on the three-dimensional model data stored in the image data storage unit 130. The model forming unit 155 forms a three-dimensional model of the object 10 using the image data stored in the image data storage unit 130. A specific three-dimensional model creation arithmetic expression is, for example, the one of the applicant of the present invention. JP-A-2002-352224, JP-A-2003-42730 and JP-A-2003-65737 according to the proposal are described. The model forming unit 155 is supplied as, for example, a 3D measurement system PI-3000V2 (product name) by the applicant of the present application, and is a system capable of performing 3D measurement and modeling all at once from aerial photographs to digital camera images. The recognizing unit 150 extracts a region where the model forming unit 155 cannot form a three-dimensional model of the target object 10, and particularly recognizes the direction of the surface where the image information of the target object 10 is missing or insufficient.

  The shooting instruction information unit 160 obtains shooting instruction information related to shooting of the non-modeled portion of the target object 10 recognized by the recognition unit 150. The obtained shooting instruction information includes, for example, the measurement condition display unit 146, The image is displayed on the shooting position input screen 148 and the object model display screen 149 viewed from the shooting position. The imaging instruction information includes geodetic position information for measuring the object 10, the distance between the object 10 and the imaging unit 110, the focal length of the imaging unit 110, and the like. Preferably, in the measurement condition display unit 146, the shooting instruction information calculated by the shooting instruction information unit 160 may be displayed in a graphic manner while being overlapped during the display of the plan view or the three-dimensional model in the range including the target object 10.

  The measurement setting unit 170 sets measurement conditions necessary for forming a three-dimensional model of the object 10, and the measurement conditions can be derived from the theory of stereo image measurement. The baseline length and overlap (overlap rate) of the photo are set. The imaging condition calculation unit 180 obtains imaging conditions that satisfy the measurement conditions set by the measurement setting unit 170. For example, the imaging condition calculation unit 180 captures the next image taken by the imaging unit 110. Find shooting conditions. The shooting conditions should be consistent with the shooting instruction information obtained by the shooting instruction information unit 160.

  The reference point display position calculation unit 190 performs real-time display with the reference point superimposed on the object 10 on the finder image displayed on the monitor image display unit 142. The calculation of the superimposition state is performed by calculating whether the reference point coordinates input in advance as the display position on the image of the finder with respect to the image information of the finder of the photographing unit 110 is calculated by the reference point display position calculation unit 190. And do it.

  FIG. 4 is a diagram illustrating an example of a screen displayed on the liquid crystal display panel, and is used as a monitor image display unit 142, a model display unit 144, and a measurement condition display unit 146. In the monitor image display unit 142, image information of the finder of the photographing unit 110 is sent to a computer constituting the arithmetic processing unit 200 via various interfaces and displayed on the liquid crystal display panel. The reference point display position calculation unit 190 may display the reference point related to the object 10 included in the image information of the finder in a superimposed state. Thereby, how many reference points appear on the monitor image display unit 142 can be determined at the time of shooting by the shooting unit 110. The monitor image display unit 142 may have a structure in which the first and second captured images captured by the imaging unit 110 and stored in the image data storage unit 130 are sequentially called and displayed.

  The apparatus shown in FIG. 4 is designed to display three screens. The left two screens are captured and stored in the image data storage unit 130, and the rightmost screen is the current shooting unit 110. The image captured in is displayed. The photographer can easily find the shooting point while checking the three screens displayed on the monitor image display unit 142 in real time. In addition, it is also possible to confirm an already captured image by sequentially displaying images stored in the image data storage unit 130 on all three screens.

  The model display unit 144 can display the result of modeling the object 10 by the model forming unit 155 in accordance with the viewpoint viewed from the shooting position and posture of the shooting unit 110. Thereby, it is possible to determine the shooting position with respect to the target object 10 while recognizing the modeling missing part of the target object 10 and confirming the shooting direction of the target object 10 to be supplemented as the shooting direction. The three-dimensional display of the target object 10 can display an image of the target object 10 reconstructed by three-dimensional measurement by using, for example, an OpenGL function, and display a wire frame or texture mapping. Thereby, an image of the object 10 can be displayed on the model display unit 144 in real time with a viewpoint and resolution. Further, since the model unformed portion of the target object 10 recognized by the recognition unit 150 is not displayed as a missing surface with respect to the target object 10 displayed by the model display unit 144, it is necessary for the photographer to replenish. It becomes a recognizable state.

  The measurement condition display unit 146 displays the position information (X, Y, Z) of the photographing unit 110 measured by the photographing position measuring unit 120, and the posture information (X, Y, Z) measured by the photographing position measuring unit 125 ( Yaw, Pitch, Law) is displayed. The measurement condition display unit 146 also includes measurement conditions set by the measurement setting unit 170, that is, a baseline length (B), a distance (H), a focal length (f), a pixel resolution (p), and accuracy (ΔXY, ΔZ ), Overlap (duplication rate), etc. are displayed.

  Further, the image data storage unit 130 holds an image photographed by the photographing unit 110 and a photographing position. The shooting condition calculation unit 180 uses the finder image (second shot image) displayed on the monitor image display unit 142 based on the information (first shot image) of the previous shot image stored in the image data storage unit 130. ) Is calculated. Since the measurement setting unit 170 is set with measurement accuracy, a baseline, a shooting position, a shooting angle, and a shooting unit interval when performing the three-dimensional measurement of the target object 10, the shooting condition calculation unit 180 is an outline of the target object 10. The accuracy is calculated and displayed on the measurement condition display unit 146.

The accuracy roughly obtained by the imaging condition calculation unit 180 is obtained by the following equation.
ΔXY = H * δp / f (1)
ΔZ = H * H * δp / (f * B) (2)
Here, δp is the pixel resolution of the photographing unit 110 or the reading resolution of the scanner, f is a focal length, and is known by the photographing unit 110 to be used. The baseline length B is calculated from the position measured from the shooting position measurement unit 120 of the shot image and the position of the shooting position measurement unit 120 to be shot next. The shooting distance H can be calculated from the position of the shooting position measurement unit 120 and the position of the reference point or 3D model if there is a reference point or 3D model at the location to be shot. If the reference point or 3D model is not shown, enter an approximate value. From the resolution δp, focal length f, base length B, and shooting distance H, which are parameters used in the above equation, approximate accuracy ΔXY and ΔZ after measurement of the stereo model that is currently being shot is calculated.

  FIG. 5 is a diagram illustrating another example of a screen displayed on the liquid crystal display panel, and is used as a measurement condition display unit 146, a shooting position input screen 148, and an object model display screen 149 viewed from the shooting position. In the shooting instruction information unit 160, shooting instruction information related to shooting of the non-modeled portion of the target object 10 recognized by the recognition unit 150 is obtained. Therefore, when an appropriate shooting position is designated from the shooting positions displayed on the shooting position input screen 148 using the input unit 132, a model image of the object viewed from the shooting position is displayed on the target model display screen 149. .

  FIG. 6 is an explanatory diagram illustrating an example of a shooting position input screen used for inputting a shooting position. The shooting position input screen 148 may be a plan view of an object displayed on a liquid crystal display panel, for example. On the shooting position input screen 148, shooting positions 1 to 6 are input using the input unit 132 such as a light pen or a mouse. Then, a 3D model of the object from the position input as the shooting position by the model forming unit 155 is formed, and a visually recognized image is displayed on the object model display screen 149 viewed from the shooting position. These shooting positions are displayed on the display unit 140 as a stereo shooting position together with the range to be modeled on the plan view as shown in FIG. 7, or on the model screen as shown in FIGS. Can be displayed to guide the photographer.

  FIG. 10 is a diagram illustrating an example of an object visual image from the designated photographing position displayed on the object model display screen 149. Reference numerals 1 to 6 in the drawing correspond to the photographing designated positions 1 to 6 in FIG. Yes. When the shooting designation positions 1 to 6 are input on the shooting position input screen 148, the object model display screen 149 displays the object image of the angle facing the shooting unit 110 from the designated shooting positions 1 to 6. A 3D image is formed and displayed on the object model display screen 149.

  A photographing operation in the apparatus configured as described above will be described. FIG. 11 is a flowchart for explaining measurement of a 3D model using images taken in the air and on the ground. First, aerial shooting is performed (S300). There are a wide variety of aerial images such as aerial photographs (for example, provided by the Geospatial Information Authority of Japan), images acquired by imaging systems such as helicopters, airships, balloons, and image data captured using paragliders with engines. Available. However, the internal orientation element in the imaging conditions of the imaging unit 110 needs to be known.

  Next, on the ground, a reference point necessary for creating the 3D model is acquired by a surveying instrument or a GPS (S310). At this time, the measurement values obtained by the surveying instrument are unified in the earth positioning coordinate system using GPS. Then, the 3D model of the object 10 is measured by the model forming unit 155 using the aerial captured image acquired in S300 (S320). In this case, the reference point acquired in S310 is used as necessary, and three-dimensional model data based on the aerial captured image created by the model forming unit 155 is stored in the image data storage unit 130.

  Next, the recognition unit 150 recognizes a model-unformed part of the object 10 based on the three-dimensional model data stored in the image data storage unit 130 (S330). Then, ground shooting is performed by the shooting unit 110 using the shooting guiding system (S340). At this time, it is preferable to take an image of the non-model-formed portion of the object 10 recognized by the recognition unit 150 using the shooting instruction information obtained by the shooting instruction information unit 160. Prior to this operation, the reference point measured in S310 and the 3D model data created in S320 are stored in the imaging guiding system in advance. Thereby, when confirming the shooting position, the monitor image display unit 142 displays the object image of the angle facing the shooting unit 110 in real time, and is measured on the target image of the monitor image display unit 142 in S310. The reference point is displayed in a superimposed state. In addition, the 3D model created in S320 is displayed on the model display unit 144 at an angle looking into the photographing unit 110. The measurement condition display unit 146 can also check the measurement accuracy and the overlap state of the monitor image display unit 142. With these functions, the photographer guides the photographing position of the photographing unit 110 to an optimum position in consideration of post-processing such as 3D model creation.

  Next, a 3D model on the ground is created by stereo measurement of the image photographed by the photographing guiding system by the model forming unit 155 (S340). Then, based on the 3D model created in S340, it is determined whether supplementary shooting of the object 10 is necessary (S350). In the necessity determination for supplementary shooting, for example, recognition of a model-unformed portion by the recognition unit 150 may be used, and a difference in resolution between the aerial shot image and the ground shot image is also taken into consideration. If supplementary shooting of the object 10 is necessary, the process returns to S330. On the other hand, if supplementary shooting of the object 10 is unnecessary, the process proceeds to S360.

  Next, the entire 3D model is created by the model forming unit 155 (S360). At this time, the model forming unit 155 bundle-adjusts the pass points and tie points of the images of each stereo model photographed in the air or on the ground (S370), so that each stereo model photographed in the air or on the ground. The coordinate system of the image is unified to create an entire 3D model. In the photographing guiding system, the 3D model formation work on the ground and the 3D model creation work of the entire images of each stereo model photographed in the air or on the ground can be automatically performed while performing bundle adjustment.

  Next, an example of creating a 3D model using an aerial photograph, an aerial image, and a ground image using the above-described imaging guiding system according to the present invention will be described. FIG. 12 is an overall site map of an object for creating a 3D model using the imaging guiding system according to the present invention. The object is an 8-story building owned by the applicant, and is located in the applicant's head office registration location. The used image is an aerial photograph (FIG. 13) as basic image information of the entire object, an image photographed by the photographing unit 110 from a powered glider for modeling from above the object (FIG. 15), and a ground side surface part of the object. Obtained by the photographing unit 110 of the photographing guiding system (FIG. 18). Each of these three types of images was created by a model forming unit 155 and fused. The reference point measured the side of the building which is the object with a surveying instrument, and converted the reference point position measured with the surveying instrument into GPS coordinates using the points measured by GPS within the site of the object.

Each imaging / analysis will be described below.
(1) Base creation by aerial photograph analysis FIG. 13 is an aerial photograph provided by the Geospatial Information Authority of Japan, which has taken around 4 km square around Itabashi Ward, including the applicant's head office registration location. A so-called general aerial photography camera can be used as the photographing camera. As the focal length, the angle of view, etc., those necessary for photographing are appropriately selected.

  FIG. 14 is an image obtained by modeling the 3D model creation target area by the model forming unit. In order to use the aerial photograph as a base (background) in the analysis of the 3D model creation target area, the film was read with a scanner, for example, with a reading resolution of 600 dpi, and the model forming unit 155 internally determined. In FIG. 15, stereo matching is not performed and simple 3D plotting is performed and used. However, when there is an undulation such as a mountain, the undulation portion of the 3D model creation target region is subjected to stereo matching processing to generate 3D. Model it.

(2) Creation of 3D Model from Aerial Shooting (Powered Glider) Image FIG. 15 is an image captured by the imaging unit 110 from a powered glider for modeling from above the object. FIG. 16 is an explanatory diagram of photography by a powered paraglider (paraglider with an engine). Powered paragliders are not aircraft, so there are few restrictions on altitude regulations, etc., and there are features that enable inexpensive and safe shooting. In addition to powered gliders, helicopters, balloons, and Cessna aircraft capable of taking large-scale aerial photographs can also be used. A consumer digital camera can be used as the camera used for shooting. .

  FIG. 17 is an explanatory diagram for explaining an example of 3D modeling of an aerial photograph. The procedure of 3D model analysis is as follows. First, in the model formation unit 155, the reference point measured by the surveying instrument is entered and orientation is performed, and the polyline measurement is performed on the edge portion of the building, which is the object, and the front yard portion (FIG. 17a). Here, polyline measurement refers to performing three-dimensional measurement while manually measuring corresponding points or semi-automatically using correlation processing, and can be performed manually or by automatic position detection. Next, for the building that is the object, the measurement of the wall surface of the building and the creation of the plane part mesh are automatically performed (FIG. 17b). And the external shape of a building is represented about the building which is a target object (FIG. 17c). Preferably, as in the second embodiment, the texture increases when a texture is pasted on a building which is an object. Then, the external shape of the 3D modeled building is integrated with the aerial photograph and fused with general image information near the object (FIG. 17d). At this time, a part of the wall surface of the building that is the object has a direction that is not photographed due to the restrictions of the aerial photograph, and is in a state lacking in 3D modeling.

(3) Addition of Ground 3D Model by Shooting Guiding System FIG. 18 shows a state in which the shooting unit 110 is shooting from the shooting direction to be supplemented with respect to the object and an additional shot image. At this time, the display screen of the imaging guiding system is as shown in FIG. 10, for example. The monitor image display unit 142 displays an image from the shooting direction to be supplemented with respect to the object. Some wall surfaces of the target building are not photographed due to aerial photography restrictions, but the image information of the missing wall surface is a precise reference point by photographing from the photographing direction to be supplemented. Obtained with information.

  FIG. 19 is an overall functional block diagram for explaining the second embodiment of the present invention. In FIG. 19, the same reference numerals are assigned to the same components as those in FIG. In the figure, the image data storage unit 130 stores three-dimensional model data relating to the object 10 and object image data. Here, the three-dimensional model data refers to data in which position data of a point image related to an object is obtained, and is obtained by, for example, a surveying instrument or a laser scanner. The object image data is simply image data for which point image position data is not obtained. The model forming unit 155 forms three-dimensional model data related to the target object 10 using the target object image data. The texture pasting unit 192 divides the three-dimensional model data related to the object 10 into a plurality of areas, and the texture of the object image data is divided into the divided areas based on the relationship between each divided area and the shooting direction. paste.

  FIG. 20 is a flowchart for explaining the second embodiment of the present invention. First, the model forming unit 155 uses the three-dimensional model data of the object 10 stored in the image data storage unit 130 and the object image data captured by the image capturing unit 110 when necessary, and uses the model of the object 10. Data is obtained (S210). The model display unit 144 displays a three-dimensional model based on the three-dimensional model data of the object 10 (S220).

  The recognition unit 150 recognizes a model-unformed part of the target object 10 based on the three-dimensional model data of the target object 10 (S230). Then, the shooting instruction information unit 160 obtains shooting instruction information related to shooting of the model-unformed part (S240). Then, the three-dimensional model and the shooting instruction information are displayed on the display unit 140 (S250). That is, the three-dimensional model is displayed on the model display unit 144, and the imaging instruction information is displayed on the measurement condition display unit 146. Then, based on the 3D model displayed on the display unit 140 and the imaging instruction information, the imaging unit 110 captures the object 10 by the photographer (S260).

  Next, the entire 3D model is created by the model forming unit 155 (S270). At this time, the model forming unit 155 bundle-adjusts the pass points and tie points of the images of each stereo model photographed in the air or on the ground (S280), so that each stereo model photographed in the air or on the ground. The coordinate system of the image is unified to create an entire 3D model. Then, the texture pasting unit 192 pastes the texture of the target object image data on the three-dimensional model data related to the target object 10 based on the relationship between each segmented area and the shooting direction. (S290). Then, a three-dimensional model display of the object is performed using the object image data to which the texture is attached (S292).

  FIG. 21 is an image of an object in which a wire frame is pasted on an aerial photograph. FIG. 22 is an image of an object in which a texture is pasted on an aerial photograph. Here, the texture and the wire frame are acquired from the 3D model obtained by the model forming unit 155 from the ground image supplemented by the imaging guiding system. Here, the texture refers to a pattern attached to the surface of a graphic or a drawing for expressing a texture in graphics or the like, and is used to express a three-dimensional feeling in a two-dimensional image of an object. The wire frame refers to a polygonal line segment diagram connecting vertices when the surface shape of an object is represented by vertices of polygons such as a large number of triangles. A part of the wall surface of the building, which is an object, has a direction that is not photographed due to aerial photography restrictions, but a 3D model is obtained by acquiring a photographed image of the object from the photographing direction to be supplemented by the photographing guiding system. By doing so, 3D modeling processing of the entire object can be performed.

  FIG. 23 is an explanatory diagram of a 3D model in which a building adjacent to an object is added. The building adjacent to the object is further added after the 3D modeling process of the entire object shown in FIG. 22 using the image taken in the air. In this way, the part to be supplemented and added in the 3D modeling process can be easily dealt with by adding an analysis from a shooting guiding system, an aerial image or an aerial photograph.

  Subsequently, a recognition method of the recognition unit 150 for recognizing an unmodeled portion of the object based on the three-dimensional model data will be described. The recognition method of the recognition unit 150 is described in step S120 in FIG. 2 and step S230 in FIG. In addition, although the model non-formation part of a target object (building) may be instruct | indicated on the screen 140 with a user interface, and a shooting position may be calculated, the method to recognize automatically is demonstrated here.

  FIG. 24 is a flowchart of missing part recognition. First, the recognizing unit 150 searches for a portion having a higher elevation from the base model (S500). For example, in Example 1, FIG. 14 corresponds to a 3D model serving as a base model created from an aerial photograph. A 3D model created by superimposing buildings on this base model is as shown in FIG. Since these are 3D models, a portion having an altitude value higher than that of the base portion corresponds to a building.

  The recognizing unit 150 extracts a figure based on the base altitude value (S510). That is, the recognizing unit 150 extracts a part higher than the altitude value as a building based on the search result of S500. As a result, the parts (1), (2), (3) and (4) in FIG. 25 are extracted as buildings.

  Next, the recognizing unit 150 recognizes the missing portion of the base model created from the aerial photograph. In the case of a building, a portion having an elevation value above the base is always closed, so the discontinuous portion becomes a missing portion. Therefore, the parts (5) and (6) in FIG. 25 are recognized as missing parts. In addition, although the missing part about the base model created from the aerial photograph can be recognized by the processing from S500 to S520, the following S530 may be performed for further certainty.

  The recognizing unit 150 checks whether there is an adjacent surface for a portion higher than the base altitude value extracted in S510 (S530). In this case, since the side surface of the building is a surface, it is checked whether there is a surface in the parts (1) to (5) in FIG. FIG. 26 is a 3D model diagram illustrating a recognition state of a missing portion due to a three-dimensional shape. By performing the processing in S530, the area corresponding to the missing part in (5) and (6) of FIG. 25 is recognized. Although an example of the recognition method has been described here, various methods exist for the method of recognizing the missing portion of the base model created from the aerial photograph by the recognition unit 150.

Next, a description will be given of how to obtain photographing information related to photographing a model-unformed part. The method for obtaining the photographing information is described in step S130 in FIG. 2 and step S240 in FIG. For example, in the case of creating a 3D model, if the required accuracy (δxy, δz) in modeling the building is input, the camera shooting distance H and the camera can be calculated by the following equations (3) and (4). The distance B is calculated.
H = δxy · f / δp (3)
B = H · H · δp / (f · δz) (4)
It becomes. At this time, the focal length f of the camera and the pixel resolution δp are known. Then, if H and B are normal directions with respect to the missing part, the photographing position can be obtained as shown in FIG.

  FIG. 28 is an overall configuration diagram for explaining a third embodiment of the present invention. In the first embodiment, a combination of a digital camera and a notebook computer is shown as shown in FIG. However, in this embodiment, a GPS, a digital camera, and a small host PC are integrally configured. For a small host PC, a portable information processing terminal such as a PDA is used. Since the functions of the GPS, the digital camera, and the small host PC are the same as those described in the first embodiment, the details thereof are omitted.

  As described above, when bundle adjustment is performed on an aerial photograph and a photographed image from the ground by the photographing unit 110 of the photographing guiding system, it is important to supplement image data from a photographing direction that is missing with respect to the object. . According to this embodiment, the recognition unit 150 recognizes a missing shooting direction with respect to the object, and points to be taken care of when supplementing the missing shooting direction. The 3D model can be efficiently constructed from the air and the ground. In addition, according to the present embodiment, bundle adjustment is performed on images captured with a wide variety of imaging units 110 and resolutions, thereby ensuring consistency of various types of image measurement data, and high performance. A textured 3D model can be created from DSM (Digital Stereo Matching) data measured by stereo matching and displayed from various viewpoint positions at various resolutions.

It is a whole functional block explaining one embodiment of the present invention. It is a flowchart explaining the 1st model formation method of this invention. It is a system configuration figure explaining one embodiment of the present invention. It is a figure explaining an example of the screen displayed on a liquid crystal display panel. It is a figure explaining another example of the screen displayed on a liquid crystal display panel. It is explanatory drawing which shows an example of the imaging | photography position input screen used for the input of an imaging | photography position. It is the top view which showed the calculated | required imaging | photography position. It is an example of the 3D model figure which showed the imaging | photography position calculated | required. It is an example of the 3D model figure which showed the imaging | photography position calculated | required. It is a figure which shows an example of the target object visual recognition image from the designated imaging | photography position. It is a flowchart explaining the measurement of 3D model by the image image | photographed in the air and the ground. It is the whole site map of the target of 3D model creation using the photography guiding system concerning the present invention. This is an aerial photograph provided by the Geospatial Information Authority of Japan, which has been photographed about 4 km square around Itabashi-ku, Tokyo, including the applicant's head office location. It is the image which modeled the 3D model creation object area | region by the model formation part. It is the image image | photographed by the imaging | photography part 110 from the power glider for modeling from the sky of a target object. It is explanatory drawing of photography by a powered paraglider (paraglider with an engine). It is explanatory drawing explaining an example of 3D modeling of an aerial photograph. A state in which the photographing unit 110 is photographing from the photographing direction to be supplemented with respect to the object and an additional photographed image are shown. It is a whole functional block explaining the 2nd Embodiment of this invention. It is a flowchart explaining the 2nd model formation method of this invention. It is a block diagram. It is the image of the target object which stuck the wire frame on the aerial photograph. It is the image of the target object which stuck the texture on the aerial photograph. It is explanatory drawing of the 3D model which added the building adjacent to a target object. It is a flowchart which recognizes a missing part. It is the top view which showed the missing part at the time of extracting a target object (building). It is the 3D model figure which showed the missing part at the time of extracting a target object (building). It is the top view which showed the calculated | required imaging | photography position. It is a whole block diagram explaining the 3rd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Measuring object 110 Image pick-up part 120 Image | photographing position measurement part 130 Image data memory | storage part 140 Display part 150 Recognition part 155 Model formation part 160 Shooting instruction | indication information part 170 Measurement setting part 180 Shooting condition calculation part

Claims (7)

An imaging unit for imaging the object;
An image data storage unit for storing three-dimensional model data of the object;
A display unit for displaying a three-dimensional model based on the three-dimensional model data of the object;
A recognition unit for recognizing a model-unformed portion of the object based on the three-dimensional model data stored in the image data storage unit or a user instruction ;
A photographing instruction information unit for obtaining photographing instruction information including a photographing position required when supplementary photographing is performed on the non-model-formed portion recognized by the recognition unit ;
With
The display unit is configured to graphically display the shooting instruction information including the shooting position obtained by the shooting instruction information unit while displaying a plan view or a three-dimensional model of the range including the object. The model forming apparatus is characterized in that the object is photographed by the photographing unit in accordance with the photographing instruction information obtained by the photographing instruction information unit. An imaging unit for imaging the object;
An image data storing unit for storing the captured object image data in the previous SL imaging unit;
From the object image data stored in the image data storage unit, and the model forming unit for obtaining a three-dimensional model data of the object;
A display unit that displays a three-dimensional model superimposed on a base having an altitude value based on the three-dimensional model data of the object;
A recognition unit for extracting a figure of the object based on the elevation value of the base based on the three-dimensional model data and recognizing an unmodeled portion of the object;
A photographing instruction information unit for obtaining photographing instruction information including a photographing position required when supplementary photographing is performed on the non-model-formed portion recognized by the recognition unit ;
With
In the display unit, the imaging instruction information including the imaging position obtained by the imaging instruction information unit is displayed in a graphic manner while being overlapped during the display of a plan view or a three-dimensional model in a range including the object. Model forming device. Further, the 3D model data formed by the model forming unit is divided into a plurality of areas, and the texture of the object image data is divided into the divided areas based on the relationship between the divided areas and the shooting directions. A texture pasting part to be pasted;
The model forming apparatus according to claim 2, wherein the display unit displays a textured model formed by the texture pasting unit. 4. The model image of an object viewed from the shooting position is displayed on the display unit when an appropriate shooting position is designated from the shooting position displayed on the display unit. 5. Model forming device. And a measurement setting unit in which measurement conditions relating to imaging of the object by the imaging unit are set;
An imaging condition calculation unit for obtaining imaging conditions that satisfy the measurement conditions set by the measurement setting unit;
The model formation apparatus of any one of Claim 1 thru | or 4 provided with these. Displaying a three-dimensional model of the object based on the three-dimensional model data of the object stored in the image data storage unit;
Recognizing an unmodeled portion of the object based on the 3D model data or user instructions ;
Obtaining photographing instruction information including a photographing position required for supplementary photographing of the recognized unmodeled portion;
The imaging instruction information including the imaging position is graphically displayed while being superimposed on a plan view or a three-dimensional model in a range including the object;
A model formation method in which each process is executed by a computer;
A model forming method, wherein the object is photographed by a photographing unit in accordance with the photographing instruction information. Obtaining three-dimensional model data of the object from the object image data photographed by the photographing unit;
Displaying a three-dimensional model superimposed on a base having an elevation value based on the three-dimensional model data of the object;
Based on the three-dimensional model data, the base of the elevation value extracting shape of the object as a reference, recognizes the unmodeled portion of the object;
Obtaining photographing instruction information including a photographing position required for supplementary photographing of the recognized unmodeled portion;
The imaging instruction information including the imaging position is graphically displayed while being superimposed on a plan view or a three-dimensional model in a range including the object ;
A model formation method in which each process is executed by a computer.