WO2015165987A1 - Method for setting an image sensor of a vehicle camera of a motor vehicle into an image capturing state, camera system as well as motor vehicle - Google Patents

Method for setting an image sensor of a vehicle camera of a motor vehicle into an image capturing state, camera system as well as motor vehicle Download PDF

Info

Publication number
WO2015165987A1
WO2015165987A1 PCT/EP2015/059374 EP2015059374W WO2015165987A1 WO 2015165987 A1 WO2015165987 A1 WO 2015165987A1 EP 2015059374 W EP2015059374 W EP 2015059374W WO 2015165987 A1 WO2015165987 A1 WO 2015165987A1
Authority
WO
WIPO (PCT)
Prior art keywords
image sensor
temperature
image
threshold value
setting
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/EP2015/059374
Other languages
French (fr)
Inventor
Patrick Eoghan Denny
Brian Michael Thomas DEEGAN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Connaught Electronics Ltd
Original Assignee
Connaught Electronics Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Connaught Electronics Ltd filed Critical Connaught Electronics Ltd
Publication of WO2015165987A1 publication Critical patent/WO2015165987A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/67—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response
    • H04N25/671—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/40—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the details of the power supply or the coupling to vehicle components
    • B60R2300/404—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the details of the power supply or the coupling to vehicle components triggering from stand-by mode to operation mode

Definitions

  • the present invention relates to a method for setting an image sensor of a vehicle camera of a motor vehicle into an image capturing state, in which the image sensor provides image data.
  • the present invention relates to a camera system for a motor vehicle including a vehicle camera comprising an image sensor adapted to provide image data in an image capturing state, including a temperature sensing device for sensing a current temperature of the image sensor, and including a controller.
  • the invention relates to a motor vehicle with such a camera system.
  • Camera systems for motor vehicles are already known from the prior art. As is known, several cameras can be employed in a motor vehicle, wherein it becomes more and more usual nowadays to use a camera assembly with at least two vehicle cameras for a camera system of a motor vehicle, which each capture an environmental region of the motor vehicle. For example, four cameras can be employed, which capture the entire
  • an overall image presentation can be provided, such as for example the so- called "bird eye view”.
  • This image presentation represents a plan view on the motor vehicle as well as an environment from a bird's eye view, for example based on a reference point located directly above the vehicle.
  • the provision of such an environmental presentation from the images of several cameras is for example known from the document US 201 1/0156887.
  • a vehicle camera includes an image sensor, by which image data can be provided in an image capturing state.
  • This image data can for example be transmitted to a display in the interior of the motor vehicle and there be displayed to a vehicle occupant.
  • the image data can be transmitted to a computing device for further processing. If the image sensor is activated and the image sensor has an increased temperature, the provided image data or video data can have a poor quality due to the thermal noise.
  • an image sensor is activated or transferred into an image capturing state and the image data is subsequently displayed to a user or supplied to a computing device, even if it has poor quality. This can result in the computing device or a driver assistance system coupled to the computing device using incorrect image or video data as the basis for further calculations. If such image data is output on a display, it can appear aesthetically disturbing for a user or a vehicle occupant.
  • this object is solved by a method, by a camera system as well as by a motor vehicle having the features according to the respective independent claims.
  • Advantageous implementations of the invention are the subject matter of the dependent claims, of the description and of the figures.
  • the method according to the invention for setting an image sensor of a vehicle camera of a motor vehicle into an image capturing state includes sensing a current temperature of the image sensor by means of a temperature sensing device of the vehicle camera before setting the image sensor into the image capturing state, comparing the sensed temperature of the image sensor to a preset, first threshold value for the temperature by means of a controller, and if the sensed temperature is greater than the first threshold value, prohibiting setting the image sensor into the image capturing state, and if the sensed temperature is less than the first threshold value, continuing setting the image sensor into the image capturing state, wherein the continuation of setting includes that the sensed temperature of the image sensor is compared to at least one second threshold value lower than the first one, and an operating mode for the image capturing state of the image sensor is selected from at least two present operating modes depending on the comparison.
  • the invention is based on the realization that the image sensor of a vehicle camera can have a temperature, which is above the intended operating temperature. In this operating state, the thermal noise usually increases in providing the image data or video data. This can result in the image data or video data being provided with decreased quality.
  • the current temperature of the image sensor is checked before setting the image sensor into the image capturing state. For example, this can be effected with a corresponding temperature sensor, which is integrated in the image sensor.
  • a first preset threshold value can be stored in a memory of the controller.
  • the controller which can be disposed either within the vehicle camera or external to camera, compares the sensed temperature to the first threshold value. If the sensed temperature of the image sensor exceeds the first threshold value, setting the image sensor into the image capturing state is not continued.
  • the first threshold value for the temperature is for example selected such that operation of the image sensor is no longer reasonable here because the thermal noise is too great.
  • the sensed temperature is less than the first threshold value
  • setting the image sensor into the image capturing state is further continued.
  • the sensed, current temperature of the image sensor is compared to at least one second threshold value, which is less compared to the first threshold value.
  • an operating mode for the image capturing state of the image sensor is selected from at least two preset operating modes.
  • the image sensor can be correspondingly operated depending on its current temperature.
  • setting the image sensor into the image capturing state includes booting the image sensor.
  • the image sensor can include a sensor chip for example formed as a CCD sensor or as a CMOS sensor and an image processing unit.
  • booting the image sensor can relate to the image processing unit, which is for example formed as a computing device or as a microcontroller.
  • booting or start-up of the image sensor can be prohibited if the image sensor has an excessive operating temperature. As a result of this, unusable image data can be prevented from being provided with the image sensor.
  • the image sensor is operated in a first operating mode of the at least two preset operating modes if the sensed temperature is less than the second threshold value, and when the image sensor is operated in the second operating mode of the at least two preset operating modes if the sensed temperature is greater than the second threshold value.
  • the image sensor can be operated in at least two different operating modes depending on its current temperature.
  • the determination of the operating mode can be effected during booting of the image sensor or the image processing unit of the image sensor. In this manner, the optimum operating state for the image sensor can already be correspondingly selected upon starting the image sensor.
  • the image sensor is operated in the first operating mode in a basic setting.
  • the image sensor is operated according to the manufacturer's specifications if the currently sensed temperature of the image sensor is less than the second preset threshold value.
  • the second threshold value can be selected such that the image sensor usually has no or only negligible thermal noise in providing the image data at a temperature less than the second threshold value. In this manner, a reliable operation of the image sensor can be guaranteed.
  • At least one operating parameter of the image sensor is varied compared to the basic setting. If the temperature of the image sensor exceeds the second threshold value, usually, it can be expected that thermal noise occurs and thus the image data or video data is affected.
  • the operating parameters of the image sensor can be adapted compared to a usually used operating mode. This allows safe operation of the image sensor and thus of the vehicle camera.
  • the temperature of the image sensor is preset between the first and the at least one second threshold value and the at least one operating parameter of the image sensor is varied for each of the threshold values of the plurality of threshold values.
  • a plurality of threshold values for the temperature is preset and an own operating mode for the image capturing state of the image sensor is provided for each of the threshold values. In this manner, the image sensor can be operated in differentiated manner depending on its current temperature.
  • a frame rate, a gain and/or an exposure time are varied as the at least one operating parameter of the image sensor.
  • a lower frame rate can be adjusted with increasing temperature of the image sensor.
  • the gain of the image sensor can be increased and/or the exposure time can be increased with increasing temperature.
  • the influences of the thermal noise can be compensated for.
  • the current temperature of a component of the vehicle camera different from the image sensor is determined and compared to the sensed temperature of the image sensor.
  • the temperature of a component of the vehicle camera can for example be sensed, which is located next to the image sensor.
  • the temperature of a component of the vehicle camera is sensed, which is thermally coupled to the image sensor.
  • the temperature of a component or of a chip next to the image sensor can be sensed.
  • the temperature of a component can be determined, the temperature of which is a priori known.
  • the measurement result of the temperature sensing device by which the current temperature of the image sensor is sensed, can be made plausible based on the sensed temperature of the component.
  • a warning indication is output if a difference between the sensed temperature of the image sensor and the sensed temperature of the component is greater than a predetermined nominal difference.
  • a predetermined nominal difference it can for example be taken into account that the image sensor and the further component of the vehicle camera are disposed spatially separated from each other. Alternatively or additionally, it can be taken into account that the image sensor can for example faster heat than the component in the operation.
  • it can be provided in addition to a warning indication that the measurement of the temperature of the image sensor and/or the measurement of the temperature of the component as well as the comparison thereof are repeated. Thus, the temperature of the image sensor can be reliably determined.
  • the temperature of the image sensor sensed with the temperature sensing device is output by the vehicle camera.
  • the sensed temperature of the image sensor can be output by the controller.
  • the measured values of the temperature of the image sensor are stored in a memory of the controller or of the vehicle camera.
  • the sensed temperature can be displayed on a display in the interior of the vehicle. In particular, it can be displayed when the temperature exceeds the first threshold value and the image sensor is not activated as a result of this. In this manner, a driver of the motor vehicle can be informed that the vehicle camera currently is not available.
  • the camera system according to the invention for a motor vehicle includes a vehicle camera including an image sensor, which is adapted to provide image data in an image capturing state, a temperature sensing device for sensing a current temperature of the image sensor and a controller.
  • the controller is formed for performing the method according to the invention.
  • the controller can be disposed within a housing of the vehicle camera. Alternatively thereto, the controller can be positioned outside of the housing of the vehicle camera in the motor vehicle.
  • the motor vehicle according to the invention includes the camera system according to the invention.
  • the motor vehicle is formed as a passenger car.
  • the camera system preferably captures an environmental region of the motor vehicle.
  • FIG. 1 in schematic illustration a block diagram of a vehicle camera of a motor vehicle according to an embodiment of the invention.
  • Fig. 2 a flow diagram of a method for setting an image sensor of the vehicle camera into an image capturing mode according to an embodiment of the invention.
  • Fig. 1 shows a block diagram of a vehicle camera 1 according to an embodiment of the present invention.
  • the vehicle camera 1 is installed in a motor vehicle, in particular a passenger car.
  • the vehicle camera 1 can also be a part of a camera system for example including several vehicle cameras 1 .
  • the vehicle camera 1 is formed for providing a temporal sequence of image data of an environmental region of the motor vehicle.
  • the vehicle camera 1 includes a communication interface 2 connected to a communication bus 3 of the motor vehicle.
  • the vehicle camera 1 can communicate with vehicle-side components via the communication bus 3.
  • the vehicle camera 1 is formed as a video camera. It includes an image sensor 9 disposed behind a lens 5.
  • the image sensor 9 in turn includes a sensor chip 4, which can for example be formed as a CCD sensor or CMOS sensor.
  • the image sensor 9 includes an image processing unit 8 formed for correspondingly processing the sensor data provided by the sensor chip 4 and outputting the processed image data.
  • the image processing unit 8 can for example be formed as a digital signal processor (DSP) or as a microcontroller.
  • DSP digital signal processor
  • a corresponding operating system can be operated on the image processing unit 8.
  • an image processing program can be run on the image processing unit 8, by which the sensor data of the sensor chip 4 can be processed.
  • the image sensor 9 is coupled to a controller 6 receiving the image data of the image sensor 9.
  • the controller 6 is also coupled to the communication interface 2, via which the controller 6 communicates with the vehicle components.
  • Various operating parameters of the image sensor 9 can be adjusted by means of the controller 6. Among other things, a gain factor G, a frame rate FR as well as an exposure time ET of the image sensor 9 are adjusted by means of the controller 6.
  • the vehicle camera 1 includes a temperature sensing device 7, which senses the current temperature T of the image sensor 9 and communicates it to the controller 6.
  • the temperature sensing device 7 can for example be a temperature sensor. As shown in the present embodiment, the temperature sensing device 7 can be disposed separate from the image sensor 9. Alternatively thereto, the temperature sensor can be integrated in the sensor chip 4 of the image sensor 9. Furthermore, it is also possible to
  • the temperature sensing device 7 is then integrated in the controller 6.
  • Fig. 2 shows a flow diagram of a method for setting the image sensor 9 into an image capturing state, in which it provides image data.
  • Setting the image sensor 9 into the image capturing state can include booting or start-up of the image processing unit 8.
  • the operating system in the image processing unit 8 can be started.
  • a first step S1 the method is started.
  • the image sensor 9 is supplied with electrical energy.
  • the electrical energy is selected such that sensing the current temperature T of the image sensor 9 is allowed, but image data is not yet transmitted with the image processing unit 8.
  • only the temperature sensing device 7 can be supplied with electrical energy.
  • step S1 can be performed if the image sensor 9 is currently deactivated. However, it is also conceivable to perform step S1 if the image sensor 9 or the vehicle camera 1 are currently active. Exceptions can optionally be provided in performing a diagnostic mode.
  • a step S2 the current temperature T of the image sensor 9, in particular of the sensor chip 4, is determined by means of the temperature sensing device 7. Therein, a sensor value characterizing the current, sensed temperature of the image sensor 9, is
  • the first threshold value Td is selected such that operation of the image sensor 9 at a temperature greater than the first threshold value Td is not reasonable since the image data provided with the image sensor 9 has poor quality as a result of the thermal noise.
  • the first threshold value Td can be stored in a memory of the controller 6.
  • step S3 If the query in step S3 reveals that the current temperature T of the image sensor 9 is greater than the first threshold value Td, the method is continued in step S4. If the current temperature T exceeds the first threshold value Td, the starting operation or the booting operation of the image sensor 9 or of the image processing unit 8 is canceled.
  • a corresponding output is effected that the image sensor 9 is overheated.
  • a corresponding output - for example via a display in the interior of the motor vehicle - is effected that the image sensor 9 is overheated and currently image data cannot be provided.
  • step S5 the method can be continued in the optional step S5.
  • the temperature Tnc of a component of the vehicle camera 1 can additionally be examined, which is located next to the image sensor 9, for example on a common circuit board. Furthermore, it can be checked if the current, sensed temperature T of the image sensor 9 is comparable to the current temperature Tnc of the component. For example, it can be examined if a difference of the sensed temperature T of the image sensor 9 and the sensed temperature Tnc of the component exceeds a predetermined nominal difference. If this is the case, the method is continued with step S6. In step S6, an output is effected that the image sensor 9 has an abnormal temperature.
  • a method step S7 follows to the previously described method step S4 and the method step S6, in which the temperature T of the image sensor 9 is continuously determined at predetermined points of time. Thereby, after cancelation of the booting operation, it can be checked if the image sensor 9 has cooled down. In this manner, it can be recognized as fast as possible if the image sensor 9 is again operable and can be transferred into the image capturing state.
  • step S8 the method is continued with method step S8.
  • the method step S8 can also directly follow to step S3 if the optional method step S5 is not taken into account. Presently, this is indicated by the arrow 10.
  • step S8 it is checked if the current temperature T of the image sensor 9 is greater than a threshold value T1 . Therein, the threshold value T1 is selected such that it is less than the first threshold value Td.
  • step S9 the image sensor 9 is operated in a predetermined operating state with respect to the image capturing state. In this operating state, at least one operating parameter of the image sensor 9 is varied compared to a basic setting of the image sensor 9. In particular, the image sensor 9 can be operated such that further heating of the image sensor 9 is prohibited.
  • the operating parameter of the image sensor 9 can for example be - as described above - a gain factor G, the frame rate FR and/or the exposure time ET. In the operation of the image sensor in one of the predetermined operating modes, the frame rate FR can be reduced compared to a basic setting of the image sensor 9. In the same manner, the gain factor G and/or the exposure time ET can be greater.
  • the operating parameters of the image sensor 9 can be adapted to the increased operating temperature thereof.
  • step S8 If the examination in step S8 reveals that the current temperature T of the image sensor 9 is less than the threshold value T1 , the method is continued according to step S10.
  • step S1 1 it is checked if the current temperature T is less than a further threshold value T2. If this is the case, the method is continued with step S1 1 .
  • the image sensor 9 is operated in a further operating mode for the image capturing state, in which at least one operating parameter of the image sensor 9 is varied compared to a basic setting.
  • step S10 If the query in step S10 reveals that the current temperature T of the image sensor 9 is less than the threshold value T2, it can be examined for further threshold values with respect to the temperature whether or not they are exceeded. For each of the threshold values, a corresponding operating mode for the image capturing state of the image sensor 9 can be provided.
  • a step S12 it is checked if the current temperature T of the image sensor 9 is less than a second threshold value Tn. If this is the case, the image sensor 9 is operated in an associated operating mode with respect to the image capturing state (step S13). If the query in step S12 reveals that the current temperature T is less than the second threshold value Tn, the method is continued in step S14. In this case, the image sensor 9 is operated according to its basic setting (default mode).
  • the image sensor 9 is operated in one of the above describe operating modes.
  • Corresponding sensor data can be provided by the sensor chip 4 and communicated to the image processing unit 8.
  • the image data is further distributed to the components of the motor vehicle via the communication bus 3.
  • images of the environment of the motor vehicle can be provided to the vehicle occupants on a display.
  • the image data can be supplied to a driver assistance system, which assists the driver in driving the motor vehicle.
  • the image sensor 9 can be correspondingly heated before performing the booting operation of the image sensor. This can be effected before start-up of the vehicle camera 1 .
  • a function test can be performed in predetermined service intervals.
  • the image sensor 9 can be prevented from being operated at an excessive temperature. This can entail that incorrect image data is not transmitted from the image sensor 9 to the controller 6 and further to the motor vehicle via the communication interface 2 as a result of the thermal noise. By the method, it can be ensured that incorrect image data is not presented on a display of the vehicle.
  • controller 6 or a further component of the motor vehicle performs incorrect calculations based on falsified image data.
  • the data is used for driver assistance systems, thus, safety in the operation of the motor vehicle can be guaranteed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

The invention relates to a method for setting an image sensor (9) of a vehicle camera (1) of a motor vehicle into an image capturing state, in which the image sensor provides image data, including the steps of: sensing a current temperature (T) of the image sensor (9) by means of a temperature sensing device (7) of the vehicle camera (1) before setting the image sensor (9) into the image capturing state (S2), comparing the sensed temperature (T) of the image sensor (9) to a preset, first threshold value (Td) for the temperature by means of a controller (6) (S3), and if the sensed temperature (T) is greater than the first threshold value (Td), prohibiting setting the image sensor (9) into the image capturing state, and if the sensed temperature (T) is less than the first threshold value (Td), continuing setting the image sensor into the image capturing state, wherein the continuation of setting includes that the sensed temperature (T) is compared to at least one second threshold value (Tn) lower than the first one (S12) and an operating mode for the image capturing state of the image sensor (9) is selected from at least two preset operating modes depending on the comparison (S13).

Description

Method for setting an image sensor of a vehicle camera of a motor vehicle into an image capturing state, camera system as well as motor vehicle
The present invention relates to a method for setting an image sensor of a vehicle camera of a motor vehicle into an image capturing state, in which the image sensor provides image data. Moreover, the present invention relates to a camera system for a motor vehicle including a vehicle camera comprising an image sensor adapted to provide image data in an image capturing state, including a temperature sensing device for sensing a current temperature of the image sensor, and including a controller. In addition, the invention relates to a motor vehicle with such a camera system.
Camera systems for motor vehicles are already known from the prior art. As is known, several cameras can be employed in a motor vehicle, wherein it becomes more and more usual nowadays to use a camera assembly with at least two vehicle cameras for a camera system of a motor vehicle, which each capture an environmental region of the motor vehicle. For example, four cameras can be employed, which capture the entire
environment around the motor vehicle. From the images of all of the cameras, for example, an overall image presentation can be provided, such as for example the so- called "bird eye view". This image presentation represents a plan view on the motor vehicle as well as an environment from a bird's eye view, for example based on a reference point located directly above the vehicle. The provision of such an environmental presentation from the images of several cameras is for example known from the document US 201 1/0156887.
A vehicle camera includes an image sensor, by which image data can be provided in an image capturing state. This image data can for example be transmitted to a display in the interior of the motor vehicle and there be displayed to a vehicle occupant. Alternatively or additionally, the image data can be transmitted to a computing device for further processing. If the image sensor is activated and the image sensor has an increased temperature, the provided image data or video data can have a poor quality due to the thermal noise.
Currently, an image sensor is activated or transferred into an image capturing state and the image data is subsequently displayed to a user or supplied to a computing device, even if it has poor quality. This can result in the computing device or a driver assistance system coupled to the computing device using incorrect image or video data as the basis for further calculations. If such image data is output on a display, it can appear aesthetically disturbing for a user or a vehicle occupant.
According to the known solutions, such effects are ignored or not recognized. The result is a reduced performance for the display on the one hand and for the performance indicators of the computing devices on the other hand. For example, an image processing algorithm running on the computing device cannot or not sufficiently be performed. This entails risks in particular for safety-relevant systems such as for example driver assistance systems of the motor vehicle.
It is the object of the invention to demonstrate a solution, how in a method of the initially mentioned kind a vehicle camera can be more reliably operated compared to the prior art.
According to the invention, this object is solved by a method, by a camera system as well as by a motor vehicle having the features according to the respective independent claims. Advantageous implementations of the invention are the subject matter of the dependent claims, of the description and of the figures.
The method according to the invention for setting an image sensor of a vehicle camera of a motor vehicle into an image capturing state, in which the image sensor provides image data, includes sensing a current temperature of the image sensor by means of a temperature sensing device of the vehicle camera before setting the image sensor into the image capturing state, comparing the sensed temperature of the image sensor to a preset, first threshold value for the temperature by means of a controller, and if the sensed temperature is greater than the first threshold value, prohibiting setting the image sensor into the image capturing state, and if the sensed temperature is less than the first threshold value, continuing setting the image sensor into the image capturing state, wherein the continuation of setting includes that the sensed temperature of the image sensor is compared to at least one second threshold value lower than the first one, and an operating mode for the image capturing state of the image sensor is selected from at least two present operating modes depending on the comparison.
The invention is based on the realization that the image sensor of a vehicle camera can have a temperature, which is above the intended operating temperature. In this operating state, the thermal noise usually increases in providing the image data or video data. This can result in the image data or video data being provided with decreased quality.
According to the invention, the current temperature of the image sensor is checked before setting the image sensor into the image capturing state. For example, this can be effected with a corresponding temperature sensor, which is integrated in the image sensor. A first preset threshold value can be stored in a memory of the controller. The controller, which can be disposed either within the vehicle camera or external to camera, compares the sensed temperature to the first threshold value. If the sensed temperature of the image sensor exceeds the first threshold value, setting the image sensor into the image capturing state is not continued. Therein, the first threshold value for the temperature is for example selected such that operation of the image sensor is no longer reasonable here because the thermal noise is too great.
In case the sensed temperature is less than the first threshold value, setting the image sensor into the image capturing state is further continued. As a further step, herein, the sensed, current temperature of the image sensor is compared to at least one second threshold value, which is less compared to the first threshold value. Depending on the comparison, an operating mode for the image capturing state of the image sensor is selected from at least two preset operating modes. Thus, the image sensor can be correspondingly operated depending on its current temperature. By the method according to the invention, image or video data of the image sensor unusable as a result of the thermal noise can be prevented from being used. In particular in safety-relevant systems such as for example driver assistance systems of the motor vehicle, incorrect calculations or false diagnoses can therefore be prevented.
In an embodiment, setting the image sensor into the image capturing state includes booting the image sensor. The image sensor can include a sensor chip for example formed as a CCD sensor or as a CMOS sensor and an image processing unit. Therein, booting the image sensor can relate to the image processing unit, which is for example formed as a computing device or as a microcontroller. Thus, booting or start-up of the image sensor can be prohibited if the image sensor has an excessive operating temperature. As a result of this, unusable image data can be prevented from being provided with the image sensor.
Furthermore, it has proven advantageous when the image sensor is operated in a first operating mode of the at least two preset operating modes if the sensed temperature is less than the second threshold value, and when the image sensor is operated in the second operating mode of the at least two preset operating modes if the sensed temperature is greater than the second threshold value. Thus, the image sensor can be operated in at least two different operating modes depending on its current temperature. The determination of the operating mode can be effected during booting of the image sensor or the image processing unit of the image sensor. In this manner, the optimum operating state for the image sensor can already be correspondingly selected upon starting the image sensor.
Preferably, the image sensor is operated in the first operating mode in a basic setting. In other words, the image sensor is operated according to the manufacturer's specifications if the currently sensed temperature of the image sensor is less than the second preset threshold value. Therein, the second threshold value can be selected such that the image sensor usually has no or only negligible thermal noise in providing the image data at a temperature less than the second threshold value. In this manner, a reliable operation of the image sensor can be guaranteed.
Advantageously, in the at least one second operating mode, at least one operating parameter of the image sensor is varied compared to the basic setting. If the temperature of the image sensor exceeds the second threshold value, usually, it can be expected that thermal noise occurs and thus the image data or video data is affected. For this state, the operating parameters of the image sensor can be adapted compared to a usually used operating mode. This allows safe operation of the image sensor and thus of the vehicle camera.
In an advantageous embodiment, a plurality of further threshold values for the
temperature of the image sensor is preset between the first and the at least one second threshold value and the at least one operating parameter of the image sensor is varied for each of the threshold values of the plurality of threshold values. In other words, already while booting the image sensor, a plurality of threshold values for the temperature is preset and an own operating mode for the image capturing state of the image sensor is provided for each of the threshold values. In this manner, the image sensor can be operated in differentiated manner depending on its current temperature.
Preferably, a frame rate, a gain and/or an exposure time are varied as the at least one operating parameter of the image sensor. For example, a lower frame rate can be adjusted with increasing temperature of the image sensor. In the same manner, the gain of the image sensor can be increased and/or the exposure time can be increased with increasing temperature. In this manner, the influences of the thermal noise can be compensated for. In an embodiment, the current temperature of a component of the vehicle camera different from the image sensor is determined and compared to the sensed temperature of the image sensor. Thus, the temperature of a component of the vehicle camera can for example be sensed, which is located next to the image sensor. Therein, it is also conceivable that the temperature of a component of the vehicle camera is sensed, which is thermally coupled to the image sensor. For example, the temperature of a component or of a chip next to the image sensor can be sensed. Moreover, the temperature of a component can be determined, the temperature of which is a priori known. The
temperature of a thermal resistance next to the image sensor can also be sensed. Thus, the measurement result of the temperature sensing device, by which the current temperature of the image sensor is sensed, can be made plausible based on the sensed temperature of the component.
Preferably, a warning indication is output if a difference between the sensed temperature of the image sensor and the sensed temperature of the component is greater than a predetermined nominal difference. In presetting the nominal difference, it can for example be taken into account that the image sensor and the further component of the vehicle camera are disposed spatially separated from each other. Alternatively or additionally, it can be taken into account that the image sensor can for example faster heat than the component in the operation. Therein, it can be provided in addition to a warning indication that the measurement of the temperature of the image sensor and/or the measurement of the temperature of the component as well as the comparison thereof are repeated. Thus, the temperature of the image sensor can be reliably determined.
Advantageously, the temperature of the image sensor sensed with the temperature sensing device is output by the vehicle camera. In particular, the sensed temperature of the image sensor can be output by the controller. Therein, it is also conceivable that the measured values of the temperature of the image sensor are stored in a memory of the controller or of the vehicle camera. Alternatively thereto, the sensed temperature can be displayed on a display in the interior of the vehicle. In particular, it can be displayed when the temperature exceeds the first threshold value and the image sensor is not activated as a result of this. In this manner, a driver of the motor vehicle can be informed that the vehicle camera currently is not available.
The camera system according to the invention for a motor vehicle includes a vehicle camera including an image sensor, which is adapted to provide image data in an image capturing state, a temperature sensing device for sensing a current temperature of the image sensor and a controller. The controller is formed for performing the method according to the invention. Therein, the controller can be disposed within a housing of the vehicle camera. Alternatively thereto, the controller can be positioned outside of the housing of the vehicle camera in the motor vehicle.
The motor vehicle according to the invention includes the camera system according to the invention. In particular, the motor vehicle is formed as a passenger car. The camera system preferably captures an environmental region of the motor vehicle.
The preferred embodiments presented with respect to the method according to the invention and the advantages thereof correspondingly apply to the camera system according to the invention as well as to the motor vehicle according to the invention.
Further features of the invention are apparent from the claims, the figures and the description of figures. All of the features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and/or shown in the figures alone are usable not only in the respectively specified combination, but also in other combinations or else alone.
Now, the invention is explained in more detail based on a preferred embodiment as well as with reference to the attached drawings.
There show:
Fig. 1 in schematic illustration a block diagram of a vehicle camera of a motor vehicle according to an embodiment of the invention; and
Fig. 2 a flow diagram of a method for setting an image sensor of the vehicle camera into an image capturing mode according to an embodiment of the invention.
Fig. 1 shows a block diagram of a vehicle camera 1 according to an embodiment of the present invention. The vehicle camera 1 is installed in a motor vehicle, in particular a passenger car. The vehicle camera 1 can also be a part of a camera system for example including several vehicle cameras 1 . The vehicle camera 1 is formed for providing a temporal sequence of image data of an environmental region of the motor vehicle. The vehicle camera 1 includes a communication interface 2 connected to a communication bus 3 of the motor vehicle. The vehicle camera 1 can communicate with vehicle-side components via the communication bus 3.
The vehicle camera 1 is formed as a video camera. It includes an image sensor 9 disposed behind a lens 5. The image sensor 9 in turn includes a sensor chip 4, which can for example be formed as a CCD sensor or CMOS sensor. Moreover, the image sensor 9 includes an image processing unit 8 formed for correspondingly processing the sensor data provided by the sensor chip 4 and outputting the processed image data. The image processing unit 8 can for example be formed as a digital signal processor (DSP) or as a microcontroller. A corresponding operating system can be operated on the image processing unit 8. Moreover, an image processing program can be run on the image processing unit 8, by which the sensor data of the sensor chip 4 can be processed.
The image sensor 9 is coupled to a controller 6 receiving the image data of the image sensor 9. The controller 6 is also coupled to the communication interface 2, via which the controller 6 communicates with the vehicle components. Various operating parameters of the image sensor 9 can be adjusted by means of the controller 6. Among other things, a gain factor G, a frame rate FR as well as an exposure time ET of the image sensor 9 are adjusted by means of the controller 6.
Moreover, the vehicle camera 1 includes a temperature sensing device 7, which senses the current temperature T of the image sensor 9 and communicates it to the controller 6. The temperature sensing device 7 can for example be a temperature sensor. As shown in the present embodiment, the temperature sensing device 7 can be disposed separate from the image sensor 9. Alternatively thereto, the temperature sensor can be integrated in the sensor chip 4 of the image sensor 9. Furthermore, it is also possible to
computationally determine the temperature T based on the image data, which is provided with the image sensor 9. In the latter case, the temperature sensing device 7 is then integrated in the controller 6.
Fig. 2 shows a flow diagram of a method for setting the image sensor 9 into an image capturing state, in which it provides image data. Setting the image sensor 9 into the image capturing state can include booting or start-up of the image processing unit 8. Herein, for example, the operating system in the image processing unit 8 can be started. In a first step S1 , the method is started. Herein, the image sensor 9 is supplied with electrical energy. Therein, the electrical energy is selected such that sensing the current temperature T of the image sensor 9 is allowed, but image data is not yet transmitted with the image processing unit 8. Also, optionally, only the temperature sensing device 7 can be supplied with electrical energy. In particular, step S1 can be performed if the image sensor 9 is currently deactivated. However, it is also conceivable to perform step S1 if the image sensor 9 or the vehicle camera 1 are currently active. Exceptions can optionally be provided in performing a diagnostic mode.
In a step S2, the current temperature T of the image sensor 9, in particular of the sensor chip 4, is determined by means of the temperature sensing device 7. Therein, a sensor value characterizing the current, sensed temperature of the image sensor 9, is
communicated by the temperature sensing device 7 to the controller 6. In a subsequent step S3, it is checked by means of the controller 6 if the current, sensed temperature T of the image sensor 9 is greater than a first threshold value Td for the temperature. Therein, the first threshold value Td is selected such that operation of the image sensor 9 at a temperature greater than the first threshold value Td is not reasonable since the image data provided with the image sensor 9 has poor quality as a result of the thermal noise. For example, the first threshold value Td can be stored in a memory of the controller 6.
If the query in step S3 reveals that the current temperature T of the image sensor 9 is greater than the first threshold value Td, the method is continued in step S4. If the current temperature T exceeds the first threshold value Td, the starting operation or the booting operation of the image sensor 9 or of the image processing unit 8 is canceled.
Furthermore, a corresponding output is effected that the image sensor 9 is overheated. In addition, it can be provided in step S4 that a corresponding output - for example via a display in the interior of the motor vehicle - is effected that the image sensor 9 is overheated and currently image data cannot be provided.
If the query in step S3 reveals that the current temperature T is less than the first threshold value Td, the method can be continued in the optional step S5. For this, the temperature Tnc of a component of the vehicle camera 1 can additionally be examined, which is located next to the image sensor 9, for example on a common circuit board. Furthermore, it can be checked if the current, sensed temperature T of the image sensor 9 is comparable to the current temperature Tnc of the component. For example, it can be examined if a difference of the sensed temperature T of the image sensor 9 and the sensed temperature Tnc of the component exceeds a predetermined nominal difference. If this is the case, the method is continued with step S6. In step S6, an output is effected that the image sensor 9 has an abnormal temperature. Herein too, a corresponding output on a display of the motor vehicle can be effected. A method step S7 follows to the previously described method step S4 and the method step S6, in which the temperature T of the image sensor 9 is continuously determined at predetermined points of time. Thereby, after cancelation of the booting operation, it can be checked if the image sensor 9 has cooled down. In this manner, it can be recognized as fast as possible if the image sensor 9 is again operable and can be transferred into the image capturing state.
Subsequent thereto, the method is continued with method step S8. The method step S8 can also directly follow to step S3 if the optional method step S5 is not taken into account. Presently, this is indicated by the arrow 10. In step S8, it is checked if the current temperature T of the image sensor 9 is greater than a threshold value T1 . Therein, the threshold value T1 is selected such that it is less than the first threshold value Td.
If the current temperature T of the image sensor 9 is greater than the threshold value T1 , the method is continued according to step S9. In step S9, the image sensor 9 is operated in a predetermined operating state with respect to the image capturing state. In this operating state, at least one operating parameter of the image sensor 9 is varied compared to a basic setting of the image sensor 9. In particular, the image sensor 9 can be operated such that further heating of the image sensor 9 is prohibited. The operating parameter of the image sensor 9 can for example be - as described above - a gain factor G, the frame rate FR and/or the exposure time ET. In the operation of the image sensor in one of the predetermined operating modes, the frame rate FR can be reduced compared to a basic setting of the image sensor 9. In the same manner, the gain factor G and/or the exposure time ET can be greater. Thus, the operating parameters of the image sensor 9 can be adapted to the increased operating temperature thereof.
If the examination in step S8 reveals that the current temperature T of the image sensor 9 is less than the threshold value T1 , the method is continued according to step S10.
Herein, it is checked if the current temperature T is less than a further threshold value T2. If this is the case, the method is continued with step S1 1 . Herein, the image sensor 9 is operated in a further operating mode for the image capturing state, in which at least one operating parameter of the image sensor 9 is varied compared to a basic setting.
If the query in step S10 reveals that the current temperature T of the image sensor 9 is less than the threshold value T2, it can be examined for further threshold values with respect to the temperature whether or not they are exceeded. For each of the threshold values, a corresponding operating mode for the image capturing state of the image sensor 9 can be provided.
Thereafter, in a step S12, it is checked if the current temperature T of the image sensor 9 is less than a second threshold value Tn. If this is the case, the image sensor 9 is operated in an associated operating mode with respect to the image capturing state (step S13). If the query in step S12 reveals that the current temperature T is less than the second threshold value Tn, the method is continued in step S14. In this case, the image sensor 9 is operated according to its basic setting (default mode).
After completion of the booting operation, the image sensor 9 is operated in one of the above describe operating modes. Corresponding sensor data can be provided by the sensor chip 4 and communicated to the image processing unit 8. The image data is further distributed to the components of the motor vehicle via the communication bus 3. For example, images of the environment of the motor vehicle can be provided to the vehicle occupants on a display. In addition, the image data can be supplied to a driver assistance system, which assists the driver in driving the motor vehicle.
In order to check the functionality of the vehicle camera 1 in performing the method, the image sensor 9 can be correspondingly heated before performing the booting operation of the image sensor. This can be effected before start-up of the vehicle camera 1 . In addition, such a function test can be performed in predetermined service intervals.
By the method according to Fig. 2, the image sensor 9 can be prevented from being operated at an excessive temperature. This can entail that incorrect image data is not transmitted from the image sensor 9 to the controller 6 and further to the motor vehicle via the communication interface 2 as a result of the thermal noise. By the method, it can be ensured that incorrect image data is not presented on a display of the vehicle.
Furthermore, it can be prohibited that the controller 6 or a further component of the motor vehicle performs incorrect calculations based on falsified image data. In particular, in case the data is used for driver assistance systems, thus, safety in the operation of the motor vehicle can be guaranteed.

Claims

Claims
1 . Method for setting an image sensor (9) of a vehicle camera (1 ) of a motor vehicle into an image capturing state, in which the image sensor provides image data, including the steps of:
- sensing a current temperature (T) of the image sensor (9) by means of a
temperature sensing device (7) of the vehicle camera (1 ) before setting the image sensor (9) into the image capturing state (S2),
- comparing the sensed temperature (T) of the image sensor (9) to a preset, first threshold value (Td) for the temperature by means of a controller (6) (S3), and
- if the sensed temperature (T) is greater than the first threshold value (Td), prohibiting setting the image sensor (9) into the image capturing state, and
- if the sensed temperature (T) is less than the first threshold value (Td), continuing setting the image sensor into the image capturing state,
- wherein the continuation of setting includes that the sensed temperature (T) is compared to at least one second threshold value (Tn) lower than the first one (S12) and an operating mode for the image capturing state of the image sensor (9) is selected from at least two preset operating modes depending on the comparison (S13)
and setting the image sensor (9) into the image capturing state includes booting the image sensor (9).
2. Method according to claim 1 ,
characterized in that
the image sensor (9) is operated in a first operating mode of the at least two preset operating modes if the sensed temperature (T) is less than the second threshold value (Tn) (S14), and that the image sensor (9) is operated in a second operating mode of the at least two preset operating modes if the sensed temperature (T) is greater than the second threshold value (Tn) (S13).
3. Method according to claim 2,
characterized in that
the image sensor (9) is operated in the first operating mode in a basic setting.
4. Method according to claim 3,
characterized in that
in the at least one second operating mode, at least one operating parameter of the image sensor (9) is varied compared to the basic setting.
5. Method according to claim 4,
characterized in that
between the first and the at least one second threshold value (Td, Tn), a plurality of further threshold values (T1 , T2) for the temperature of the image sensor (9) is preset, and for each threshold value (T1 , T2) of the plurality of threshold values, the at least one operating parameter of the image sensor (9) is varied.
6. Method according to claim 4 or 5,
characterized in that
a frame rate (FR), a gain (G) and/or an exposure time (ET) are varied as the at least one operating parameter of the image sensor (9).
7. Method according to any one of the preceding claims,
characterized in that
a current temperature (Tnc) of a component of the vehicle camera (1 ) different from the image sensor (9) is determined and compared to the sensed temperature (T) of the image sensor (S5).
8. Method according to claim 7,
characterized in that
a warning indication is output if a difference between the sensed temperature (T) of the image sensor (9) and the sensed temperature (Tnc) of the component is greater than a predetermined nominal difference (S6).
9. Method according to any one of the preceding claims,
characterized in that
if setting the image sensor (9) into the image capturing state is prohibited, subsequently, the current temperature (T) of the image sensor (9) is continuously determined at predetermined points of time (S7).
10. Method according to any one of the preceding claims,
characterized in that
the sensed temperature (T) of the image sensor (9) is output by the vehicle camera (1 )-
1 1 . Camera system for a motor vehicle including a vehicle camera (1 ) comprising an image sensor (9) adapted to provide image data in an image capturing state, including a temperature sensing device (7) for sensing a current temperature (T) of the image sensor (9) and including a controller (6),
characterized in that
the controller (6) is adapted to perform a method according to any one of the preceding claims.
12. Motor vehicle, in particular passenger car, with a camera system according to claim 1 1 .
PCT/EP2015/059374 2014-04-30 2015-04-29 Method for setting an image sensor of a vehicle camera of a motor vehicle into an image capturing state, camera system as well as motor vehicle Ceased WO2015165987A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102014106034.2A DE102014106034B4 (en) 2014-04-30 2014-04-30 Method for setting an image sensor of a vehicle camera of a motor vehicle into an image recording state, camera system and motor vehicle
DE102014106034.2 2014-04-30

Publications (1)

Publication Number Publication Date
WO2015165987A1 true WO2015165987A1 (en) 2015-11-05

Family

ID=53039425

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2015/059374 Ceased WO2015165987A1 (en) 2014-04-30 2015-04-29 Method for setting an image sensor of a vehicle camera of a motor vehicle into an image capturing state, camera system as well as motor vehicle

Country Status (2)

Country Link
DE (1) DE102014106034B4 (en)
WO (1) WO2015165987A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114979453A (en) * 2021-02-20 2022-08-30 Oppo广东移动通信有限公司 Shooting method and device and electronic equipment

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6800682B2 (en) * 2016-09-30 2020-12-16 キヤノン株式会社 Imaging equipment, imaging systems and mobiles

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005175961A (en) * 2003-12-11 2005-06-30 Canon Inc Imaging apparatus, imaging sensitivity control method, shutter control method, and program
US20100060738A1 (en) * 2008-09-08 2010-03-11 Toyota Jidosha Kabushiki Kaisha Road surface division mark recognition apparatus, and lane departure prevention apparatus
US20110001640A1 (en) * 2008-01-31 2011-01-06 Toyota Jidosha Kabushiki Kaisha Operation control map, display unit, and white line detection apparatus
US20110156887A1 (en) 2009-12-30 2011-06-30 Industrial Technology Research Institute Method and system for forming surrounding seamless bird-view image
US20120026379A1 (en) * 2010-08-02 2012-02-02 Canon Kabushiki Kaisha Image capturing apparatus, control method therefor, and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002090851A (en) 2000-09-20 2002-03-27 Eastman Kodak Japan Ltd Electronic camera
DE102012011886A1 (en) 2012-06-15 2013-12-19 Connaught Electronics Ltd. Method for operating camera of motor vehicle, involves detecting image of surrounding area of motor vehicle by image sensor of camera, where image processing algorithm is executed on basis of image by image processing unit

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005175961A (en) * 2003-12-11 2005-06-30 Canon Inc Imaging apparatus, imaging sensitivity control method, shutter control method, and program
US20110001640A1 (en) * 2008-01-31 2011-01-06 Toyota Jidosha Kabushiki Kaisha Operation control map, display unit, and white line detection apparatus
US20100060738A1 (en) * 2008-09-08 2010-03-11 Toyota Jidosha Kabushiki Kaisha Road surface division mark recognition apparatus, and lane departure prevention apparatus
US20110156887A1 (en) 2009-12-30 2011-06-30 Industrial Technology Research Institute Method and system for forming surrounding seamless bird-view image
US20120026379A1 (en) * 2010-08-02 2012-02-02 Canon Kabushiki Kaisha Image capturing apparatus, control method therefor, and storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114979453A (en) * 2021-02-20 2022-08-30 Oppo广东移动通信有限公司 Shooting method and device and electronic equipment

Also Published As

Publication number Publication date
DE102014106034A1 (en) 2015-11-05
DE102014106034B4 (en) 2022-02-10

Similar Documents

Publication Publication Date Title
CN104271400B (en) In-vehicle camera controls device
US8115809B2 (en) Vehicle surroundings monitoring apparatus
US20160262682A1 (en) Driver monitoring apparatus
US9978188B2 (en) Vehicle electronic control device and system with off-board diagnostic function
JP2001088609A (en) Power supply control method and device for vehicle-mounted camera
US11285779B2 (en) Moving body control apparatus
EP2858048A1 (en) Method for utilizing a camera system of a motor vehicle as an anti-theft alarm system, camera system and motor vehicle
CN108924835B (en) Vehicle control system, method and safety control unit
WO2015165987A1 (en) Method for setting an image sensor of a vehicle camera of a motor vehicle into an image capturing state, camera system as well as motor vehicle
JP6678844B2 (en) Abnormality detection device and abnormality detection method
US11376920B2 (en) Moving body control apparatus
CN110228418B (en) Control method, device and system for vehicle interior rearview mirror and vehicle
JP5652323B2 (en) Vehicle imaging device
US10864921B2 (en) Onboard apparatus
WO2018116803A1 (en) Vehicle display apparatus
US20200290542A1 (en) Occupant monitoring device for vehicle and traffic system
CN109890664B (en) Method and apparatus for operating a safety system of a motor vehicle and safety system
CN110557558B (en) Object monitoring device
JP5902051B2 (en) Image monitoring apparatus, vehicle, program, and failure determination method
US20050222725A1 (en) Vehicle-mounted equipment control apparatus
EP2815930A1 (en) Arrangement, vehicle and method for control of vehicle-airbag-status display
JP2024053194A (en) Abnormality determination method, abnormality determination program, and passenger monitoring device
JP2019074963A (en) Device and system for detecting specific body part
JP2026508833A (en) System for operating heating devices for cameras in autonomous vehicles
WO2021255991A1 (en) Stereo camera

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15719712

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15719712

Country of ref document: EP

Kind code of ref document: A1