JP2003092687A5 - - Google Patents

Description

【０１１１】
これと同じ処理を緑成分と青成分について実行し、次に、各色成分間での特徴ベクトルの内積を求める。すなわち、赤成分と緑成分との間の特徴ベクトルの内積ｃｏｒｒ＿ｒｇは数２式に表すように求められ、緑成分と青成分との間の特徴ベクトルの内積ｃｏｒｒ＿ｇｂは数３式に表すように求められ、青成分と赤成分との間の特徴ベクトルの内積ｃｏｒｒ＿ｂｒは数４式に表すように求められる。
【０１１２】
【数２】

【０１１３】
【数３】

【０１１４】
【数４】

【０１１５】
ベクトルの内積は、両ベクトルの類似度を表すといえ、その値は「０」〜「１」となる。従って、しきい値ＣＯＲＲとして「０．７」を定めたとして、それぞれの内積ｃｏｒｒ＿ｒｇ，ｃｏｒｒ＿ｇｂ，ｃｏｒｒ＿ｂｒのうちいずれか一つでもこのしきい値ＣＯＲＲ以下のものがあれば類似具合が低いものと判断してステップＳ１０６の非処理を実行することにする。
【０１１６】
本実施形態においては、この特徴ベクトルの内積に基づく処理が類似具合判断手段を構成しており、特徴ベクトルに基づく内積の演算であれば手法が確立しており判断も容易である。しかしながら、むろんこの例に限るものではない。例えば、全階調範囲を四つの領域に分割しているが、これら以上とすることは自由であるし、度数分布の両端位置や、標準偏差、尖度などの統計的手法を用いてその近似度を求めることも可能である。
【０１１７】
ここで、このような統計的手法を用いて近似度を求める具体的手法について説明する。統計的手法の一例には、分布の代表値を利用するものがあげられる。いま、変数として平均値、中央値、標準偏差（分散）の差の絶対値を赤成分と緑成分、緑成分と青成分、青成分と赤成分との間で求めておく。そして、平均値と標準偏差について赤成分と緑成分の差の絶対値をＡｖｅ＿ｒｇ，Ｓｔｄ＿ｒｇとしたとき、赤成分と緑成分の間の評価関数として
ｈ（ｒｇ）＝（１−Ａｖｅ＿ｒｇ／２５５）×（１−Ｓｔｄ＿ｒｇ／２５５）と設定する。また、同様に緑成分と青成分の間の評価関数として、
ｈ（ｇｂ）＝（１−Ａｖｅ＿ｇｂ／２５５）×（１−Ｓｔｄ＿ｇｂ／２５５）と設定するとともに、青成分と赤成分との間の評価関数として、
ｈ（ｂｒ）＝（１−Ａｖｅ＿ｂｒ／２５５）×（１−Ｓｔｄ＿ｂｒ／２５５）と設定する。分布が似ている場合には平均値、中央値、標準偏差（分散）は殆ど同じになり、各変数の差も殆ど「０」になるから、評価関数ｈの値は「１」に近くなる。分布が異なる場合は差も大きくなり、評価関数ｈの値も小さくなる。従って、評価関数と実験で求められたしきい値とを比較することにより補正を行うか否かを決定することができるようになる。むろん、この場合に平均値の代わりに中央値を利用することも可能であるし、評価関数の具体例はこれに限られるものでもない。
【０１１８】
以上のような処理により各色成分毎に求めた度数分布についてある程度の類似具合が見つけられたら各度数分布から成分毎の特性を見出せるものと判断でき、かかる特性に基づいて各成分間の均一化を図る。なお、上述した各種の判断の結果、ステップＳ１０６にて非処理を実行してフラグをセットしている場合もある。このような場合には、ステップＳ２０２にて同フラグを参照しているので、以下の均一化の処理をすることなく本処理を終了する。
【０１１９】
特性の均一化の処理では、最初にオフセットの算出と修正を行う。いわゆる狭義の意味での色ずれの修正に該当し、特性の均一化を図るオフセットが本実施形態で実効値を構成する。本来であれば、図２０に示すように被写体の色成分とＲＧＢの各成分との間には正比例の関係がなければならないが、撮像素子の特性などによって各成分毎に変換特性がずれていることがある。従来は、このようなずれを通常の画像から判断することはできなかった。しかしながら、各色成分毎の度数分布を取ったときに概ね類似しているとすれば、これは本来的に一致すべきと判断できるので、各成分毎のずれであるオフセット量を検出できることになる。
【０１２０】
本実施形態においては、ステップＳ２０４にてこのオフセット量を求め、ステップＳ２０６ではかかるオフセット量を考慮して色ずれを修正するために利用するテーブルを作成している。
【０１２１】
ＲＧＢの階調データ（Ｒp,Ｇp,Ｂp ）を利用している場合、テレビジョンなどでは全体の輝度ｙｐを
ｙp＝０．３０Ｒp＋０．５９Ｇp＋０．１１Ｂp …（２）
として求めている。すなわち、緑成分が最も輝度に影響を及ぼしており、その意味で緑成分に対する他の色成分のずれを修正するのが全体の画像イメージを変化させないというメリットがある。
【０１２２】
一方、各色成分毎の度数分布のずれを求めるには、この度数分布の特徴部分を考慮することが望ましい。このため、本実施形態においては、上述した端部処理を施した度数分布の上端（Ｒｍａｘ，Ｇｍａｘ，Ｂｍａｘ）と度数分布上におけるメジアン（Ｒｍｅｄ，Ｇｍｅｄ，Ｂｍｅｄ）とを利用している。両端位置は分布を判断する意味で有効である。ただし、下端位置については元々ずれの影響が分かりにくい範囲であるため、敢えて省略している。これにより、ずれの影響が大きい範囲で得られるずれだけを重視した修正が可能となる。分布の中央となるメジアンは極端な画素があったとしても度数分布の山の位置を示すことができる。この場合、かかる山の部分は画像のイメージに大きく影響を与える部分でもあるので特性を把握する意味で効果的である。
【０１２３】
このようにして得られた、青緑赤についての上端（Ｒｍａｘ，Ｇｍａｘ，Ｂｍａｘ）とメジアン（Ｒｍｅｄ，Ｇｍｅｄ，Ｂｍｅｄ）から緑成分に対する他の色成分のずれｄＲｍａｘ，ｄＢｍａｘ，ｄＲｍｅｄ，ｄＢｍｅｄを次式に基づいて求める。
【０１２４】
ｄＲｍａｘ＝Ｇｍａｘ−Ｒｍａｘ …（３）
ｄＢｍａｘ＝Ｇｍａｘ−Ｂｍａｘ …（４）
ｄＲｍｅｄ＝Ｇｍｅｄ−Ｒｍｅｄ …（５）
ｄＢｍｅｄ＝Ｇｍｅｄ−Ｂｍｅｄ …（６）
そして、これらを参考として赤成分用オフセットｄＲと青成分用オフセットｄＢとを次式のようにして求める。
【０１２５】
ｄＲ＝（ｄＲｍａｘ＋ｄＲｍｅｄ）／２ …（７）
ｄＢ＝（ｄＢｍａｘ＋ｄＢｍｅｄ）／４ …（８）
ただし、−１２＜ｄＲ，ｄＢ＜１２とする。このように制限するのは度数分布だけで完全な色再現性を修正できる訳ではない場合に、この一例だけで大きく度数分布を修正してしまわないようにするためである。むろん、この範囲については実験的な経験より適当な値を定めればよい。また、分母の相違は（２）色に基づく影響の相違に対応しており、実験等に基づいて適宜変更可能である。
【０１２６】
これらはオフセット量に過ぎないから、実際の統計値を次式に基づいて修正して新たな上端Ｒｍａｘ２，Ｂｍａｘ２とメジアンＲｍｅｄ２，Ｂｍｅｄ２と下端Ｒｍｉｎ２，Ｂｍｉｎ２とをする。
【０１２７】
Ｒｍａｘ２＝Ｒｍａｘ＋ｄＲ …（９）
Ｒｍｅｄ２＝Ｒｍｅｄ＋ｄＲ …（１０）
Ｒｍｉｎ２＝Ｒｍｉｎ＋ｄＲ …（１１）
Ｂｍａｘ２＝Ｂｍａｘ＋ｄＢ …（１２）
Ｂｍｅｄ２＝Ｂｍｅｄ＋ｄＢ …（１３）
Ｂｍｉｎ２＝Ｂｍｉｎ＋ｄＢ …（１４）
なお、この対応関係は緑成分に対する赤成分と青成分のオフセット量に過ぎないことは明らかであり、階調値によって変化しているわけではない。従って、現実の各画素における画像データについては一律に当該オフセット量を加算すれば足りる。
【０１２８】
ただし、後述するように本実施形態においては他の要因に基づく画像データの修正も行なっており、個別に修正することは演算時間を要して不利である。従って、演算の効率化を図るべく、本実施形態においてはステップＳ２０６にて変換前のＲＧＢの階調データ（Ｒ1,Ｇ1,Ｂ1）に対する変換後のＲＧＢの階調データ（Ｒ2,Ｇ2,Ｂ2）という対応関係を表すテーブルを形成し、現実に画像データを修正するのは最後に一回だけとなるようにしている。
【０１２９】
一方、上述した実施例では、ベクトルの内積に基づいて類似度を求めるとともに同類似度をしきい値ＣＯＲＲ（「０．７」）と比較して特性の均一化を行うか否かを決めている。従って、ベクトルの内積としきい値とがほぼ一致するような場合には、同じ画像でも周囲に付加されるビットの影響を受けて異なる判断結果がなされることになりかねない。
【０１３０】
むろん、特性の均一化を行うと判断されると上述したようなオフセット量が加算され、特性の均一化を行なわないと判断されると上述したようなオフセット量が加算されないことになるので、しきい値を挟んで大きな差が生じる。
【０１３１】
このようにして結果が大きく変わってしまうのを防止する手法として連続的に変化する窓関数を利用することが効果的である。
【０１３２】
ｘ＝ｍｉｎ（ｃｏｒｒ＿ｒｇ，ｃｏｒｒ＿ｇｂ，ｃｏｒｒ＿ｂｒ） …（１４１）
とおくとともに窓関数ｆ（ｘ）は、
ｘ＜０．５ のとき ｆ（ｘ）＝０
０．５≦ｘ≦０．７ のとき ｆ（ｘ）＝５・ｘ−２．５
０．７＜ｘ のとき ｆ（ｘ）＝１
とし、上記しきい値ＣＯＲＲを「０．５」とする。窓関数ｆ（ｘ）の変化状況を図３０に示しており、ｘ＜０．５において一定の「０」となり、０．５≦ｘ≦０．７において「０」から「１．０」へと直線的に増加し、０．７＜ｘにおいて一定の「１．０」となる。そして、（７）式と（８）式において赤成分用オフセットｄＲと青成分用オフセットｄＢとを求めていたのをこのｆ（ｘ）との積に改める。すなわち、
ｄＲ＝ｆ（ｘ）・（ｄＲｍａｘ＋ｄＲｍｅｄ）／２ …（７）’
ｄＢ＝ｆ（ｘ）・（ｄＢｍａｘ＋ｄＢｍｅｄ）／４ …（８）’
とする。図５および図６に示すフローチャートにおいては類似度がしきい値を越えていなければ非処理としてしまうが、このようにオフセットに対して窓関数を乗算する場合は、図３１に示すフローチャートを実行する。すなわち、類似度に基づいて非処理とすることはやめ、ステップＳ２０５で窓関数を利用したオフセット量の算出を実行し、この算出したオフセット量を利用する。むろん、上述したようにｘ＜０．７において窓関数ｆ（ｘ）が急激に「０」へ近づくため、しきい値付近であった画像が処理によって大きく変化するということはなくなるし、類似度が低いものについてはオフセットも殆ど「０」となって悪影響は与えない。また、このようにして算出したオフセット量ｄＲ，ｄＢに基づいて（９）式〜（１４）式も算出される。
【０１３３】
窓関数ｆ（ｘ）は上述した関数に限られるものでないことは明らかである。上述した例では、相関係数の最小値に基づいて窓関数の値を変化させているが、最小値に限られる必要はない。例えば、
ｆ（ｃｏｒｒ＿ｒｇ，ｃｏｒｒ＿ｇｂ，ｃｏｒｒ＿ｂｒ） …（１４２）としてもよい。また、さらに一般的にして、
ｆ（統計量）
としても良い。この場合、統計量には最小値、最大値、メジアン、標準偏差などが該当する。
【０１３４】
一方、窓関数は窓を開け閉めするごとくある領域において演算値を有効化させるとともに別の領域において演算値を無効化させるように機能する。今、色かぶりをその原因別に分類すると、上述した画像入力装置１０のハードウェア性能などによる場合と、夕焼けなどによる意図的な色かぶりと、タングステンランプなどを利用した特殊な照明による色かぶりとに分類できる。そして、夕焼けなどによる意図的な色かぶりについては特性の統一化を図る必要がないことは上述したとおりであるが、タングステンランプなどを利用した特殊な照明による色かぶりの場合は特性の統一化を図ることも有意義である。
【０１３５】
夕焼けなどによる意図的な色かぶりとタングステンランプなどを利用した特殊な照明による色かぶりは、上述した相関係数から判断できる。すなわち、相関係数が極端に低い写真はこのような特殊な照明に起因するものであることが多く、窓関数を図３２に示すように変形した。
【０１３６】
すなわち、ｘ＜０．５の領域を変形させ、
ｘ＜０．１ のとき ｆ（ｘ）＝１．０
０．１≦ｘ＜０．３ のとき ｆ（ｘ）＝−５・ｘ＋１．５
０．３≦ｘ０．５ のとき ｆ（ｘ）＝０
とした。これにより、ｘが「０．３」以下となると窓関数ｆ’（ｘ）は再び直線的に上昇し、ｘが「０．１」以下の領域で同窓関数ｆ’（ｘ）は一定の「１．０」となる。むろん、窓関数が「１．０」となったときには各成分の分布状況に基づいて演算された大きなオフセット量がそのまま適用され、照明の影響を打ち消して特性の均一化が図られる。
【０１３７】
なお、窓関数は遷移領域で直線的に変化する必要はなく、単調増加もしくは単調減少するような曲線的なものであっても構わない。
【０１３８】
また、図５に示すようにステップＳ１１６にて類似度をチェックし、類似度が低い場合にステップＳ１０６にて非処理を実行してフラグをセットしたり、あるいは、図３１に示すようにステップＳ１１６を実行することなくステップＳ２０５にて窓関数を利用してオフセット量を算出することにより、実質的には類似度が低いときに特性の均一化を図らないようにしているので、これらのソフトウェア処理及びこれを実現するハードウェアなどによって修正制御手段を構成していると言える。
【０１３９】
一方、度数分布の広がり方に差がある場合にはこれを均一化させることが有効である。本実施形態においては、この度数分布の広がり方を均一化させつつ、度数分布を可能な範囲で拡大させて各成分毎にコントラストを強調させている。
【０１４０】
コントラストの強調は、階調範囲が「０」〜「２５５」としたときに、変換前の各色成分（Ｒ1 ，Ｇ1 ，Ｂ1 ）と各成分の最大値Ｒｍａｘ２，Ｇｍａｘ，Ｂｍａｘ２と最小値Ｒｍｉｎ２，Ｇｍｉｎ，Ｂｍｉｎ２から変換先の各色成分（Ｒ2 ，Ｇ2 ，Ｂ2 ）を次式に基づいて求める。
【０１４１】
Ｒ2＝far×Ｒ1＋fbr …（１５）
Ｇ2＝fag×Ｇ1＋fbg …（１６）
Ｂ2＝fab×Ｂ1＋fbb …（１７）
ただし
far＝２５５／（Ｒｍａｘ２−Ｒｍｉｎ２） …（１８）
fag＝２５５／（Ｇｍａｘ −Ｇｍｉｎ ） …（１９）
fav＝２５５／（Ｂｍａｘ２−Ｂｍｉｎ２） …（２０）
fbr＝−far×Ｒｍｉｎ２あるいは２５５−far×Ｒｍａｘ２ …（２１）
fbg＝−fag×Ｇｍｉｎ あるいは２５５−fag×Ｇｍａｘ２ …（２２）
fbb＝−fab×Ｂｍｉｎ２あるいは２５５−fab×Ｂｍａｘ２ …（２３）
また、上記変換式にてＲ2,Ｇ2,Ｂ2 ＜０ならばＲ2,Ｇ2,Ｂ2 ＝０とし、Ｒ2,Ｇ2,Ｂ2 ＞２５５ならばＲ2,Ｇ2,Ｂ2 ＝２５５とする。ここにおける、far,fag,fab は傾きであり、fbr,fbg,fbb はオフセットといえる。この変換式によれば、図２２に示すように、あるせまい幅を持った輝度分布を再現可能な範囲まで広げることができる。なお、基本的に輝度の分布範囲の拡大においては、画素数が変化するわけではないので、ヒストグラムの面積は一致する。ただし、このように再現可能な範囲を最大限に利用して輝度分布の拡大を図った場合、ハイライト部分が白く抜けてしまったり、ハイシャドウ部分が黒くつぶれてしまうことが起こる。
これを防止するため本実施形態においては、拡大する階調範囲を制限している。
すなわち、全階調範囲の上端と下端に拡大しない範囲として階調値で「５」だけ残している。この結果、変換式のパラメータは次式のようになる。
【０１４２】
far＝２４５／（Ｒｍａｘ２−Ｒｍｉｎ２） …（２４）
fag＝２４５／（Ｇｍａｘ −Ｇｍｉｎ ） …（２５）
fav＝２４５／（Ｂｍａｘ２−Ｂｍｉｎ２） …（２６）
fbr＝５−far×Ｒｍｉｎ２あるいは２５０−far×Ｒｍａｘ２ …（２７）
fbg＝５−fag×Ｇｍｉｎ あるいは２５０−fag×Ｇｍａｘ２ …（２８）
fbb＝５−fab×Ｂｍｉｎ２あるいは２５０−fab×Ｂｍａｘ２ …（２９）
そして、この場合には階調「５」未満と、階調「２５０」以上については変換を行わないようにする。
【０１４３】
なお、本実施形態においては、ハイライト部分とハイシャドウ部分とを保持するために一律に階調範囲の端部から階調値にして「５」の範囲を非拡大領域としているが、ハイライト部分やハイシャドウ部分を比較的再現しやすいような画像出力装置であればその範囲を狭くしても良いし、再現力がさらに弱い場合にはより範囲を大きくするようにしても良い。また、一律に拡大しないのではなく、ボーダー領域で徐々に拡大率を制限するようにしていっても良い。
【０１４４】
また、図２３（ａ）には画像の輝度分布が狭い場合を示しているが、これまで述べたようにして輝度分布の拡大率（far,fag,fabに対応）を適用してしまうと、再現可能な範囲に合わせて非常に大きな拡大率が得られる場合も生じてくる。
すると、夕方のような薄暮の状態では最も明るい部分から暗い部分までのコントラストの幅が狭くて当然であるのに、この画像についてコントラストを大きく拡大しようとする結果、昼間の画像のように変換されてしまいかねない。このような変換は希望されないので、拡大率には制限を設けていおき、far,fag,fabが１．５（〜２）以上とはならないように制限する。これにより、薄暮は薄暮なりに表現されるようになる。
【０１４５】
拡大率に制限を設けない場合を図２３（ａ）の一点鎖線に示しており、変換後には再現可能な範囲で余分な部分は残っていない。しかしながら、拡大範囲を制限する場合には、同図（ｂ）の二点鎖線で示すように、変換後の分布をどこに持ってくるかの自由度が生じてしまい、場合によっては全体的に明るくなりすぎたり、暗くなり過ぎたりしかねない。従って、このような場合には、変換前における階調範囲内において上端側と下端側に残っている残余の領域の割合（ｍ１：ｍ２）が、変換後において上端側と下端側に残っている残余の領域の割合（ｎ１：ｎ２）と一致するように変換すればよい。
【０１４６】
以上のようにしてパラメータfar,fag,fab,fbr,fbg,fbbを得る処理をステップＳ２０８にて実行し、続くステップＳ２１０ではステップＳ２０６と同様に変換テーブルを作成する。このときの変換テーブルは図２１に示す変換後の成分値（Ｒ1，Ｇ1，Ｂ1）を入力として対応関係を演算し、変換テーブルにおける変換後の成分値（Ｒ1，Ｇ1，Ｂ1）と入れ換える。これにより、同変換テーブルを参照すればステップＳ２０４にて求めたオフセット量の加算とステップＳ２０８にて求めたコントラストの強調処理という二つの処理を同時に実施することになる。
【０１４７】
一方、度数分布の各色成分間のずれとして、もう一つ、全体的な明るさという要素が残る。従って、ステップＳ２１４にてこの明るさのずれを均一化させるγ補正をかけるため、ステップＳ２１２にてγを算出する。例えば、図２４にて実線で示す赤成分の度数分布の山が全体的に暗い側に寄っていたり、同図にて一点鎖線で示す青成分の度数分布の山が全体的に明るい側に寄っていたりした場合に、同図鎖線で示す緑成分の度数分布の山のように全体的に中央に寄るように修正移動させると良い。
【０１４８】
各種の実験を行った結果、本実施形態においては、度数分布におけるメジアンを基準に判断し、同メジアンが「８５」未満である場合に暗い画像と判断して以下のγ値に対応するγ補正で明るくする。
【０１４９】
γr＝Ｒｍｅｄ２／８５ …（３０）
γg＝Ｇｍｅｄ／８５ …（３１）
γb＝Ｂｍｅｄ２／８５ …（３２）
あるいは、
γr＝（Ｒｍｅｄ２／８５）**（１／２） …（３３）
γg＝（Ｇｍｅｄ／８５）**（１／２） …（３４）
γb＝（Ｂｍｅｄ２／８５）**（１／２） …（３５）
とする。
【０１５０】
この場合、γr，γg，γb＜０．７となっても、γr，γg，γb＝０．７とする。このような限界を設けておかないと夜の画像が昼間のようになってしまうからである。なお、明るくしすぎると全体的に白っぽい画像になってコントラストが弱い画像になりやすいため、彩度を合わせて強調するなどの処理が好適である。
【０１５１】
一方、メジアンが「１２８」より大きい場合に明るい画像と判断して以下のγ値に対応するγ補正で暗くする。
【０１５２】
γr＝Ｒｍｅｄ２／１２８ …（３６）
γg＝Ｇｍｅｄ／１２８ …（３７）
γb＝Ｂｍｅｄ２／１２８ …（３８）
あるいは、
γr＝（Ｒｍｅｄ２／１２８）**（１／２） …（３９）
γg＝（Ｇｍｅｄ／１２８）**（１／２） …（４０）
γb＝（Ｂｍｅｄ２／１２８）**（１／２） …（４１）
とする。この場合、γr，γg，γb＞１．３となっても、γr，γg，γb＝１．３として暗くなり過ぎないように限界を設けておく。なお、暗くしすぎると色が乗りすぎて濃い画像になるので、合わせて彩度強調を弱くするなどの処理が好適である。ただし、明るい背景の中の被写体に対してはこのような暗くする処理はかえって悪影響を及ぼす場合もある。例えば、空が画像の半分をしめるような風景画像や晴れた日の記念写真などでは、ただでさえ逆光で顔が暗くつぶれ気味であることが多いからである。これらの画像の場合は暗い部分と明るい部分とが混じっているので各色成分の標準偏差を求めると比較的高い値となっていることが多い。従って、そのような標準偏差が「７０」よりも大きいような場合には暗くするためのγ補正を行わないようにすることも可能である。γ補正をした場合における対応関係を図２５に示しており、γr，γg，γb＜１であれば上方に膨らむカーブとなり、γr，γg，γb＞１であれば下方に膨らむカーブとなる。なお、明るさの修正に関しては必ずしも度数分布に基づく必要はなく、他の要素から明るさを判断して修正するようにしても良い。
【０１５３】
以上のようにして決定したγr，γg，γbに対してγ補正するには次式のようにする。変換前の階調値ｒ0,ｇ0,ｂ0に対して変換後の階調値Ｒ1,Ｇ1,Ｂ1は、
Ｒ1＝２５５＊（ｒ0／２５５）**γr …（４２）
Ｇ1＝２５５＊（ｇ0／２５５）**γg …（４３）
Ｂ1＝２５５＊（ｂ0／２５５）**γb …（４４）
なお、このγ補正も図２１に示す変換テーブルに対して実行する。すなわち、コントラスト強調の場合と同様に図２１に示す変換後の成分値（Ｒ1，Ｇ1，Ｂ1）を入力として対応関係を演算し、変換テーブルにおける変換後の成分値（Ｒ1，Ｇ1，Ｂ1）と入れ換える。これにより、同変換テーブルを参照すればオフセット量の加算とコントラストの強調処理に加えて明度の修正の処理を同時に実施することになる。
【０１５４】
そして、最後に、ステップＳ２１６にて画像データの変換を行い、全画素の画像データ（ｒｍ，ｇｍ，ｂｍ）について図２１に示す変換テーブルを参照し、変換後の画像データ（Ｒｍ，Ｇｍ，Ｂｍ）を得るという処理を繰り返すことになる。
【０１５５】
本実施形態においては、オフセット量による修正と、コントラストの強調と、明るさの修正とをこの順番に実施しているが、必ずしもこの全てを実施しなければならないわけではないし、また、個々の修正手法も適宜変更して実施することもできる。
【０１５６】
例えば、上述した実施形態においては、コントラストの強調処理においては、変換前の成分値に対して直線的な対応関係をなす変換式で修正を行うようにしているが、より滑らかな変換となるように、図２６に示すようないわゆるＳ字カーブの変換を行うようにしても良い。また、この場合、度数分布の広がりを両端位置で判断するのではなく、分布の散らばり具合を表す標準偏差の概念を利用することも可能である。以下、標準偏差を利用したＳ字カーブの対応関係でコントラストを強調する例を示す。なお、各成分毎の度数分布に基づいて変換は行うものの、演算手法は全てにおいて共通するため、輝度の表現で表し、変換前の輝度ｙより変換後の輝度Ｙを得る手順を示しておく。
【０１５７】
標準偏差については二つの考え方があるが本実施形態においては、次式に基づいて演算する。
【０１５８】
【数５】

【０１５９】
ｙp：各画素の変換前の輝度
ｙm：各画素の変換前の輝度の平均値
標準偏差は輝度分布の広がり量に対応するものであるが、広がり量を表す意味では分散を利用してもよい。また、本実施形態のように全体としての階調数が２５６階調となっているので、分布の尖度ｋから広がり量を求めることも可能である。
【０１６０】
【数６】

【０１６１】
なお、ここにいうｋ＝３の尖度が正規分布の広がり量に相当する。
【０１６２】
このようにして求めた輝度分布の広がり量である標準偏差σに基づいて、コントラストの強調は分布密度の大きい範囲に多くの階調数を与えつつ分布密度の小さい範囲に少ない階調数を割り当てることでもあり、図２６に示すように入出力の対応関係がいわゆるＳ字カーブとなると変換前に割り当てられている階調範囲rng0に対して変換後に割り当てられる階調範囲RNG1は大きくなり、割り当てられた階調数が多くなったことになる。一方、入力における低輝度側と高輝度側における階調範囲rng0を外れた範囲についていえば、変換後に割り当てられる階調範囲は少なくなったことになる。
【０１６３】
本実施形態においては、階調範囲の中心位置ｙmidを「１２８」として、この中心位置ｙmid以下ではγ１を与えるとともに、中心位置ｙmid より大きい範囲ではγ２を与えたγ補正を行うものとし、このγ１，γ２を標準偏差σに基づいて定める。すなわち、
ｙ≦１２８では、
γ１＝（σstd_limit／σ）**fc …（４７）
ｙ＞１２８では、
γ２＝（σ／σstd_limit）**fc …（４８）
とし、ステップＳ２０４にてこれらのパラメータ演算を実行する。ここにおいて、σstd_limitとfcは変換結果を考慮して実験的に求めて与えたパラメータであり、本実施形態においてはσstd_limitを「１２８」とするとともにfcを「０．１」としている。標準偏差σは概して「１２８」よりも小さな値となるからこれらの関係式では標準偏差σが大きいと、γ２とγ１はそれぞれ「１」に近づくことになり、Ｓ字カーブの傾斜は緩やかになる。これは、広がり量が大きいときに中心位置ｙmidを中心とする階調範囲rng0に対して変換先の階調範囲RNG0はさほど広くならないことを意味しており、より具体的には画像データの輝度が広く分布しているときには輝度範囲を拡大するような変換を行わないことを意味する。
これに対して、標準偏差σが小さいと、γ２とγ１はそれぞれ「１」から離れることになり、Ｓ字カーブの傾斜は急になる。これは、広がり量が小さいときに中心位置ｙmidを中心とする階調範囲rng0に対して変換先の階調範囲RNG1が広く拡大されることを意味しており、より具体的には画像データの輝度が狭い範囲にしか分布していないときには輝度範囲を拡大させる変換を行なうことを意味する。
【０１６４】
本実施形態においては、Ｓ字カーブの対応関係をγ補正によって成立させているが、図２７には、階調「０」、下方側四分点ｙq1、中心位置ｙmid、上方側四分点ｙq3、階調「２５５」という五点を基準点としつつ、階調「０」と中心位置ｙmidと階調「２５５」に対してはＹ＝ｙとしつつ、下方側四分点ｙq1と上方側四分点ｙq3における変換点を標準偏差に基づいて決定する。そして、これらの五点を結ぶ対応関係をスプライン補間演算やニュートン補間で求める。むろん、中心位置ｙmidから下方側の三点や上方側の三点をそれぞれスプライン補間演算やニュートン補間で求めるようにしてもよい。
【０１６５】
すなわち、図２８に示すように、ステップＳ２３０にて各成分毎に標準偏差σr，σg，σb を求め、ステップＳ２３２にて各色成分毎のγ１，γ２を求め、ステップＳ２３４にてγ１，γ２を利用した対応関係に基づいて変換テーブルを作成する。これらのステップＳ２３０〜Ｓ２３４を図６に示すフローチャートにおけるステップＳ２０８〜Ｓ２１４の処理に代えて実行する。
【０１６６】
次に、上記構成からなる本実施形態の動作を順を追って説明する。
【０１６７】
スキャナ１１などで写真を撮像したとすると、同写真をＲＧＢの階調データで表した画像データがコンピュータ２１に取り込まれ、ＣＰＵは図５及び図６に示す画像処理のプログラムを実行して画像データの色再現性を修正する処理を実行する。
【０１６８】
まず、ステップＳ１０２では画像データを所定の誤差内となる範囲で間引き、選択した画素について各色成分毎に度数分布を求める。このままの度数分布を使用することはできず、まず、画像が白黒のような二値画像でないかステップＳ１０４にて判断するとともに、ステップＳ１０８では自然画か否かを判断する。二値画像である場合や自然画でない場合などを除き、ステップＳ１１０では画像データに枠部がないか判断し、枠部があれば除いた後、ステップＳ１１４にて度数分布の両端の不明瞭領域を取り除く。この状態で各色成分毎の度数分布について特徴ベクトルを求め、特徴ベクトル同士の内積から度数分布の類似度をチェックする。各色成分毎の度数分布があまり似ていないときには、元の画像データにおいて意図的にカラーバランスがずれていることの裏付けとなり、特性を均一化させる処理は行わない。しかしながら、所定のしきい値との比較においてある程度の類似性が見られる場合にはカラーバランスがずれてしまっているものと判断して、以下のような特性の均一化を図る。
【０１６９】
すなわち、ステップＳ２０４では度数分布の上端とメジアンを利用して緑成分に対する赤成分オフセットｄＲと青成分用オフセットｄＢとを求め、ステップＳ２０６では最後の画像データ変換のための変換テーブルを形成する。続いて、ステップＳ２０８では各色成分毎にコントラストの均一化を図りつつ同コントラストを強調するためのパラメータを演算し、ステップＳ２１０では同パラメータに基づいてコントラスト強調させつつ各色成分毎のコントラストのバランスを一致させるための変換テーブルを作成する。そして、ステップＳ２１２では各色成分毎の明るさを均一化させるためのγ補正のパラメータを演算し、かかるγ補正を施すことになる変換テーブルを作成する。
【０１７０】
最後に、ステップＳ２１６にて上述したようにして作成されている変換テーブルを参照して全画素についての画像データを変換する。
【０１７１】
この場合は類似性があるか否かに応じてステップＳ２０２で処理を分けているが、図３１に示すフローチャートを実行する場合には類似具合に応じてオフセット量という実効値を変化させ、実質的に特性の均一化を図ったり図らなかったりする。
【０１７２】
むろん、上述したように二値画像や自然画でない場合などにおいてはかかる画像処理は行われないが、本発明の画像処理が行われた場合には、写真の状態では色ずれなどの入力機器に起因するような色再現性の悪かった画像データであるにもかかわらず、各色成分毎に色ずれとコントラストと明るさとが均一化するとともにコントラストを強調されてメリハリのある良好な画像が極めて容易に得られるようになる。
【０１７３】
なお、上述した実施形態においては、いくつかのパラメータを一定としているが、コンピュータ２１上では所定のＧＵＩを介してユーザーが選択できるようにしても良い。
【０１７４】
このように、ステップＳ１０２で間引きするなどしながら画像データについて各色成分毎の度数分布を求め、ステップ１１６にて度数分布間に類似性があるか否かを判断し、類似性が低くなければ本来的に度数分布から見出される特性は一致するものと判断して、ステップＳ２０４〜Ｓ２１６にてオフセット修正やコントラスト強調や明るさ修正によってずれを修正することにより、色再現性の悪い画像データであってもメリハリのある良好な画像としつつ、かかる作業を自動化し、非熟練者でも容易にカラーバランスの修正を行うことができるようになる。
【図面の簡単な説明】
【図１】本発明の一実施形態にかかる画像処理装置が適用される画像処理システムのブロック図である。
【図２】同画像処理装置の具体的ハードウェア構成例を示すブロック図である。
【図３】本発明の画像処理装置の他の適用例を示す概略ブロック図である。
【図４】本発明の画像処理装置の他の適用例を示す概略ブロック図である。
【図５】本発明の画像処理装置における度数分布検出手段と類似具合判断手段とに相当するフローチャートである。
【図６】本発明の画像処理装置におけるオフセット量修正手段とコントラスト修正手段と明るさ修正手段とに相当するフローチャートである。
【図７】変換元の画像における座標を示す図である。
【図８】サンプリング周期を示す図である。
【図９】サンプリング画素数を示す図である。
【図１０】変換元の画像とサンプリングされる画素の関係を示す図である。
【図１１】度数分布を検出するための配列変数を示す図である。
【図１２】グレー画像抽出間引き処理のフローチャートである。
【図１３】白黒の画像を示す図である。
【図１４】白黒の画像の輝度分布を示す図である。
【図１５】非自然画と自然画の場合の各色成分毎の度数分布の状態を示す図である。
【図１６】枠部のある画像を示す図である。
【図１７】枠部のある画像の度数分布を示す図である。
【図１８】度数分布の端部処理と端部処理にて得られる端部を示す図である。
【図１９】各色成分毎の特徴ベクトルとするための要素の抽出方法を示す図である。
【図２０】色再現性の修正の必要のないリニアな関係を示す図である。
【図２１】度数分布に基づいて画像データを変換する際の変換テーブルを示す図である。
【図２２】度数分布の拡大と全階調の範囲を示す図である。
【図２３】コントラストの拡大率に制限を与える場合を示す図である。
【図２４】明るさの均一化を図る必要のある度数分布を示す図である。
【図２５】γ補正で変更される変換関係を示す図である。
【図２６】Ｓ字カーブの対応関係でコントラストを強調させる変換関係を示す図である。
【図２７】特定した変換点を補間法で接続する場合の変換関係を示す図である。
【図２８】Ｓ字カーブでコントラストと明るさを修正する場合の部分フローチャートである。
【図２９】画像処理プログラムを記録する媒体から同プログラムをハードディスクに転送する状態を示す図である。
【図３０】利用する窓関数の変化状況を示すグラフである。
【図３１】窓関数を利用してオフセット量を調整する場合における画像処理プログラムのフローチャートである。
【図３２】利用する他の窓関数の変化状況を示すグラフである。
【符号の説明】
１０…画像入力装置
１１…スキャナ
１１ｂ…スキャナ
１２…デジタルスチルカメラ
１２ａ…デジタルスチルカメラ
１２ｂ…デジタルスチルカメラ
１３ｂ…モデム
２０…画像処理装置
２１…コンピュータ
２２…ハードディスク
３０…画像出力装置
３１…プリンタ
３１ａ…プリンタ
３１ｂ…プリンタ
３２…ディスプレイ
３２ａ…ディスプレイ
４１…フレキシブルディスク
４２…ＣＤ−ＲＯＭ
４３…ハードディスク
４４…ＲＯＭ
４５…ＲＡＭ
４６ａ…通信回線
４６ｂ…モデム
４６ｃ…ファイルサーバPatent application title: IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND MEDIUM CONTAINING IMAGE PROCESSING PROGRAM
[Claim of claim]
[Claim 1]Each component value of the same image data is input for image data in which each pixel in an image consisting of a plurality of pixels is represented by gradation table color data consisting of substantially equal color components, and predetermined conversion processing is performed and output. An image processing apparatus configured to perform conversion to correct color balance in relation to an input and an output.
A statistic based on the distribution of gradation table color data is determined for each color component of the target pixel while moving the target pixel for each pixel to be processed, and between the color components based on the same statistic for each color component An image processing apparatus comprising: characteristic equalization means for obtaining the deviation of the characteristic of the distribution at the point and making the characteristic among the color components uniform based on the deviation of the characteristic of the obtained distribution.
[Claim 2]In the image processing apparatus according to claim 1, the characteristic uniforming unit specifies the conversion relationship of the gradation value for each color component, and converts the color component value for each pixel to obtain the characteristics between the color components. Uniform theAn image processing apparatus characterized by
[Claim 3]Claim 1 or 2The image processing apparatus according to any one of the preceding claims, wherein the characteristic uniforming unit targets the pixels to which the gradation table color data of each color component approximates the distribution.
[Claim 4]The image processing apparatus according to any one of claims 1 to 3, wherein the characteristic uniforming unit determines the characteristic from a predetermined position in the distribution, and shifts the respective components so as to make the characteristic uniform. An image processing apparatus characterized by obtaining an offset amount corresponding to and correcting each component value.
[Claim 5]5. The image processing apparatus according to claim 4, wherein the characteristic uniformizing unit determines an end position of the distribution range as the characteristic of the distribution.
[6]The image processing apparatus according to any one of claims 4 and 5, wherein the characteristic uniforming unit determines a substantially central position of the distribution as the characteristic of the distribution.
[7]The image processing apparatus according to any one of claims 1 to 6, wherein the characteristic equalizing means substantially uniforms the spread of the distribution for each color component.
[Claim 8]8. The image processing apparatus according to claim 7, wherein said characteristic uniforming means enlarges both ends of the spread of said distribution within an effective gradation range.
[9]In the image processing apparatus according to the seventh aspect, the characteristic uniformizing unit gives a large number of gradations to a large range of distribution density based on the spread amount of the same distribution, and reduces few gradations to a small range of distribution density. An image processing apparatus characterized by assigning a number.
10.The image processing apparatus according to any one of claims 1 to 9, wherein the characteristic uniforming unit equalizes the brightness based on the distribution of each component.
11.The image processing apparatus according to claim 10, wherein the characteristic uniformizing unit determines the contrast of the image by comparing the substantially central position of the distribution with a predetermined gradation.
[12]12. The image processing apparatus according to claim 10, wherein the characteristic uniformizing unit equalizes the brightness and darkness of the image by γ correction for each component based on the judgment result of the brightness and darkness of the image. An image processing apparatus characterized by
[13]Each component value of the same image data is input for image data in which each pixel in an image consisting of a plurality of pixels is represented by gradation table color data consisting of substantially equal color components, and predetermined conversion processing is performed and output. When performing conversion to correct the color balance based on the relationship between input and output, the target pixel is moved with each pixel as the processing target, A statistic based on the distribution of tone color data is determined for each color component of the pixel of interest, and a deviation of the distribution characteristic among the color components is determined based on the same statistic for each color component, and the determined distribution An image processing method characterized in that the characteristics among the color components are made uniform based on the deviation of the characteristics.
14.A medium on which a program for performing image processing by a computer is recorded, and each component value of the image data representing the pixel data in the image composed of a plurality of pixels represented by gradation table color data composed of roughly equal color components In performing conversion to correct the color balance based on the relationship between the input and the output by performing predetermined conversion processing by inputting the same target pixel while moving each pixel as a processing target The statistic based on the distribution of tone color data is determined for each color component of each color component, the deviation of the distribution characteristic among the color components is determined based on the same statistic amount for each color component, and the determined distribution characteristic It shows the equalization of the characteristics between each color component based on the deviation.A medium having an image processing program recorded thereon.
Detailed Description of the Invention
[0001]
Field of the Invention
The present invention relates to an image processing apparatus, an image processing method, and a medium storing an image processing program, and more particularly, to an image processing apparatus, an image processing method, and an image processing program storing an image processing program for correcting color balance.
[0002]
[Prior Art]
Conventionally, correction of color balance often refers to correction of so-called color cast or the like. That is, it is correction processing for color misregistration in an image input device or the like.
[0003]
For example, in a digital still camera or the like, image data is output as RGB (red-green-blue) gradation table color data. In this case, depending on the characteristics of the lens and the CCD element, a specific color, for example, one having a so-called color deviation in which red is emphasized more than the actual color can be seen.
[0004]
Conventionally, when it is desired to perform some correction on image data having such color misregistration, the data is read into image processing software etc., and the operator weakens the component value of the emphasized color through trial and error through a predetermined operation. ing.
[0005]
[Problems to be solved by the invention]
The conventional correction method described above has the following problems.
[0006]
Since the operator makes corrections by trial and error, the accuracy may be poor and it may be difficult for non-experts.
[0007]
In addition, since the predetermined component value is uniformly increased or decreased, it may not be said that correction is properly performed over the entire gradation.
[0008]
Furthermore, efficient correction can not be said to be possible when elements other than mere color shift are included.
[0009]
In addition, it is basically impossible to determine whether the gradation color data for each component representing the color of each pixel is correct unless comparison with a standard color or the like is performed. It must be said that it is extremely difficult to automatically determine whether or not there is a color shift, given that it may be more natural to emphasize certain components by external factors. Absent.
[0010]
The present invention has been made in view of the above problems, and provides an image processing apparatus, an image processing method, and an image processing method capable of automatically correcting correction of color reproducibility such as so-called color shift and comprehensively correcting color balance. It aims at offer of the medium which recorded the program.
[0011]
[Means for Solving the Problems]
In order to achieve the above object, the invention according to claim 1 isEach component value of the same image data is input for image data in which each pixel in an image consisting of a plurality of pixels is represented by gradation table color data consisting of substantially equal color components, and predetermined conversion processing is performed and output. The image processing apparatus is configured to perform conversion that corrects color balance based on the relationship between input and output, and moves the pixel of interest while processing the pixel of interest, and for each color component of the pixel of interest A statistic based on the distribution of gradation color data is determined, and a deviation of the distribution characteristic among the color components is determined based on the same statistic for each color component, and the determined distribution characteristicA configuration is provided including a property equalizing means for equalizing the characteristics among the color components based on the deviation.
[0012]
In the invention according to claim 1 configured as described above, gradation color data consisting of substantially equal color components are used.When image data representing each pixel is present, each component value of the image data is input, subjected to predetermined conversion processing, and output, thereby performing conversion for correcting color balance based on the relationship between input and output. However, at this time, the characteristic uniforming unit obtains a statistic based on the distribution of gradation table color data for each color component of the pixel of interest while moving the pixel of interest with each pixel as a processing target. The deviation of the characteristic of the distribution among the color components is determined based on the same statistical value for each component, and the characteristic among the color components is made uniform based on the deviation of the characteristic of the determined distribution.
Further, in the invention according to claim 2, the characteristic equalizing means specifies the conversion relationship of the gradation value for each color component, and converts each color component value for each pixel to obtain characteristics of each color component. Do the equalizationing.
[0013]
Taking a digital still camera as an example, conventionally, to correct color balance means whether or not the conversion characteristics of color components in each pixel coincide with each other, whether the component value tends to be emphasized for each color. I was adjusting it. On the other hand, in the present invention, the distribution of gradation table color data is determined for each color component, and the characteristics are found in a situation separated for each color component, and such characteristics are made uniform. That is, it can be understood that an image having a clear image without color misregistration as an image can be obtained when the characteristics of the distribution of gradation color data match for each color component regardless of the contents of the image. Homogenization is determined based on the distribution. The term "homogenization" as used herein does not mean homogenization in a strict sense, but it may be at least as uniform as the tendency.
[0014]
In this sense, the distribution may be any distribution that can find the distribution state of each component in the image data, and various statistical methods including frequency distribution can also be adopted. For example, it is possible to use secondary processing data such as standard deviation, median, or mode derived from frequency distribution. In this case, the frequency distribution has an advantage of easy processing and simplification.
[0015]
By the way, it may be natural that the characteristics of the distributions do not match. For example, in the case of an evening scene, there is no unnaturalness even if only the red system color is used. In such a case, it is more unnatural to equalize the characteristics determined from the distribution of each color component. For this reason,The characteristic equalization means may be configured to include correction control means for judging the degree of similarity between the color components based on the distribution, and not making the correction if the degree of similarity is small.
[0016]
Configured as aboveIfFirst, the correction control means of the property equalizing means determines the degree of similarity between the color components. In the case of an evening scene when the color components of red (R), green (G) and blue (B) are used, a biased distribution distributed only to the red component is obtained, and the blue and green components The distribution decreases extremely. In such a case, it is natural that the characteristics of the distribution do not become uniform, the degree of similarity between each color component is extremely small, and the characteristics are not homogenized. On the other hand, when the color components appear on average, it is judged that the degree of similarity of the distribution is large, and in such a case, bias unique to the image input device etc. is realized by making the mutual characteristics uniform. It is also possible to reduce.
[0017]
In determining the similarity of the distributions, it is not necessary to compare individually using the frequency distribution at all gradations, and as a suitable example in such a case,The correction control means may be configured to divide the possible gradation range of the gradation color data into a plurality of regions and determine the degree of similarity between the color components by comparing the distribution of each region.
[0018]
Configured as aboveIfIn determining the similarity of the distribution, the possible gradation range of the gradation color data is divided into a plurality of regions, and the distribution for each region is compared among the color components. This reduces the effort to compare the distribution seen for each tone.
[0019]
On the other hand, various methods can be employed to determine the degree of similarity of distribution, and one example isThe correction control means may be configured to determine the similarity based on an inner product of vectors having the distribution of each color component as an element.
[0020]
The inner product of vectors can be easily done to see the correlation between each other. Therefore, in order to adopt the method, the distribution itself is considered as a vector element, and an inner product is obtained between each color. When the degree of similarity is the largest, it becomes "1", and when the degree of similarity is small, it approaches "0". In order to consider the distribution as a vector element, it is not necessary to make it an element for all tone numbers.,allThe distribution in the region where the gradation range is classified into several may be used as the element of the vector.
[0021]
When the degree of similarity is small, it is possible to compare the degree of similarity with a predetermined threshold value and branch the determination based on the comparison result as a method for preventing equalization of the characteristics. On the other hand, it is also possible to adopt a method of controlling the effective value as a method for equalizing the characteristics when the degree of similarity is large and not for equalizing the characteristics when the degree of similarity is small. As such an example,The characteristic equalization means utilizes an effective value for achieving the uniformity of the characteristic, and the correction control means substantially enables or invalidates the correction by changing the effective value. The effective value may be changed continuously.
[0022]
Configured as aboveIfThe effective value for achieving the uniformity of the characteristics is determined, and the correction is substantially enabled or invalidated by changing the effective value. A window function etc. is effective for such control, and more specifically, the window function that takes "1" when it is desired to be valid and "0" when it is desired to be invalidated is integrated to the effective value do it.
[0023]
On the other hand, if it changes discontinuously in the area | region which changes from "1" to "0", or "0" to "1" in this example, judgment of the degree of similarity will be delicate depending on how to take in the gray scale color data. May change to Then, even if the image is the same, the judgment results may be opposite. Therefore, as a countermeasure in such a case, the effective value is made to change continuously. Of course, the window function can be modified in various ways, and depending on the factor, it is possible to make the effective value effective in a region of small similarity, and vice versa, in order to treat exceptionally. is there. Of course, the effective value here refers to a value widely involved in correction, such as a correction amount, a correction amount, or an offset amount, and this effective value is calculated also for the window function described above. It may be a single variable used in the process.
[0024]
Also, in general, the fact that the degree of similarity is small is not the color shift due to the device. However, special lighting may change the degree of similarity extremely. Therefore, even if the degree of similarity is small, it may be effective to equalize the characteristics depending on the cause. Therefore, it is generally possible not to attempt to make the characteristics uniform when the degree of similarity is small, but to make the characteristics uniform even if the degree of similarity is small depending on the cause. For example, when the degree of similarity is extremely small, it is possible to make the characteristics uniform, and to realize this, it is also possible to use a window function corresponding to the degree of similarity.
[0025]
Incidentally, with regard to the idea of unifying the characteristics on the basis of the degree of similarity between the color components as described above, the method of unifying the characteristics is not necessarily limited. Therefore, as a suitable example corresponding to the present idea,Each component value of the image data is input for image data in which an image is represented as a set of pixels in a dot matrix form by gray scale color data consisting of similar color components, and subjected to predetermined conversion processing and output. Is an image processing apparatus in which conversion for correcting color balance is performed based on the relationship between input and output, and the degree of similarity of the distribution of gradation table color data is determined for each color component, and the degree of similarity is small In this case, the color balance may not be corrected.
[0026]
Also,Each component value of the image data is input for image data in which an image is represented as a set of pixels in a dot matrix form by gray scale color data consisting of similar color components, and subjected to predetermined conversion processing and output. When performing conversion that corrects the color balance based on the relationship between input and output, find the degree of similarity of the distribution of tone color data for each color component, and do not correct the color balance if this degree of similarity is small. In the invention of a medium storing a program for image processing by a computer or a computer, image data in which an image is represented as a set of pixels in a dot matrix by gray scale color data consisting of substantially equal color components is the same. By inputting each component value of the image data, performing predetermined conversion processing, and outputting it, the relationship between input and output In carrying out the transformation to correct the over balance, the degree of similarity calculated in the distribution of the respective color components for each the tone color specification data may be configured to not modify the color balance when the similarity degree is smaller.
[0027]
Configured in this wayIfThe degree of similarity of the distribution of gradation color data is determined for each color component, and the color balance is not corrected if the degree of similarity is small.
[0028]
On the other hand, when obtaining the distribution as described above, it is not always necessary to treat all the pixels equally. And, as an example, claimThe third aspect of the present invention is the image processing apparatus according to any one of the first and second aspects, wherein the characteristic uniformizing means is an image in which the gradation table color data of each color component is similar.The element is the target of the above distribution.
[0029]
Essentially, a person can look at an image to determine a color shift if the color that is supposed to be achromatic is colored. That is, when looking at only red ones, it can not be determined whether or not it is a color shift unless the original color is known. Therefore, it can be said that it is effective to look at the dispersion of the characteristics for only the achromatic color pixels in order to examine the balance between each color.
For this reason, the characteristic equalizing means determines, for each pixel, whether or not the gradation table color data of each color component are close to each other, and counts as a target of distribution only if they are similar. The characteristics are made uniform based on the obtained distribution.
[0030]
Whether or not the gradation color data of the color component in each pixel is similar may be determined by obtaining the maximum value and the minimum value and using the difference therebetween. Also, it is possible not to handle extreme data because it is likely to be saturated.
[0031]
By the way, as an example for achieving uniform characteristics, the claimsThe invention of claim 4 relates to the above-mentioned claim 1 to claim 3In the image processing apparatus, the characteristic uniforming unit determines the characteristic from the predetermined position in the distribution, determines an offset amount corresponding to the deviation between the components so as to make the characteristic uniform, and determines each component value. As a configuration to correct
[0032]
Claim configured as above4In the present invention, including the process of obtaining the distribution for each color component, several positions such as the upper end and the lower end, the average value, the median, and the mode value can be obtained. It is possible to find Then, since it is considered that the deviation between these components is a characteristic that can be one factor of the specific color deviation, the offset amount corresponding to the same deviation is determined to correct each component value.
[0033]
A claim as an example of a specific positionThe invention according to claim 5 relates to the fourth aspect.In the image processing apparatus, the characteristic uniformizing unit may be configured to determine an end position of the distribution range as the characteristic of the distribution.
[0034]
Claim configured as above5In the present invention, the upper end and the lower end positions, which are the end positions of the range of distribution, are regarded as the characteristics of the distribution, and the deviation between the components is determined.
[0035]
If you look at the individual distributions, it will be easy to determine uniformly if the extremely diverse distributions are also at the upper end position or the lower end position, and it will be easy to acquire the characteristics.
[0036]
Also, as another example, claimsAccording to the sixth aspect of the present invention, theIn the image processing apparatus according to any one of the above, the characteristic uniformizing unit is configured to determine the approximate center position of the distribution as the characteristic of the distribution.
[0037]
Claim configured as above6In the present invention, a position such as an average value, a median or a mode value, which is a substantially central position of the distribution, is regarded as a characteristic of the distribution, and the deviation between the components is determined.
[0038]
Similarly, extremely diverse distributions can be easily determined uniformly if they are at positions such as an average value, a median, or a mode value, which is a substantially central position, and acquisition of characteristics becomes easy.
[0039]
In general images, although there is a difference in the absolute value of the component for each component, there is often no significant change in the spread of the distribution. Therefore, it can be said that it is natural to equalize the spread amount in each component. For this reason, claimsThe invention according to claim 7 relates to the above-mentioned claim 1 to claim 6In the image processing apparatus according to any one of the above, the property uniformizing unit is configured to substantially uniform the spread of the distribution for each color component.
[0040]
Claim configured as above7In the present invention, the spread of the distribution for each component value is determined, and conversion processing is performed so as to make the distribution substantially uniform.
[0041]
The idea of the spread of distribution can be grasped in various ways, and it is not necessary to be limited to a specific one, and as an example, claimsThe invention according to claim 8 relates to the seventh aspect.In the image processing apparatus, the characteristic uniformizing unit is configured to expand both ends of the spread of the distribution within an effective gradation range.
[0042]
Claim configured as above8In the present invention, since the distribution has a mountain shape and the width of the distribution can be known at both ends of the foot, the width is converted so as to be expanded within an effective gradation range. For example, if the width of the distribution is only a part of the effective gradation width, it can be said that the image is weak in contrast, but in such a case, the distribution extends over the entire width of the effective gradation range. To expand. If the distribution expands so as to spread over the full width of the effective gradation range for each color component, the characteristic of the width of the distribution is made uniform. Note that the width of the distribution in this case may not only be the width of the actual distribution, but may be such that the error range or the like is omitted by cutting both end portions by a predetermined amount or the like.
[0043]
Also, as another example, claimsThe invention according to claim 9 relates to the seventh aspect.In the image processing apparatus, the characteristic uniformizing unit assigns a small number of gradations to the small distribution density range while giving a large number of gradations to the large distribution density range based on the spread amount of the distribution. As.
[0044]
Claim configured as above9In the present invention, the characteristic equalization means determines the spread amount of the distribution for each component, and based on the determined distribution, the distribution density is small while giving many gradation numbers to the large range of the distribution density for each component. Assign a small number of gradations to the range. The concept of the spread amount corresponds to the spread of the distribution as well as the width of the distribution, and in the case of mathematical expression, it may be a standard deviation, a dispersion or a kurtosis. When the characteristics as the spread amount are made to approach each other so as to match each component, if conversion is performed so that the large part of the distribution density is spread within the allowable range, the spread amount mismatched for each component Since it becomes possible to view and the distribution is distributed as much as possible using the effective range without concentrating on the partial range, the contrast is generally enhanced. Of course, the spread amount may be other than the standard deviation, the variance, or the kurtosis, and also in these cases, the calculation in the strict meaning is not required, and the calculation may be simplified.
[0045]
Furthermore, claimsThe invention according to 10 relates to the above-mentioned claims 1 to 9In the image processing apparatus according to any one of the above, the characteristic equalization means is configured to equalize the brightness based on the distribution of each component.
[0046]
Claim configured as above10In the invention according to the above, since the brightness may be non-uniform in the distribution of each component and thus it may appear as a color shift, the characteristic uniforming means equalizes the brightness based on the distribution of each component. Let The brightness in this case represents the overall position of the distribution, and for example, the distribution is considered to be bright if it is located on the overall bright side, and it is considered dark if it is located on the overall dark side.
[0047]
Of course, the method of determining the overall brightness can be changed as appropriate, and various methods can be adopted as a method of correcting whether to brighten or darken. For example, claimThe invention according to claim 11 corresponds to the above claim 10.In the image processing apparatus, the characteristic uniformizing unit may be configured to determine the contrast of the image by comparing the substantially central position of the distribution with a predetermined gradation.
[0048]
In the invention as set forth above, the substantially central position of the distribution is compared with the predetermined gradation of the effective gradation range, and the contrast of the image is judged based on the comparison result.
[0049]
For example, although the median obtained when obtaining the distribution is provided with the condition as the central position of the distribution, whether the median as a whole is brighter depending on whether the median is larger or smaller than the median in all gradation numbers It is possible to determine whether it is dark.
[0050]
Also, claimsThe invention according to claim 12 corresponds to the above claim 10 or claim 11In the image processing apparatus, the characteristic uniformizing unit is configured to make the brightness and darkness of the image uniform by the γ correction for each component based on the judgment result of the brightness and darkness of the image.
[0051]
Claim configured as above12In the present invention, after the brightness and darkness of the image is determined by various methods, the brightness and darkness of the image are equalized by γ correction for each component based on the determination result. For example, if the image looks dark, the entire image is corrected brightly by γ correction with γ <1, and the same median approaches the median value of all gradation numbers, and if it appears bright, γ> 1. With the γ correction, the overall correction will be dark, and the same median will approach the median of the total number of gradations.
[0052]
The method of determining the deviation of color balance based on the comparison of the distribution of each color component in this manner is not necessarily limited to a device having a substantial effect, and one example is the claimThe invention according to 13 relates to predetermined conversion processing by inputting each component value of image data in which each pixel in an image consisting of a plurality of pixels is represented by gradation table color data consisting of substantially equal color components. When performing conversion to correct the color balance based on the relationship between input and output by performing the output, the target pixel is moved with each pixel as the processing target, and the gradation table for each color component of the target pixel is moved. The statistic based on the distribution of color data is determined, and the deviation of the characteristic of the distribution among the color components is determined based on the same statistic for each color component, and the characteristic of the determined distributionIt is configured to make the characteristics of each color component uniform based on the deviation.
[0053]
That is, the present invention is not necessarily limited to a device having substance, and there is no difference in that the method is effective.
[0054]
By the way, such an image processing apparatus may exist alone or may be used in a state of being incorporated in a certain device. The idea of the invention includes various aspects. Also, it can be changed as appropriate, such as software or hardware.
[0055]
As an example,A printer driver that converts each pixel in an image composed of a plurality of pixels into image data corresponding to printing ink based on image data represented by gradation table color data composed of roughly equal color components, and causes a predetermined color printer to print Even in this case, while moving the pixel of interest with each pixel as the processing target, a statistic based on the distribution of gradation table color data is obtained for each color component of the pixel of interest, and each color is calculated based on the same statistic for each color component. Deviation of distribution characteristics between components is determined, and uniformity of characteristics among color components is uniformed based on the determined deviation of distribution characteristics.Can be configured.
[0056]
In such a case, the printer driver converts the input image data in correspondence to the printing ink. At this time, the distribution is obtained for each color component in the image data, and the characteristic found from the distribution is each color component Convert and print to be uniform between the two. That is, by comparing the distribution obtained for each color component and performing correction such that the distribution matches, the color balance as a whole is adjusted, and good color development is achieved in each color of each pixel.
[0057]
In the case of software of an image processing apparatus as an embodiment of the concept of the invention, it naturally can be said to be present and used also on a recording medium on which such software is recorded.
[0058]
As an example, claimsThe invention according to 14 is a medium on which a program for performing image processing by a computer is recorded, and the same applies to image data in which each pixel in an image composed of a plurality of pixels is represented by gradation table color data composed of roughly equal color components. A target pixel is moved with each pixel as the processing target in performing conversion for correcting the color balance based on the relationship between the input and the output by inputting each component value of the image data, performing predetermined conversion processing, and outputting it. While doing this, a statistic based on the distribution of tone color data is determined for each color component of the same pixel of interest, and the deviation of the distribution characteristic among the color components is determined based on the same statistic for each color component Of the characteristics of the distributed distributionIt is configured to make the characteristics of each color component uniform based on the deviation.
[0059]
Of course, the recording medium may be a magnetic recording medium or a magneto-optical recording medium, and any recording medium developed in the future can be considered in the same way. In addition, the replication stages of primary and secondary copies are the same without any question..
[0060]
Furthermore, even when a part is software and a part is realized by hardware, there is no difference in the concept of the invention, and a part is stored on a recording medium, and as needed, as appropriate. It may be in a form to be read. Furthermore, it is needless to say that the present invention is also applicable to image processing apparatuses such as color facsimile machines, color copiers, color scanners, digital still cameras, digital video cameras and the like.
[0061]
【Effect of the invention】
As described above, according to the present invention, the concept of making the distribution uniform for each component can automate the judgment of color deviation that is fundamentally difficult to judge, and image processing that can adjust the color balance more easily. An apparatus can be provided.
[0062]
Also,If the degree of similarity of distribution is small, no attempt is made to make the characteristics uniform, and if the color balance is naturally deviated, the characteristics will be made uniform to prevent an unnatural image. The gradation range is divided into a plurality of regions to obtain the distribution to facilitate the comparison, or the similarity is determined based on the inner product of vectors having the distribution for each color component as an element, It is also possible to make judgment easier, and to make it difficult to generate discontinuities in the image processing result by continuously changing the effective value for achieving uniform characteristics.it can.
[0063]
Furthermore, claims3According to the present invention, since pixels which are inherently approximate to gradation color data are picked up and used for judgment of the characteristics, it is effective for finding the dispersion of the characteristics.
[0064]
Furthermore, claims4According to the present invention, the offset amount between the components can be determined relatively easily because the positional deviation of the distribution is used.
[0065]
Furthermore, claims5According to the present invention, the end of the distribution can be easily determined uniformly, which makes the determination easy.
[0066]
Furthermore, claims6According to the invention, since the deviation at the position where the distribution density is high is used, the error in the correction of the whole deviation is small.
[0067]
Furthermore, claims7According to the present invention, in order to make the spread of the distribution uniform, colors can be efficiently developed when effective gradation numbers are not effectively used for each color component, and sharpening can be used it can.
[0068]
Furthermore, claims8According to the present invention, since the distribution is expanded using both ends of the spread of the distribution that is easy to determine, the configuration for the determination becomes easy.
[0069]
Furthermore, claims9According to the present invention, since the spread amount by the mathematical method or the like is used, more accurate judgment can be made.
[0070]
Furthermore, claims10According to the present invention, it is possible to eliminate color misregistration that causes only one of the components to protrude by making the brightness of each component uniform.
[0071]
Furthermore, claims11According to the present invention, it is easy to determine the brightness with the approximate median value of the distribution.
[0072]
Furthermore, claims12According to the invention, the brightness is changed by changing the brightness by the γ correction, and the γ correction is widely used, so the configuration becomes easy.
[0073]
Furthermore, claims13According to the invention, an image processing method is capable of automating the judgment of the color shift which is fundamentally difficult to judge by the concept of making the distribution of each component uniform and to adjust the color balance more easily. Can be provided.
[0074]
And claim 14According to the present invention, it is possible to provide a medium storing a program for performing such image processing.
[0075]
Besides, regardless of the method of homogenizing the characteristics, when the similarity of the distribution is small, the characteristics of the color balance are naturally deviated as a matter of course by making it difficult to achieve the uniformity of the characteristics. It is also possible to prevent the image from becoming unnaturalit can.
[0076]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described based on the drawings.
[0077]
FIG. 1 is a block diagram showing an image processing system according to an embodiment of the present invention, and FIG. 2 is a block diagram showing a specific hardware configuration example.
[0078]
In the figure, the image input device 10 outputs image data to the image processing device 20 by capturing an image or the like, and the image processing device 20 performs image processing such as equalizing characteristics to the image output device 30. The image output device 30 outputs an image with enhanced contrast.
[0079]
Here, a specific example of the image input device 10 corresponds to the scanner 11, the digital still camera 12 or the video camera 14, and a specific example of the image processing device 20 corresponds to a computer system including the computer 21 and the hard disk 22 etc. A specific example of the output device 30 corresponds to the printer 31, the display 32, and the like. Of course, the present invention is also applicable to color copiers and color facsimile machines.
[0080]
In the present image processing system, since an image with poor color reproducibility represented by color shift is to be corrected, it is image data obtained by capturing a picture with the scanner 11 as the image input device 10, The image data captured by the digital still camera 12 or the moving image captured by the video camera 14 is to be processed, and is input to a computer system as the image processing apparatus 20. The calculation speed of the input image of the video camera 14 may not be in time. In such a case, the first condition setting requiring operation time is performed for each shooting scene, and it is possible to cope with it by performing only image conversion of each frame under the same condition setting during shooting.
[0081]
The image processing apparatus 20 includes at least frequency distribution detection means for determining frequency distribution for each color component, similarity condition determination means for determining similarity of frequency distribution for each detected color component, and each color component from frequency distribution Offset amount correction means that performs correction to determine and equalize deviations of the image, contrast correction means that performs correction to determine and equalize differences in contrast intensity for each color component from the frequency distribution, and color components from the frequency distribution A brightness correction unit is configured to perform correction to determine and equalize the difference in brightness for each. Of course, the image processing apparatus 20 may be other color conversion means for correcting differences in color depending on the model, or may be resolution conversion means for converting the resolution corresponding to each model. . In this example, the computer 21 executes a program of each image processing stored in the internal ROM or the hard disk 22 while using a RAM or the like. Also, when recording these programs, as shown in FIG. 29, portable media such as the flexible disk 41 and the CD-ROM 42 as well as the state of being installed on the hard disk 43 as well as portable media such as It is also possible to use an IC card having the ROM 44 and the RAM 45, or use the communication line 46a as a medium via the modem 46b or the like. In this case, a file server 46c is connected to the end of the communication line 46a, and the file server 46c provides predetermined software.
[0082]
The execution result of this image processing program is obtained as sharp image data with corrected color reproducibility as described later, and printing is performed by the printer 31 which is the image output device 30 based on the obtained image data, or the same. It is displayed on the display 32 which is the image output device 30. More specifically, this image data is gradation data of “256” gradations for each of RGB (green, blue, red), and the image has a height direction and a width direction (width). ) Are configured as dot matrix data arranged in a grid pattern.
[0083]
In this embodiment, a computer system is incorporated between image input / output devices to perform image processing, but such a computer system is not necessarily required, and a digital still camera as shown in FIG. A system may be used in which an image processing apparatus in the sense of correcting color reproducibility and the like is incorporated in 12a, and the converted image data is displayed on the display 32a or printed on the printer 31a. Further, as shown in FIG. 4, in the printer 31b which inputs and prints image data without passing through a computer system, the image data inputted through the scanner 11b, the digital still camera 12b or the modem 13b, etc. is automatically made. It is also possible to configure to correct the color reproducibility.
[0084]
Among the image processing executed by the computer 21, the processing corresponding to the frequency distribution detection means and the similarity determination means is shown in FIG. 5, and the offset amount correction performed when the similarity for each color component is not small. A process corresponding to the means, the contrast correction means, and the brightness correction means is shown in FIG. It should be noted that the similarity determination means also constitutes control means for controlling the execution of the subsequent processing based on the similarity in a broad sense.
[0085]
FIG. 5 mainly corresponds to the process of detecting the frequency distribution for each color component. First, the pixels to be distributed will be described.
[0086]
Although the frequency distribution may be determined for all the pixels, it is not necessary to necessarily determine the frequency distribution for all the pixels because the purpose is to determine the tendency of the characteristics. Therefore, it is possible to perform thinning to such an extent that it falls within a certain error range. In the present embodiment, as shown in step S102 of FIG. 5, thinning processing is performed to thin the target pixels. According to the statistical error, the error with respect to the number of samples N can be approximately expressed as 1 / (N ** (1/2)). However, ** represents a power. Therefore, in order to perform processing with an error of about 1%, N = 10000.
[0087]
As shown in FIG. 7, in the case of a bitmap image, it is configured as a two-dimensional dot matrix consisting of predetermined dots in the vertical direction and predetermined dots in the horizontal direction, and this bitmap screen has (width) × (height) The sampling period ratio is given by
ratio = min (width, height) / A + 1 ... (1)
I assume. This min (width, height) is either the width or the height, whichever is smaller, and A is a constant. Further, the sampling period ratio mentioned here indicates how many pixels are to be sampled, and the circled pixels in FIG. 8 indicate the case where the sampling period ratio = 2. That is, sampling of one pixel is performed every two pixels in the vertical direction and horizontal direction, and sampling is performed every other pixel. The number of sampling pixels in one line when A = 200 is as shown in FIG.
[0088]
As apparent from the figure, it is understood that the number of samples is at least 100 at least when there is a width of 200 pixels or more, except in the case of sampling period ratio = 1 where sampling is not to be performed. Therefore, in the case of 200 pixels or more in the vertical direction and the horizontal direction, (100 pixels) × (100 pixels) = (10000 pixels) is secured, and the error can be made 1% or less.
[0089]
The reason why min (width, height) is used as a reference here is as follows. For example, assuming that width >> height as in the bitmap image shown in FIG. 10A, if the sampling period ratio has been determined with the longer width, as shown in FIG. However, it may happen that only two lines at the top and bottom are extracted in the vertical direction. However, if the sampling cycle ratio is determined based on the smaller one as min (width, height), as shown in FIG. 6C, thinning is performed to include the middle part even in the smaller vertical direction. Will be able to
[0090]
Of course, with regard to the pixels sampled in this manner, as shown in FIG. 11, the component variables (CNT_R, CNT_G, CNT_B) prepared corresponding to all the gradations of RGB are counted for each component value. And determine the frequency distribution.
[0091]
In this example, the frequency distribution is determined by thinning out the pixels in the vertical direction and the horizontal direction with an accurate sampling period. This is suitable for processing while thinning out sequentially input pixels. However, when all the pixels are input, the coordinates may be randomly designated in the vertical direction or the horizontal direction to select the pixels. In this case, when the necessary minimum number of pixels such as 10000 pixels is determined, the process of extracting randomly is repeated until it reaches 10000 pixels, and the extraction may be stopped when it becomes 10000 pixels.
[0092]
Moreover, it is also possible to perform thinning which specified an object pixel to simple thinning processing which carries out thinning without specifying such an object pixel. That is, if the difference between the RGB components of each pixel is small, it is close to gray, and it can be said that it is easy to judge the color characteristics of the input device if only gray pixels are extracted and frequency distributions are compared. is there. FIG. 12 shows a process of extracting and thinning out such gray pixels. In step S302, the largest component is specified among the components in the target pixel, and in step S304, the smallest component is specified. Then, in step S306, the maximum component difference, which is the difference between the maximum value and the minimum value, is calculated, and in step S308 it is determined whether the maximum component difference falls within a range smaller than a predetermined threshold. Do. As an example, it is assumed that the maximum component difference is within “52” gradations as long as the pixel is close to gray. Then, the frequency distribution is counted in step S310 for the pixels within such a gradation. On the other hand, if it exceeds such a gradation, it will not be used for counting noting that it is not gray. Thereafter, the target pixel is moved to the next pixel in step S312, and the above-described processing is repeated until it is determined to be the final pixel in step S314.
[0093]
Although the frequency distribution is determined for the selected pixel by thinning out in this way, it may not always be appropriate to correct the image using this frequency distribution, so three items will be checked below.
[0094]
The first is the case where the image is a binary image such as a black and white image. If it is a binary image including a black and white image, the concept of correction of color reproducibility is inappropriate. Assuming that there is a black and white image as shown in FIG. 13, the frequency distribution for each color component with respect to this image is concentrated at both ends within the gradation number allocation range as shown in FIG. Also, it basically concentrates on gradation "0" and gradation "255".
[0095]
Therefore, when performing the black and white check in step S104, it can be determined whether the sum of the number of pixels of gradation "0" and gradation "255" matches the number of pixels selected by thinning out for each color component. . Then, in the case of a black-and-white image, non-processing is executed in step S106 in order to interrupt the processing without executing the following processing. In the present embodiment, since the process of the former stage for obtaining the frequency distribution and the process of the latter stage for correcting the image data are roughly divided, in this non-processing, a flag is set so as not to execute the luminance conversion process of the latter stage. The distribution extraction process has been completed.
[0096]
The binary data is not limited to black and white, but may be colored binary data. Also in such a case, the process for correcting the color reproducibility is not necessary in the same manner, and if the distribution state is checked and the distribution becomes two colors, the processing may be interrupted as binary data. For example, in the case of two colors of a certain intermediate color and black, the distribution is concentrated to two gradations even in the frequency distribution for each color component. On the other hand, in the case of two colors of blue and black, the distribution is concentrated in two gradations only in the frequency distribution of the blue component, and the distribution is concentrated only in the gradation "0" in the red component and the green component. That is, in the case of binary data, the concentration of the distribution is less than or equal to two gradations, and therefore, it is a binary image by scanning array variables and counting how many gradations other than “0” are present. It can be determined whether or not.
[0097]
The second is to consider whether the image is a business graph or a natural image. It is needless to say that color reproduction needs to be corrected in natural pictures, but it is natural that there is a bias in the colors originally used for things like business graphs and pictures, and such It is unreasonable to perform processing to make the characteristics of each color component uniform from the frequency distribution of the image. Therefore, in step S108, it is checked whether the image is a natural image.
[0098]
In natural paintings, the number of colors is extremely high, including shadows, but in some types of paintings such as business graphs and drawings, the number of colors is often limited. Therefore, if the number of colors is small, it can be determined that the image is not a natural image. When each component of RGB has 256 gradations, it can represent 16.7 million colors, and if it is intended to accurately determine the number of colors, the color is used to determine which one of the 16.7 million colors is used. It is not practical because it is necessary to prepare as many array variables as possible. On the other hand, since the frequency distribution has already been obtained, it can be determined whether it is the number of colors of the natural image by judging how many gradations are effectively used for each color component.
[0099]
FIGS. 15A to 15C show an example of the frequency distribution in the case of the business graph, and FIGS. 15D to 15F show an example of the frequency distribution in the case of a natural picture. As is clear from this example, in non-natural images, the frequency distribution has a linear spectrum. In the processing in the computer 21, for each color component, the number of gradations whose frequency is not "0" is counted over all gradations, and they are summed up. In the case of a natural image, it is considered to be dispersed almost uniformly over all gradations, and the count value is often “768 (= 256 × 3)”, and in the case of a business graph, each color component Even if the "20" gradation is used, the count value is only about "60 (= 20.times.3)". Therefore, if "200" is set as the threshold value for determining whether the image is a natural image or not, if it is "200" or less, it may be determined as a non-natural image. If it is determined that the image is not natural, non-processing is executed in step S106. Of course, the threshold value "200" can be changed as appropriate.
[0100]
Moreover, it is also possible to judge whether the frequency distribution is a linear spectrum or not by the adjacent ratio of gradation values whose frequency is not "0". That is, it is determined whether or not there is a distribution between adjacent tone values that are tone values whose frequency is not "0". If at least one of two adjacent tone values is adjacent, nothing is done, and counting is performed if they are not adjacent. As a result, the number of tone values that are not "0" and the count value It should be judged by the ratio. For example, if the number of gradation values that are not "0" is "64" and the number of non-adjacent ones is "64", it is understood that the distribution is in a linear spectrum.
[0101]
Furthermore, when the image processing program is being executed via the operating system, it is also possible to judge by the extension of the image file. Among bit map files, file compression is performed particularly on a photographic image, and an implicit extension is often used to indicate the compression method. For example, if the extension is "JPG", it is known that the file is compressed in the JPEG format.
Since the operating system manages the file name, if the printer driver or the like issues an inquiry to the operating system, the extension of the file will be answered. If there is, the following process may be executed. In addition, if it is an extension specific to a business graph such as “XLS”, it can be determined as a non-natural image, and non-processing may be selected as described above.
[0102]
The third consideration is whether or not there is a frame around the image as shown in FIG. If such a frame is white or black, the frequency distribution appears in the form of a linear spectrum at both ends within the gradation number allocation range as shown in FIG. It also appears as a smooth frequency distribution inside other.
[0103]
Of course, since it is appropriate not to put the frame part in the frequency distribution, in the check of the frame part in step S108, the sum of the number of pixels of the gradation "0" and the gradation "255" is sufficiently large. It is determined whether or not the selected number of pixels matches, and if affirmative, it is determined that there is a frame portion, and frame portion processing is performed in step S112. In this frame processing, in order to ignore the frame, the number of pixels of gradation "0" and gradation "255" in the frequency distribution is set to "0". Thus, the following processing can be handled in the same manner as one without a frame.
[0104]
In this example, a white or black frame is targeted, but it is also conceivable that there is a frame of a specific color. In such a case, a protruding linear spectrum appears in the original smooth curve drawn by the frequency distribution. Therefore, linear spectrum-like ones where a large difference occurs between adjacent luminance values may be considered as a frame part and not be a target of frequency distribution. In this case, since it is possible that the color is used outside the frame portion, an average may be calculated and assigned to the frequencies of the adjacent tone values.
[0105]
After the above consideration, if it is not non-processing, both ends of the frequency distribution are obtained in step S114. The frequency distribution in the natural pattern often appears roughly in the shape of a mountain as shown in FIG. Of course, the position and shape are various. The base of such frequency distribution changes as it approaches “0” without limit in statistical terms. Therefore, in order to specify a certain frequency distribution, it becomes important to specify both ends of the mountain, but if we impose the condition that the frequency becomes "0", then any distribution characteristic will It may be a match.
[0106]
Therefore, in the distribution range, a portion that is inward by a certain distribution ratio from the side with the largest luminance and the side with the smallest luminance is defined as both ends of the distribution. In the present embodiment, as shown in FIG. 18, this distribution ratio is set to 0.5%. Of course, this ratio can be changed as appropriate. As described above, by cutting the upper end and the lower end by a certain distribution ratio, it is possible to ignore white spots and black spots caused due to noise and the like. On the other hand, if such processing is not performed, even if there is a white point or a black point even at one point, it becomes both ends of the frequency distribution, so in many cases the lowest end is the gradation "0" and the highest end Becomes gray scale "255". However, such a problem is eliminated by setting the end portion to a portion that is 0.5% inward from the both end portions.
[0107]
In the actual processing, 0.5% of the number of pixels to be processed (the total number of pixels selected in the thinning process or the total number of pixels corresponding to the frame portion deleted) is calculated. The respective frequencies are accumulated while moving inward from the point in order, and a tone value having a value of 0.5% is obtained. The upper ends of the RGB color components are called Rmax, Gmax, and Bmax, and the lower ends thereof are called Rmin, Gmin, and Bmin.
[0108]
As described above, it may be natural that the frequency distribution of each color component is uneven. And, in such a case, correction of color reproducibility should not be made either. It is determined that if the frequency distribution of each color component is similar to a certain degree, the frequency distribution should be uniform, and if the frequency distributions are not similar, it should not be uniform. it can.
[0109]
Therefore, in the present embodiment, the similarity of the frequency distribution for each color component is checked in step S116. Now, assuming that the frequency distribution for each color component appears as shown in FIG. 19, the whole gradation range is divided into four regions ([0 to 63], [64 to 127], [128 to 192], [193 ̃255]), and consider feature vectors whose elements are frequencies belonging to each region. Taking the red component as an example, when the frequency in each region is represented by r64, r128, r192, r255 and the total number of effective pixels is r_pixel, the feature vector of the red component can be represented by Equation 1.
[0110]
[Equation 1]

[0111]
The same processing is performed on the green and blue components, and then the inner product of feature vectors among color components is determined. That is, the inner product corr_rg of the feature vector between the red component and the green component is determined as expressed by Equation 2, and the inner product corr_gb of the feature vector between the green component and the blue component is calculated as expressed by Equation 3. The inner product corr_br of the feature vector between the blue component and the red component is determined as expressed by Equation 4.
[0112]
[Equation 2]

[0113]
[Equation 3]

[0114]
[Equation 4]

[0115]
It can be said that the inner product of the vectors represents the similarity of both vectors, and its value is “0” to “1”. Therefore, assuming that "0.7" is defined as the threshold value CORR, it is determined that the similarity is low if any one of the inner products corr_rg, corr_gb and corr_br is smaller than the threshold value CORR. Then, the non-processing of step S106 is performed.
[0116]
In the present embodiment, the process based on the inner product of the feature vectors constitutes the similarity condition judging means, and the method is established and the judgment is easy if the inner product is calculated based on the feature vectors. However, of course, it is not limited to this example. For example, although the whole gradation range is divided into four regions, it is free to use more than these, and approximation using statistical methods such as both end positions of the frequency distribution, standard deviation, kurtosis and the like It is also possible to determine the degree.
[0117]
Here, a specific method for obtaining the degree of approximation using such a statistical method will be described. An example of the statistical method is one using a representative value of distribution. Now, as a variable, the absolute value of the difference between the mean value, the median value, and the standard deviation (variance) is determined among the red component and the green component, the green component and the blue component, and the blue component and the red component. Then, when the absolute value of the difference between the red component and the green component for the average value and the standard deviation is Ave_rg, Std_rg, as an evaluation function between the red component and the green component
Set h (rg) = (1−Ave_rg / 255) × (1−Std_rg / 255). Also, similarly, as an evaluation function between the green component and the blue component,
As h (gb) = (1-Ave_gb / 255) × (1-Std_gb / 255), and as an evaluation function between the blue component and the red component,
Set h (br) = (1-Ave_br / 255) × (1-Std_br / 255). If the distributions are similar, the mean value, median value, and standard deviation (variance) will be almost the same, and the difference between each variable will be almost "0", so the value of the evaluation function h will be close to "1" . When the distributions are different, the difference increases and the value of the evaluation function h also decreases. Therefore, it is possible to determine whether or not to make a correction by comparing the evaluation function with the threshold value obtained by experiment. Of course, in this case, it is also possible to use the median instead of the average, and the specific example of the evaluation function is not limited to this.
[0118]
If a certain degree of similarity is found in the frequency distribution obtained for each color component by the above processing, it can be determined that the characteristics of each component can be found from each frequency distribution, and uniformity among the components is determined based on such characteristics. Plan. In addition, as a result of various judgments mentioned above, non-processing may be performed in step S106 and the flag may be set. In such a case, since the same flag is referred to in step S202, the present processing ends without performing the following equalization processing.
[0119]
In the process of equalizing the characteristics, offset calculation and correction are performed first. This corresponds to correction of color misregistration in a so-called narrow sense, and an offset for achieving uniform characteristics forms an effective value in this embodiment. Normally, as shown in FIG. 20, the color component of the subject and each component of RGB must have a direct proportional relationship, but the conversion characteristic is shifted for each component due to the characteristic of the image pickup element etc. Sometimes. Conventionally, such deviation can not be determined from a normal image. However, if the frequency distribution for each color component is taken as roughly similar, since it can be judged that they should be basically identical, it is possible to detect the offset amount which is the deviation for each component.
[0120]
In the present embodiment, the offset amount is obtained in step S204, and a table used to correct the color misregistration in consideration of the offset amount is created in step S206.
[0121]
When using RGB gradation data (Rp, Gp, Bp), the entire luminance yp is used in television etc.
yp = 0.30 Rp + 0.59 Gp + 0.11 Bp (2)
As you are seeking. That is, the green component most affects the brightness, and there is an advantage that correcting the deviation of the other color component with respect to the green component does not change the entire image.
[0122]
On the other hand, in order to obtain the deviation of the frequency distribution for each color component, it is desirable to consider the characteristic part of this frequency distribution. Therefore, in the present embodiment, the upper end (Rmax, Gmax, Bmax) of the frequency distribution subjected to the end processing described above and the median (Rmed, Gmed, Bmed) on the frequency distribution are used. Both end positions are effective in the sense of judging the distribution. However, the lower end position is originally omitted because it is a range in which the influence of the deviation is originally difficult to understand. As a result, it is possible to make a correction that emphasizes only the deviation obtained in the range where the influence of the deviation is large. The median at the center of the distribution can indicate the position of the peak of the frequency distribution even if there are extreme pixels. In this case, such a mountain portion is a portion that greatly affects the image of the image, and thus is effective in grasping the characteristics.
[0123]
Thus obtained deviations dRmax, dBmax, dRmed, and dBmed of the other color components from the upper end (Rmax, Gmax, Bmax) and the median (Rmed, Gmed, Bmed) for blue-green-red to the following formulas Ask based on
[0124]
dRmax = Gmax-Rmax (3)
dBmax = Gmax−Bmax (4)
dRmed = Gmed-Rmed (5)
dBmed = Gmed-Bmed (6)
Then, with reference to these, the red component offset dR and the blue component offset dB are determined as in the following equation.
[0125]
dR = (dRmax + dRmed) / 2 (7)
dB = (dBmax + dBmed) / 4 (8)
However, −12 <dR, dB <12. This limitation is to prevent the frequency distribution from being largely corrected by this one example only when the frequency distribution alone can not correct the perfect color reproducibility. Of course, the appropriate value may be set for this range based on experimental experience. Further, the difference in the denominator corresponds to (2) the difference in the influence based on the color, and can be appropriately changed based on an experiment or the like.
[0126]
Since these are only offset amounts, the actual statistical values are corrected based on the following equation to obtain new upper ends Rmax2 and Bmax2, medians Rmed2 and Bmed2, and lower ends Rmin2 and Bmin2.
[0127]
Rmax2 = Rmax + dR (9)
Rmed2 = Rmed + dR (10)
Rmin2 = Rmin + dR (11)
Bmax2 = Bmax + dB (12)
Bmed2 = Bmed + dB (13)
Bmin2 = Bmin + dB (14)
It is apparent that this correspondence relationship is only an offset amount of the red component and the blue component with respect to the green component, and does not change with the gradation value. Therefore, it is sufficient to uniformly add the offset amount for image data in each actual pixel.
[0128]
However, as described later, in the present embodiment, correction of image data based on other factors is also performed, and individual correction is disadvantageous because it requires calculation time. Therefore, in the present embodiment, RGB gradation data (R2, G2, B2) after conversion with respect to RGB gradation data (R1, G1, B1) before conversion in step S206 in order to improve calculation efficiency. The table representing the correspondence relationship is formed, and the image data is actually corrected only once at the end.
[0129]
On the other hand, in the embodiment described above, the similarity is determined based on the inner product of the vectors, and the similarity is compared with the threshold value CORR (“0.7”) to determine whether to equalize the characteristics. There is. Therefore, in the case where the inner product of vectors and the threshold value substantially match, different judgment results may be made under the influence of bits added to the periphery even in the same image.
[0130]
Of course, the offset amount as described above is added when it is determined that the characteristic equalization is performed, and the offset amount as described above is not added when it is determined that the characteristic equalization is not performed. There is a large difference across the threshold.
[0131]
It is effective to use a continuously changing window function as a method to prevent such a large change in the result.
[0132]
x = min (corr_rg, corr_gb, corr_br) ... (141)
The window function f (x) is
When x <0.5 f (x) = 0
When 0.5 ≦ x ≦ 0.7 f (x) = 5 · x−2.5
When 0.7 <x f (x) = 1
The threshold value CORR is set to “0.5”. The change state of the window function f (x) is shown in FIG. 30, and becomes constant “0” at x <0.5, and changes from “0” to “1.0” at 0.5 ≦ x ≦ 0.7. And increases linearly, and becomes constant "1.0" at 0.7 <x. Then, the fact that the offset dR for the red component and the offset dB for the blue component in the equations (7) and (8) is determined is converted to the product of f (x). That is,
dR = f (x) (dRmax + dRmed) / 2 (7) '
dB = f (x) · (dB max + dBmed) / 4 (8) '
I assume. In the flowcharts shown in FIG. 5 and FIG. 6, if the similarity does not exceed the threshold, it is regarded as non-processing, but in the case of multiplying the offset by the window function in this way, the flowchart shown in FIG. . That is, non-processing is not performed based on the degree of similarity, the calculation of the offset amount using the window function is executed in step S205, and the calculated offset amount is used. Of course, as described above, since the window function f (x) rapidly approaches “0” at x <0.7, the image which has been in the vicinity of the threshold is not greatly changed by the processing, and the similarity The offset is also almost "0" for those with a low. Further, the equations (9) to (14) are also calculated based on the offset amount dR, dB calculated in this manner.
[0133]
It is obvious that the window function f (x) is not limited to the above-mentioned function. In the example described above, the value of the window function is changed based on the minimum value of the correlation coefficient, but it is not necessary to be limited to the minimum value. For example,
f (corr rg, corr gb, corr br) ... (142). And, more generally,
f (statistics)
As well. In this case, the statistic corresponds to the minimum value, the maximum value, the median, the standard deviation, and the like.
[0134]
On the other hand, the window function functions to validate the calculated value in one area so as to open and close the window and to invalidate the calculated value in another area. Now, if the color cast is classified according to the cause, it is classified into the case based on the hardware performance of the image input device 10 mentioned above, the intentional color cast due to sunset etc. and the color cast due to special lighting using tungsten lamp etc. It can be classified. And, as mentioned above, there is no need to unify the characteristics for intentional color cast due to sunset etc., but in the case of color cast due to special lighting using tungsten lamp etc. It is also meaningful to plan.
[0135]
The intentional color cast due to sunset and the like and the color cast due to special lighting using a tungsten lamp etc. can be judged from the above-mentioned correlation coefficient. That is, photographs with extremely low correlation coefficients are often attributed to such special illumination, and the window function is deformed as shown in FIG.
[0136]
That is, the region of x <0.5 is deformed,
When x <0.1 f (x) = 1.0
When 0.1 ≦ x <0.3 f (x) = − 5 · x + 1.5
When 0.3 ≦ x0.5 f (x) = 0
And Thus, the window function f '(x) rises linearly again when x is less than or equal to "0.3", and the window function f' (x) is constant in the region where x is less than or equal to "0.1". It will be 1.0. Of course, when the window function becomes "1.0", the large offset amount calculated based on the distribution state of each component is applied as it is, and the influence of the illumination is canceled to make the characteristics uniform.
[0137]
Note that the window function does not have to change linearly in the transition region, and may be curved such as monotonously increasing or monotonously decreasing.
[0138]
In addition, as shown in FIG. 5, the similarity is checked in step S116, and when the similarity is low, non-processing is executed in step S106 to set a flag, or as shown in FIG. By calculating the offset amount using the window function in step S205 without executing the process, it is possible to prevent the equalization of the characteristics substantially when the similarity is low, so these software processing It can be said that the correction control means is configured by hardware or the like that realizes this.
[0139]
On the other hand, when there is a difference in how the frequency distribution spreads, it is effective to make this uniform. In the present embodiment, while making the spread of the frequency distribution uniform, the frequency distribution is expanded as much as possible, and the contrast is enhanced for each component.
[0140]
When the gradation range is “0” to “255”, the contrast enhancement is performed for each color component (R1, G1, B1) before conversion and the maximum value Rmax2, Gmax, Bmax2 and the minimum value Rmin2, Gmin of each component. , Bmin2 to obtain each color component (R2, G2, B2) of the conversion destination based on the following equation.
[0141]
R2 = far × R1 + fbr (15)
G2 = fag × G1 + fbg (16)
B2 = fab × B1 + fbb (17)
However
far = 255 / (Rmax2-Rmin2) (18)
fag = 255 / (Gmax-Gmin) (19)
fav = 255 / (Bmax2-Bmin2) (20)
fbr = −far × Rmin2 or 255−far × Rmax2 (21)
fbg = -fag x Gmin or 255-fag x Gmax 2 (22)
fbb = −fab × Bmin 2 or 255−fab × Bmax 2 (23)
In addition, if R2, G2, B2 <0 in the above conversion equation, R2, G2, B2 =0If R2, G2, B2> 255, then R2, G2, B2 = 255. Here, far, fag, fab are inclinations, and fbr, fbg, fbb are offsets. According to this conversion equation, as shown in FIG. 22, the luminance distribution having a certain narrow width can be expanded to a reproducible range. Basically, the distribution of luminanceRange ofSince the number of pixels does not change in the enlargement, the areas of the histograms coincide. However, when the luminance distribution is expanded by making the best use of the reproducible range as described above, the highlight portion may be whitened or the high shadow portion may be blackened.
In order to prevent this, in the present embodiment, the gradation range to be expanded is limited.
That is, only "5" as the gradation value is left as the range which is not expanded at the upper end and the lower end of the entire gradation range. As a result, the parameters of the conversion equation become as follows.
[0142]
far = 245 / (Rmax2-Rmin2) (24)
fag = 245 / (Gmax-Gmin) (25)
fav = 245 / (Bmax2-Bmin2) (26)
fbr = 5-far x Rmin 2 or 250-far x Rmax 2 (27)
fbg = 5-fag × Gmin or 250-fag × Gmax 2 (28)
fbb = 5−fab × Bmin 2 or 250−fab × Bmax 2 (29)
Then, in this case, conversion is not performed for the gradation less than "5" and the gradation "250" or more.
[0143]
In the present embodiment, in order to hold the highlight portion and the high shadow portion, the range of “5” is set as the non-enlarged area, with the gradation value from the end of the gradation range uniformly. The range may be narrowed as long as the image output device can relatively easily reproduce a portion or a high shadow portion, or the range may be further enlarged if the reproducibility is further weak. Also, the enlargement ratio may be gradually limited in the border area rather than uniformly.
[0144]
Further, FIG. 23A shows the case where the luminance distribution of the image is narrow, but if the enlargement ratio of the luminance distribution (corresponding to far, fag, fab) is applied as described above, In some cases, a very large magnification can be obtained in accordance with the reproducible range.
Then, although the width of the contrast from the brightest part to the dark part is narrow in the dusk state such as the evening, it is converted as a daytime image as a result of trying to greatly expand the contrast of this image. It may be possible. Since such conversion is not desired, the enlargement ratio is set at a limit so that far, fag and fab do not exceed 1.5 (2 or more). By this, dusk comes to be expressed in dusk.
[0145]
The case where the restriction on the enlargement ratio is not provided is shown by the alternate long and short dash line in FIG. 23A, and after conversion, no extra portion remains within the reproducible range. However, in the case of limiting the expansion range, as indicated by the two-dot chain line in FIG. 7B, the degree of freedom of where to bring the converted distribution is generated, and in some cases, the whole is bright. It may become too dark or too dark. Therefore, in such a case, the ratio (m1: m2) of the remaining area remaining on the upper end side and the lower end side in the gradation range before conversion remains on the upper end side and the lower end side after conversion It may be converted to match the ratio of the remaining area (n1: n2).
[0146]
As described above, the process for obtaining the parameters far, fag, fab, fbr, fbg, fbb is executed in step S208, and in the subsequent step S210, a conversion table is created in the same manner as step S206. The conversion table at this time uses the component values (R1, G1, B1) after conversion shown in FIG. 21 as input to calculate the correspondence, and replaces the component values (R1, G1, B1) after conversion in the conversion table. As a result, referring to the conversion table, two processes of adding the offset amount obtained in step S204 and enhancing the contrast obtained in step S208 are simultaneously performed.
[0147]
On the other hand, one more element of overall brightness remains as a deviation between each color component of the frequency distribution. Accordingly, in order to make the γ correction to equalize the brightness deviation in step S214, γ is calculated in step S212. For example, the peak of the frequency distribution of the red component indicated by the solid line in FIG. 24 is generally darker, or the peak of the frequency distribution of the blue component indicated by the alternate long and short dashed line in FIG. It is preferable that the correction movement is performed so that the entire image is closer to the center as a mountain of the frequency distribution of the green component indicated by the dashed line in FIG.
[0148]
As a result of conducting various experiments, in this embodiment, judging based on the median in the frequency distribution, if the median is less than “85”, it is judged as a dark image and the γ correction corresponding to the following γ value Make it brighter.
[0149]
γr = Rmed 2/85 (30)
γg = Gmed / 85 (31)
γb = Bmed 2/85 (32)
Or
γr = (Rmed 2/85) ** (1/2) (33)
γg = (Gmed / 85) ** (1/2) (34)
γb = (Bmed 2/85) ** (1/2) (35)
I assume.
[0150]
In this case, even if γr, γg, γb <0.7, γr, γg, γb = 0.7. If such a limit is not set, the night image will be like daytime. If the image is too bright, the overall image is whitish and the image tends to have a weak contrast. Therefore, processing such as emphasizing saturation is preferable.
[0151]
On the other hand, when the median is larger than “128”, it is judged as a bright image and darkened by γ correction corresponding to the following γ value.
[0152]
γr = Rmed 2/128 (36)
γg = Gmed / 128 (37)
γb = Bmed 2/128 (38)
Or
γr = (Rmed 2/128) ** (1/2) (39)
γg = (Gmed / 128) ** (1/2) (40)
γb = (Bmed 2/128) ** (1/2) (41)
I assume. In this case, even if γr, γg, γb> 1.3, a limit is provided so as not to be too dark as γr, γg, γb = 1.3. If the image is too dark, the color is too thick and the image becomes dark. Therefore, processing such as weakening the saturation enhancement is preferable. However, such a darkening process may adversely affect the subject in a bright background. For example, in the case of a landscape image in which the sky shows half of the image or a commemorative photo of a sunny day, the face is often dark and prone to collapse due to even back light. In the case of these images, since dark parts and light parts are mixed, the standard deviation of each color component is often relatively high. Therefore, it is possible not to perform γ correction for darkening when such standard deviation is larger than “70”. The corresponding relationship in the case where the γ correction is made is shown in FIG. 25. When γr, γg, γb <1, it becomes a curve which bulges upward, and when γr, γg, γb> 1, it becomes a curve which swells downward. The correction of the brightness does not necessarily have to be based on the frequency distribution, and the brightness may be determined from other factors and corrected.
[0153]
To perform γ correction on γr, γg, and γb determined as described above, the following equation is used. The gradation values R1, G1, B1 after conversion with respect to the gradation values r0, g0, b0 before conversion are
R1 = 255 * (r0 / 255) ** γr (42)
G1 = 255 * (g0 / 255) ** γg (43)
B1 = 255 * (b0 / 255) ** γb (44)
This γ correction is also performed on the conversion table shown in FIG. That is, as in the case of contrast enhancement, the correspondence is calculated using the component values (R1, G1, B1) after conversion shown in FIG. 21 as input, and the component values (R1, G1, B1) after conversion in the conversion table. Replace it. As a result, referring to the conversion table, in addition to the addition of the offset amount and the enhancement of the contrast, the correction of brightness is simultaneously performed.
[0154]
Finally, the image data is converted in step S216, and the image data (rm, gm, bm) of all pixels is referred to the conversion table shown in FIG. 21, and the converted image data (Rm, Gm, Bm) is obtained. Will be repeated.
[0155]
In this embodiment, the correction by the offset amount, the enhancement of the contrast, and the correction of the brightness are performed in this order, but not all of them have to be performed, and individual corrections The method can also be implemented with appropriate changes.
[0156]
For example, in the above-described embodiment, in contrast enhancement processing, correction is performed using a conversion formula that has a linear correspondence with component values before conversion, but smoother conversion may be obtained. Alternatively, so-called S-curve conversion as shown in FIG. 26 may be performed. In this case, it is also possible to use the concept of standard deviation that represents the degree of dispersion of the distribution, instead of judging the spread of the frequency distribution at both end positions. Hereinafter, an example in which the contrast is emphasized in the correspondence relationship of the S-shaped curve using the standard deviation will be shown. Note that although conversion is performed based on the frequency distribution for each component, since the calculation method is common to all, it is expressed by expression of luminance, and a procedure for obtaining luminance Y after conversion from luminance y before conversion is shown.
[0157]
There are two ways of thinking about the standard deviation, but in this embodiment, it is calculated based on the following equation.
[0158]
[Equation 5]

[0159]
yp: luminance before conversion of each pixel
ym: Average value of luminance before conversion of each pixel
The standard deviation corresponds to the amount of spread of the luminance distribution, but dispersion may be used in the meaning of the amount of spread. Further, since the number of gradations as a whole is 256 as in this embodiment, it is possible to obtain the amount of spread from the kurtosis k of the distribution.
[0160]
[Equation 6]

[0161]
The kurtosis of k = 3 referred to here corresponds to the spread amount of the normal distribution.
[0162]
Based on the standard deviation σ, which is the spread amount of the luminance distribution obtained in this way, the contrast enhancement assigns a small number of gradations to the small distribution density range while giving many gradation numbers to the large distribution density range. Also, as shown in FIG. 26, when the input / output correspondence relationship is a so-called S-shaped curve, the gradation range RNG1 allocated after conversion becomes larger than the gradation range rng0 allocated before conversion. This means that the number of gradations has increased. On the other hand, regarding the range outside the gradation range rng0 on the low luminance side and the high luminance side at the input, the gradation range to be allocated after conversion is reduced.
[0163]
In this embodiment, assuming that the center position ymid of the gradation range is “128”, γ1 is given below this center position ymid, and γ correction is performed with γ2 given in the range larger than the center position ymid. , Γ2 are determined based on the standard deviation σ. That is,
For y ≦ 128,
γ1 = (σstd_limit / σ) ** fc (47)
For y> 128,
γ2 = (σ / σstd_limit) ** fc (48)
Then, in step S204, these parameter calculations are performed. Here, σ std_limit and fc are parameters obtained experimentally by giving consideration to the conversion result, and in the present embodiment, σ std_limit is set to “128” and fc is set to “0.1”. Since the standard deviation σ is generally smaller than “128”, if the standard deviation σ is large in these relational expressions, γ2 and γ1 will each approach “1”, and the slope of the S-shaped curve will be gentle. . This means that the conversion target gradation range RNG0 is not so wide as to the gradation range rng0 centered on the central position ymid when the expansion amount is large, and more specifically, the luminance of the image data When it is widely distributed, it means that the conversion which expands the luminance range is not performed.
On the other hand, when the standard deviation σ is small, γ2 and γ1 will both be away from “1”, and the slope of the S-shaped curve will be steep. This means that when the spread amount is small, the conversion-target gradation range RNG1 is widely expanded with respect to the gradation range rng0 centered on the central position ymid, and more specifically, the image data When the luminance is distributed only in a narrow range, it means that conversion is performed to expand the luminance range.
[0164]
In the present embodiment, the correspondence relationship of the S-shaped curve is established by γ correction, but in FIG. 27, the gradation “0”, the lower quadrant yq1, the center position ymid, and the upper quadrant yq3 For the gradation "0", the center position ymid and the gradation "255" while setting the five points of the gradation "255" as the reference points, the lower quadrant yq1 and the upper four are made Y. The conversion point at the division point yq3 is determined based on the standard deviation. Then, the correspondence relationship connecting these five points is obtained by spline interpolation operation or Newton interpolation. Of course, three points on the lower side and three points on the upper side from the center position ymid may be obtained by spline interpolation or Newton interpolation, respectively.
[0165]
That is, as shown in FIG. 28, standard deviations .sigma.r, .sigma.g, .sigma.b are obtained for each component in step S230, .gamma.1 and .gamma.2 for each color component are obtained in step S232, and .gamma.1 and .gamma.2 are used in step S234. Create a conversion table based on the correspondence relationship. These steps S230 to S234 are executed instead of the processes of steps S208 to S214 in the flowchart shown in FIG.
[0166]
Next, the operation of this embodiment having the above configuration will be described in order.
[0167]
Assuming that a photograph is taken by the scanner 11 or the like, image data representing the photograph in the form of gradation data of RGB is taken into the computer 21, and the CPU executes the image processing program shown in FIG. 5 and FIG. Execute processing to correct the color reproducibility of
[0168]
First, in step S102, image data is thinned out within a predetermined error range, and a frequency distribution is obtained for each color component for the selected pixel. The frequency distribution as it is can not be used. First, it is determined in step S104 whether the image is a binary image such as black and white, and it is determined in step S108 whether it is a natural image or not. In step S110, it is determined whether there is a frame portion in the image data except in the case of a binary image or a non-natural image, and if there is a frame portion, the unclear regions at both ends of the frequency distribution are removed in step S114. Get rid of In this state, a feature vector is obtained for the frequency distribution for each color component, and the similarity of the frequency distribution is checked from the inner product of the feature vectors. If the frequency distributions of the respective color components are not very similar, it means that the original image data is intentionally out of color balance, and processing for uniforming the characteristics is not performed. However, if a certain degree of similarity is found in comparison with a predetermined threshold value, it is determined that the color balance has deviated, and the following characteristics are made uniform.
[0169]
That is, in step S204, the red component offset dR and the blue component offset dB for the green component are obtained using the upper end of the frequency distribution and the median, and in step S206, a conversion table for the final image data conversion is formed. Subsequently, in step S208, a parameter for emphasizing the contrast is calculated while achieving equalization of contrast for each color component, and in step S210 the balance of contrast for each color component is matched while emphasizing contrast based on the same parameter. Create a conversion table for Then, in step S212, a parameter of γ correction is calculated to make the brightness of each color component uniform, and a conversion table to be subjected to such γ correction is created.
[0170]
Finally, the image data for all pixels is converted with reference to the conversion table created as described above in step S216.
[0171]
In this case, the processing is divided in step S202 according to whether or not there is similarity, but when the flowchart shown in FIG. 31 is executed, the effective value of the offset amount is changed according to the similarity, Attempt to make the characteristics uniform or not.
[0172]
Of course, as described above, such image processing is not performed when the image is not a binary image or a natural image, but when the image processing of the present invention is performed, it is possible Despite the image data having poor color reproducibility, the color shift, the contrast and the brightness are made uniform for each color component, and the contrast is enhanced, and a good image with sharpness is extremely easily obtained. It will be obtained.
[0173]
In the above-described embodiment, although some parameters are fixed, the user may be able to select on the computer 21 via a predetermined GUI.
[0174]
As described above, the frequency distribution for each color component is obtained for the image data while thinning out in step S102, and it is determined in step 116 whether the frequency distributions have similarity or not. It is determined that the characteristics found from the frequency distribution coincide with each other, and the image data is poor in color reproducibility by correcting the deviation by offset correction, contrast enhancement, or brightness correction in steps S204 to S216. While making a good image with sharpness, such work can be automated, and even non-experts can easily correct the color balance.
Brief Description of the Drawings
FIG. 1 is a block diagram of an image processing system to which an image processing apparatus according to an embodiment of the present invention is applied.
FIG. 2 is a block diagram showing an example of a specific hardware configuration of the image processing apparatus.
FIG. 3 is a schematic block diagram showing another application example of the image processing apparatus of the present invention.
FIG. 4 is a schematic block diagram showing another application example of the image processing apparatus of the present invention.
FIG. 5 is a flowchart corresponding to frequency distribution detection means and similarity determination means in the image processing apparatus of the present invention.
FIG. 6 is a flowchart corresponding to offset amount correction means, contrast correction means, and brightness correction means in the image processing apparatus of the present invention.
FIG. 7 is a diagram showing coordinates in an image of a conversion source.
FIG. 8 is a diagram showing a sampling cycle.
FIG. 9 is a diagram showing the number of sampling pixels.
FIG. 10 is a diagram showing the relationship between a conversion source image and pixels to be sampled.
FIG. 11 is a diagram showing arrangement variables for detecting a frequency distribution.
FIG. 12 is a flowchart of gray image extraction thinning processing;
FIG. 13 shows a black and white image.
FIG. 14 is a view showing a luminance distribution of a black and white image.
FIG. 15 is a diagram showing the state of frequency distribution for each color component in the case of a non-natural image and a natural image.
FIG. 16 is a view showing an image with a frame portion.
FIG. 17 is a diagram showing a frequency distribution of an image having a frame portion.
FIG. 18 is a view showing an end processing of the frequency distribution and an end obtained by the end processing.
FIG. 19 is a diagram showing a method of extracting an element to obtain a feature vector for each color component.
FIG. 20 is a diagram showing a linear relationship without the need for correction of color reproducibility.
FIG. 21 is a diagram showing a conversion table when converting image data based on frequency distribution.
FIG. 22 is a diagram showing the expansion of frequency distribution and the range of all gradations.
FIG. 23 is a view showing a case where the enlargement ratio of contrast is limited.
FIG. 24 is a diagram showing a frequency distribution required to make the brightness uniform.
FIG. 25 is a diagram showing a conversion relationship changed by γ correction.
FIG. 26 is a diagram showing a conversion relationship in which the contrast is enhanced in the correspondence relationship of the S-shaped curve.
FIG. 27 is a diagram showing a conversion relationship in the case of connecting the specified conversion points by interpolation.
FIG. 28 is a partial flow chart in the case of correcting contrast and brightness with an S-shaped curve.
FIG. 29 is a diagram showing a state in which the program is transferred from the medium for recording the image processing program to the hard disk.
FIG. 30 is a graph showing how the window function used changes.
FIG. 31 is a flowchart of an image processing program in the case of adjusting an offset amount using a window function.
FIG. 32 is a graph showing the change of another window function to be used.
[Description of the code]
10: Image input device
11: Scanner
11b: Scanner
12: Digital still camera
12a ... digital still camera
12b ... digital still camera
13b ... modem
20: Image processing apparatus
21 ... computer
22: Hard disk
30: Image output device
31 ... Printer
31a ... Printer
31b ... Printer
32 ... display
32a ... display
41 ... flexible disk
42 ... CD-ROM
43: Hard disk
44 ... ROM
45 ... RAM
46a ... communication line
46b ... modem
46c ... file server