JP2004013306A

JP2004013306A - Similarity computing device, index data generating device, video or audio database device, similarity computing method, index data generating method, content representation data storage device, and recording medium

Info

Publication number: JP2004013306A
Application number: JP2002162957A
Authority: JP
Inventors: Akio Yamada; 山田　昭雄
Original assignee: NEC Corp
Current assignee: NEC Corp
Priority date: 2002-06-04
Filing date: 2002-06-04
Publication date: 2004-01-15

Abstract

<P>PROBLEM TO BE SOLVED: To improve retrieval precision of a similarity retrieval signal system using a plurality of signal feature quantities by performing normalization wherein characteristics of the individual signal feature quantities are reflected when total similarity (distance) is found from distances between the individual signal feature quantities. <P>SOLUTION: After a distance distribution state computation part 12 statistically processes distances calculated by the individual feature quantities, a distance distribution index 105 as a parameter for normalization is computed and an individual feature quantity distance normalization part 13 performs a normalization process corresponding to statistic characteristics of the individual feature quantities. <P>COPYRIGHT: (C)2004,JPO

Description

【０００１】
【発明の属する技術分野】
本発明は、類似度計算装置、インデックスデータ生成装置、映像または音声データベース装置、類似度計算方法、インデックスデータ生成方法、内容表現データ記憶装置、および、記録媒体に関し、特に、映像・音声類似検索のための類似度計算装置、インデックスデータ生成装置、映像または音声データベース装置、類似度計算方法、インデックスデータ生成方法、内容表現データ記憶装置、および、記録媒体に関する。
【０００２】
【従来の技術】
映像または音声信号の特徴をインデックス信号としてデータベースに格納し、インデックス信号間の距離を測ることによって映像または音声信号間の類似度をもとめ、これにより類似映像・音声検索を実装する方式が広く使われている。インデックス信号として利用可能な信号特徴量には様々なものがあり、それらを複数利用することによって類似判定精度を高めることができる。複数の信号特徴量を用いる手法として例えば、特開２０００−２８５１４１、特開２０００−１６３５７６、特開２０００−０４８１８１、特開平１１−０３９３２５、特開平０９−２０４４４０、特開平０９−１０１９７０があげられる。なお以下では類似度の代わりに一般に広く使われる距離測定（どのくらい似ていないかを示す）によって代用する。例えば距離の逆数をとることによって類似度を求めることが可能である。
【０００３】
特開２０００−２８５１４１では、複数の利用可能な信号特徴の種別を利用者に提示して、それらの中から利用者が選択したものを用いる。距離判定においては、選択した信号特徴量それぞれの信号特徴量間距離を合計することによって総合的な類似度を算出する。算出に際しては、それぞれの個別信号特徴量の重要性が異なることから、ユーザによって重みづけという概念を導入することを許容している。
【０００４】
特開２０００−１６３５７６は、これとは少し異なる概念の元に設計されており、映像信号を領域分割した上で、個々の領域ごとに特徴量を計算し、それらを領域面積により重み付け加算することによって全体的な距離を計算している。この例では、特開２０００−２８５１４１でユーザに一任した重要度判定を領域の面積により自動計算することが特徴となっている。特開平１１−０３９３２５は、重み付けに学習効果を導入するもので学習過程を通してユーザが好ましいと望むように重み付けを設定している。特開平０９−２０４４４０も同様の考え方に基づくもので事前学習を行う代わりに検索結果を表示したもののうちどれをユーザが選択したかを追跡し、ユーザが重要視していると考えられる特徴量に対して重みをつける。とく開閉０９−１０１９７０もまったく同様の構図の上で設計されており、重みを次第に重くすることによって、過度の重み付けを防いでおり、学習をリアルタイムに実施することと等価である。
【０００５】
特開２０００−０４８１８１も自動的な重み付けを工夫する手法で、問い合わせ画像の彩度及びテクスチャ強度に比例して重み付け値を設定している。
【０００６】
【発明が解決しようとする課題】
これらの手法は複数の特徴量を併用する場合の相互関係を重み付けという概念で規定する手法であるが、本来特徴量はそれぞれ独立に設計されており、ある特徴量は０−１００、別の特徴量は０−５といったレンジで変化する。上記の例ではこれらは全て重み付けという概念で吸収しているが、特開２０００−０４８１８１が指摘している通り、本来ダイナミックレンジの異なる特徴量を正規化した上でユーザ嗜好を反映する重み係数を導入することが望ましい。なぜならば特定の嗜好を持たない一般的な状況下においては、正規化された特徴量の合計が総合的な特徴量を示すからである。特開２０００−０４８１８１は正規化した上で重み付け導入をうたっているが、この発明では重みつけ設計の方法に主眼がおかれ正規化の手法は外部で定義されることを前提としている。正規化の手法として容易に類推できる方法はそれぞれの特徴量が持つ理論的なダイナミックレンジにより除算して、特徴量を０−１の範囲内で変化するように帰着させる手法である。一般論としてこの手法はあらゆる場面で適用可能であるが、実際にこれを適用した場合に十分な性能が得られないという問題がある。
【０００７】
これは理論上のダイナミックレンジと一般的な映像音声信号から発生する信号特徴量の値域が必ずしも一致しないためで０−１内に分布させたつもりが特徴量によっては０．５〜０．８等の狭い範囲内での分布となっている可能性があるためである。
【０００８】
複数の特徴量を用いる類似検索システムにおいて、適切な正規化手法を提供することにある。
【０００９】
映像音声信号のデータベースは本来何らかの目的を持ってコンテンツ登録がなされている。類似検索の目的はこの登録されたコンテンツを効率的に整理する手段を与えることである。そこで本発明では、データベース登録された信号　がなるべく距離空間内で均等分布して分類が容易になるように、登録された信号の特徴量分布から実効的なダイナミックレンジを測定してその値を正規化に用いることを特徴とする。また、本発明は、正規化を実施する手法としては特徴量空間で行う手法と距離空間で行う手法の２種類を提供し、アプリケーションが要求する装置規模、計算リソース、精度に応じてこれらの使い分けを可能にすることを特徴とする。
【００１０】
【課題を解決するための手段】
本発明の第１の類似度計算装置は、問い合わせ信号となる映像または音声信号と、データベースにあらかじめ登録された複数の映像または音声信号間の距離を算出する類似度計算装置において、前記問い合わせ信号から複数の信号特徴量を算出する問い合わせ信号特徴量抽出部と、前記抽出した信号特徴量とデータベースに登録された個々の映像または音声信号の複数の映像特徴量からそれぞれの信号特徴量間の距離を計算する個別信号特徴量間距離算出部と、前記個別信号特徴量間距離算出部が出力する問い合わせ信号と個々のデータベース登録信号間の個別特徴量間距離群を信号特徴量種別毎に統計処理し、信号特徴量毎の距離分布状況を示す統計量を出力する特徴量間距離分布状況計算部と、前記個別信号特徴量間距離算出部が算出した個別信号特徴量間距離を前記距離分布状況を示す統計量を用いて正規化する個別信号特徴量間距離正規化部と、正規化された個別信号特徴量間距離から総合的な信号間類似度を求める信号間類似度算出部からなることを特徴とする。
【００１１】
本発明の第２の類似度計算装置は、前記第１の類似度計算装置であって、類似度計算装置において、前記信号特徴毎の距離分布状況を示す統計量が個別信号特徴量間距離の最小値と最大値からなることを特徴とする。
【００１２】
本発明の第３の類似度計算装置は、前記第２の類似度計算装置であって、類似度計算装置において、前記個別信号特徴量間距離正規化部が、個別特徴量間距離を前記最大値と前記最小値の差分値で除算することにより正規化を実施することを特徴とする。
【００１３】
本発明の第４の類似度計算装置は、前記第１の類似度計算装置であって、類似度計算装置において、前記信号特徴毎の距離分布状況を示す統計量が個別信号特徴量間距離の平均値と分散値からなることを特徴とする。
【００１４】
本発明の第５の類似度計算装置は、前記第４の類似度計算装置であって、類似度計算装置において、前記個別信号特徴量間距離正規化部が、個別特徴量間距離から前記平均値を減算した上で減算結果を前記分散値で除算することにより正規化を実施することを特徴とする。
【００１５】
本発明の第６の類似度計算装置は、２つの映像または音声信号間の類似度を複数の信号特徴量と複数の信号特徴量それぞれに対応する正規化指標を用いて計算する類似度計算装置において、第一の映像または音声信号の個々の信号特徴量を対応する正規化指標を用いて正規化する第一の信号特徴量正規化部と、第二の映像または音声信号の個々の信号特徴量を対応する正規化指標を用いて正規化する第二の信号特徴量正規化部と、第一の映像または音声信号の個々の正規化された信号特徴量と、第二の映像または音声信号の対応する正規化された信号特徴量との間でそれぞれ距離計算を行う個別信号特徴量間距離算出部と、前記個別信号特徴量間距離から総合的な信号間類似度を求める信号間類似度算出部からなることを特徴とする。
【００１６】
本発明の第７の類似度計算装置は、前記第６の類似度計算装置であって、類似度計算装置において、入力である正規化指標が信号特徴量の最小値及び最大値であることを特徴とする。
【００１７】
本発明の第８の類似度計算装置は、前記第７の類似度計算装置であって、類似度計算装置において、個別信号特徴量正規化部は、個別信号特徴量を前記最大値と前記最小値の差分値で除算することにより正規化を実施することを特徴とする。
【００１８】
本発明の第９の類似度計算装置は、前記第６の類似度計算装置であって、類似度計算装置において、入力である正規化指標が信号特徴量の分散値及び平均値であることを特徴とする。
【００１９】
本発明の第１０の類似度計算装置は、前記第９の類似度計算装置であって、類似度計算装置において、個別信号特徴量正規化部は、個別信号特徴量から前記平均値を減算した上で減算結果を分散値で除算することにより正規化を実施することを特徴とする。
【００２０】
本発明の第１１の類似度計算装置は、２つの映像または音声信号間の類似度を複数の信号特徴量と複数の信号特徴量それぞれに対応する正規化指標を用いて計算する類似度計算装置において、第一の映像または音声信号の個々の信号特徴量と、第二の映像または音声信号の対応する信号特徴量との間でそれぞれ距離計算を行う個別信号特徴量間距離算出部と、算出された前記信号特徴量間距離それぞれを対応する正規化指標を用いて正規化し正規化された個別信号特徴量間距離を計算する個別信号特徴量間距離正規化部と、前記正規化された個別信号特徴量間距離から総合的な信号間類似度を求める信号間類似度算出部からなることを特徴とする。
【００２１】
本発明の第１２の類似度計算装置は、前記第１１の類似度計算装置であって、類似度計算装置において、入力である正規化指標が信号特徴量の最小値及び最大値であることを特徴とする。
【００２２】
本発明の第１３の類似度計算装置は、前記第１２の類似度計算装置であって、類似度計算装置において、個別信号特徴量間距離正規化部は、個別信号特徴量間距離を前記最大値と前記最小値の差分値で除算することにより正規化を実施することを特徴とする。
【００２３】
本発明の第１４の類似度計算装置は、前記第１１の類似度計算装置であって、類似度計算装置において、入力である正規化指標が信号特徴量の平均値及び分散値であることにより正規化を実施することを特徴とする。
【００２４】
本発明の第１５の類似度計算装置は、前記第１４の類似度計算装置であって、類似度計算装置において、個別信号特徴量間距離正規化部は、個別信号特徴量間距離から前記平均値を減算した上で減算結果を分散値で除算することにより正規化を実施することを特徴とする。
【００２５】
本発明の第１のインデックスデータ生成装置は、映像または音声信号の複数の信号特徴量を格納する映像または音声データベースのインデックスデータ生成装置において、入力される映像または音声信号から複数の信号特徴量を抽出する信号特徴量抽出部に加えて、抽出された特徴量と登録済みの特徴量から、個々の特徴量それぞれについてその統計量を算出する特徴量分布状況算出部と、前記統計量を映像または音声データベースに出力する特徴量分布状況記録部からなることを特徴とする。
【００２６】
本発明の第２のインデックスデータ生成装置は、インデックスデータ生成装置において、統計量としてそれぞれの特徴量の平均値および分散値を用いることを特徴とする。
【００２７】
本発明の第３のインデックスデータ生成装置は、前記第１のインデックスデータ生成装置であって、インデックスデータ生成装置において、統計量としてそれぞれの特徴量の最大値および最小値を用いることを特徴とする。
【００２８】
本発明の第１の映像または音声データベース装置は、映像または音声信号の複数の信号特徴量を格納する映像または音声データベース装置において、登録されているここの映像信号から抽出した複数の信号特徴量統計量に加えて、それぞれの特徴量の統計的性質をあらわす指標をも格納することを特徴とする。
【００２９】
本発明の第２の映像または音声データベース装置は、前記第１の映像または音声データベース装置であって、映像または音声データベース装置において、特徴量の統計的性質をあらわす指標としてそれぞれの特徴量の平均値及び分散値を用いることを特徴とする。
【００３０】
本発明の第３の映像または音声データベース装置は、前記第１の映像または音声データベース装置であって、映像または音声データベース装置において、特徴量の統計的性質をあらわす指標としてそれぞれの特徴量の最小値および最大値を用いることを特徴とする。
【００３１】
本発明の第１の類似度計算方法は、問い合わせ信号となる映像または音声信号と、データベースにあらかじめ登録された複数の映像または音声信号間の距離を算出する類似度計算方法において、前記問い合わせ信号から複数の信号特徴量を算出する問い合わせ信号特徴量抽出過程と、前記抽出した信号特徴量とデータベースに登録された個々の映像または音声信号の複数の映像特徴量からそれぞれの信号特徴量間の距離を計算する個別信号特徴量間距離算出過程と、前記個別信号特徴量間距離算出部が出力する問い合わせ信号と個々のデータベース登録信号間の個別特徴量間距離群を信号特徴量種別毎に統計処理し、信号特徴量毎の距離分布状況を示す統計量を出力する特徴量間距離分布状況計算過程と、前記個別信号特徴量間距離算出仮定で算出した個別信号特徴量間距離を前記距離分布状況を示す統計量を用いて正規化する個別信号特徴量間距離正規化過程と、正規化された個別信号特徴量間距離から総合的な信号間類似度を求める信号間類似度算出過程からなることを特徴とする。
【００３２】
本発明の第２の類似度計算方法は、前記第１の類似度計算方法であって、類似度計算方法において、前記信号特徴毎の距離分布状況を示す統計量が個別信号特徴量間距離の最小値と最大値からなることを特徴とする。
【００３３】
本発明の第３の類似度計算方法は、前記第２の類似度計算方法であって、類似度計算方法において、前記個別信号特徴量間距離正規化過程が、個別特徴量間距離を前記最大値と前記最小値の差分値で除算することにより正規化を実施することを特徴とする。
【００３４】
本発明の第４の類似度計算方法は、前記第１の類似度計算方法であって、類似度計算方法において、前記信号特徴毎の距離分布状況を示す統計量が個別信号特徴量間距離の平均値と分散値からなることを特徴とする。
【００３５】
本発明の第５の類似度計算方法は、前記第４の類似度計算方法であって、類似度計算方法において、前記個別信号特徴量間距離正規化過程が、個別特徴量間距離から前記平均値を減算した上で減算結果を前記分散値で除算することにより正規化を実施することを特徴とする。
【００３６】
本発明の第６の類似度計算方法は、２つの映像または音声信号間の類似度を複数の信号特徴量と複数の信号特徴量それぞれに対応する正規化指標を用いて計算する類似度計算方法において、第一の映像または音声信号の個々の信号特徴量を対応する正規化指標を用いて正規化する第一の信号特徴量正規化過程と、第二の映像または音声信号の個々の信号特徴量を対応する正規化指標を用いて正規化する第二の信号特徴量正規化過程と、第一の映像または音声信号の個々の正規化された信号特徴量と、第二の映像または音声信号の対応する正規化された信号特徴量との間でそれぞれ距離計算を行う個別信号特徴量間距離算出過程と、前記個別信号特徴量間距離から総合的な信号間類似度を求める信号間類似度算出過程からなることを特徴とする。
【００３７】
本発明の第７の類似度計算方法は、前記第６の類似度計算方法であって、類似度計算方法において、入力である正規化指標が信号特徴量の最小値及び最大値であることを特徴とする。
【００３８】
本発明の第８の類似度計算方法は、前記第７の類似度計算方法であって、類似度計算方法において、個別信号特徴量正規化過程は、個別信号特徴量を前記最大値と前記最小値の差分値で除算することにより正規化を実施することを特徴とする。
【００３９】
本発明の第９の類似度計算方法は、前記第６の類似度計算方法であって、類似度計算方法において、入力である正規化指標が信号特徴量の分散値及び平均値であることを特徴とする。
【００４０】
本発明の第１０の類似度計算方法は、前記第９の類似度計算方法であって、類似度計算方法において、個別信号特徴量正規化過程は、個別信号特徴量から前記平均値を減算した上で減算結果を分散値で除算することにより正規化を実施することを特徴とする。
【００４１】
本発明の第１１の類似度計算方法は、２つの映像または音声信号間の類似度を複数の信号特徴量と複数の信号特徴量それぞれに対応する正規化指標を用いて計算する類似度計算方法において、第一の映像または音声信号の個々の信号特徴量と、第二の映像または音声信号の対応する信号特徴量との間でそれぞれ距離計算を行う個別信号特徴量間距離算出過程と、算出された前記信号特徴量間距離それぞれを対応する正規化指標を用いて正規化し正規化された個別信号特徴量間距離を計算する個別信号特徴量間距離正規化過程と、前記正規化された個別信号特徴量間距離から総合的な信号間類似度を求める信号間類似度算出過程からなることを特徴とする。
【００４２】
本発明の第１２の類似度計算方法は、前記第１１の類似度計算方法であって、類似度計算方法において、入力である正規化指標が信号特徴量の最小値及び最大値であることを特徴とする。
【００４３】
本発明の第１３の類似度計算方法は、前記第１２の類似度計算方法であって、類似度計算方法において、個別信号特徴量間距離正規化過程は、個別信号特徴量間距離を前記最大値と前記最小値の差分値で除算することにより正規化を実施することを特徴とする。
【００４４】
本発明の第１４の類似度計算方法は、前記第１１の類似度計算方法であって、類似度計算方法において、入力である正規化指標が信号特徴量の平均値及び分散値であることを特徴とする。
【００４５】
本発明の第１５の類似度計算方法は、前記第１４の類似度計算方法であって、類似度計算方法において、個別信号特徴量間距離正規化過程は、個別信号特徴量間距離から前記平均値を減算した上で減算結果を分散値で除算することにより正規化を実施することを特徴とする。
【００４６】
本発明の第１のインデックスデータ生成方法は、映像または音声信号の複数の信号特徴量を格納する映像または音声データベースのインデックスデータ生成方法において、入力される映像または音声信号から複数の信号特徴量を抽出する信号特徴量抽出過程に加えて、抽出された特徴量と登録済みの特徴量から、個々の特徴量それぞれについてその統計量を算出する特徴量分布状況算出過程と、前記統計量を映像または音声データベースに出力する特徴量分布状況記録過程からなることを特徴とする。
【００４７】
本発明の第２のインデックスデータ生成方法は、前記第１のインデックスデータ生成方法であって、インデックスデータ生成方法において、統計量としてそれぞれの特徴量の平均値および分散値を用いることを特徴とする。
【００４８】
本発明の第３のインデックスデータ生成方法は、前記第１のインデックスデータ生成方法であって、統計量としてそれぞれの特徴量の最大値および最小値を用いることを特徴とする。
【００４９】
本発明の第１の映像または音声内容表現データ記憶装置は、映像または音声信号の複数の信号特徴量を格納する映像または音声信号の内容表現データ記憶装置において、登録されているここの映像信号から抽出した複数の信号特徴量統計量に加えて、それぞれの特徴量の統計的性質をあらわす指標もが含まれることを特徴とする。
【００５０】
本発明の第２の映像または音声内容表現データ記憶装置は、前記第１の映像または音声内容表現データ記憶装置であって、特徴量の統計的性質をあらわす指標としてそれぞれの特徴量の平均値及び分散値を用いることを特徴とする。
【００５１】
本発明の第３の映像または音声内容表現データ記憶装置は、前記第１の映像または音声内容表現データ記憶装置であって、
特徴量の統計的性質をあらわす指標としてそれぞれの特徴量の最小値および最大値を用いることを特徴とする。
【００５２】
本発明の記録媒体は、前記請求項類似度計算方法または前記信号特徴量インデックス作成方法を格納したことを特徴とする。
【００５３】
【発明の実施の形態】
次に、本発明の実施の形態について図面を参照して詳細に説明する。図１は距離空間で特徴量の正規化を行う手法の一実施例である。
この実施例は問い合わせ信号１０１とデータベースに登録された信号の特徴量である登録信号特徴量１０３から登録信号と問い合わせ信号の間の信号類似度１０７（距離）を出力するもので、信号特徴量抽出部１０、個別特徴量間距離算出部１１、距離分布状況計算部１２、個別特徴量間距離正規化部１３、信号類似度算出部１４からなる。
信号特徴量抽出部１０は、問い合わせ信号１０１から複数種類の問い合わせ信号特徴量１０２を抽出する。以下、抽出されたＮ種類の特徴量を｛Ｆ_ｉ：（ｉ＝１，２，・・・，Ｎ）｝とする。個々の特徴量としては、たとえばアイ・エス・オー、アイ・イー・シー１５９３８マルチメディア・コンテンツ・ディスクリプション・インタフェイス（ＩＳＯ／ＩＥＣ　１５９３８　Ｍｕｌｔｉｍｅｄｉａ　Ｃｏｎｔｅｎｔ　Ｄｅｓｃｒｉｐｔｉｏｎ　Ｉｎｔｅｒｆａｃｅ）に記載の信号特徴量を利用することができる。個々の信号特徴量がベクトル量である場合には、スカラ量であるその構成要素それぞれを「個々の信号特徴量」として独立に扱ってもよい。データベースにはＭ個の登録信号それぞれに対する特徴量である登録信号特徴量１０３｛Ｇ（ｊ）_ｉ：（ｉ＝１，２，・・・，Ｎ）（ｊ＝１，２，・・・，Ｍ）｝が登録されており、個別特徴量間距離算出部１１は、抽出された問い合わせ信号特徴量１０２と、登録信号特徴量１０３の間で、それぞれの特徴量の種類ごとに距離計算を行う。この結果得られる個別特徴量間距離１０４を｛ｄ（問い合わせ信号、第ｊ登録信号）_ｉ：（ｉ＝１，２，・・・，Ｎ）（ｊ＝１，２，・・・，Ｍ）｝と表記する。個別特徴間距離１０４は、距離分布状況計算部１２と個別特徴量間距離正規化部１３に送られる。距離分布状況計算部１２は、個別特徴間距離１０４から個別特徴間距離１０４の統計的性質を示す距離分布指標１０５を生成するもので、距離分布指標１０５としては、例えば最小値、最大値、分散値、平均値など任意の統計的性質を示す指標を利用可能である。
【００５４】
個別特徴量間距離正規化部１３は、個別特徴量間距離１０４と距離分布指標１０５から正規化された個別特徴量間距離１０６を求める。例えば個別特徴量間距離１０４において個々の特徴量種別毎の距離分布がガウス分布をしている場合には、平均値と分散値を用いて（ｄ（問い合わせ信号、第ｊ登録信号）_ｉ−第ｉ特徴量間距離の平均値）／第ｉ特徴量間距離の分散値により正規化を実現できる。同様に例えば個別特徴量間距離１０４において個々の特徴量種別毎の距離分布が一様分布をしているならば、最小値と最大値を用いて、ｄ（問い合わせ信号、第ｊ登録信号）_ｉ／（第ｉ特徴量間距離の最大値−第ｉ特徴量間距離の最小値）により正規化を実現できる。
【００５５】
正規化処理結果である正規化された個別特徴量間距離１０６を総合し、信号類似度算出部１４で問い合わせ信号と登録済み信号間の信号類似度１０７｛Ｄ（問い合わせ信号、第ｊ登録信号）_ｉ：（ｊ＝１，２，・・・，Ｍ）｝を生成する。正規化された個別特徴量間距離１０６から信号類似度１０７を求めるには、個々の距離の絶対値和や二乗和等様々な手法を利用することができる。
【００５６】
この発明は従来手法と比較して恣意的に設定した重み付け値を実施する代わりに距離分布状況計算部１２が生成する距離分布指標１０５を用いて個別特徴間距離正規化部１３で正規化を行うことが特徴であるが、信号類似度算出部１４の手前に従来手法で使われている重み付け回路を追加して挿入しても問題なく、既存の任意の手法と組み合わせ可能である。
【００５７】
図２は特徴量空間で特徴量の正規化を行う手法の一実施例である。
この実施例は問い合わせ信号１０１とデータベースに登録された信号の特徴量である登録信号特徴量１０３、および個々の特徴量の正規化方法を規定する特徴量正規化指標１０８から登録信号と問い合わせ信号の間の信号類似度１０７（距離）を出力するもので、信号特徴量抽出部１０、問い合わせ個別特徴量正規化部１５、登録個別特徴量正規化部１６、個別特徴量間距離算出部１１、信号類似度算出部１４からなる。
【００５８】
信号特徴量抽出部１０は、問い合わせ信号１０１から複数種類の問い合わせ信号特徴量１０２を抽出する。データベースには第一の実施例で示した登録信号特徴量１０３の他に、個々の特徴量をどのように正規化すべきかを示す特徴量正規化指標１０８｛Ｌ_ｉ：（ｉ＝１，２，・・・，Ｎ）｝が登録されている。問い合わせ信号特徴量１０２は問い合わせ個別特徴量正規化部１５で、登録信号特徴量１０３は登録個別特徴量正規化部１６で、それぞれ正規化され、正規化された問い合わせ信号特徴量１０９と正規化された登録信号特徴量１１０が生成される。特徴量正規化指標１０８は、登録済みの信号特徴量の統計的特徴を示すもので、例えば個々の特徴量種別における最小値、最大値、分散値、平均値など任意の統計的性質を示す指標を利用可能である。正規化処理を実施する問い合わせ個別特徴量正規化部１５または登録個別特徴量正規化部１６における正規化の手法として、例えば登録信号特徴量１０３がガウス分布をしている場合には、平均値と分散値を用いて、（第ｉ特徴量−第ｉ特徴量の平均値）／第ｉ特徴量の分散値により、正規化を実現できる。
【００５９】
同様に例えば個々の特徴量が一様分布をしているならば、最小値と最大値を用いて、第ｉ特徴量／（第ｉ特徴量の最大値−第ｉ特徴量の最小値）により正規化を実現できる。
【００６０】
個々の特徴量は既に正規化されているため、個々の特徴量種別ごとに単純距離計算を個別特徴量距離算出部１１で計算し、その結果である個別特徴量間距離１１１を信号類似度算出部１４でスカラー量とすることによって総合的な類似度が計算できる。個別特徴量間距離１１１から総合的な信号間距離（類似度に反比例）を求めるには、個々の距離の絶対値和や二乗和等様々な手法を利用することができる。
【００６１】
なお本実施例では個別特徴量間距離算出部１１において、個別特徴量間距離１１１を算出する前に、問い合わせ個別特徴量正規化部１５および登録個別特徴量正規化部１６にて正規化を実施しているが、正規化処理の線形性から自明なように、問い合わせ信号特徴量１０２と登録信号特徴量１０３を直接個別特徴量間距離算出部１１に入力した後に、個別特徴量間距離算出部１１の出力を特徴量正規化指標１０８で個々の要素毎に正規化しても同様の効果が得られる。
【００６２】
図３は特徴量空間で正規化を実施する場合に必要となる正規化指標をも併せ持ったデータベースを構築するための、インデックスデータ生成システムの一実施例である。
【００６３】
この実施例は登録信号特徴量１０３をすでに保持するデータベースに新規登録信号１１２から取り出したインデックス信号を追加し、あわせてデータベースに記録される特徴量正規化指標１０８を出力するシステムで、信号特徴量抽出部１０と特徴量分布状況計算部１７から構成される。
【００６４】
新規登録信号１１２は信号特徴量抽出部１０に入力され新規登録信号特徴量１１３が生成される。生成された新規登録信号特徴量１１３はデータベースに登録するためにシステムから出力されると共に、特徴量分布状況計算部１７にも同時に入力される。特徴量分布状況計算部１７はデータベースに登録されている信号の特徴量である登録信号特徴量１０３をすべて読み込み、新規登録信号特徴量１１３とあわせて統計処理をして特徴量正規化指標１０８を生成する。前述のように特徴量正規化指標１０８は登録済みの信号特徴量の統計的特徴を示すもので、例えば均一分布する特徴量種別における最小値、最大値やガウス分布する特徴量種別における分散値、平均値など任意の統計的性質を示す指標を利用可能である。データベースは登録信号特徴量１０３のほかに特徴量正規化指標１０８をも保持しているが、新規に信号登録をする際には本実施例を用いて保持している特徴量正規化指標１０８を毎回更新する。
【００６５】
【発明の効果】
以上説明したように、本発明においては、次のような効果を奏する。
【００６６】
第一の効果は、複数の信号特徴量を併用する検索システムで異なる値域を持つ特徴量間の適切な相対評価ができることにある。
【００６７】
正規化は一般的な状況下ではなく、データベースに最適化された状態でなされるので、データベースに登録されている信号をより適切に整理することが可能になる。
第二の効果は、距離空間ではなく特徴量空間での正規化を実行することで、距離計算毎に正規化パラメータ計算処理を繰り返す処理を省略することができ、システムの規模が小規模にできることにある。
【図面の簡単な説明】
【図１】本発明の実施の形態の距離空間で適応的正規化を実施する構成のブロック図である。
【図２】本発明の実施の形態の特徴量空間で適応的正規化処理を行う構成のブロック図である。
【図３】本発明の実施の形態のインデックスデータ生成システムのブロック図である。
【符号の説明】
１０　　信号特徴量抽出部
１１　　個別特徴量間距離算出部
１２　　距離分布状況計算部
１３　　個別特徴量間距離正規化部
１４　　信号類似度算出部
１５　　問い合わせ個別特徴量正規化部
１６　　登録個別特徴量正規化部
１７　　特徴量分布状況計算部
１０１　　問い合わせ信号
１０２　　問い合わせ信号特徴量
１０３　　登録信号特徴量
１０４　　個別特徴量間距離
１０５　　距離分布指標
１０６　　正規化された個別特徴量間距離
１０７　　信号類似度
１０８　　特徴量正規化指標
１０９　　正規化された問い合わせ信号特徴量
１１０　　正規化された登録信号特徴量
１１１　　個別特徴量間距離
１１２　　新規登録信号
１１３　　新規登録信号特徴量[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a similarity calculation device, an index data generation device, a video or audio database device, a similarity calculation method, an index data generation method, a content expression data storage device, and a recording medium. Calculation device, index data generation device, video or audio database device, similarity calculation method, index data generation method, content expression data storage device, and recording medium for the same.
[0002]
[Prior art]
Widely used is a method of storing the characteristics of video or audio signals as an index signal in a database, measuring the distance between the index signals to determine the similarity between the video or audio signals, and implementing a similar video / audio search. ing. There are various signal feature amounts that can be used as index signals, and similarity determination accuracy can be improved by using a plurality of them. Examples of the method using a plurality of signal feature amounts include JP-A-2000-285141, JP-A-2000-163576, JP-A-2000-048181, JP-A-11-039325, JP-A-09-204440, and JP-A-09-101970. In the following, instead of the similarity, a commonly used distance measurement (indicating how dissimilar) is used. For example, it is possible to obtain the similarity by taking the reciprocal of the distance.
[0003]
In Japanese Patent Application Laid-Open No. 2000-285141, a plurality of types of available signal features are presented to a user, and a type selected by the user from among them is used. In the distance determination, the total similarity is calculated by summing the distances between the signal feature amounts of the selected signal feature amounts. At the time of calculation, the importance of each individual signal feature is different, so that the concept of weighting is allowed to be introduced by the user.
[0004]
Japanese Patent Application Laid-Open No. 2000-163576 is designed based on a concept slightly different from that described above, in which a video signal is divided into regions, a feature amount is calculated for each region, and these are weighted and added by a region area. To calculate the overall distance. This example is characterized in that the importance determination determined by the user in JP-A-2000-285141 is automatically calculated based on the area of the region. Japanese Patent Application Laid-Open No. H11-039325 introduces a learning effect to weighting, and sets weighting as desired by a user through a learning process. Japanese Patent Laid-Open No. 09-204440 is based on the same idea, and instead of performing pre-learning, traces which one of the displayed search results is selected by the user, and calculates the feature amount considered to be important by the user. Weighting. In particular, the opening / closing 09-101970 is also designed on the same composition, and the weight is gradually increased to prevent excessive weighting, which is equivalent to performing learning in real time.
[0005]
Japanese Patent Application Laid-Open No. 2000-048181 also employs a method of devising automatic weighting, and sets a weighting value in proportion to the saturation and texture strength of an inquiry image.
[0006]
[Problems to be solved by the invention]
These methods define the mutual relationship in the case of using a plurality of feature amounts in combination with the concept of weighting, but the feature amounts are originally designed independently. One feature amount is 0-100, and another feature amount is different. The amount varies in a range such as 0-5. In the above example, these are all absorbed by the concept of weighting. However, as pointed out in Japanese Patent Application Laid-Open No. 2000-048181, a weighting factor that reflects the user's preference after normalizing feature amounts that originally have different dynamic ranges is used. It is desirable to introduce. This is because, under a general situation without a particular preference, the sum of normalized feature amounts indicates an overall feature amount. Japanese Patent Application Laid-Open No. 2000-048181 claims that weighting is introduced after normalization. However, in the present invention, it is assumed that the emphasis is on the weighting design method and that the normalization method is externally defined. A method that can be easily analogized as a normalization method is a method of dividing the characteristic amount by the theoretical dynamic range of each characteristic amount and reducing the characteristic amount so as to change within a range of 0-1. As a general theory, this method can be applied in every situation, but there is a problem that sufficient performance cannot be obtained when it is actually applied.
[0007]
This is because the theoretical dynamic range and the value range of signal features generated from general video and audio signals do not always match, and it is intended to be distributed within 0-1. This is because there is a possibility that the distribution is within a narrow range.
[0008]
An object of the present invention is to provide an appropriate normalization method in a similarity search system using a plurality of feature amounts.
[0009]
The database of the video and audio signals is originally registered as a content for some purpose. The purpose of the similarity search is to provide a means for efficiently organizing the registered contents. Therefore, in the present invention, the effective dynamic range is measured from the characteristic amount distribution of the registered signal, and the value is normalized so that the signal registered in the database is evenly distributed in the metric space as much as possible to facilitate classification. It is characterized in that it is used for conversion. Further, the present invention provides two types of techniques for performing normalization, a technique performed in a feature quantity space and a technique performed in a metric space. These techniques are selectively used depending on the device scale, calculation resources, and accuracy required by an application. Is made possible.
[0010]
[Means for Solving the Problems]
A first similarity calculation device of the present invention is a similarity calculation device for calculating a distance between a video or audio signal serving as an inquiry signal and a plurality of video or audio signals registered in advance in a database, An inquiry signal feature amount extraction unit for calculating a plurality of signal feature amounts, and a distance between each signal feature amount from the extracted signal feature amounts and a plurality of video feature amounts of individual video or audio signals registered in the database. Calculating a distance between individual signal feature values to be calculated, and statistically processing a group of distances between individual feature values between the inquiry signal output by the individual signal feature value distance calculation unit and each database registration signal for each signal feature type. The inter-feature distance distribution situation calculating section that outputs a statistic indicating the distance distribution situation for each signal feature quantity, and the individual signal inter-feature distance calculating section calculates An inter-individual signal feature distance normalizing unit for normalizing the inter-individual signal feature distance using a statistic indicating the distance distribution state; and a comprehensive inter-signal similarity based on the normalized inter-individual signal feature distance. Is obtained by calculating a similarity between signals.
[0011]
The second similarity calculation device of the present invention is the first similarity calculation device, wherein in the similarity calculation device, a statistic indicating a distance distribution state for each signal feature is a distance of an individual signal feature amount. It is characterized by comprising a minimum value and a maximum value.
[0012]
The third similarity calculation device of the present invention is the second similarity calculation device, wherein in the similarity calculation device, the inter-individual signal feature amount distance normalizing unit sets the inter-individual feature amount distance to the maximum. Normalization is performed by dividing by a difference value between the value and the minimum value.
[0013]
The fourth similarity calculation device of the present invention is the first similarity calculation device, wherein the statistic indicating the distance distribution status for each signal feature is the difference between the individual signal feature amounts. It is characterized by comprising an average value and a variance value.
[0014]
A fifth similarity calculating apparatus according to the present invention is the fourth similarity calculating apparatus, wherein the individual signal feature amount distance normalizing section calculates the average from the individual feature amount distance. Normalization is performed by subtracting a value and dividing the subtraction result by the variance.
[0015]
A sixth similarity calculating apparatus according to the present invention calculates a similarity between two video or audio signals using a plurality of signal features and a normalization index corresponding to each of the plurality of signal features. A first signal feature amount normalization unit that normalizes individual signal features of a first video or audio signal using a corresponding normalization index, and an individual signal feature of a second video or audio signal. A second signal feature normalization unit for normalizing the quantity using a corresponding normalization index, individual normalized signal features of the first video or audio signal, and a second video or audio signal An individual signal feature distance calculating unit that calculates a distance between the corresponding normalized signal feature and a signal similarity that obtains an overall signal similarity from the individual signal feature distance. It is characterized by comprising a calculating unit.
[0016]
A seventh similarity calculating apparatus according to the present invention is the sixth similarity calculating apparatus, wherein in the similarity calculating apparatus, the input normalization index is a minimum value and a maximum value of the signal feature amount. Features.
[0017]
An eighth similarity calculating apparatus according to the present invention is the seventh similarity calculating apparatus, wherein the individual signal feature quantity normalizing unit determines the individual signal feature quantity by the maximum value and the minimum value. It is characterized in that normalization is performed by dividing by a value difference value.
[0018]
A ninth similarity calculating apparatus according to the present invention is the sixth similarity calculating apparatus, wherein in the similarity calculating apparatus, the input normalization index is a variance value and an average value of the signal feature amount. Features.
[0019]
A tenth similarity calculation device according to the present invention is the ninth similarity calculation device, wherein the individual signal feature amount normalization unit subtracts the average value from the individual signal feature amount. The above is characterized in that normalization is performed by dividing the subtraction result by the variance value.
[0020]
An eleventh similarity calculating apparatus according to the present invention calculates a similarity between two video or audio signals using a plurality of signal features and a normalization index corresponding to each of the plurality of signal features. An individual signal feature distance calculation unit that calculates a distance between an individual signal feature of the first video or audio signal and a corresponding signal feature of the second video or audio signal, A normalized individual signal feature distance calculating unit that normalizes each of the calculated signal feature distances using a corresponding normalization index and calculates a normalized individual signal feature distance, and the normalized individual feature distance. It is characterized by comprising an inter-signal similarity calculating unit for obtaining a total inter-signal similarity from the inter-signal feature amount distance.
[0021]
A twelfth similarity calculating apparatus according to the present invention is the eleventh similarity calculating apparatus, wherein in the similarity calculating apparatus, the input normalization index is a minimum value and a maximum value of the signal feature amount. Features.
[0022]
A thirteenth similarity calculation device of the present invention is the twelfth similarity calculation device, wherein the individual signal feature quantity distance normalization unit sets the individual signal feature quantity distance to the maximum. Normalization is performed by dividing by a difference value between the value and the minimum value.
[0023]
A fourteenth similarity calculating apparatus according to the present invention is the eleventh similarity calculating apparatus, wherein in the similarity calculating apparatus, the input normalization index is an average value and a variance value of the signal feature amount. It is characterized in that normalization is performed.
[0024]
A fifteenth similarity calculation device of the present invention is the fourteenth similarity calculation device, wherein the individual signal feature amount distance normalizing section calculates the average from the individual signal feature amount distance. Normalization is performed by subtracting a value and dividing the subtraction result by a variance value.
[0025]
A first index data generation device of the present invention is a video or audio database index data generation device that stores a plurality of signal feature amounts of a video or audio signal. In addition to the extracted signal feature amount extraction unit, a feature amount distribution situation calculation unit that calculates the statistic amount of each individual feature amount from the extracted feature amount and the registered feature amount, It is characterized by comprising a feature amount distribution status recording unit for outputting to a voice database.
[0026]
A second index data generation device according to the present invention is characterized in that in the index data generation device, an average value and a variance value of each feature amount are used as statistics.
[0027]
A third index data generation device according to the present invention is the first index data generation device, wherein the index data generation device uses a maximum value and a minimum value of each feature amount as statistics. .
[0028]
A first video or audio database device of the present invention is a video or audio database device for storing a plurality of signal characteristic amounts of a video or audio signal, wherein a plurality of signal characteristic amount statistics extracted from the registered video signal are stored. In addition to the quantities, an index indicating the statistical property of each feature quantity is also stored.
[0029]
The second video or audio database device of the present invention is the first video or audio database device, wherein in the video or audio database device, an average value of each feature amount is used as an index indicating a statistical property of the feature amount. And a variance value.
[0030]
A third video or audio database device according to the present invention is the first video or audio database device, wherein the video or audio database device has a minimum value of each feature value as an index indicating a statistical property of the feature value. And the maximum value is used.
[0031]
A first similarity calculation method of the present invention is a similarity calculation method for calculating a distance between a video or audio signal serving as an inquiry signal and a plurality of video or audio signals registered in advance in a database, An inquiry signal feature amount extraction process of calculating a plurality of signal feature amounts, and a distance between respective signal feature amounts from the extracted signal feature amounts and a plurality of video feature amounts of individual video or audio signals registered in the database. Calculating the distance between the individual signal features to be calculated, and statistically processing the distance group between the individual features between the inquiry signal and the individual database registration signal output by the distance between individual signal features calculation unit for each signal feature type. A feature value distance distribution situation calculation step of outputting a statistic indicating a distance distribution situation for each signal feature quantity, and the individual signal feature quantity distance calculation assumption An individual signal feature distance normalization process for normalizing the calculated distance between individual signal features using a statistic indicating the distance distribution state, and a total signal interval from the normalized individual signal feature distance. It is characterized in that it comprises a process of calculating a similarity between signals for obtaining a similarity.
[0032]
A second similarity calculation method according to the present invention is the first similarity calculation method, wherein in the similarity calculation method, a statistic indicating a distance distribution state for each of the signal features is a distance between individual signal feature amounts. It is characterized by comprising a minimum value and a maximum value.
[0033]
A third similarity calculation method according to the present invention is the second similarity calculation method, wherein the individual signal feature distance separation normalizing step includes setting the individual feature distance to the maximum. Normalization is performed by dividing by a difference value between the value and the minimum value.
[0034]
A fourth similarity calculation method according to the present invention is the first similarity calculation method, wherein in the similarity calculation method, a statistic indicating a distance distribution state for each of the signal features is a distance between individual signal feature amounts. It is characterized by comprising an average value and a variance value.
[0035]
A fifth similarity calculation method according to the present invention is the fourth similarity calculation method, wherein in the similarity calculation method, the step of normalizing the distance between individual signal features is performed based on the distance between individual features. Normalization is performed by subtracting a value and dividing the subtraction result by the variance.
[0036]
A sixth similarity calculation method according to the present invention calculates a similarity between two video or audio signals using a plurality of signal features and a normalization index corresponding to each of the plurality of signal features. A first signal feature amount normalization step of normalizing individual signal features of the first video or audio signal using the corresponding normalization index, and individual signal features of the second video or audio signal. A second signal feature normalization process for normalizing the quantity using a corresponding normalization index, individual normalized signal features of the first video or audio signal, and a second video or audio signal. A distance calculation process for calculating the distance between the individual signal feature amounts corresponding to the normalized signal feature amounts corresponding thereto, and a signal similarity degree for obtaining an overall signal similarity from the distance between the individual signal feature amounts. Characterized by a calculation process
[0037]
The seventh similarity calculation method of the present invention is the sixth similarity calculation method, wherein in the similarity calculation method, the input normalization index is a minimum value and a maximum value of the signal feature amount. Features.
[0038]
An eighth similarity calculation method according to the present invention is the seventh similarity calculation method, wherein in the similarity calculation method, the individual signal feature amount normalizing step includes: setting the individual signal feature amount to the maximum value and the minimum value. It is characterized in that normalization is performed by dividing by a value difference value.
[0039]
A ninth similarity calculation method according to the present invention is the sixth similarity calculation method, wherein in the similarity calculation method, the input normalization index is a variance value and an average value of the signal feature amount. Features.
[0040]
A tenth similarity calculation method according to the present invention is the ninth similarity calculation method, wherein in the similarity calculation method, the individual signal feature quantity normalizing step is obtained by subtracting the average value from the individual signal feature quantity. The above is characterized in that normalization is performed by dividing the subtraction result by the variance value.
[0041]
An eleventh similarity calculation method according to the present invention calculates a similarity between two video or audio signals using a plurality of signal feature amounts and a normalization index corresponding to each of the plurality of signal feature amounts. An individual signal feature distance calculation step of calculating a distance between each signal feature of the first video or audio signal and a corresponding signal feature of the second video or audio signal, Normalizing each of the obtained signal feature amount distances using a corresponding normalization index to calculate a normalized individual signal feature amount distance, and the normalized individual feature distance distance process. It is characterized in that it comprises an inter-signal similarity calculating step of obtaining a total inter-signal similarity from the distance between signal feature amounts.
[0042]
A twelfth similarity calculation method according to the present invention is the eleventh similarity calculation method, wherein in the similarity calculation method, the input normalization index is a minimum value and a maximum value of the signal feature amount. Features.
[0043]
A thirteenth similarity calculation method according to the present invention is the twelfth similarity calculation method, wherein in the similarity calculation method, the distance between individual signal features is normalized by setting the distance between individual signal features to the maximum. Normalization is performed by dividing by a difference value between the value and the minimum value.
[0044]
A fourteenth similarity calculation method according to the present invention is the eleventh similarity calculation method, wherein in the similarity calculation method, the input normalization index is an average value and a variance value of the signal feature amount. Features.
[0045]
A fifteenth similarity calculation method of the present invention is the fourteenth similarity calculation method, wherein the individual signal feature distance separation normalizing step comprises calculating the average from the individual signal feature distance. Normalization is performed by subtracting a value and dividing the subtraction result by a variance value.
[0046]
According to a first index data generation method of the present invention, in the index data generation method of a video or audio database storing a plurality of signal characteristic amounts of a video or audio signal, a plurality of signal characteristic amounts are input from an input video or audio signal. In addition to the signal feature value extraction process to be extracted, a feature value distribution situation calculation process of calculating a statistic for each feature value from the extracted feature value and the registered feature value, and It is characterized by comprising a feature amount distribution status recording step of outputting to a voice database.
[0047]
A second index data generation method according to the present invention is the first index data generation method, wherein in the index data generation method, an average value and a variance value of each feature amount are used as statistics. .
[0048]
A third index data generation method according to the present invention is the first index data generation method, wherein a maximum value and a minimum value of each feature amount are used as statistics.
[0049]
A first video or audio content expression data storage device of the present invention is a video or audio signal content expression data storage device that stores a plurality of signal feature amounts of a video or audio signal. It is characterized in that in addition to the extracted plurality of signal feature amount statistics, an index indicating the statistical property of each feature amount is also included.
[0050]
The second video or audio content expression data storage device of the present invention is the first video or audio content expression data storage device, wherein the average value of each characteristic amount and the average value of each characteristic amount are used as indices indicating the statistical properties of the characteristic amounts. It is characterized by using a variance value.
[0051]
The third video or audio content expression data storage device of the present invention is the first video or audio content expression data storage device,
It is characterized in that the minimum value and the maximum value of each feature amount are used as indices indicating the statistical properties of the feature amounts.
[0052]
A recording medium according to the present invention is characterized by storing the claim similarity calculation method or the signal feature index creation method.
[0053]
BEST MODE FOR CARRYING OUT THE INVENTION
Next, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 shows an embodiment of a method for normalizing feature values in a metric space.
In this embodiment, a signal similarity 107 (distance) between a registration signal and an inquiry signal is output from an inquiry signal 101 and a registered signal characteristic amount 103 which is a characteristic amount of a signal registered in a database. It comprises a unit 10, an inter-individual feature distance calculating unit 11, a distance distribution situation calculating unit 12, an inter-individual feature amount distance normalizing unit 13, and a signal similarity calculating unit 14.
The signal characteristic amount extraction unit 10 extracts a plurality of types of inquiry signal characteristic amounts 102 from the inquiry signal 101. Hereinafter, the extracted N types of feature amounts are denoted by ｛F _i : (I = 1, 2,..., N)}. As each feature, for example, a signal feature described in ISO / IEC 15938 Multimedia Content Description Interface (ISO / IEC 15938) is used. Can be. When the individual signal feature amounts are vector amounts, each of the constituent elements that are scalar amounts may be independently treated as “individual signal feature amounts”. The registered signal feature amount 103 @ G (j), which is a feature amount for each of the M registered signals, is stored in the database. _i : (I = 1, 2,..., N) (j = 1, 2,..., M)} are registered. Distance calculation is performed between the quantity 102 and the registration signal feature quantity 103 for each type of feature quantity. The distance 104 between individual features obtained as a result is represented by ｛d (inquiry signal, j-th registration signal). _i : (I = 1, 2,..., N) (j = 1, 2,..., M)}. The inter-individual feature distance 104 is sent to the distance distribution situation calculation unit 12 and the inter-individual feature distance normalization unit 13. The distance distribution situation calculation unit 12 generates a distance distribution index 105 indicating the statistical property of the inter-individual feature distance 104 from the inter-individual feature distance 104. Examples of the distance distribution index 105 include a minimum value, a maximum value, and a variance. An index indicating any statistical property such as a value and an average value can be used.
[0054]
The inter-individual feature distance normalization unit 13 obtains a normalized inter-feature distance 106 from the inter-individual feature distance 104 and the distance distribution index 105. For example, when the distance distribution for each feature amount type has a Gaussian distribution at the individual feature amount distance 104, the average value and the variance value are used to obtain (d (inquiry signal, j-th registration signal) _i Normalization can be realized by the variance value of (the average value of the i-th feature amount distance) / the i-th feature amount distance. Similarly, for example, if the distance distribution for each characteristic amount type is uniform at the distance 104 between individual characteristic amounts, d (inquiry signal, j-th registration signal) is obtained using the minimum value and the maximum value. _i / (Maximum value of distance between ith feature values-minimum value of distance between ith feature values) can realize normalization.
[0055]
The normalized inter-characteristic distance 106, which is the result of the normalization processing, is integrated, and the signal similarity calculator 14 calculates the signal similarity 107 ｛D between the inquiry signal and the registered signal (inquiry signal, j-th registration signal). _i : (J = 1, 2,..., M)}. In order to obtain the signal similarity 107 from the normalized inter-characteristic distance 106, various methods such as sum of absolute values and sum of squares of the individual distances can be used.
[0056]
According to the present invention, the individual feature distance normalization unit 13 performs normalization using the distance distribution index 105 generated by the distance distribution situation calculation unit 12 instead of arbitrarily setting a weight value compared to the conventional method. However, there is no problem even if a weighting circuit used in the conventional method is additionally inserted in front of the signal similarity calculation unit 14, and it can be combined with any existing method.
[0057]
FIG. 2 shows an embodiment of a method for normalizing a feature in a feature space.
In this embodiment, a registration signal and an inquiry signal are obtained from an inquiry signal 101, a registration signal characteristic amount 103 which is a characteristic amount of a signal registered in a database, and a characteristic amount normalization index 108 which specifies a method for normalizing each characteristic amount. It outputs a signal similarity 107 (distance) between the signal characteristic amount extraction unit 10, the query individual characteristic amount normalization unit 15, the registered individual characteristic amount normalization unit 16, the inter-individual characteristic amount distance calculation unit 11, the signal A similarity calculation unit 14 is provided.
[0058]
The signal characteristic amount extraction unit 10 extracts a plurality of types of inquiry signal characteristic amounts 102 from the inquiry signal 101. In the database, in addition to the registration signal feature amount 103 shown in the first embodiment, a feature amount normalization index 108 ｛L indicating how to normalize each feature amount. _i : (I = 1, 2,..., N)} are registered. The inquiry signal characteristic amount 102 is normalized by an inquiry individual characteristic amount normalization unit 15, and the registered signal characteristic amount 103 is normalized by a registered individual characteristic amount normalization unit 16 to a normalized inquiry signal characteristic amount 109. The registered signal feature value 110 is generated. The feature amount normalization index 108 indicates a statistical feature of a registered signal feature amount, and is an index indicating an arbitrary statistical property such as a minimum value, a maximum value, a variance value, and an average value in each feature amount type. Is available. As a normalization method in the query individual feature amount normalization unit 15 or the registered individual feature amount normalization unit 16 that performs the normalization process, for example, when the registered signal feature amount 103 has a Gaussian distribution, the average value and Using the variance value, normalization can be realized by the variance value of (the i-th feature amount−the average value of the i-th feature amount) / the i-th feature amount.
[0059]
Similarly, for example, if the individual feature values have a uniform distribution, the minimum value and the maximum value are used to calculate the ith feature value / (the maximum value of the ith feature value−the minimum value of the ith feature value). Normalization can be realized.
[0060]
Since the individual feature values have already been normalized, simple distance calculation is performed by the individual feature value distance calculation unit 11 for each feature value type, and the resulting distance 111 between individual feature values is calculated by signal similarity calculation. By using the scalar amount in the unit 14, the overall similarity can be calculated. In order to obtain the total distance between signals (inversely proportional to the similarity) from the distance 111 between individual feature amounts, various methods such as the sum of absolute values and the sum of squares of the individual distances can be used.
[0061]
In this embodiment, before calculating the inter-individual feature amount distance 111 in the inter-individual feature amount calculation unit 11, normalization is performed by the inquiry individual feature amount normalization unit 15 and the registered individual feature amount normalization unit 16. However, as is obvious from the linearity of the normalization process, after inputting the inquiry signal feature amount 102 and the registered signal feature amount 103 directly to the inter-individual feature amount distance calculation unit 11, the inter-individual feature amount distance calculation unit The same effect can be obtained by normalizing the output of No. 11 for each element using the feature quantity normalization index 108.
[0062]
FIG. 3 shows an embodiment of an index data generation system for constructing a database that also has a normalization index required when performing normalization in the feature space.
[0063]
This embodiment is a system that adds an index signal extracted from a new registration signal 112 to a database that already holds the registration signal feature amount 103 and outputs a feature amount normalization index 108 recorded in the database. It comprises an extraction unit 10 and a feature amount distribution situation calculation unit 17.
[0064]
The newly registered signal 112 is input to the signal feature amount extraction unit 10, and a newly registered signal feature amount 113 is generated. The generated newly registered signal feature amount 113 is output from the system for registration in the database, and is also input to the feature amount distribution situation calculation unit 17 at the same time. The feature amount distribution situation calculation unit 17 reads all the registered signal feature amounts 103 which are the feature amounts of the signals registered in the database, performs statistical processing together with the newly registered signal feature amount 113, and calculates the feature amount normalized index 108. Generate. As described above, the feature amount normalization index 108 indicates a statistical feature of a registered signal feature amount. For example, a minimum value and a maximum value in a uniformly distributed feature amount type, a variance value in a Gaussian distributed feature amount type, An index indicating an arbitrary statistical property such as an average value can be used. Although the database also holds a feature quantity normalization index 108 in addition to the registered signal feature quantity 103, when newly registering a signal, the feature quantity normalization index 108 held using this embodiment is used. Update every time.
[0065]
【The invention's effect】
As described above, the present invention has the following effects.
[0066]
A first effect is that an appropriate relative evaluation between features having different value ranges can be performed in a search system using a plurality of signal features in combination.
[0067]
Since the normalization is performed not in a general situation but in a state optimized for the database, it is possible to more appropriately arrange the signals registered in the database.
The second effect is that by performing normalization in the feature space instead of the metric space, the process of repeating the normalization parameter calculation process for each distance calculation can be omitted, and the system scale can be reduced. It is in.
[Brief description of the drawings]
FIG. 1 is a block diagram of a configuration for performing adaptive normalization in a metric space according to an embodiment of the present invention.
FIG. 2 is a block diagram of a configuration for performing adaptive normalization processing in a feature space according to the embodiment of this invention.
FIG. 3 is a block diagram of an index data generation system according to the embodiment of the present invention.
[Explanation of symbols]
10 Signal feature extraction unit
11 Distance calculation unit between individual features
12 Distance distribution situation calculation part
13 Individual feature distance distance normalization unit
14 signal similarity calculator
15 Query individual feature normalization unit
16 Registered individual feature normalization unit
17 Feature distribution status calculator
101 Inquiry signal
102 Inquiry signal features
103 Registered signal features
104 Distance between individual features
105 Distance distribution index
106 Normalized distance between individual features
107 Signal similarity
108 Feature Normalization Index
109 Normalized interrogation signal features
110 Normalized registration signal features
111 Distance between individual features
112 New registration signal
113 Newly registered signal features

Claims

In a similarity calculation device that calculates the distance between a plurality of video or audio signals registered in advance in a database, the video or audio signal serving as an inquiry signal,
An inquiry signal characteristic amount extraction unit that calculates a plurality of signal characteristic amounts from the inquiry signal,
An individual signal feature distance calculation unit that calculates a distance between each signal feature from the plurality of video features of the individual video or audio signals registered in the extracted signal feature and the database,
Statistical processing is performed on the inter-individual feature distance group between the inquiry signal and the individual database registration signal output by the inter-individual signal feature distance calculation unit for each signal feature type, and the statistics indicating the distance distribution status for each signal feature amount A feature amount distance distribution status calculation unit for outputting a quantity,
An inter-individual signal feature amount distance normalization unit that normalizes the inter-individual signal feature amount distance calculated by the inter-individual signal feature amount calculation unit using a statistic indicating the distance distribution situation;
A similarity calculating apparatus comprising an inter-signal similarity calculating unit that obtains a total inter-signal similarity from a normalized distance between individual signal features.

In the similarity calculation device,
The similarity calculation apparatus according to claim 1, wherein the statistic indicating the distance distribution state for each signal feature includes a minimum value and a maximum value of the distance between individual signal features.

In the similarity calculation device,
3. The degree of similarity according to claim 2, wherein the inter-individual signal feature amount distance normalizing unit performs normalization by dividing the inter-individual feature amount distance by a difference value between the maximum value and the minimum value. Computing device.

In the similarity calculation device,
The similarity calculation device according to claim 1, wherein the statistic indicating the distance distribution state for each signal feature comprises an average value and a variance value of the distance between individual signal features.

In the similarity calculation device,
5. The normalization unit according to claim 4, wherein the inter-individual signal feature distance normalizing unit subtracts the average value from the inter-individual feature distance and divides the subtraction result by the variance. A similarity calculation device as described.

In a similarity calculation device that calculates a similarity between two video or audio signals using a plurality of signal feature amounts and a normalization index corresponding to each of the plurality of signal feature amounts,
A first signal feature amount normalization unit that normalizes each signal feature amount of the first video or audio signal using a corresponding normalization index,
A second signal feature amount normalization unit that normalizes an individual signal feature amount of the second video or audio signal using a corresponding normalization index,
Between individual signal features that perform distance calculations between the individual normalized signal features of the first video or audio signal and the corresponding normalized signal features of the second video or audio signal, respectively. A distance calculator,
A similarity calculation device comprising an inter-signal similarity calculation unit for obtaining a total inter-signal similarity from the distance between the individual signal feature amounts.

In the similarity calculation device,
7. The similarity calculating apparatus according to claim 6, wherein the input normalization index is a minimum value and a maximum value of the signal feature amount.

In the similarity calculation device,
The similarity calculation device according to claim 7, wherein the individual signal feature amount normalization unit performs normalization by dividing the individual signal feature amount by a difference value between the maximum value and the minimum value.

In the similarity calculation device,
7. The similarity calculation device according to claim 6, wherein the input normalization index is a variance value and an average value of the signal feature amount.

In the similarity calculation device,
10. The similarity calculation according to claim 9, wherein the individual signal feature quantity normalization unit performs normalization by subtracting the average value from the individual signal feature quantity and dividing the subtraction result by a variance value. apparatus.

In a similarity calculation device that calculates a similarity between two video or audio signals using a plurality of signal feature amounts and a normalization index corresponding to each of the plurality of signal feature amounts,
Individual signal feature amount of the first video or audio signal, and an individual signal feature amount distance calculation unit that performs a distance calculation between the corresponding signal feature amount of the second video or audio signal,
An inter-individual signal feature amount distance normalization unit that calculates a normalized inter-individual signal feature amount distance by normalizing each of the calculated inter-signal feature amount distances using a corresponding normalization index,
A similarity calculating apparatus, comprising: an inter-signal similarity calculating unit that obtains an overall inter-signal similarity from the normalized distance between individual signal features.

In the similarity calculation device,
12. The similarity calculation device according to claim 11, wherein the input normalization index is a minimum value and a maximum value of the signal feature amount.

In the similarity calculation device,
13. The similarity according to claim 12, wherein the inter-individual signal feature distance normalization unit performs normalization by dividing the inter-individual signal feature distance by a difference value between the maximum value and the minimum value. Computing device.

In the similarity calculation device,
12. The similarity calculating apparatus according to claim 11, wherein the normalization is performed based on an average value and a variance value of the signal feature amount, which are input as the normalization index.

In the similarity calculation device,
15. The inter-individual-signal-feature-value-distance normalizing unit performs normalization by subtracting the average value from the inter-individual-signal-feature-value distance, and dividing the subtraction result by a variance value. Similarity calculator.

In a video or audio database index data generation device that stores a plurality of signal features of the video or audio signal,
In addition to the signal feature amount extraction unit that extracts a plurality of signal feature amounts from the input video or audio signal,
A feature amount distribution status calculation unit that calculates a statistic of each of the feature amounts from the extracted feature amounts and the registered feature amounts;
An index data generation device, comprising: a feature amount distribution status recording unit that outputs the statistics to a video or audio database.

In the index data generation device,
An index data generation device characterized by using an average value and a variance value of each feature amount as a statistic.

In the index data generation device,
17. The index data generation device according to claim 16, wherein a maximum value and a minimum value of each feature amount are used as the statistics.

In a video or audio database device that stores a plurality of signal features of a video or audio signal,
A video or audio database apparatus, which stores, in addition to a plurality of signal feature quantity statistics extracted from the registered video signal, an index indicating a statistical property of each feature quantity.

In a video or audio database device,
20. The video or audio database apparatus according to claim 19, wherein an average value and a variance value of each of the feature amounts are used as indices indicating the statistical properties of the feature amounts.

In a video or audio database device,
20. The video or audio database device according to claim 19, wherein a minimum value and a maximum value of each feature amount are used as indices indicating the statistical properties of the feature amounts.

In a similarity calculation method for calculating the distance between a plurality of video or audio signals registered in advance in a database, the video or audio signal serving as an inquiry signal,
An inquiry signal feature amount extraction step of calculating a plurality of signal feature amounts from the inquiry signal,
An individual signal feature distance calculating step of calculating a distance between respective signal features from a plurality of video features of the individual video or audio signals registered in the extracted signal feature and the database,
Statistical processing is performed on the inter-individual feature distance group between the inquiry signal and the individual database registration signal output by the inter-individual signal feature distance calculation unit for each signal feature type, and the statistics indicating the distance distribution status for each signal feature amount And calculating the distance between the individual signal features, which calculates the distance between the individual signal features by using the statistic indicating the distance distribution status. A similarity calculation method characterized by comprising an inter-quantity distance normalization step and an inter-signal similarity calculation step of obtaining a comprehensive inter-signal similarity from the normalized individual signal feature amount distance.

In the similarity calculation method,
23. The similarity calculation method according to claim 22, wherein the statistic indicating the distance distribution state for each signal feature includes a minimum value and a maximum value of the distance between individual signal features.

In the similarity calculation method,
24. The degree of similarity according to claim 23, wherein the normalizing step between individual signal features performs normalization by dividing the distance between individual features by a difference value between the maximum value and the minimum value. Method of calculation.

In the similarity calculation method,
23. The similarity calculation method according to claim 22, wherein the statistic indicating the distance distribution state for each signal feature comprises an average value and a variance of the distance between individual signal features.

In the similarity calculation method,
26. The inter-individual signal feature distance normalization step performs normalization by subtracting the average value from the inter-individual feature distance and dividing the result of the subtraction by the variance. The similarity calculation method described.

In a similarity calculation method of calculating a similarity between two video or audio signals using a plurality of signal feature amounts and a normalization index corresponding to each of the plurality of signal feature amounts,
A first signal feature amount normalization step of normalizing individual signal features of the first video or audio signal using a corresponding normalization index,
A second signal feature amount normalization process of normalizing the individual signal feature amount of the second video or audio signal using a corresponding normalization index,
Between individual signal features that perform distance calculations between the individual normalized signal features of the first video or audio signal and the corresponding normalized signal features of the second video or audio signal, respectively. Distance calculation process,
A similarity calculation method characterized by comprising an inter-signal similarity calculation step of obtaining a total inter-signal similarity from the individual signal feature distance.

In the similarity calculation method,
28. The similarity calculation method according to claim 27, wherein the input normalization index is a minimum value and a maximum value of the signal feature amount.

In the similarity calculation method,
29. The similarity calculation method according to claim 28, wherein in the individual signal feature amount normalization step, normalization is performed by dividing the individual signal feature amount by a difference value between the maximum value and the minimum value.

In the similarity calculation method,
28. The similarity calculation method according to claim 27, wherein the input normalization index is a variance value and an average value of the signal feature amount.

In the similarity calculation method,
31. The similarity calculation according to claim 30, wherein in the individual signal feature amount normalization process, the normalization is performed by subtracting the average value from the individual signal feature amount and dividing the subtraction result by a variance value. Method.

In a similarity calculation method of calculating a similarity between two video or audio signals using a plurality of signal feature amounts and a normalization index corresponding to each of the plurality of signal feature amounts,
Individual signal feature amount of the first video or audio signal, and individual signal feature amount distance calculation step of performing a distance calculation between the corresponding signal feature amount of the second video or audio signal,
An individual signal feature amount distance normalization process of calculating the normalized individual signal feature amount distance by normalizing each of the calculated signal feature amount distances with a corresponding normalization index,
A similarity calculation method characterized by comprising an inter-signal similarity calculation step of obtaining a total inter-signal similarity from the normalized distance between individual signal features.

In the similarity calculation method,
33. The similarity calculation method according to claim 32, wherein the input normalization index is a minimum value and a maximum value of the signal feature amount.

In the similarity calculation method,
34. The degree of similarity according to claim 33, wherein the normalizing step between the individual signal feature amounts performs the normalization by dividing the distance between the individual signal feature amounts by a difference value between the maximum value and the minimum value. Method of calculation.

In the similarity calculation method,
33. The similarity calculation method according to claim 32, wherein the input normalization index is an average value and a variance value of the signal feature amount.

In the similarity calculation method,
36. The normalization process of the distance between individual signal features, wherein the normalization is performed by subtracting the average value from the distance between the individual signal features and dividing the subtraction result by a variance value. Similarity calculation method.

In the method of generating index data of a video or audio database that stores a plurality of signal features of a video or audio signal,
In addition to the signal feature extraction process of extracting a plurality of signal features from the input video or audio signal,
A feature amount distribution situation calculation process of calculating a statistic of each of the individual feature amounts from the extracted feature amounts and the registered feature amounts;
A method for generating index data, comprising a feature amount distribution status recording step of outputting the statistics to a video or audio database.

In the index data generation method,
38. The index data generation method according to claim 37, wherein an average value and a variance value of each feature amount are used as the statistic.

In the index data generation device,
39. The index data generation method according to claim 38, wherein a maximum value and a minimum value of each feature amount are used as the statistics.

In a video or audio signal content representation data storage device that stores a plurality of signal feature amounts of the video or audio signal,
A video or audio content expression data storage device characterized in that, in addition to a plurality of signal feature amount statistics extracted from the registered video signal, an index indicating a statistical property of each feature amount is included. .

In the content expression data storage device,
41. The content expression data storage device according to claim 40, wherein an average value and a variance value of each feature amount are used as indices indicating the statistical properties of the feature amounts.

In the content expression data storage device,
41. The content expression data storage device according to claim 40, wherein the minimum value and the maximum value of each feature amount are used as indices indicating the statistical properties of the feature amounts.

A storage medium storing the similarity calculation method or the signal feature index creation method according to any one of claims 22 to 39.