Analysis of chernoff criterion for linear dimensionality reduction

Jing Peng, Stefan Robila, Wei Fan, Guna Seetharaman

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

1 Citation (Scopus)

Abstract

Well known linear discriminant analysis (LDA) based on the Fisher criterion is incapable of dealing with heteroscedasticity in data. However, in many practical applications we often encounter heteroscedastic data, i.e., within class scatter matrices can not be expected to be equal. A technique based on the Chernoff criterion for linear dimensionality reduction has been proposed recently. The technique extends well-known Fisher's LDA and is capable of exploiting information about heteroscedasticity in the data. While the Chernoff criterion has been shown to outperform the Fisher's, a clear understanding of its exact behavior is lacking. In addition, the criterion, as introduced, is rather complex, thereby making it difficult to clearly state its relationship to other linear dimensionality techniques. In this paper, we show precisely what can be expected from the Chernoff criterion and its relations to the Fisher criterion and Fukunaga-Koontz transform. Furthermore, we show that a recently proposed decomposition of the data space into four subspaces is incomplete. We provide arguments on how to best enrich the decomposition of the data space in order to account for heteroscedasticity in the data.

Original languageEnglish
Title of host publication2010 IEEE International Conference on Systems, Man and Cybernetics, SMC 2010
Pages3014-3021
Number of pages8
DOIs
StatePublished - 1 Dec 2010
Event2010 IEEE International Conference on Systems, Man and Cybernetics, SMC 2010 - Istanbul, Turkey
Duration: 10 Oct 201013 Oct 2010

Publication series

NameConference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
ISSN (Print)1062-922X

Other

Other2010 IEEE International Conference on Systems, Man and Cybernetics, SMC 2010
CountryTurkey
CityIstanbul
Period10/10/1013/10/10

Fingerprint

Discriminant analysis
Decomposition

Keywords

  • Chernoff distance
  • Dimensionality reduction
  • Linear discriminant analysis

Cite this

Peng, J., Robila, S., Fan, W., & Seetharaman, G. (2010). Analysis of chernoff criterion for linear dimensionality reduction. In 2010 IEEE International Conference on Systems, Man and Cybernetics, SMC 2010 (pp. 3014-3021). [5641971] (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics). https://doi.org/10.1109/ICSMC.2010.5641971
Peng, Jing ; Robila, Stefan ; Fan, Wei ; Seetharaman, Guna. / Analysis of chernoff criterion for linear dimensionality reduction. 2010 IEEE International Conference on Systems, Man and Cybernetics, SMC 2010. 2010. pp. 3014-3021 (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics).
@inproceedings{9c0401241c3b44a1938de03d9cf8e86b,
title = "Analysis of chernoff criterion for linear dimensionality reduction",
abstract = "Well known linear discriminant analysis (LDA) based on the Fisher criterion is incapable of dealing with heteroscedasticity in data. However, in many practical applications we often encounter heteroscedastic data, i.e., within class scatter matrices can not be expected to be equal. A technique based on the Chernoff criterion for linear dimensionality reduction has been proposed recently. The technique extends well-known Fisher's LDA and is capable of exploiting information about heteroscedasticity in the data. While the Chernoff criterion has been shown to outperform the Fisher's, a clear understanding of its exact behavior is lacking. In addition, the criterion, as introduced, is rather complex, thereby making it difficult to clearly state its relationship to other linear dimensionality techniques. In this paper, we show precisely what can be expected from the Chernoff criterion and its relations to the Fisher criterion and Fukunaga-Koontz transform. Furthermore, we show that a recently proposed decomposition of the data space into four subspaces is incomplete. We provide arguments on how to best enrich the decomposition of the data space in order to account for heteroscedasticity in the data.",
keywords = "Chernoff distance, Dimensionality reduction, Linear discriminant analysis",
author = "Jing Peng and Stefan Robila and Wei Fan and Guna Seetharaman",
year = "2010",
month = "12",
day = "1",
doi = "10.1109/ICSMC.2010.5641971",
language = "English",
isbn = "9781424465880",
series = "Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics",
pages = "3014--3021",
booktitle = "2010 IEEE International Conference on Systems, Man and Cybernetics, SMC 2010",

}

Peng, J, Robila, S, Fan, W & Seetharaman, G 2010, Analysis of chernoff criterion for linear dimensionality reduction. in 2010 IEEE International Conference on Systems, Man and Cybernetics, SMC 2010., 5641971, Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics, pp. 3014-3021, 2010 IEEE International Conference on Systems, Man and Cybernetics, SMC 2010, Istanbul, Turkey, 10/10/10. https://doi.org/10.1109/ICSMC.2010.5641971

Analysis of chernoff criterion for linear dimensionality reduction. / Peng, Jing; Robila, Stefan; Fan, Wei; Seetharaman, Guna.

2010 IEEE International Conference on Systems, Man and Cybernetics, SMC 2010. 2010. p. 3014-3021 5641971 (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics).

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

TY - GEN

T1 - Analysis of chernoff criterion for linear dimensionality reduction

AU - Peng, Jing

AU - Robila, Stefan

AU - Fan, Wei

AU - Seetharaman, Guna

PY - 2010/12/1

Y1 - 2010/12/1

N2 - Well known linear discriminant analysis (LDA) based on the Fisher criterion is incapable of dealing with heteroscedasticity in data. However, in many practical applications we often encounter heteroscedastic data, i.e., within class scatter matrices can not be expected to be equal. A technique based on the Chernoff criterion for linear dimensionality reduction has been proposed recently. The technique extends well-known Fisher's LDA and is capable of exploiting information about heteroscedasticity in the data. While the Chernoff criterion has been shown to outperform the Fisher's, a clear understanding of its exact behavior is lacking. In addition, the criterion, as introduced, is rather complex, thereby making it difficult to clearly state its relationship to other linear dimensionality techniques. In this paper, we show precisely what can be expected from the Chernoff criterion and its relations to the Fisher criterion and Fukunaga-Koontz transform. Furthermore, we show that a recently proposed decomposition of the data space into four subspaces is incomplete. We provide arguments on how to best enrich the decomposition of the data space in order to account for heteroscedasticity in the data.

AB - Well known linear discriminant analysis (LDA) based on the Fisher criterion is incapable of dealing with heteroscedasticity in data. However, in many practical applications we often encounter heteroscedastic data, i.e., within class scatter matrices can not be expected to be equal. A technique based on the Chernoff criterion for linear dimensionality reduction has been proposed recently. The technique extends well-known Fisher's LDA and is capable of exploiting information about heteroscedasticity in the data. While the Chernoff criterion has been shown to outperform the Fisher's, a clear understanding of its exact behavior is lacking. In addition, the criterion, as introduced, is rather complex, thereby making it difficult to clearly state its relationship to other linear dimensionality techniques. In this paper, we show precisely what can be expected from the Chernoff criterion and its relations to the Fisher criterion and Fukunaga-Koontz transform. Furthermore, we show that a recently proposed decomposition of the data space into four subspaces is incomplete. We provide arguments on how to best enrich the decomposition of the data space in order to account for heteroscedasticity in the data.

KW - Chernoff distance

KW - Dimensionality reduction

KW - Linear discriminant analysis

UR - http://www.scopus.com/inward/record.url?scp=78751554089&partnerID=8YFLogxK

U2 - 10.1109/ICSMC.2010.5641971

DO - 10.1109/ICSMC.2010.5641971

M3 - Conference contribution

SN - 9781424465880

T3 - Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics

SP - 3014

EP - 3021

BT - 2010 IEEE International Conference on Systems, Man and Cybernetics, SMC 2010

ER -

Peng J, Robila S, Fan W, Seetharaman G. Analysis of chernoff criterion for linear dimensionality reduction. In 2010 IEEE International Conference on Systems, Man and Cybernetics, SMC 2010. 2010. p. 3014-3021. 5641971. (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics). https://doi.org/10.1109/ICSMC.2010.5641971