Chernoff dimensionality reduction-where fisher meets FKT

Jing Peng, Guna Seetharaman, Wei Fan, Stefan Robila, Aparna Varde

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

3 Citations (Scopus)

Abstract

Well known linear discriminant analysis (LDA) based on the Fisher criterion is incapable of dealing with heteroscedasticity in data. However, in many practical applications we often encounter heteroscedastic data, i.e., within-class scatter matrices can not be expected to be equal. A technique based on the Chernoff criterion for linear dimensionality reduction has been proposed recently. The technique extends well-known Fisher's LDA and is capable of exploiting information about heteroscedasticity in the data. While the Chernoff criterion has been shown to outperform the Fisher's, a clear understanding of its exact behavior is lacking. In addition, the criterion, as introduced, is rather complex, making it difficult to clearly state its relationship to other linear dimensionality reduction techniques. In this paper, we show precisely what can be expected from the Chernoff criterion and its relations to the Fisher criterion and Fukunaga-Koontz transform. Furthermore, we show that a recently proposed decomposition of the data space into four subspaces is incomplete. We provide arguments on how to best enrich the decomposition of the data space in order to account for heteroscedasticity in the data. Finally, we provide experimental results validating our theoretical analysis.

Original languageEnglish
Title of host publicationProceedings of the 11th SIAM International Conference on Data Mining, SDM 2011
Pages271-282
Number of pages12
StatePublished - 1 Dec 2011
Event11th SIAM International Conference on Data Mining, SDM 2011 - Mesa, AZ, United States
Duration: 28 Apr 201130 Apr 2011

Other

Other11th SIAM International Conference on Data Mining, SDM 2011
CountryUnited States
CityMesa, AZ
Period28/04/1130/04/11

Fingerprint

Discriminant analysis
Decomposition

Keywords

  • Chernoff distance
  • Dimension reduction
  • FKT
  • LDA

Cite this

Peng, J., Seetharaman, G., Fan, W., Robila, S., & Varde, A. (2011). Chernoff dimensionality reduction-where fisher meets FKT. In Proceedings of the 11th SIAM International Conference on Data Mining, SDM 2011 (pp. 271-282)
Peng, Jing ; Seetharaman, Guna ; Fan, Wei ; Robila, Stefan ; Varde, Aparna. / Chernoff dimensionality reduction-where fisher meets FKT. Proceedings of the 11th SIAM International Conference on Data Mining, SDM 2011. 2011. pp. 271-282
@inproceedings{619cc8257a0c49fc980a7bb80109c70d,
title = "Chernoff dimensionality reduction-where fisher meets FKT",
abstract = "Well known linear discriminant analysis (LDA) based on the Fisher criterion is incapable of dealing with heteroscedasticity in data. However, in many practical applications we often encounter heteroscedastic data, i.e., within-class scatter matrices can not be expected to be equal. A technique based on the Chernoff criterion for linear dimensionality reduction has been proposed recently. The technique extends well-known Fisher's LDA and is capable of exploiting information about heteroscedasticity in the data. While the Chernoff criterion has been shown to outperform the Fisher's, a clear understanding of its exact behavior is lacking. In addition, the criterion, as introduced, is rather complex, making it difficult to clearly state its relationship to other linear dimensionality reduction techniques. In this paper, we show precisely what can be expected from the Chernoff criterion and its relations to the Fisher criterion and Fukunaga-Koontz transform. Furthermore, we show that a recently proposed decomposition of the data space into four subspaces is incomplete. We provide arguments on how to best enrich the decomposition of the data space in order to account for heteroscedasticity in the data. Finally, we provide experimental results validating our theoretical analysis.",
keywords = "Chernoff distance, Dimension reduction, FKT, LDA",
author = "Jing Peng and Guna Seetharaman and Wei Fan and Stefan Robila and Aparna Varde",
year = "2011",
month = "12",
day = "1",
language = "English",
isbn = "9780898719925",
pages = "271--282",
booktitle = "Proceedings of the 11th SIAM International Conference on Data Mining, SDM 2011",

}

Peng, J, Seetharaman, G, Fan, W, Robila, S & Varde, A 2011, Chernoff dimensionality reduction-where fisher meets FKT. in Proceedings of the 11th SIAM International Conference on Data Mining, SDM 2011. pp. 271-282, 11th SIAM International Conference on Data Mining, SDM 2011, Mesa, AZ, United States, 28/04/11.

Chernoff dimensionality reduction-where fisher meets FKT. / Peng, Jing; Seetharaman, Guna; Fan, Wei; Robila, Stefan; Varde, Aparna.

Proceedings of the 11th SIAM International Conference on Data Mining, SDM 2011. 2011. p. 271-282.

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

TY - GEN

T1 - Chernoff dimensionality reduction-where fisher meets FKT

AU - Peng, Jing

AU - Seetharaman, Guna

AU - Fan, Wei

AU - Robila, Stefan

AU - Varde, Aparna

PY - 2011/12/1

Y1 - 2011/12/1

N2 - Well known linear discriminant analysis (LDA) based on the Fisher criterion is incapable of dealing with heteroscedasticity in data. However, in many practical applications we often encounter heteroscedastic data, i.e., within-class scatter matrices can not be expected to be equal. A technique based on the Chernoff criterion for linear dimensionality reduction has been proposed recently. The technique extends well-known Fisher's LDA and is capable of exploiting information about heteroscedasticity in the data. While the Chernoff criterion has been shown to outperform the Fisher's, a clear understanding of its exact behavior is lacking. In addition, the criterion, as introduced, is rather complex, making it difficult to clearly state its relationship to other linear dimensionality reduction techniques. In this paper, we show precisely what can be expected from the Chernoff criterion and its relations to the Fisher criterion and Fukunaga-Koontz transform. Furthermore, we show that a recently proposed decomposition of the data space into four subspaces is incomplete. We provide arguments on how to best enrich the decomposition of the data space in order to account for heteroscedasticity in the data. Finally, we provide experimental results validating our theoretical analysis.

AB - Well known linear discriminant analysis (LDA) based on the Fisher criterion is incapable of dealing with heteroscedasticity in data. However, in many practical applications we often encounter heteroscedastic data, i.e., within-class scatter matrices can not be expected to be equal. A technique based on the Chernoff criterion for linear dimensionality reduction has been proposed recently. The technique extends well-known Fisher's LDA and is capable of exploiting information about heteroscedasticity in the data. While the Chernoff criterion has been shown to outperform the Fisher's, a clear understanding of its exact behavior is lacking. In addition, the criterion, as introduced, is rather complex, making it difficult to clearly state its relationship to other linear dimensionality reduction techniques. In this paper, we show precisely what can be expected from the Chernoff criterion and its relations to the Fisher criterion and Fukunaga-Koontz transform. Furthermore, we show that a recently proposed decomposition of the data space into four subspaces is incomplete. We provide arguments on how to best enrich the decomposition of the data space in order to account for heteroscedasticity in the data. Finally, we provide experimental results validating our theoretical analysis.

KW - Chernoff distance

KW - Dimension reduction

KW - FKT

KW - LDA

UR - http://www.scopus.com/inward/record.url?scp=84873201295&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9780898719925

SP - 271

EP - 282

BT - Proceedings of the 11th SIAM International Conference on Data Mining, SDM 2011

ER -

Peng J, Seetharaman G, Fan W, Robila S, Varde A. Chernoff dimensionality reduction-where fisher meets FKT. In Proceedings of the 11th SIAM International Conference on Data Mining, SDM 2011. 2011. p. 271-282