Chernoff distance and Relief feature selection

Jing Peng, Guna Seetharaman

Research output: Contribution to conferencePaperResearchpeer-review

2 Citations (Scopus)

Abstract

In classification, a large number of features often make the design of a classifier difficult and degrades its performance. In such situations, feature selection or dimensionality reduction methods play an important role in building classifiers by significantly reducing the number of features. There are many dimensionality reduction techniques for classification in the literature. The most popular one is Fisher's linear discriminant analysis (LDA). For two class problems, LDA simply tries to separate class means as much as possible. For the multi-class case, linear reduction does not guarantee to capture all the relevant information for a classification task. To address this problem, a multi-class problem is cast into a binary problem. The objective becomes to find a subspace where the two classes are well separated. This formulation not only simplifies the problem but also works well in practice. However, it lacks theoretical justification. We show in this paper the connection between the above formulation and RELIEF, thereby providing a sound basis for observed benefits associated with this formulation. Experimental results are provided that corroborate with our analysis.

Original languageEnglish
Pages3493-3496
Number of pages4
DOIs
StatePublished - 1 Dec 2012
Event2012 32nd IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2012 - Munich, Germany
Duration: 22 Jul 201227 Jul 2012

Other

Other2012 32nd IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2012
CountryGermany
CityMunich
Period22/07/1227/07/12

Fingerprint

landform
Feature extraction
Discriminant analysis
discriminant analysis
Classifiers
Acoustic waves
analysis
method

Keywords

  • Chernoff distance
  • Classification
  • Dimensionality reduction
  • Relief

Cite this

Peng, J., & Seetharaman, G. (2012). Chernoff distance and Relief feature selection. 3493-3496. Paper presented at 2012 32nd IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2012, Munich, Germany. https://doi.org/10.1109/IGARSS.2012.6350667
Peng, Jing ; Seetharaman, Guna. / Chernoff distance and Relief feature selection. Paper presented at 2012 32nd IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2012, Munich, Germany.4 p.
@conference{9ccf3a8211b4408cac07eb0b597e1a57,
title = "Chernoff distance and Relief feature selection",
abstract = "In classification, a large number of features often make the design of a classifier difficult and degrades its performance. In such situations, feature selection or dimensionality reduction methods play an important role in building classifiers by significantly reducing the number of features. There are many dimensionality reduction techniques for classification in the literature. The most popular one is Fisher's linear discriminant analysis (LDA). For two class problems, LDA simply tries to separate class means as much as possible. For the multi-class case, linear reduction does not guarantee to capture all the relevant information for a classification task. To address this problem, a multi-class problem is cast into a binary problem. The objective becomes to find a subspace where the two classes are well separated. This formulation not only simplifies the problem but also works well in practice. However, it lacks theoretical justification. We show in this paper the connection between the above formulation and RELIEF, thereby providing a sound basis for observed benefits associated with this formulation. Experimental results are provided that corroborate with our analysis.",
keywords = "Chernoff distance, Classification, Dimensionality reduction, Relief",
author = "Jing Peng and Guna Seetharaman",
year = "2012",
month = "12",
day = "1",
doi = "10.1109/IGARSS.2012.6350667",
language = "English",
pages = "3493--3496",
note = "null ; Conference date: 22-07-2012 Through 27-07-2012",

}

Peng, J & Seetharaman, G 2012, 'Chernoff distance and Relief feature selection' Paper presented at 2012 32nd IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2012, Munich, Germany, 22/07/12 - 27/07/12, pp. 3493-3496. https://doi.org/10.1109/IGARSS.2012.6350667

Chernoff distance and Relief feature selection. / Peng, Jing; Seetharaman, Guna.

2012. 3493-3496 Paper presented at 2012 32nd IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2012, Munich, Germany.

Research output: Contribution to conferencePaperResearchpeer-review

TY - CONF

T1 - Chernoff distance and Relief feature selection

AU - Peng, Jing

AU - Seetharaman, Guna

PY - 2012/12/1

Y1 - 2012/12/1

N2 - In classification, a large number of features often make the design of a classifier difficult and degrades its performance. In such situations, feature selection or dimensionality reduction methods play an important role in building classifiers by significantly reducing the number of features. There are many dimensionality reduction techniques for classification in the literature. The most popular one is Fisher's linear discriminant analysis (LDA). For two class problems, LDA simply tries to separate class means as much as possible. For the multi-class case, linear reduction does not guarantee to capture all the relevant information for a classification task. To address this problem, a multi-class problem is cast into a binary problem. The objective becomes to find a subspace where the two classes are well separated. This formulation not only simplifies the problem but also works well in practice. However, it lacks theoretical justification. We show in this paper the connection between the above formulation and RELIEF, thereby providing a sound basis for observed benefits associated with this formulation. Experimental results are provided that corroborate with our analysis.

AB - In classification, a large number of features often make the design of a classifier difficult and degrades its performance. In such situations, feature selection or dimensionality reduction methods play an important role in building classifiers by significantly reducing the number of features. There are many dimensionality reduction techniques for classification in the literature. The most popular one is Fisher's linear discriminant analysis (LDA). For two class problems, LDA simply tries to separate class means as much as possible. For the multi-class case, linear reduction does not guarantee to capture all the relevant information for a classification task. To address this problem, a multi-class problem is cast into a binary problem. The objective becomes to find a subspace where the two classes are well separated. This formulation not only simplifies the problem but also works well in practice. However, it lacks theoretical justification. We show in this paper the connection between the above formulation and RELIEF, thereby providing a sound basis for observed benefits associated with this formulation. Experimental results are provided that corroborate with our analysis.

KW - Chernoff distance

KW - Classification

KW - Dimensionality reduction

KW - Relief

UR - http://www.scopus.com/inward/record.url?scp=84873162255&partnerID=8YFLogxK

U2 - 10.1109/IGARSS.2012.6350667

DO - 10.1109/IGARSS.2012.6350667

M3 - Paper

SP - 3493

EP - 3496

ER -

Peng J, Seetharaman G. Chernoff distance and Relief feature selection. 2012. Paper presented at 2012 32nd IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2012, Munich, Germany. https://doi.org/10.1109/IGARSS.2012.6350667