Discriminant learning analysis

Jing Peng, Peng Zhang, Norbert Riedel

Research output: Contribution to journalArticle

19 Citations (Scopus)

Abstract

Linear discriminant analysis (LDA) as a dimension reduction method is widely used in classification such as face recognition. However, it suffers from the small sample size (SSS) problem when data dimensionality is greater than the sample size, as in images where features are high dimensional and correlated. In this paper, we propose to address the SSS problem in the framework of statistical learning theory. We compute linear discriminants by regularized least squares regression, where the singularity problem is resolved. The resulting discriminants are complete in that they include both and information. We show that our proposal and its nonlinear extension belong to the same framework where powerful classifiers such as support vector machines are formulated. In addition, our approach allows us to establish an error bound for LDA. Finally, our experiments validate our theoretical analysis results.

Original languageEnglish
Pages (from-to)1614-1625
Number of pages12
JournalIEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Volume38
Issue number6
DOIs
StatePublished - 8 Dec 2008

Fingerprint

Discriminant Analysis
Discriminant analysis
Sample Size
Learning
Face recognition
Support vector machines
Classifiers
Least-Squares Analysis
Experiments

Keywords

  • Dimensionality reduction
  • Discriminant learning analysis (DLA)
  • Feature extraction
  • Linear discriminant analysis (LDA)
  • Regularized least squares (RLS)
  • Small sample size (SSS) problem

Cite this

Peng, Jing ; Zhang, Peng ; Riedel, Norbert. / Discriminant learning analysis. In: IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics. 2008 ; Vol. 38, No. 6. pp. 1614-1625.
@article{a01d4164d5524a14b2934208504b079d,
title = "Discriminant learning analysis",
abstract = "Linear discriminant analysis (LDA) as a dimension reduction method is widely used in classification such as face recognition. However, it suffers from the small sample size (SSS) problem when data dimensionality is greater than the sample size, as in images where features are high dimensional and correlated. In this paper, we propose to address the SSS problem in the framework of statistical learning theory. We compute linear discriminants by regularized least squares regression, where the singularity problem is resolved. The resulting discriminants are complete in that they include both and information. We show that our proposal and its nonlinear extension belong to the same framework where powerful classifiers such as support vector machines are formulated. In addition, our approach allows us to establish an error bound for LDA. Finally, our experiments validate our theoretical analysis results.",
keywords = "Dimensionality reduction, Discriminant learning analysis (DLA), Feature extraction, Linear discriminant analysis (LDA), Regularized least squares (RLS), Small sample size (SSS) problem",
author = "Jing Peng and Peng Zhang and Norbert Riedel",
year = "2008",
month = "12",
day = "8",
doi = "10.1109/TSMCB.2008.2002852",
language = "English",
volume = "38",
pages = "1614--1625",
journal = "IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics",
issn = "1083-4419",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "6",

}

Discriminant learning analysis. / Peng, Jing; Zhang, Peng; Riedel, Norbert.

In: IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, Vol. 38, No. 6, 08.12.2008, p. 1614-1625.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Discriminant learning analysis

AU - Peng, Jing

AU - Zhang, Peng

AU - Riedel, Norbert

PY - 2008/12/8

Y1 - 2008/12/8

N2 - Linear discriminant analysis (LDA) as a dimension reduction method is widely used in classification such as face recognition. However, it suffers from the small sample size (SSS) problem when data dimensionality is greater than the sample size, as in images where features are high dimensional and correlated. In this paper, we propose to address the SSS problem in the framework of statistical learning theory. We compute linear discriminants by regularized least squares regression, where the singularity problem is resolved. The resulting discriminants are complete in that they include both and information. We show that our proposal and its nonlinear extension belong to the same framework where powerful classifiers such as support vector machines are formulated. In addition, our approach allows us to establish an error bound for LDA. Finally, our experiments validate our theoretical analysis results.

AB - Linear discriminant analysis (LDA) as a dimension reduction method is widely used in classification such as face recognition. However, it suffers from the small sample size (SSS) problem when data dimensionality is greater than the sample size, as in images where features are high dimensional and correlated. In this paper, we propose to address the SSS problem in the framework of statistical learning theory. We compute linear discriminants by regularized least squares regression, where the singularity problem is resolved. The resulting discriminants are complete in that they include both and information. We show that our proposal and its nonlinear extension belong to the same framework where powerful classifiers such as support vector machines are formulated. In addition, our approach allows us to establish an error bound for LDA. Finally, our experiments validate our theoretical analysis results.

KW - Dimensionality reduction

KW - Discriminant learning analysis (DLA)

KW - Feature extraction

KW - Linear discriminant analysis (LDA)

KW - Regularized least squares (RLS)

KW - Small sample size (SSS) problem

UR - http://www.scopus.com/inward/record.url?scp=57049189783&partnerID=8YFLogxK

U2 - 10.1109/TSMCB.2008.2002852

DO - 10.1109/TSMCB.2008.2002852

M3 - Article

C2 - 19022731

AN - SCOPUS:57049189783

VL - 38

SP - 1614

EP - 1625

JO - IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

JF - IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

SN - 1083-4419

IS - 6

ER -